RefineDataMCP

RefineData MCP is a Model Context Protocol server that gives AI agents (Claude Code, Cursor, ChatGPT) a toolbox for data preparation, anonymization, analysis, and visualization. It leans on Polars and DuckDB so the heavy tools stream data out-of-core instead of loading everything into RAM.

Tools

Tool	Purpose	Engine	Scales beyond RAM?
analyze_data	Row counts, schema, numeric stats	DuckDB	✅ out-of-core
query_db	Run SQL on PostgreSQL/SQLite → file	DuckDB	✅ out-of-core
process_data	Dedup + missing-value handling	Polars (lazy/streaming)	◐ streamed
simplify_quant_data	Percentiles, skew, correlations	Polars (lazy/streaming)	◐ streamed
filter_toxic_data	Drop toxic rows (batched inference)	Detoxify	✗ in-memory
anonymize_data	Mask PII in text columns	Presidio NLP	✗ in-memory
scrape_web_data	Extract the main HTML table from a URL	requests + BeautifulSoup	✗ in-memory
visualize_dataset	Heatmap/scatter/boxplot/bar → image	Matplotlib/Seaborn	✗ in-memory

◐ = the streamable parts of the plan run in chunks; the result is materialized. The PII/toxicity/scraping/viz tools load the working set into memory by design.

Installation

Core install (only analyze_data + query_db) is lightweight. Heavy features are optional extras, so you don't pull in PyTorch/spaCy/Matplotlib unless you need them.

python -m venv venv && . venv/Scripts/activate   # Windows
# source venv/bin/activate                       # Linux/Mac

pip install -e .                # core (DuckDB + Polars)
pip install -e ".[scraping]"    # + web scraper
pip install -e ".[viz]"         # + visualizer
pip install -e ".[pii]"         # + PII anonymizer
pip install -e ".[toxicity]"    # + toxicity filter
pip install -e ".[all]"         # everything

If you call a tool whose extra isn't installed, it returns a clear message telling you which pip install to run.

The PII tool also needs a spaCy model:

python -m spacy download en_core_web_sm

Supported Python: 3.11–3.12 (the heavy ML deps don't ship 3.13/3.14 wheels yet).

Configuration for MCP clients

{
  "mcpServers": {
    "RefineDataMCP": {
      "command": "refinedata-mcp",
      "cwd": "/absolute/path/to/refinedata-mcp"
    }
  }
}

refinedata-mcp is installed as a console script. Alternatively use "command": "python", "args": ["-m", "refinedata_mcp"].

Environment variables

Variable	Default	Effect
REFINEDATA_LOG_LEVEL	INFO	Log verbosity (logs go to stderr)
REFINEDATA_MAX_FILE_SIZE_MB	1024	Reject in-memory loads larger than this (0 = off)
REFINEDATA_WORKDIR	unset	Sandbox: restrict all file reads/writes to this dir
REFINEDATA_HTTP_TIMEOUT	15	Web scraper request timeout (seconds)
REFINEDATA_ALLOW_PRIVATE_URLS	false	Allow scraping private/loopback IPs (SSRF override)
REFINEDATA_DB_READ_ONLY	true	Attach external databases read-only
REFINEDATA_DB_TIMEOUT	60	Best-effort DB query timeout (seconds, 0 = off)
REFINEDATA_PII_LANGUAGE	en	Presidio scan language
REFINEDATA_DETOXIFY_MODEL	original	Detoxify model variant

Security notes

• The web scraper blocks requests to private/loopback/link-local/metadata IPs (SSRF guard). Override only for trusted local use with REFINEDATA_ALLOW_PRIVATE_URLS=true.
• query_db executes the SQL you pass it and attaches databases read-only by default.
• Set REFINEDATA_WORKDIR to confine file access to a single directory.
• This server is built for a single trusted user (your agent). Do not expose it to untrusted input without adding allow-lists.

Testing

pip install -e ".[dev]"
pytest

Tests for optional features importorskip their dependencies, so the suite runs (skipping what isn't installed) at any install level. See ROADMAP.md for what's done and what's next.