Data Platform

txn2/mcp-data-platform

GitHub license Go Reference MCP Release CI CodeQL codecov OpenSSF Best Practices OpenSSF Scorecard Signed by Cosign Docker Benchmark Report: Knowledge Layer Benchmark Report: Knowledge Use

Documentation | Installation | Quick Start | Go Library

Your AI assistant can run SQL. But it doesn't know that cust_id contains PII, that the table was deprecated last month, or who to ask when something breaks.

mcp-data-platform fixes that. It is a single MCP server that connects AI assistants to your data infrastructure and enriches every response with business context from your semantic layer: query a table and get its meaning, owners, quality scores, and deprecation warnings in the same call.

It is a platform, not just a bridge. The same endpoint gives agents persistent memory and a governed path to write knowledge back to the catalog, proxies third-party MCP servers and REST APIs through one authentication, persona, and audit pipeline, and ships a web portal where AI-generated assets are saved, organized into collections, and shared with teammates.

---

Do you need DataHub? Only for cross-enrichment

Cross-enrichment is what DataHub is for: point the platform at it as the semantic layer, then add Trino for SQL and S3 for object storage when you're ready. Learn why this stack.

Everything else runs without it. DataHub is an adapter behind a provider interface, not a substrate the platform is built on. Omit the semantic: block and the semantic provider resolves to a noop, the server starts normally, and the gateways, knowledge layer, memory, portal, and search/fetch run on PostgreSQL alone. What you give up is cross-enrichment and the datahub_* tools, not the platform: semantic:, query:, storage:, and toolkits: are independent config blocks, so Trino and S3 stay available on their own terms and simply stop being enriched. See Deployment Shapes for exactly what each shape includes and leaves out.

---

Why

AI assistants are powerful at querying data, but they work blind. When an agent asks "What's in the orders table?", it gets column names and types. It doesn't know that customer_id is PII, that the table is deprecated in favor of orders_v2, that the quality score dropped last week, or who to contact when something looks wrong.

# Without mcp-data-platform
─────────────────────────────────────────────────────────────────────
User:      "Describe the orders table"
AI:        Queries Trino → gets columns and types
User:      "Who owns this data?"
AI:        Queries DataHub → finds owners
User:      "Is this table still active?"
AI:        Queries DataHub again → finds deprecation status
User:      "What does customer_id actually mean?"
AI:        Queries DataHub again → finds column descriptions
─────────────────────────────────────────────────────────────────────
4 round trips. Context scattered across conversations. Easy to miss warnings.

# With mcp-data-platform
─────────────────────────────────────────────────────────────────────
User:      "Describe the orders table"
AI:        Gets everything in one response:
           → Schema: columns and types
           → ⚠️ DEPRECATED: Use orders_v2 instead
           → Owners: Data Platform Team
           → Tags: pii, financial
           → Quality Score: 87%
           → Column meanings and business definitions
─────────────────────────────────────────────────────────────────────
1 call. Complete context. Warnings front and center.

How It Works

sequenceDiagram
    participant AI as AI Assistant
    participant P as mcp-data-platform
    participant T as Trino
    participant D as DataHub

    AI->>P: trino_describe_table "orders"
    P->>T: DESCRIBE orders
    T-->>P: columns, types
    P->>D: Get semantic context
    D-->>P: description, owners, tags, quality, deprecation
    P-->>AI: Schema + Full Business Context

The platform intercepts tool responses at the protocol level and enriches them with context from the other services. This cross-enrichment is bidirectional:

• Trino → DataHub: query results include owners, tags, glossary terms, deprecation warnings, quality scores
• DataHub → Trino: search results include query availability and sample SQL
• S3 ↔ DataHub: object listings include matching dataset metadata, and dataset searches show storage availability

Does it work? Measured effectiveness

The knowledge layer is not just a design claim; it is benchmarked. On knowledge-trap questions, the ones an agent answers plausibly but wrongly without business context, connecting the agent to the platform's semantic knowledge layer lifts accuracy from 42.7% (raw data tools) to 98.7%, a +56-point gain (95% CI +44 to +67). On plain lookups and arithmetic, where no business context is needed, the platform and bare tools are statistically tied, so the gain is specific to knowledge-gated questions, not a blanket accuracy boost.

This is an arm-vs-arm result on a single pinned model (same model, same tasks, only the platform configuration changes). It measures the knowledge layer specifically, that is cross-enrichment, search, and the memory/apply_knowledge lifecycle, not the whole platform. Every number is recomputed from committed raw data by a notebook that needs no API key. See the full, citable Benchmark Report (four-arm ablation, cold-start learning curve, lifecycle scorecard, threats to validity) and the benchmark report series, which indexes every study's protocol, toolchain, and archived run data.

Features

Each feature links to its full documentation.

Semantic data access

Feature	Description
Cross-enrichment	Business context added to every tool response automatically, with session dedup to save tokens
Lineage inheritance	Column descriptions inherited from upstream datasets via DataHub lineage
Universal search	One search tool fans a query across the catalog, knowledge pages, memory, insights, assets, prompts, and APIs; fetch dereferences any result
Workflow gating	Session-aware guidance that steers agents to discovery before SQL, with escalating warnings
Tools	Full tool reference for Trino, DataHub, S3, knowledge, memory, portal, and gateway toolkits

Knowledge and memory

Feature	Description
Memory layer	Persistent agent memory across sessions, PostgreSQL + pgvector, hybrid semantic/lexical recall
Knowledge capture	Agents record domain insights during sessions; approved knowledge is written back to DataHub or canonical knowledge pages
Governance workflow	Human-in-the-loop review, approve/reject, changeset tracking, and rollback for every applied change
Managed resources	Human-uploaded reference files (playbooks, samples, templates) served to agents as MCP resources

Gateways and extensibility

Feature	Description
MCP gateway	Re-expose any third-party MCP server through the platform's auth, persona, and audit pipeline
API gateway	Proxy REST/HTTP APIs (Salesforce, Google, GitHub, Stripe) with four tools instead of one tool per endpoint
API catalogs	Versioned OpenAPI bundles shared across connections, with semantic endpoint ranking
REST invoke shim	Call gateway endpoints from NiFi, Airflow, or curl under the same auth and audit pipeline
Self-configuration	Admins manage personas, connections, and prompts by asking the agent instead of clicking
MCP Apps	Interactive UI panels rendered inline in the MCP host
Go library	Import the platform as a library: custom toolkits, providers, and middleware

Security and operations

Feature	Description
Authentication	Fail-closed model: OIDC (Keycloak, Auth0, Okta, Azure AD) and API keys for service accounts
OAuth 2.1 server	A broker, not an identity provider: authorization server with PKCE and Dynamic Client Registration toward MCP clients, delegating every human login upstream to your IdP
Outbound OAuth	OAuth to upstream MCPs and APIs with encrypted refresh tokens that survive restarts
Personas	Role-mapped allow/deny tool and connection filtering, default-deny; roles that match no persona reach nothing, in the portal as well as over MCP
Audit logging	Every tool call logged to PostgreSQL with identity, persona, sanitized parameters, and timing
Observability	Prometheus metrics and optional OpenTelemetry distributed tracing
Session externalization	PostgreSQL-backed sessions for zero-downtime restarts, horizontal scaling, and live tool-inventory updates
Explicit session handles	platform_info mints a session_id the agent threads on every call, making orientation unskippable and readying the platform for the sessionless MCP 2026-07-28 protocol
Multi-provider	Multiple instances of each service behind one endpoint, with isolated failure domains
Operating modes	Standalone (no database) or file + database with live config overrides resolved per read
Deployment shapes	Which backends you need: the semantic stack for cross-enrichment, PostgreSQL alone for the gateways and knowledge layer, or both
Email notifications	Branded emails for shares and feedback: admin-configured SMTP, per-user preferences (immediate, daily digest, or off), durable queue with retries, a per-share notify toggle and optional plain-text note, and delivery history for admins and for each recipient

The Portal

A built-in web portal serves both operators and end users. Enable with portal.enabled: true.

For operators: dashboards with activity timelines and performance percentiles, a searchable audit log, an interactive tool explorer with per-persona visibility and inline test runs, knowledge insight governance, connection and persona management, API keys, and indexing health. See the Admin Portal guide.

For users: AI-generated assets (reports, charts, documents) are saved from any session with the save_asset tool, organized into shareable collections, and shared with teammates or through public links. A prompt library, feedback threads on any asset, and personal knowledge and activity views round out the User Portal.

Quick Start

Install (see all methods: Homebrew, Docker, source):

go install github.com/txn2/mcp-data-platform/cmd/mcp-data-platform@latest

Create a minimal configuration. This one wires the semantic layer, which is what cross-enrichment needs; ${VAR} references are expanded from the environment:

# platform.yaml
server:
  name: mcp-data-platform
  transport: stdio

semantic:
  provider: datahub
  instance: primary

toolkits:
  datahub:
    enabled: true
    instances:
      primary:
        url: "${DATAHUB_URL}"
        token: "${DATAHUB_TOKEN}"
    default: primary

Wire it to Claude Code:

claude mcp add data-platform \
  -e DATAHUB_URL=https://datahub.example.com/api/graphql \
  -e DATAHUB_TOKEN=$TOKEN \
  -- mcp-data-platform --config platform.yaml

Starting without a warehouse or catalog? Swap the semantic: and toolkits: blocks above for a database and the API toolkit, and the gateways, knowledge layer, memory, portal, and search/fetch come up on PostgreSQL alone:

# platform.yaml
server:
  name: mcp-data-platform
  transport: http
  address: ":8080"

database:
  dsn: "${DATABASE_URL}"

# API connections are authored in the admin portal, not in YAML.
toolkits:
  api:
    enabled: true

Deployment Shapes covers the full configuration, what each shape includes, and what it leaves out.

For a hosted deployment, run --transport http and enable the built-in OAuth 2.1 server so Claude and other MCP clients sign in through your identity provider. That server is a broker, not an identity provider: it hands every human login to your IdP. See Configuration, Deployment (Docker Compose, Kubernetes), and the OAuth 2.1 Server guide.

Security

The platform implements a fail-closed security model: missing or invalid credentials deny access, never bypass. Personas are default-deny, Trino and S3 support enforced read-only mode, and metadata is sanitized against prompt injection. See the Auth Overview and MCP Defense: A Case Study in AI Security for the architecture rationale.

Transport	Authentication	TLS
stdio	Not required (local execution)	N/A
HTTP	Required (Bearer token or API key)	Strongly recommended

A broker, not an identity provider

mcp-data-platform is an OAuth 2.1 broker, not an identity provider. No person authenticates to it: there is no login form, no user password to verify, and no MFA. A human's identity comes from your existing IdP (Keycloak, Auth0, Okta, Azure AD) over OIDC. /authorize redirects the browser there and refuses the flow outright when no upstream IdP is configured, and the roles and email that person is authorized against are the ones the IdP asserts. Service accounts authenticate with API keys instead, and their roles come from local configuration.

It stores no human passwords, and no migration in the tree defines a password column. The secrets it does hold are machine credentials, held the way an auditor would want: API keys and the client secrets Dynamic Client Registration issues to MCP client software are bcrypt hashes, the authorization codes and tokens the platform itself issues are SHA-256 digests, and refresh tokens for upstream services are encrypted at rest (AES-256-GCM when ENCRYPTION_KEY is set; the server warns loudly at startup when it is not).

The platform presents an authorization server toward MCP clients because the MCP specification requires a discoverable authorization server supporting Dynamic Client Registration, which upstream IdPs generally do not expose. The broker shape is what the spec requires, not a decision to reimplement identity.

The parts an auditor reaches for first:

Concern	Implementation
redirect_uri matching	Exact match for non-loopback; RFC 8252 section 7.3 handling for loopback (pkg/oauth/storage.go)
DCR abuse	Plain HTTP to non-loopback hosts refused regardless of configuration; private-use schemes excluded from AllowAllRedirectURIs, so the unauthenticated registration endpoint never hands out scheme hijacking by default (pkg/oauth/dcr.go)
Brute force and registration flood	Per-IP token-bucket limits on /token and /register, applied before the bcrypt work they would otherwise burn (pkg/oauth/ratelimit.go)
Authorization	Deny-before-allow, default deny, fail-closed on unresolved persona (pkg/persona/filter.go)
Prompt injection carried in catalog metadata	Untrusted descriptions, tags, and owner notes are sanitized before they reach the model, and detected attempts are logged (pkg/semantic/sanitize.go, pkg/semantic/injection_logger.go)

Engineering posture

More than 1.25 lines of test code per line of production Go, with the security-critical packages carrying the highest ratios in the tree: pkg/oauth and pkg/middleware are both above 2:1. Fuzz suites cover pkg/oauth, pkg/auth, pkg/platform, and pkg/middleware. Every PR passes race-detector tests, golangci-lint, gosec, and Semgrep and CodeQL SAST, under a coverage floor enforced in CI. Release artifacts are Cosign-signed with GitHub build-provenance attestations, and supply-chain posture is tracked by OpenSSF Scorecard.

Those ratios are claims a reader can check, so they are kept true mechanically rather than by hand: make posture-check recomputes them and fails when the tree crosses a stated line.

Ecosystem

mcp-data-platform is the orchestration layer for a suite of open-source MCP servers that also run standalone:

• txn2/mcp-datahub: DataHub metadata: search, lineage, glossary, domains, tags, ownership
• txn2/mcp-trino: Trino distributed SQL with configurable timeouts and row limits
• txn2/mcp-s3: S3 object storage: buckets, prefixes, objects, presigned URLs

See Ecosystem for how they compose.

Documentation

Full documentation lives at mcp-data-platform.txn2.com.

• Server Guide: architecture, configuration, deployment
• Cross-Enrichment: how automatic enrichment works
• Authentication: OIDC, API keys, OAuth 2.1
• Knowledge Capture and Memory: the agent knowledge loop
• Go Library: build custom MCP servers (API stability policy)
• Tools API Reference: complete tool specifications
• Examples Gallery: real-world configurations
• Troubleshooting: common issues and debugging

Development

go build -o mcp-data-platform ./cmd/mcp-data-platform   # build
go test -race ./...                                     # tests
make verify                                             # full CI-equivalent suite
make osv                                                # osv-scanner, informational (mirrors OpenSSF Scorecard)

make verify runs govulncheck, which does reachability analysis and reports only vulnerabilities your code actually calls. make osv runs osv-scanner the way OpenSSF Scorecard does, flagging every vulnerable package in the dependency graph regardless of reachability. It is informational and not part of verify; suppressions for non-reachable and test-only findings are documented with justification and expiry in osv-scanner.toml.

The React admin portal lives under ui/. Its CI job runs npm run lint, which enforces per-function complexity budgets that mirror the Go gates (complexity <= 10 ≈ gocyclo <= 10, cognitive-complexity <= 15 ≈ gocognit <= 15) plus an import-cycle rule. See ui/README.md for the thresholds and the ratchet baseline.

One browser suite also lives outside make verify: make frontend-e2e-public-viewer renders the public share viewer's client-rendered content families — HTML, JSX, markdown, SVG and a collection item — against a live stack and fails on anything the viewer's Content-Security-Policy blocks. It needs make frontend-build, a running server (make dev) and network egress to esm.sh, so it is run on demand rather than per commit — see ui/e2e/public-viewer/README.md.

Two measurement harnesses live outside make verify (each is its own Go module): test/load measures throughput and resource limits ("how much"), and bench/ measures agent effectiveness — arm-ablated accuracy and efficiency with audit-derived metrics ("how well"). Run them via make load-* and make bench-* targets.

Contributions for bug fixes, tests, and documentation are welcome. Please run make verify (formatting, race-detected tests, coverage, linting, security scanning) before opening a pull request.

License

Apache License 2.0

---

Open source by Craig Johnston, sponsored by Deasil Works, Inc. and Plexara