Conformance Core

mcp-conformance

Independent conformance testing for the Model Context Protocol. Record a trace of any MCP session — in any language, over any transport — and find out exactly which of the spec's requirements it met, which it broke, and why.

Status: 0.5.1 on crates.io (cargo install mcp-trace-validator), published with SLSA build-provenance attestations. Pre-1.0, so the API and the verdicts it produces may still change between minor releases — the changelog says so explicitly when they do.

---

The problem

The Model Context Protocol is a specification: a long list of normative requirements, each a MUST, SHOULD, or MAY that a conforming implementation is expected to honor. If you build an MCP server or client, how do you actually know it conforms?

Today there is essentially one answer — the official conformance suite, which drives live scenarios written in TypeScript. That suite is the authority, and it is invaluable. But it leaves a real gap:

• Nothing takes a recording of an MCP session — whatever language produced it, whatever transport it crossed — and checks it, requirement by requirement, against the spec.
• In Rust there is no reference everything server or host to measure against at all.

What this is

mcp-conformance is the missing half — a toolkit built around three verbs:

• Capture a trace of an MCP session: a plain JSON Lines file, one event per line.
• Validate it offline and get requirement-level findings — the exact spec clause, the offending message, and a plain-language reason — as human text, machine JSON, or JUnit XML for CI.
• Calibrate against the authority: the reference server and host bundled here are driven by the official suite on every CI run, and this toolkit's verdicts are diffed against the official runner's. A disagreement fails the build.

That last point is the whole game. A conformance verdict is only worth as much as its credibility, so these verdicts are continuously checked against the recognized authority — deterministic, reproducible from a committed file, and defensible rather than a "trust us."

See it work

Install the validator and point it at a recorded session:

$ mcp-trace-validator validate session.jsonl
MCP trace validation — revision 2025-11-25
  PASS  BASE-001 (MUST)
  ...
  FAIL  LIFE-001 (MUST)
        seq 0: first message is a "tools/list" request, expected "initialize"
totals: 10 pass, 1 fail, 1 warn, 87 excluded, 0 unsupported, 14 not applicable, 29 not observed
verdict: fail

The exit code is documented (0 pass, 1 findings, 2 bad invocation, 3 malformed trace), so this drops straight into CI. A trace that judges no requirement at all — the shape a broken capture step produces — is a bad invocation rather than a pass: an empty recording is not a conforming session, and a green build is the wrong thing to tell you about one. And a recording of another revision — a 2026-07-28 session judged against the default 2025-11-25 registry — says so in the report rather than quietly reporting the clauses the two revisions disagree about as violations.

Note what the totals do not say. A clause the session never came near is not observed — never a pass. A trace that opens a connection and stops has complied with nothing; reporting it as 100-odd passes would be a score, not a verdict, and the whole tool is worth exactly as much as that distinction.

The one idea: capture, then judge

The validator is a pure function — a slice of trace events in, a report out — with no network, no clock, and no I/O of its own. Whoever owns the socket (the reference server's session tap, the host's capture wrapper, or any external proxy) records the trace and assigns the ordering; the validator's only job is to judge it. The judge also never links the SDK it judges, so its verdicts stay independent of any one implementation's interpretation of the spec.

That separation is what buys determinism, replayability, and language/transport independence: the same trace yields a byte-identical report forever, on any platform — a regression is a diff, not a flake. The design and its trade-offs are written up for an external audience in docs/design/trace-validation.md.

The toolkit

Crate	What it gives you
mcp-conformance-core	The spec as data. A requirement registry whose every entry carries a verbatim spec quote, an RFC 2119 level, an optional capability gate, and either a mechanical check or a documented reason it cannot be judged from a trace (the SEP-2484 traceability shape) — covering the 2025-11-25 core protocol surface. Plus the JSON Lines trace schema and RFC 8785 canonical JSON. Serde only; it links no protocol SDK.
mcp-trace-validator	The validator and its CLI. Replay a trace; get findings with the spec clause and the offending event seq, as human text, JSON, or JUnit, with documented exit codes. Every check is falsified by at least one committed violation trace in corpus/ — a check that cannot fail is not a check.
mcp-everything-server	The reference server, on rmcp (the official Rust SDK). It passes the official suite's full 2025-11-25 server surface — 40/40 checks — over stdio and policy-gated streamable HTTP, with a default-secure Host/Origin policy that closes the CVE-2026-42559 DNS-rebinding class by construction. --protocol-version 2026-07-28 serves the stateless surface instead (SEP-2575: no initialize, no sessions, per-request _meta, SEP-2549 caching hints, SEP-2322 MRTR for server-to-client requests, subscriptions/listen), over stdio and HTTP alike — the suite's 2026-07-28 scenarios score 41 passing / 0 failing against that mode (the same scenarios score 37 passing / 4 failing against the 2025-11-25 mode, each failure the caching hints this revision adds), and five committed captures — a conforming session over each transport, a probe session of deliberately malformed requests, and the official runner's two — evidence 114 of the 125 judgeable clauses between them, with everything else reported not observed rather than counted as a pass. Its tap records each session as a trace for the calibration check. Offered upstream as rust-sdk#902.
mcp-reference-host	The reference host (an MCP client). It passes all four of the official suite's 2025-11-25 client scenarios at the pinned version — bounded tool-use loops over both real transports (child-process stdio and streamable HTTP), scriptable sampling / elicitation / roots for CI with zero model-provider network use, and host-side trace capture with redaction by construction.

Requirement coverage

The table is generated from the registry by cargo xtask coverage and verified in CI — the numbers cannot drift from the data:

Area	Requirements	Checked	Excluded	Capability-gated
BASE	25	12	13	0
LIFE	18	10	8	0
TRAN	49	12	37	0
TOOL	15	9	6	13
RES	10	3	7	6
PROM	10	5	5	7
LOG	5	1	4	4
COMP	5	1	4	3
PAGE	5	2	3	0
Total	142	55	87	33

Revision 2025-11-25: 142 requirements — 55 judged by 51 distinct trace checks (every check falsified by a committed violation trace, and every check examining a real subject on at least one of them), 87 carrying documented exclusions explaining why a recorded trace cannot judge them. A requirement is reported pass only where the session carried something it binds to: a capability-gated clause the session never negotiated reports not-applicable, and a clause whose subject matter never appeared reports not-observed. Neither is a vacuous pass.

A requirement gated on a capability that was never negotiated is reported not-applicable, and one whose subject matter the session never carried is reported not-observed — neither is ever a pass. Every check counts the subjects it considered, so the report can tell "complied with" from "never came up"; inflating a score with vacuous checks is exactly how a conformance tool loses credibility.

The trace format

A trace is JSON Lines: one event per line, each carrying a capture-assigned seq (the only ordering authority), a direction, a transport, and a kind — message events hold the JSON-RPC payload verbatim; http events record conformance-relevant headers, a response's status, and a client request's method; lifecycle events mark transport open/close. This session reuses a request ID:

{"seq":0,"direction":"client-to-server","transport":"stdio","kind":"message","payload":{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":"2025-11-25","capabilities":{},"clientInfo":{"name":"my-host","version":"1.0.0"}}}}
{"seq":1,"direction":"server-to-client","transport":"stdio","kind":"message","payload":{"jsonrpc":"2.0","id":1,"result":{"protocolVersion":"2025-11-25","capabilities":{"tools":{}},"serverInfo":{"name":"my-server","version":"1.0.0"}}}}
{"seq":2,"direction":"client-to-server","transport":"stdio","kind":"message","payload":{"jsonrpc":"2.0","method":"notifications/initialized"}}
{"seq":3,"direction":"client-to-server","transport":"stdio","kind":"message","payload":{"jsonrpc":"2.0","id":1,"method":"tools/list"}}

and the validator answers with the violated clause, verbatim from the spec via the registry, addressed to the offending event:

  FAIL  BASE-003 (MUST NOT)
        seq 3: request "tools/list" reuses id 1, already used by the same party at seq 0
totals: 17 pass, 1 fail, 0 warn, 87 excluded, 0 unsupported, 6 not applicable, 31 not observed
verdict: fail

The six not-applicable rows are the capability-gated requirements this session never negotiated (the resources and prompts clauses), and the thirty not-observed rows are the clauses whose subject matter never appeared — nothing was paginated, no binary content was sent, no error was returned. Neither is reported as a pass. corpus/ holds complete annotated sessions for every area.

Documentation

• The book — architecture, the trace format, the corpus guide, and conformance results, collected as an mdBook (book/) that builds on every push and deploys to https://mcp-conformance.com.
• API docs for every crate on docs.rs.
• The engineered plan — charter, verified ecosystem register, architecture, conformance strategy, engineering standards, security model, roadmap, and decision records — in docs/plan/. Every claim is verified and dated.

Why it exists

Conformance is the load-bearing mechanism of MCP's maturity model: SEP-1730 gates an SDK's tier standing on its conformance pass rate, and SEP-2484 gates spec finalization on conformance scenarios. The official suite executes live scenarios from TypeScript; nothing in any language validates recorded traces against the spec's normative requirements, and no Rust everything server exists. This project builds that missing half — upstream-first (anything generically useful is offered to the official repositories first), calibrated against the official suite, and engineered to the standard set by a2a-rust and held by CI: clippy pedantic + nursery at -D warnings on stable and MSRV across three platforms, property and golden-corpus tests, diff-scoped mutation gates with a zero-surviving-mutants standard on every shipped crate, fuzzing, a sanitization pass, and cargo deny on every push — plus scheduled gates that catch what a green PR cannot: byte-identical reports on big-endian and 32-bit targets, a build at the oldest dependency versions the manifests claim to support, and a ratcheted measurement of how much of the next spec revision the reference server already satisfies (conformance/draft-readiness.json).

The full reasoning, with every claim verified and dated, is in docs/plan/00-charter.md.

Contributing

CONTRIBUTING.md has the gates — cargo xtask ci runs them all locally — and SECURITY.md has the vulnerability process. Anything generically useful to the official MCP SDKs belongs upstream first; the engagement backlog is docs/plan/07-ecosystem-engagement.md.

License

MIT