sumo-qa MCP

tests PyPI Python License: Apache 2.0

A QA-native MCP server and skills library: analyze a repo, map risks to tests and evidence, plan and scaffold tests, strengthen weak suites, and review diffs before merge with senior-QA discipline.

[!IMPORTANT] sumo-qa is an advisor, not an oracle. Like any AI tool it can be wrong. Your judgment and your team's standards are the final word.

🚀 New here? 5-minute demo →

One install line, then the QA loop on your own repo: map it, then review a change for a merge verdict or audit the whole repo for a strategy.

Why it exists

AI writes more of your code every month, and it's confident about all of it: "done", "looks good", "safe to merge". But confidence isn't evidence. It will call a change safe off CI that ran an hour ago, write tests that pass by restating the code they're meant to check, and declare work finished without running anything. You can't tell what was actually verified from what merely sounds verified, and you're still the one who has to answer for what shipped.

sumo-qa exists to make AI-written code something you can stand behind. It holds any agent, in any host and on any stack, to a senior QA's discipline on its own work, and makes it show the evidence: risks tied to specific files and lines, each mapped to a test that ran this turn, and a safe-to-merge verdict you could hand a reviewer instead of "the model said it was fine". The same standard every time, not each model's own idea of "tested".

Ask a stock assistant to QA a change and you get the junior answer: "add unit tests, consider edge cases, maybe test performance." sumo-qa makes it work like the senior who has to sign off on what ships:

• Reads the repo first (what it does, which tests cover which code), then names risks tied to specific files and lines, not vague categories
• Picks a real design technique per risk from an ISTQB-grounded catalogue, instead of generating tests by vibe
• Runs your suite fresh in the current turn before any "safe to merge" claim, and refuses "CI was green earlier"
• Holds TDD's red phase before any production code, and keeps production code locked while it strengthens tests against mutation survivors
• Won't ship a plan or a verdict without measurable, checkable criteria, and will say "no tests needed here" when that's the honest call

The discipline lives in a library of skill files, each followed literally; every one has an Iron Law and a HARD-GATE callout the LLM can't talk past. Skills route automatically from natural-language prompts; you don't need to remember to invoke them.

Install

pip install sumo-qa && sumo-qa-install

Then restart your host or open a fresh chat. sumo-qa-install configures every MCP host it detects. Windows, single-host targeting, the one-command install.sh / install.ps1 wrappers, plugin installs, and updating are all in docs/INSTALL.md.

Verify

sumo-qa-doctor

Green means you're wired. The checks are read-only, and each failure prints the exact Fix: to run. Then ask any host "load the QA classifications"; the canonical names back confirm it end to end. (Troubleshooting →)

Host support

Every host calls the same MCP server and reads the same SKILL.md files. What differs is how each host exposes them. That's a host-API difference, not a sumo-qa choice.

sumo-qa-install sets these hosts up. It wires Claude Code and VS Code directly; for the JetBrains hosts it prints the exact config for you to paste:

Host	Slash	Setup
Claude Code	/sumo-qa-deciding-approach (hyphens)	sumo-qa-install --claude-code
VS Code + Copilot (Agent mode, Claude Sonnet 4.5 or equivalent)	Natural language	sumo-qa-install --vscode --workspace <repo> writes .vscode/mcp.json
JetBrains AI Assistant	/sumo_qa_deciding_approach (underscores)	One-time UI setup; sumo-qa-install --jetbrains prints the fields to paste
JetBrains Junie	Natural language	Create ~/.junie/mcp/sumo-qa.json (global) or <repo>/.junie/mcp/ (per project) with the command path sumo-qa-install --jetbrains prints (details)

In Claude Code, type / then sumo-qa- to see the skills as hyphenated entries (symlinked into ~/.claude/skills/). The same skills are also registered through MCP with underscores (/sumo_qa_load_classifications, /sumo_qa_find_test_data); both routes call the same SKILL.md.

Natural language works everywhere: "Review my changes", "plan QA for this story", "load the QA classifications" all route by tool description. Slash and natural-language paths produce the same result.

Other MCP hosts (Cursor, Codex, OpenCode, Gemini CLI, etc.): pip install sumo-qa ships a standard stdio MCP server, so it should work with anything that speaks MCP. Follow your host's MCP-server setup docs and point it at the absolute path of the sumo-qa script. These hosts are explicitly not yet verified end-to-end by us, so we don't ship per-host instructions for them.

Host adapter folders

sumo-qa ships generated plugin manifest folders (.claude-plugin/ for Claude Code, .codex-plugin/ for Codex, not yet verified end-to-end), built from a single canonical source in pyproject.toml and drift-checked in CI on every PR. Architecture, per-host install status, and how to add a host: docs/host-adapters.md.

See it in action

Transcripts showing the workflow on real code: diff reviews refusing to call safe-to-merge from stale CI, TDD cycles with the red output surfaced verbatim, mutation survivors walked one at a time, formal test plans gated on entry/exit criteria, and the case where the right answer is "no tests needed, stop here":

• tests/scenarios/worked-examples/: start with 02 review my changes for a representative end-to-end
• tests/scenarios/SCENARIOS.md: the scenario specs (prompt → expected shape → anti-patterns the skill prevents)

What's included

Three layers: the host LLM follows skills, which cite knowledge and standards to produce the output. The architecture doc has the diagram and the full data flow.

Layer	What
Skills (skills/)	Iron-Law procedures across the QA lifecycle: deciding approach, preparing for work, TDD scaffolding, diff review, strengthening tests, finding test data, answering testing questions, repo strategy, plus the planning → parallel subagent execution → finishing chain.
MCP entry points	A thin tool surface: skill tools, knowledge loaders, a capabilities-discovery tool, repo-map tools, test-data tools, an ingestion tool, and external-skill lifecycle tools. Each is file IO or small deterministic logic; no inference.
Progressive skill loading	A read-only loader that fetches a skill in slices (routing manifest → section → module → full body), so a host pays the routing slice on each revisit, not the whole body. See docs/TOOLS.md and docs/SKILLS.md.
Knowledge catalogues (knowledge/)	Classifications, approaches, principles, techniques the agent picks from instead of recalling from training data. Editable as plain markdown. (Specialty-tool picks are deliberately not catalogued; observe the risk surface and web-search current options instead.)

Run it from the terminal

Beyond the host integration, sumo-qa ships terminal commands for the QA-native repo loop:

sumo-qa analyze            # map the current repo into .sumo-qa/repo-map.json
sumo-qa status             # is the map present and current against HEAD? what next?
sumo-qa report             # compose the .sumo-qa artifacts into qa-report.html

All take an optional [path] and --json; report renders honest not-available states for anything missing. (Bare sumo-qa launches the MCP server; sumo-qa-doctor runs diagnostics.)

When sumo-qa doesn't fit

If your QA intent has no native fit (Playwright E2E, accessibility audits, k6 load testing, type checking), sumo-qa searches for an external skill through its MCP server, offers a [y/N] install gate, installs through the Skills CLI, then loads the installed SKILL.md back into the conversation.

The host never runs npx directly; four MCP tools own the lifecycle (search → [y/N] gate → install → load), and search returns the Skills CLI output verbatim, so there's no parser to drift. Node.js is required; if npx is missing the tool returns an actionable error rather than elevating. Any machine-level install the external skill suggests is translated to sumo-qa's repo-pinned, CI-reproducible standard, and sumo-qa keeps its confirmation gates, test evidence, and risk-to-test mapping.

Support

Filing a clear issue gets it fixed faster. Pick the template that matches the problem:

Symptom	Template
pip install / sumo-qa-install failed, or first-run setup is broken	Install / setup problem
Install worked, but the host (Claude Code, VS Code + Copilot, JetBrains, Cursor, …) does not surface tools or skills correctly	Host compatibility problem
The wrong sumo-qa skill ran for a prompt (or none ran when one should have)	Skill routed wrong
A skill ran, but its QA output was generic, wrong, or missed something	QA output quality issue
You want a new workflow, skill, or host integration	Feature / workflow request
Reproducible defect that does not fit the above	Bug report

License

Apache 2.0. See NOTICE for attribution requirements that apply to forks and redistributors.

More docs

• AGENTS.md: AI-agent bootstrap and per-host setup
• docs/ARCHITECTURE.md: three layers, host delivery, knowledge authority
• docs/SKILLS.md: every skill with its Iron Law
• docs/TOOLS.md: every MCP entry point
• docs/INSTALL.md: per-host install detail and troubleshooting
• docs/CONTENT-FORMATS.md: schemas + worked examples for adding team standards, knowledge, change rules, and test data (incl. swapping ISTQB out)
• docs/CONFIGURATION.md: env vars
• docs/DEVELOPMENT.md: local dev
• docs/TEST-DATA.md: known-good test-data catalogue
• docs/REPO-MAP.md: QA-native repo-map artifact under .sumo-qa/: schema, scanner, and the scan / diff-impact / query tools that consume it (issues #155, #156)
• docs/RISK-LEDGER.md: risk-to-test traceability ledger: the structured appendix to the markdown-first verdict, its row schema and evidence-status vocabulary, and when not to use it (issue #144)
• docs/SCORECARD.md: QA readiness scorecard: composes the risk ledger + context bundle + optional coverage/mutation into a derived readiness recommendation (ready / blocked / insufficient_evidence / ready-with-accepted-residuals): an evidence summary, not a predictive quality score (issue #151)
• docs/GATE-EVIDENCE.md: evidence-backed gate reporting: the gate-status and evidence-source vocabulary, the schema and validators that reject an unsupported "tests passed" / "safe to merge" claim, and the honesty boundary the skills now carry (issue #213)
• docs/RELIABILITY-HARDENING.md: the epic #211 control matrix: for each reliability weakness (prompt dependence, verbosity fatigue, enforcement uncertainty, shallow static analysis) the merged control, its proof artifacts, the backward-compatible default, and the accepted residual limit (issue #211)
• docs/EXPORT.md: deterministic export of already-structured QA test cases to versioned JSON, a markdown table, or (flat-only) CSV: the case schema, the format set, the side-effect-free contract, and the import-mapping caveat (issue #148)
• docs/QA-REPORT.md: local QA report: the static .sumo-qa/qa-report.html page composed from the persisted artifacts, the four honest artifact states, and a readiness verdict derived by the #151 QaScorecard engine (issue #157)
• docs/PERSONA.md: optional Sumo-sensei voice (off by default)