Protocol Tester

SafeSkill 87/100

Security Audit CI License: MIT Node.js No Secrets CodeQL

Universal test suite for MCP servers with dual-rail payment testing.

Tests x402 (USDC on Base) and MPP (USDC on Tempo) protocols simultaneously across all tools on any MCP-compatible server.

Features

• 20-phase test suite — 920+ assertions across discovery, payments, security, anti-abuse, and agent UX
• Dual-rail payment testing — x402 + MPP in parallel with cross-rail price validation
• Full MCP protocol validation — initialize, tools/list, tools/call, resources/list, prompts/list, version negotiation
• 402 challenge validation — every tool checked for correct payment challenge fields (payTo, network, asset, amount)
• Payment security — replay attacks, race conditions, double-spend, amount manipulation, stale challenges
• Advanced security — SSRF, timing attacks, CORS, header injection, fuzz testing, response analysis
• Payment bypass prevention — replay, double-spend 100x, signed underpayment, cache leak, MCP session abuse, cross-rail nonce
• Resilience testing — brute force, SSL/TLS cipher/cert validation, enumeration, error cascade
• Load testing — ramp-up stress, latency percentiles (p50/p95/p99), sustained load
• Tool discovery — discover_tools prompt validation, stemming, category enumeration, abuse testing
• Platform features — usage analytics, tool quality index, batch API, cross-feature validation
• Agent experience — zero-knowledge bootstrap, description quality, error actionability, E2E lifecycle
• Provider health map — per-provider status and latency tracking
• Universal — works with ANY MCP server, not just APIbase
• Scoring — 0-100 score with A+/A/B/C/D/F grade
• CI-ready — GitHub Actions with TruffleHog, Gitleaks, CodeQL, Snyk
• Zero build — plain Node.js, no TypeScript compilation needed

Quick Start

git clone https://github.com/whiteknightonhorse/mcp-protocol-tester.git
cd mcp-protocol-tester
npm install
cp .env.example .env
# Edit .env with your server URL and wallet keys
npm test

Usage

# Full dual-rail test (all 20 phases)
npm test

# Dry run — no real payments, tests challenges and security only
npm run test:dry

# Fast mode — discovery + infrastructure + MCP only
npm run test:fast

# Security audit (basic + payment + advanced + resilience)
npm run test:security

# Payment tests only
npm run test:payments

# Custom server
API_BASE_URL=https://my-server.com npm test

# Specific phases
PHASES=0,1,7,8,9,10 npm test

# Load test with 10 concurrent requests
CONCURRENCY=10 npm test

Scoring

Grade	Score	Meaning
A+	97-100	Production-ready, all protocols verified
A	93-96	Excellent, minor recommendations
A-	90-92	Very good, few non-critical issues
B+	87-89	Good, some improvements needed
B	83-86	Functional, recommendations available
B-	80-82	Acceptable, notable gaps
C	70-79	Basic functionality, significant gaps
D	60-69	Critical issues found
F	<60	Major failures, not production-ready

Note: Any CRITICAL (500) server error automatically caps the grade at D. Skipped phases score 0%.

Phases (20)

Phase	Name	What it tests	Cost
P0	Discovery	Catalog fetch, schema validation, .well-known/*, dual-rail detection, internal tool leak check, well-known CORS	$0
P1	Infrastructure	Health, Tempo/Base RPC, USDC + gas balances, facilitator, health info leakage check	$0
P2	MPP Challenges	WWW-Authenticate: Payment on all tools + field validation (recipient, amount, chainId) + currency pin (exactly one currency, USDC.e on mainnet / pathUSD on testnet, never OUSD; RED) + challenge id and expires	$0
P3	x402 Challenges	x402 402 body on all tools + field validation (payTo, network, asset, scheme, multi-accept), challenge nonce uniqueness, Cache-Control: no-store on the 402 of each rail, maxTimeoutSeconds == 120	$0
P4	MCP Protocol	initialize, tools/list, tools/call, resources/list, prompts/list, version negotiation, JSON-RPC error codes, session fixation, oversized payload, ID type handling, paid tool without payment -> isError + _meta["x402/payment-required"] equal to the HTTP 402 accepts[], x402 payment in params._meta -> _meta["x402/payment-response"], tool-definition fingerprint (definition_sha256 recompute REST/MCP/manifest, ETag 304, tamper self-test)	~$0.001
P5	MPP Payments	Real Tempo USDC.e payments via mppx 0.13.5 + response content validation; current-client receipt with tx-hash reference (RED); legacy mppx 0.8.19 client lane; replay of a settled credential is refused (RED) and answered with a final 403 credential_replayed, charged:false (rec); credential in Payment-Authorization next to a Bearer key -> 200; both headers with credentials -> 400; mppx validate child process	~$0.01
P6	x402 Payments	Real Base USDC payments via @x402/core + response content validation	~$0.05
P7	Basic Security	Auth bypass, forged credentials, HTTP method enforcement + override bypass, Content-Type manipulation, request ID, path traversal, expanded hidden endpoints (+10), request smuggling	$0
P8	Payment Security	Replay (same/cross-tool), race condition (10 parallel), double-spend, amount manipulation (0/negative/underpay/tampered payTo/integer overflow 2^64), float precision, off-by-one (999 vs 1000), expired challenge, wrong network (testnet→mainnet), cross-rail price consistency; MPP replay answer is a final 403 credential_replayed (400 only for forged credentials)	~$0.01
P9	Advanced Security	SSRF (AWS/GCP/localhost/file:// + IPv6/octal/decimal), timing attacks, CORS + subdomain confusion + expose-headers audit, header injection (CRLF/64KB/Host), fuzz (null bytes/unicode/JSON bomb/prototype pollution), XXE injection, SSTI payloads, ReDoS, response analysis	$0
P10	Resilience	Brute force (50 keys), XFF + 5 IP spoofing headers bypass, SSL/TLS version + cipher + cert expiry + TLSv1.1 rejection, enumeration, error cascade, referrer-policy, rate limit granularity (per-key)	$0
P11	Load Test	Concurrent requests, mixed endpoints, sustained load, latency p50/p95/p99, ramp-up (1→5→10→25), large body DoS (1MB)	$0
P12	Provider Health	1 tool per provider → HEALTHY/DOWN/RATE_LIMITED + latency, provider error sanitization	$0
P13	Cache & Simulation	Cache isolation, cache leak test, cache poisoning (X-Forwarded-Host), cache key collision, User-Agent/Accept variation, REST+MCP simultaneous, error schema	$0
P14	Discover Tools	Category enumeration (21 cats), category+task combos, stemming, keyword relevance, abuse (SQLi/XSS/unicode/10k chars), truncation, performance	$0
P15	Platform Features	Usage Analytics (account.usage/tools/timeseries), Tool Quality Index (tool_quality/rankings), Batch API (call_batch + REST), recursive batch, batch tool_id injection, cross-feature validation	$0
P16	Agent Experience	Zero-knowledge bootstrap, tool description quality, error actionability (400/402/429), payment UX, response consistency, E2E lifecycle (golden path), MCP protocol completeness, error path leakage	$0
P17	Payment Bypass	MCP session payment enforcement, replay after 30s delay, signed underpayment, pay-cheap-use-expensive, cache leak across keys, price consistency, MCP batch abuse, header case/duplication, Content-Type bypass, prototype pollution, session ID entropy, double-spend 100x, cross-rail nonce, cross-chain replay, WebSocket upgrade bypass, burst 50 unpaid, chunked encoding bypass, nonce entropy, 9 path-encoding bypass vectors (%2F, %5C, raw backslash, /call/..%2F, double slash, /API/v1, %0A, /mcp%2F, /mcp/..; RED, $0)	~$0.01
P18	CDP Facilitator	PayAI health + Base mainnet + Bazaar extension, CDP auth-gated, 402 wallet/network/asset/version consistency across tools, MPP dual-rail header, health/catalog/free/paid architecture checks, facilitator TLS, wallet constant	$0
P19	Report	Score, grade, per-phase breakdown, recommendations, txt + JSON export	$0
P20	Moderation E2E	Real trigger content through a real paid call (sacrificial key only), asserts blocked + settle-on-block holds after 90s	~$0.001
P21	Appeals	/appeals/:id + /api/v1/appeals/:id path-param fuzz (5 hostile values x 2 routes), immediate liveness check after each	$0
P22	Balance Rail	Plain-Bearer call (no x402/MPP header) against a priced tool must pay from balance; balance must actually decrease (sacrificial key only)	varies
P23	Devices	device.list/state/command + safety-gate rejection of an out-of-bounds command (sacrificial key only)	$0
P24	Docs/Catalog Truth	Every static page's tools/<id>/call references cross-checked against the live catalog; homepage tool-count drift; /about and /index.md (200, content-type, canonical, forbidden phrases); payment wording on /integrator/agent-guide, /auth.md, /llms.txt, /.well-known/x402-payment.json (mppx >= 0.6.16, USDC.e, no coinbase/x402, no "memo" next to "quote")	$0
P25	Time Promises	Plants a real expiring moderation record, verifies it's actually gone after the real 14-day retention window (sacrificial key only)	~$0.001
P26	Coverage Parity	Live nginx.conf's declared public locations vs this suite's own probed-route inventory	$0
P27	Storefront Surface	Demo shop MCP initialize + tools/list, /m/<slug> + agent.json + llms.txt URLs, /shops, /integrator (+ Markdown negotiation), discover kind=merchant vs /shops parity (demo mode, GET only)	$0
P28	Shop Order Flow	Demo shop quote for demo-bundle, 402 challenge (amount, x402 payTo is not the platform wallet; MPP recipient is a well-formed address, and for apibase-demo only the not-platform-Tempo check is SKIPPED because its Tempo payout is the operator wallet by design, the stream pilot requires it: docs/integrator.md:94, src/shop/stream-session.ts:187-219; both rails), replay of the same quote -> same binding, amount mismatch -> 402 with the order left QUOTED; pay -> DELIVERED and receipt replay are SKIPPED (no demo settlement, would cost money)	$0
P29	Subscriptions	demo-daily subscribe quote + terms (daily, 72 h window), 402 challenge, refusals (non-subscription SKU, qty 2), 404 on unknown subscription routes; renewal / cancel / double-renew / past_due are SKIPPED (need a paid period 1); when the docs list the mpp_subscription pull mode, shop.subscription.get must describe it	$0
P32	Webhook Signature	Tester-local HTTP receiver: valid HMAC accepted, wrong secret and tampered body rejected, replayed delivery id ignored, timestamp older than 300 s rejected; scheme from src/shop/webhook/webhook.service.ts:37-40	$0
P34	Legal 428 Gate	/legal/index.json sha256 vs served bodies; unaccepted-merchant quote gate: machine-readable 428 once a document is past the 30-day re-accept window (before that: not refused)	$0

⚠️ Note found in passing while adding P20-P26 (not fixed — out of this pass's scope): this table's OWN numbering for P15-P18 above does not match the actual code (src/phases/16-platform-features.js is really P16 in src/index.js/src/lib/reporter.js's WEIGHTS, not the P15 this table shows — the same one-off shift that reporter.js's old labels array had, fixed there in the Э1 pass but not re-derived here). Treat src/index.js's config.phaseEnabled(N) calls and WEIGHTS in src/lib/reporter.js as the actual source of truth for phase numbers; a full renumbering pass of this table is its own task.

P20-P26 (Э3, Fable's audit 2026-09-02) need an operator-provisioned sacrificial/test identity (MODERATION_TEST_API_KEY, BALANCE_RAIL_TEST_KEY, DEVICE_TEST_API_KEY — see .env.example) and SKIP cleanly without one; they never fall back to the primary API_KEY.

Total estimated cost: ~$0.07 per full run (plus a few tenths of a cent more once P20/P22/P25 are provisioned).

Environment Variables

Variable	Required	Default	Description
API_BASE_URL	Yes	https://apibase.pro	Target server base URL
MCP_SERVER_URL	No	{API_BASE_URL}/mcp	MCP endpoint URL
API_KEY	No	(none)	API key for authenticated requests
PRIVATE_KEY	No*	(none)	Wallet private key for payments
CONCURRENCY	No	5	Parallel requests in load test
SKIP_PAYMENTS	No	false	Skip real payment tests
PHASES	No	0,1,2,...,15 (all)	Comma-separated phase numbers
MAX_TOOLS	No	0 (all)	Max tools to test (0 = unlimited)
TIMEOUT_MS	No	30000	Per-request timeout in ms
MAX_USDC_BUDGET	No	0.25	Spending cap per protocol

* PRIVATE_KEY is required only for payment phases (P5, P6, P8). The same key works on both Base and Tempo chains.

Testing Your Own MCP Server

1. Your server must support MCP protocol (Streamable HTTP at /mcp)
2. For payment tests: server must return HTTP 402 with payment challenge
3. Set API_BASE_URL in .env to your server
4. Optional: set PRIVATE_KEY for real payment tests
5. Run npm test

Minimum server requirements for each phase:

Phase	Server must support
P0	GET /api/v1/tools + /.well-known/mcp.json
P1	GET /health/ready
P2	HTTP 402 with WWW-Authenticate: Payment header
P3	HTTP 402 with x402 JSON body (x402Version: 2, accepts array)
P4	MCP Streamable HTTP (POST /mcp) with initialize, tools/list, tools/call
P5	MPP payment verification via mppx (server-side mppx >= 0.13.5 recommended; clients >= 0.6.16 supported, 0.8.19 tested as the legacy lane)
P6	x402 payment verification via @x402/core
P7-P10	Standard REST API (any MCP server)
P11	Any HTTP endpoint
P12	POST /api/v1/tools/{id}/call
P13	REST + MCP endpoints
P14	MCP prompts/get with discover_tools prompt
P15	Platform tools: account.usage, platform.tool_quality, platform.call_batch
P16	.well-known/* discovery, MCP session, tool descriptions, error messages

Reports

Reports are saved to reports/ directory (git-ignored). Each run creates:

• dual-rail-report-YYYYMMDD_HHMMSS.txt — human-readable text
• dual-rail-report-YYYYMMDD_HHMMSS.json — machine-readable JSON

JSON report structure:

{
  "timestamp": "2026-03-27T09:00:00.000Z",
  "server": "https://apibase.pro",
  "score": 85,
  "grade": "B",
  "assertions": { "total": 920, "pass": 890, "fail": 30 },
  "financial": { "x402": 0.05, "mpp": 0.01 },
  "errors": [],
  "recommendations": [],
  "failures": []
}

Payment Protocols

x402 (USDC on Base)

Standard HTTP 402 payment protocol. Client sends request, gets 402 with payment requirements (accepts array with network, asset, payTo, scheme, amount), signs USDC transfer on Base chain, retries with X-Payment + PAYMENT-SIGNATURE headers.

Validation: P3 checks every tool's 402 response for correct payTo address, network (eip155:8453), asset (USDC contract), and reasonable amount. Multi-accept entries are all validated.

MPP (USDC on Tempo)

Machine Payments Protocol by Stripe/Tempo. Client sends request, gets 402 with WWW-Authenticate: Payment header containing challenge (id, realm, method, intent, request, expires), signs payment on Tempo chain via mppx SDK, retries with Authorization: Payment <credential>.

Validation: P2 checks every tool for correct recipient, chainId (4217), and reasonable amount. P8 validates cross-rail price consistency (x402 vs MPP amounts must match).

Both protocols can coexist on the same server — this tester verifies both work correctly and consistently.

Security

This tool is safe to use

Check	Status	How
No hardcoded secrets	Verified on every commit	TruffleHog + Gitleaks scan
No code vulnerabilities	Verified on every commit	GitHub CodeQL static analysis
No dependency vulnerabilities	Weekly scan	npm audit + Snyk
No data exfiltration	Verified on every commit	Custom scan for eval(), child_process, suspicious fetch calls
No secret logging	Verified on every commit	Grep for console.log with env vars
Private keys stay local	.env in .gitignore	Keys never leave your machine
No telemetry	By design	Zero analytics, zero tracking, zero external calls except target server
Open source	MIT license	Full source code available for audit

What this tool tests on YOUR server

Category	Tests	What it looks for
Payment Replay	5	Reused payment signatures (same tool, cross-tool, modified body)
Double-Spend	2	10 + 100 parallel requests with same payment
Amount Manipulation	8	Zero, negative, underpay, tampered payTo, float precision, integer overflow (2^64), off-by-one (999 vs 1000), wrong network (testnet→mainnet)
SSRF	7	AWS/GCP metadata, localhost, file://, IPv6 [::1], octal 0177.0.0.1, decimal 2130706433
Timing Attacks	2	Valid vs invalid key timing differential
CORS	5	Evil origin, null origin, preflight with credentials, subdomain confusion, expose-headers audit
Header Injection	4	CRLF in auth, 64KB header, Host override, request smuggling (CL+TE)
Fuzz	5	Null bytes, unicode, JSON bomb, prototype pollution, MAX_SAFE_INTEGER
XXE/SSTI	4	XML entity injection, template injection {{7*7}}, ${7*7}, <%= %>
Brute Force	3	50 random keys, XFF bypass, lockout recovery
IP Spoofing	5	X-Real-IP, X-Client-IP, CF-Connecting-IP, True-Client-IP, Forwarded
SSL/TLS	4	Protocol version, cipher strength, certificate expiry, TLSv1.1 rejection
Enumeration	4	Uniform errors, hidden endpoints (+10 paths), tool ID injection, path traversal
Response Analysis	4	Stack trace leaks, X-Powered-By, Server header, internal path leakage
MCP Session	4	Payment enforcement, batch abuse, session fixation, oversized payload
Cache	4	Cross-key leak, cache headers, cache poisoning (X-Forwarded-Host), cache key collision
Signed Underpayment	1	Valid signature with tampered amount
Cross-Tool Price	1	Cheap tool payment used for expensive tool
Cross-Chain Replay	2	Tempo nonce as x402, cross-chain payment proof
Double-Spend 100x	1	100 parallel requests with single payment
Replay After Delay	1	Same payment replayed after 30 seconds
Expired Challenge	1	Payment with maxTimeoutSeconds=0
Proto Pollution	1	__proto__.isPaid=true bypass attempt
HTTP Method Override	3	X-HTTP-Method-Override, X-HTTP-Method, X-Method-Override
Content-Type Bypass	3	text/plain, text/xml, multipart with payment check
WebSocket Bypass	1	Upgrade: websocket to bypass HTTP payment middleware
Chunked Encoding	1	Transfer-Encoding: chunked to bypass payment check
Burst Attack	1	50 unpaid requests in 100ms — none should return 200
Nonce Entropy	2	Challenge nonce uniqueness + minimum length (≥20 chars)
Session Entropy	1	Session IDs >= 128-bit, unpredictable
Rate Limit	1	Per-key not per-IP (one abuser doesn't block others)
Batch Injection	2	Recursive batch, path traversal in tool_id
ReDoS	1	50K-char input within 5s timeout
CDP Facilitator	16	PayAI/CDP health, wallet consistency, network/USDC/version, Bazaar, facilitator TLS
Price Consistency	10	Catalog price matches 402 challenge for 10 tools

Verify yourself

# Check for secrets in code
grep -rn "0x[0-9a-fA-F]\{64\}" --include="*.js" src/
# Should return: no matches

# Check for eval/exec
grep -rn "eval(\|child_process\|\.exec(" --include="*.js" src/
# Should return: no matches

# Check what HTTP endpoints are called
grep -rn "fetch\|http\." --include="*.js" src/lib/
# Should only reference: config.apiUrl, config.apiBaseUrl, config.mcpServerUrl,
# facilitator.payai.network, rpc.tempo.xyz

Warning: Never commit your .env file. Never share your PRIVATE_KEY.

See docs/SECURITY.md for details on what the security test phases cover.

Project Structure

mcp-protocol-tester/
├── src/
│   ├── index.js                   # Main entry point / CLI
│   ├── phases/
│   │   ├── 00-discovery.js        # P0: Catalog, schema validation, .well-known, dual-rail
│   │   ├── 01-infrastructure.js   # P1: Health, RPC, USDC + gas balances
│   │   ├── 02-mpp-challenges.js   # P2: MPP 402 + field validation (recipient, chainId)
│   │   ├── 03-x402-challenges.js  # P3: x402 402 + field validation (payTo, network, asset)
│   │   ├── 04-mcp-protocol.js     # P4: MCP init/list/call + resources + prompts + version
│   │   ├── 05-mpp-payments.js     # P5: Real MPP payments + response validation
│   │   ├── 06-x402-payments.js    # P6: Real x402 payments + response validation
│   │   ├── 07-security.js         # P7: Auth, credentials, HTTP methods, Content-Type
│   │   ├── 08-payment-security.js # P8: Replay, race, double-spend, amounts, cross-rail
│   │   ├── 09-advanced-security.js# P9: SSRF, timing, CORS, headers, fuzz, response
│   │   ├── 10-resilience.js       # P10: Brute force, TLS cipher/cert, enumeration
│   │   ├── 11-load.js             # P11: Concurrent, sustained, percentiles, ramp-up
│   │   ├── 12-provider-health.js  # P12: Per-provider health map
│   │   ├── 13-cache-simulation.js # P13: Cache leak, User-Agent, REST+MCP, errors
│   │   ├── 14-discover-tools.js   # P14: Categories, stemming, relevance, abuse
│   │   ├── 16-platform-features.js# P15: Usage Analytics, Tool Quality, Batch API
│   │   ├── 17-agent-experience.js # P16: Bootstrap, descriptions, errors, E2E lifecycle
│   │   ├── 18-payment-bypass.js   # P17: Anti-abuse, replay, double-spend, cache leak
│   │   ├── 19-cdp-facilitator.js  # P18: CDP/PayAI dual-facilitator, wallet consistency
│   │   └── 15-report.js           # P19: Grade, breakdown, export
│   ├── lib/
│   │   ├── config.js              # dotenv + env loader
│   │   ├── http.js                # HTTP client with timeout + provider delays
│   │   ├── mpp-client.js          # MPP client wrapper (mppx 0.13.5 + legacy 0.8.19 lane)
│   │   ├── mpp-pay.js             # manual credential flow + `mppx validate` child process
│   │   ├── x402-client.js         # x402 payment wrapper (@x402/core)
│   │   ├── mcp-client.js          # MCP JSON-RPC session handler
│   │   ├── scoring.js             # Score/grade calculator
│   │   └── reporter.js            # Report formatter (txt + json) + weights
│   └── utils/
│       └── assert.js              # Body builders (30+ heuristics), 70+ known params
├── reports/                       # Generated reports (git-ignored)
├── docs/
│   ├── SCORING.md                 # Scoring methodology
│   ├── ADDING-PHASES.md           # How to add test phases
│   └── SECURITY.md                # Security test details
├── .github/workflows/
│   ├── security-audit.yml         # TruffleHog + Gitleaks + CodeQL + Snyk + custom scans
│   └── ci.yml                     # Syntax check + dry run on Node 18/20/22
├── .env.example                   # Template (no real keys)
├── .gitignore
├── package.json
├── LICENSE                        # MIT
└── README.md

Contributing

PRs welcome! See docs/ADDING-PHASES.md for how to add new test phases.

License

MIT