Alphaassay/

Signal validation & overfitting forensics for AI agents. Most trading signals are noise. AlphaAssay puts them on trial — deflated Sharpe, out-of-sample, leakage forensics — and returns signed pass/fail verdicts anyone can verify. Methodology audits, not investment advice.

docs

What this repo is. The public documentation and the registry manifest (server.json) for the AlphaAssay MCP server. The validation engine is a hosted, remote MCP — this repo is docs and metadata, not the engine source. The endpoint is live at https://mcp.alphaassay.com/mcp — the calls below are runnable.

What it does

Use this when you need to validate a trading signal, strategy or backtest before capital touches it: statistical signal validation, overfitting detection, deflated Sharpe ratio (multiple-testing aware), walk-forward and out-of-sample testing, look-ahead and data-leakage forensics, placebo tests. It returns a deterministic verdict — pass, conditional, fail or insufficient_evidence — with machine-readable failure gradients and an Ed25519-signed, replayable audit record anyone can verify.

It does not generate strategies and does not give buy/sell advice. It is a methodology audit. Verdicts are demote-only: evidence can lower a grade, never inflate one. Asset-agnostic — crypto, stocks, futures, FX.

Connect (MCP)

The server is a remote, streamable-HTTP MCP endpoint:

https://mcp.alphaassay.com/mcp

Point any MCP-capable client at it. Example client config:

{
  "mcpServers": {
    "alphaassay": {
      "type": "streamable-http",
      "url": "https://mcp.alphaassay.com/mcp"
    }
  }
}

No account is needed for the free tools (assay_demo, assay_graveyard, assay_calibration, assay_certificate_verify, assay_provider_protocol, assay_preflight). Paid tools are billed per call via x402 or credits.

Run it locally (stdio clients, Docker)

This repo ships a real local stdio server (server/index.mjs, dependency-free, Node >= 18): it defines all 21 tools statically (server/tools.json, versioned here) and executes each call against the hosted engine — the engine itself is server-side by design, so a local install can never be a locally editable examiner. Free tools run without credentials; paid tools take an optional ALPHAASSAY_API_KEY environment variable.

node server/index.mjs            # directly
docker build -t alphaassay-mcp . && docker run -i --rm alphaassay-mcp

Stdio client config:

{
  "mcpServers": {
    "alphaassay": { "command": "node", "args": ["/path/to/mcp/server/index.mjs"] }
  }
}

Try it over plain HTTP

Every capability is also a plain HTTPS API — the same request/response shape the MCP wraps. Send a golden specimen (a test signal whose correct verdict is known in advance):

curl -s https://api.alphaassay.com/v1/assay/demo \
  -H "Content-Type: application/json" \
  -d @golden_lookahead.json

Expected response (this specimen is a whipsaw with no net-of-cost edge, so it dies at the first gate — abridged; the verbatim body is in examples/golden_lookahead.expected.json):

{
  "schema": "gauntlet.v1",
  "verdict": "fail",
  "died_at": "net_edge",
  "failure_codes": ["no_net_edge"],
  "family_id": "fam_004b8867ffc7a556",
  "stages": [
    {
      "stage": "net_edge",
      "verdict": "fail",
      "evidence": { "net_sharpe_annualized": null, "net_return_total_pct": 0.0, "trades": 0, "bars": 199 },
      "detail": "No positive net-of-cost edge -- dies before any deeper test."
    }
  ],
  "budget": { "cumulative_n": null, "n_trials_effective": null, "note": "..." },
  "_meta": { "engine_version": "024739a", "api_version": "v1", "as_of": "<varies per call>" },
  "disclaimer": "Consolidated research reality-check. NOT financial advice, no order path. Demote-only."
}

stages also carries funding_edge, family_deflation, power_honesty, significance, cpcv, walk_forward, concentration, placebo, capacity and graveyard_prior. Field-by-field explanation: https://alphaassay.com/docs/verdicts. Runnable scripts for all four specimens: examples/.

Golden specimens (free known-answer tests)

Four prepared signals with a planted property and a known correct verdict — the standing offer to catch AlphaAssay being wrong before you pay it.

specimen	planted property	expected verdict
golden_clean	a genuine, persistent edge	pass · died_at: null
golden_lookahead	a whipsaw with no net-of-cost edge	fail · net_edge · no_net_edge
golden_cherry	best-of-many parameter cherry-pick	fail · family_deflation · deflated_out_at_n=50 + BACKTEST_TOO_SHORT_FOR_N=50
golden_thin	too little data to judge	insufficient_evidence

The verdict is bit-identical on every run — assert on verdict, died_at and failure_codes in CI; only the wall-clock timestamps vary (_meta.as_of, and generated_at in validator provenance). Catalog: https://alphaassay.com/docs/specimens.

Tools

tool	what it does	tier
assay_demo	free demo assay + the four golden specimens with known verdicts	free
assay_signal	full statistical battery on a signal → verdict + failure gradients + signed record	paid
assay_forensics	WHY it fails: leakage forensics on decision timestamps — look-ahead, front-loading, a three-dimensional placebo	paid
assay_backtest	overfitting forensics on backtest results (deflated Sharpe, PBO/CPCV)	paid
assay_gauntlet	the whole battery in one call — every graded stage chained into one dossier that names the first gate that killed it	paid
assay_falsify	eight adversarial attacks (execution lag, cost stress, regime split, synthetic-null worlds, drift-burst strip, …) → the survival map	paid
assay_preflight	lint a payload's form (DSL schema, OHLCV sanity, trade rows) before spending a paid check	free
assay_batch	up to 25 sweep variants in one call, each a family-ledger trial under cumulative deflation — counts, never a ranking	paid
assay_reproduce	independently recompute a claimed track record from trades + candles; grades each headline claim against disclosed tolerances	paid
assay_tradelog	interrogate a raw fill log against itself — duplicate fills, time travel, PnL contradicting its own prices; no candles needed	paid
assay_cpcv	the full purged combinatorial CV distribution behind the gauntlet's cpcv stage — quantiles, histogram, recoverable path count	paid
assay_survivors	Romano-Wolf stepwise FWER over a T×N sweep matrix — an error-budget disclosure, never a ranking	paid
assay_pbo	Probability of Backtest Overfitting over your full parameter-sweep trial matrix (CSCV, purged) — grades the selection process itself	paid
assay_register	pre-register a hypothesis with a tamper-proof timestamp, then forward-evaluate on post-cutoff data	paid
assay_verdict	fetch / replay / export an existing verdict; offline Ed25519 signature check	replay free · certificate paid
assay_var_es	exceedance backtest of ex-ante VaR/ES forecasts — exact binomial Basel traffic light + joint (VaR, ES) e-process	paid
assay_conformal	coverage audit of prediction intervals against realised outcomes — Binomial or Beta-Binomial when a split-conformal calibration size is disclosed	paid
assay_graveyard	anonymised mortality statistics per structural signal family — has this idea already died here?	free
assay_certificate_verify	offline Ed25519 verification of any AlphaAssay certificate, incl. hash-chained revocation — no platform access needed	free
assay_provider_protocol	the falsification protocol as machine-readable rules — seven tests for judging ANY signal provider, AlphaAssay included	free
assay_calibration	the public signed calibration ledger — is AlphaAssay itself trustworthy?	free

Full tool set (21 assay_* tools) and reference: https://alphaassay.com/docs.

Pricing

Paid calls are flat-priced per completed call — pass and fail cost the same, because you are buying the trial, not a flattering outcome. The free tier (demo, graveyard, calibration, verification) never costs anything. Current numbers, always live: alphaassay.com/pricing or GET https://api.alphaassay.com/v1/meta/pricing.

Verify a verdict yourself

Every verdict is bit-identically replayable, and any paid verdict can be exported as an Ed25519-signed certificate — the free demo verdict is unsigned. Check a certificate in three seconds at https://alphaassay.com/verify, or fully offline with the published public key — https://alphaassay.com/docs/verify-offline.

Docs

• Quickstart — https://alphaassay.com/docs/quickstart
• Rules for your trading agent (copy-paste) — https://alphaassay.com/docs/ai-rules
• Golden specimens — https://alphaassay.com/docs/specimens
• Verdicts & the gradient — https://alphaassay.com/docs/verdicts
• llms.txt — https://alphaassay.com/llms.txt

Publishing

Published in the official MCP registry as com.alphaassay/mcp; also listed on Glama, Smithery and the public MCP directories. Registry releases track this repository.

Licence

Apache-2.0.

---

Disclaimer. AlphaAssay is a methodology audit, not investment advice. No order path, no custody, no buy/sell signals. A pass means a signal survived a statistical trial most signals fail — never a promise of any outcome. Verdicts are demote-only.