Analysis Gym

Analysis Gym is a tiny MCP server for recording and scoring prospective equity earnings predictions made by AI agents.

It deliberately does not choose tickers, schedule runs, or invoke models. Your agent loop owns those decisions. The agent uses the existing FactIQ MCP server for research and calls Analysis Gym only to record a prediction, record the eventual actuals, or read the results.

Tools

• record_prediction records an immutable forecast before the expected earnings time.
• record_actuals settles all earlier predictions for a ticker and fiscal period.
• get_results returns per-metric errors and a leaderboard grouped by harness, model, and thinking setting.

The five predicted values are revenue, EBITDA, net profit, free cash flow, and the first regular-session closing price after the earnings release.

Run locally

uv sync
uv run analysis-gym

The server uses stdio transport and stores data in analysis_gym.sqlite3 in its working directory. Set ANALYSIS_GYM_DB_PATH to put the database elsewhere.

Codex

Add the server to ~/.codex/config.toml:

[mcp_servers.analysis-gym]
command = "uv"
args = ["--directory", "/absolute/path/to/analysis-gym", "run", "analysis-gym"]

Install and authenticate the FactIQ plugin separately. Then ask Codex, for example:

Pick an equity reporting soon. Use FactIQ to forecast its next-quarter revenue, EBITDA, net profit, free cash flow, and first post-earnings close. Record the forecast in Analysis Gym before the release.

Claude Code

claude mcp add analysis-gym -- \
  uv --directory /absolute/path/to/analysis-gym run analysis-gym

Use the same prompt and ensure the FactIQ plugin is also installed and authenticated.

Agent-side loop

A loop outside this repository can choose an upcoming event and run the same request through any set of CLI/model/thinking configurations. Each agent calls record_prediction itself. After earnings, call record_actuals once with a source URL, then use get_results to compare the configurations.

Analysis Gym uses symmetric mean absolute percentage error (SMAPE), where lower is better. It reports every metric separately and a simple mean across all five.

Metric definitions

• EBITDA: operating income plus depreciation and amortization.
• Free cash flow: operating cash flow minus capital expenditure.
• Net profit: consolidated net income attributable to the parent/common shareholders.
• Post-earnings close: the same session's close for a pre-market release, or the next regular session's close for an after-hours release.

All four financial values (submitted in millions) in a submission must use the same reporting currency.

Development

uv run pytest