Command Palette

Search for a command to run...

UnylyUnyly
Browse all

Jev Judge

FreeNot checked

Typed judgment tools for MCP agents. TypeSafe's Jev model as verify, screen, find, classify, rerank, decide, compare, extract, review, gate, and score: the mode

GitHubEmbed

About

Typed judgment tools for MCP agents. TypeSafe's Jev model as verify, screen, find, classify, rerank, decide, compare, extract, review, gate, and score: the model judges, policy decides auto, review, or escalate.

README

jev-judge-mcp, with a scales-of-justice emblem: typed judgment tools for MCP agents. Model judges, policy decides: auto, review, or escalate. Check, find, decide, ship.

CI Version Python 3.12+ MCP over stdio MIT license

An MCP server that gives your coding agent eleven judgment tools backed by TypeSafe's Jev model. The agent hands a tool some evidence and a question it can enumerate: is this claim supported, is this page safe to read, which of these files answers the question, did this patch finish the task. Jev answers with probabilities, usually in under a second. Policy turns them into one of three actions: auto (proceed), review (check it another way), or escalate (stop).

Use it for checks that have a fixed set of answers. When the step needs new text, code, or options you cannot list, the agent should write it itself.

Install

You need Python 3.12+, uv, a POSIX system (Linux or macOS), and a TypeSafe API key from console.typesafe.ai. The package is not on PyPI yet, so you run it from a clone.

git clone https://github.com/PyModel/jev-judge-mcp
cd jev-judge-mcp
uv sync --extra typesafe
Install with the installer (recommended)
uv run jev-judge-mcp setup     # verify your key, then store it
uv run jev-judge-mcp install   # add the server to your agents
uv run jev-judge-mcp doctor    # check the configuration, offline

Restart your agent. The tools show up as jev_verify, jev_gate, and so on (mcp__jev__* in Claude Code).

setup reads the key from TYPESAFE_API_KEY, or asks for it at a hidden prompt. It never takes the key as an argument, so the key stays out of your shell history. It makes one live call to check the key and writes nothing if TypeSafe rejects it. A good key goes to ~/.config/jev-mcp/key, readable only by you. The server uses that file whenever TYPESAFE_API_KEY is unset, so agents you start without exporting the key still work. When the variable is set, it wins.

install finds the agents on your machine, shows what it will change, and asks before writing. It supports Claude Code, Claude Desktop, Codex (CLI and the ChatGPT app), Cursor, OpenCode, Pi, omp, and Pythinker.

uv run jev-judge-mcp install --dry-run         # show the plan, write nothing
uv run jev-judge-mcp install -a claude-code    # one agent (repeatable)
uv run jev-judge-mcp install --remove          # undo what install wrote

Terminal agents get a reference to TYPESAFE_API_KEY, never the key itself. Desktop apps don't see your shell's environment. Claude Desktop (macOS only) is skipped unless you pass --desktop-key, which writes the key into that app's config file. The same flag writes the key into the Codex and Pythinker files when the ChatGPT or Pythinker desktop app shares them; without it, install says that app has no key. The installer warns if a file holding the key ends up readable by other users. Pi also needs its MCP adapter first: pi install npm:pi-mcp-adapter.

Register the server by hand

<uvx> is the absolute path of uvx. <spec> is your clone's absolute path plus [typesafe], for example /home/me/jev-judge-mcp[typesafe].

Claude Code (~/.claude.json), omp (~/.omp/agent/mcp.json), Cursor (~/.cursor/mcp.json), and Pi (~/.pi/agent/mcp.json) use the same shape. Claude Code and omp also add "type": "stdio". Pi also adds the three exposure keys below; without them pi-mcp-adapter keeps the server lazy and proxy-only and the tools stay out of the model's initial list (ADR-0036).

{
  "mcpServers": {
    "jev": {
      "command": "<uvx>",
      "args": ["--from", "<spec>", "jev-judge-mcp"],
      "env": {"TYPESAFE_API_KEY": "${TYPESAFE_API_KEY}"}
    }
  }
}

Pi's full entry:

{
  "mcpServers": {
    "jev": {
      "command": "<uvx>",
      "args": ["--from", "<spec>", "jev-judge-mcp"],
      "env": {"TYPESAFE_API_KEY": "${TYPESAFE_API_KEY}"},
      "lifecycle": "eager",
      "directTools": true,
      "toolPrefix": "none"
    }
  }
}

lifecycle: "eager" connects at startup, directTools: true registers every tool individually, and toolPrefix: "none" keeps the published names (jev_verify, ...), so the tools sit in the model's initial tool list, callable like any builtin.

Claude Desktop (~/Library/Application Support/Claude/claude_desktop_config.json) uses the same shape with the key itself in env. Pythinker (~/.pythinker-code/mcp.json) uses it without env, unless the Pythinker desktop app shares the file and needs the key there.

Codex CLI and the ChatGPT app share ~/.codex/config.toml:

[mcp_servers.jev]
command = "<uvx>"
args = ["--from", "<spec>", "jev-judge-mcp"]
env_vars = ["TYPESAFE_API_KEY"]

When the ChatGPT app shares that file, it also needs the key itself in an [mcp_servers.jev.env] table with TYPESAFE_API_KEY = "<key>".

OpenCode (~/.config/opencode/opencode.json):

{
  "mcp": {
    "jev": {
      "type": "local",
      "command": ["<uvx>", "--from", "<spec>", "jev-judge-mcp"],
      "environment": {"TYPESAFE_API_KEY": "{env:TYPESAFE_API_KEY}"}
    }
  }
}

What to use it for

Ask your agent in plain words. It picks the tool, or you can name it.

You want to Tool You get
Check that the agent's "done" matches the diff and the test log jev_gate one ship decision over the patch and each completion claim
Check claims in a summary or PR description against the sources jev_verify verified, contradicted, or unsupported for each claim
Screen a fetched web page for prompt injection before reading it jev_screen pass, review, block, or skip
Review a patch against the request jev_review correctness, spec match, test gaps, blast radius
Spot drift between docs and code, or a changelog and a diff jev_compare same fact, contradiction, or different facts
Find the file or note that answers a question jev_find the best match, plus whether anything matches at all
Rank search hits or grep results jev_rerank a relevance score for every candidate, sorted
Route tickets or label many items at once jev_classify one class per item from your catalog
Pick one option, or decide whether to keep waiting on a slow command jev_decide your option, or ask_user / investigate / none
Grade severity or risk on your own scale jev_score a position on your 2 to 10 levels, with the distribution; threshold it in code, since positions between levels are weakly calibrated
Pull a version, date, or price out of a document jev_extract a value copied from a match of your regex, or null

For example, "use jev_verify to check your summary against the changelog" returns one row per claim:

{
  "claim": "The setup command accepts the API key as a command-line argument.",
  "verdict": "contradicted",
  "probabilities": { "supports": 0, "contradicts": 1, "says_nothing": 0 },
  "confidence": 1,
  "action": "auto",
  "supporting_evidence": "setup.py"
}

Jev sees only what the agent passes in the call, so the agent has to include the evidence. docs/skills/jev-mcp/SKILL.md is a skill you can give your agent: it covers which tool fits which step and what to do with each action. Allow rules for Claude Code are printed by doctor, and opt-in setups for Claude Code, Codex, and Pi are in the harness samples.

Configuration

The server reads environment variables only. It does not load a .env file.

Variable Default What it does
TYPESAFE_API_KEY unset TypeSafe key; takes priority over the stored key
JEV_MCP_KEY_FILE ~/.config/jev-mcp/key where setup stores the key and the server reads it
JEV_PROVIDER auto auto takes the first provider with credentials: typesafe, openrouter, cloudflare, compatible. The reference's vercel provider (AI_GATEWAY_API_KEY) is not supported
JEV_MCP_MODEL jev-latest Jev model to ask
JEV_MCP_CACHE off replay identical requests from disk at no API cost; leave it off when answers must be fresh, and delete the directory to clear it
JEV_MCP_CACHE_DIR ~/.cache/jev-mcp where the cache lives
JEV_MCP_TRANSPORT stdio streamable-http is experimental and binds JEV_MCP_HTTP_HOST:JEV_MCP_HTTP_PORT, default 127.0.0.1:8000
JEV_MCP_LOG_LEVEL INFO logs go to stderr

Architecture

One diagram covers the whole server: the tool-call loop from tools/call to the returned action text, the fail-closed answer path, and the local CLI commands around it (install, setup, hook gate, doctor) with the stored key file and the optional response cache. Open docs/architecture.html for the interactive version (guided views, dark mode, node search, relationship tracing).

jev-judge-mcp architecture: an MCP client calls eleven Jev tools through one Toolset over stdio or streamable-http; the runtime resolves one of four providers, validation rejects malformed answers, pure policy decides auto, review, or escalate, and the result serializes back to the client. The install, setup, hook, and doctor CLIs, the stored key file, and the optional response cache surround the server.

Measured results

Two paid studies, both descriptive, with small samples and no significance test.

On 150 questions with Pi (opencode-go/deepseek-v4.1-flash), forcing a Jev call added 10.4 s median wall time per task. Letting the agent choose left Jev uncalled on all 150. Jev itself answered in 465 ms median over 157 calls. Accuracy was not measured. Details: evals/reports/bench150.md.

arm median wall s p95 called Jev agent spend
A direct 3.06 6.17 0/150 $0.0928
B automatic 2.91 8.09 0/150 $0.0955
C forced 13.95 28.67 150/150 $0.2904

The agent outcome study ran on 2026-09-23 with jev-1.13.0: three tasks, three repeats per arm, with and without Jev. Both agents solved the same pairs either way and picked the right decision on every run. Both were slower with Jev. One Pi pair is excluded because its with-Jev run never called Jev. Details, raw records, and the chart script: docs/evals/.

agent solved without / with Jev median time to correct, without / with extra wall time with Jev (paired median) spend
Claude Code (claude-sonnet-5) 6/9 / 6/9 14.6 s / 18.3 s +4.6 s $1.4953
Pi (ds4/glm-5.3-flash, local) 6/8 / 6/8 49.4 s / 127.8 s +85.8 s $0.0006

Claude Code outcome study: tasks solved, median time to a correct solution, tokens per solved task, and the paired wall-time difference, with and without Jev

Pi outcome study: tasks solved, median time to a correct solution, tokens per solved task, and the paired wall-time difference, with and without Jev

About this project

This is a Python rewrite of the TypeScript @jkudish/jev-mcp 0.5.0. The ten reference tools match it on the wire, checked by recorded parity fixtures; jev_score is an addition. Vocabulary is in docs/CONTEXT.md, decisions in docs/adr/, and security notes in SECURITY.md. Windows is not supported; the server exits at startup on a non-POSIX platform.

uv sync --all-extras
make ci      # lint, types, unit, property, policy coverage, contract, parity, security, build, smoke
make eval    # offline scorer checks

make eval-live, make security-live, and JEV_AB_LIVE=1 make ab call paid services and stay off CI. Contribution notes are in CONTRIBUTING.md.

MIT license.

from github.com/PyModel/jev-judge-mcp

Install Jev Judge in Claude Desktop, Claude Code & Cursor

Recommended · one command, every IDE
unyly install jev-judge

Installs into Claude Desktop, Claude Code, Cursor & VS Code — handles npx, uvx and build-from-source repos for you.

First time? Get the CLI: curl -fsSL https://unyly.org/install | sh

Or configure manually

Run in your terminal:

claude mcp add jev-judge -- uvx jev-judge-mcp

Step-by-step: how to install Jev Judge

FAQ

Is Jev Judge MCP free?

Yes, Jev Judge MCP is free — one-click install via Unyly at no cost.

Does Jev Judge need an API key?

No, Jev Judge runs without API keys or environment variables.

Is Jev Judge hosted or self-hosted?

Self-hosted: the server runs locally on your machine via the install command above.

How do I install Jev Judge in Claude Desktop, Claude Code or Cursor?

Open Jev Judge on unyly.org, pick your client tab (Claude Desktop, Claude Code, Cursor) and press Install — the config is generated automatically, no JSON editing.

Related MCPs

Compare Jev Judge with

Not sure what to pick?

Find your stack in 60 seconds

Author?

Embed badge for your README

Browse similar

All development MCPs