Jev Judge
БесплатноНе проверенTyped judgment tools for MCP agents. TypeSafe's Jev model as verify, screen, find, classify, rerank, decide, compare, extract, review, gate, and score: the mode
Описание
Typed judgment tools for MCP agents. TypeSafe's Jev model as verify, screen, find, classify, rerank, decide, compare, extract, review, gate, and score: the model judges, policy decides auto, review, or escalate.
README
An MCP server that gives your coding agent eleven judgment tools backed by TypeSafe's Jev model. The agent hands a tool some evidence and a question it can enumerate: is this claim supported, is this page safe to read, which of these files answers the question, did this patch finish the task. Jev answers with probabilities, usually in under a second. Policy turns them into one of three actions: auto (proceed), review (check it another way), or escalate (stop).
Use it for checks that have a fixed set of answers. When the step needs new text, code, or options you cannot list, the agent should write it itself.
Install
You need Python 3.12+, uv, a POSIX system (Linux or macOS), and a TypeSafe API key from console.typesafe.ai. The package is not on PyPI yet, so you run it from a clone.
git clone https://github.com/PyModel/jev-judge-mcp
cd jev-judge-mcp
uv sync --extra typesafe
Install with the installer (recommended)
uv run jev-judge-mcp setup # verify your key, then store it
uv run jev-judge-mcp install # add the server to your agents
uv run jev-judge-mcp doctor # check the configuration, offline
Restart your agent. The tools show up as jev_verify, jev_gate, and so on (mcp__jev__* in Claude Code).
setup reads the key from TYPESAFE_API_KEY, or asks for it at a hidden prompt. It never takes the key as an argument, so the key stays out of your shell history. It makes one live call to check the key and writes nothing if TypeSafe rejects it. A good key goes to ~/.config/jev-mcp/key, readable only by you. The server uses that file whenever TYPESAFE_API_KEY is unset, so agents you start without exporting the key still work. When the variable is set, it wins.
install finds the agents on your machine, shows what it will change, and asks before writing. It supports Claude Code, Claude Desktop, Codex (CLI and the ChatGPT app), Cursor, OpenCode, Pi, omp, and Pythinker.
uv run jev-judge-mcp install --dry-run # show the plan, write nothing
uv run jev-judge-mcp install -a claude-code # one agent (repeatable)
uv run jev-judge-mcp install --remove # undo what install wrote
Terminal agents get a reference to TYPESAFE_API_KEY, never the key itself. Desktop apps don't see your shell's environment. Claude Desktop (macOS only) is skipped unless you pass --desktop-key, which writes the key into that app's config file. The same flag writes the key into the Codex and Pythinker files when the ChatGPT or Pythinker desktop app shares them; without it, install says that app has no key. The installer warns if a file holding the key ends up readable by other users. Pi also needs its MCP adapter first: pi install npm:pi-mcp-adapter.
Register the server by hand
<uvx> is the absolute path of uvx. <spec> is your clone's absolute path plus [typesafe], for example /home/me/jev-judge-mcp[typesafe].
Claude Code (~/.claude.json), omp (~/.omp/agent/mcp.json), Cursor (~/.cursor/mcp.json), and Pi (~/.pi/agent/mcp.json) use the same shape. Claude Code and omp also add "type": "stdio". Pi also adds the three exposure keys below; without them pi-mcp-adapter keeps the server lazy and proxy-only and the tools stay out of the model's initial list (ADR-0036).
{
"mcpServers": {
"jev": {
"command": "<uvx>",
"args": ["--from", "<spec>", "jev-judge-mcp"],
"env": {"TYPESAFE_API_KEY": "${TYPESAFE_API_KEY}"}
}
}
}
Pi's full entry:
{
"mcpServers": {
"jev": {
"command": "<uvx>",
"args": ["--from", "<spec>", "jev-judge-mcp"],
"env": {"TYPESAFE_API_KEY": "${TYPESAFE_API_KEY}"},
"lifecycle": "eager",
"directTools": true,
"toolPrefix": "none"
}
}
}
lifecycle: "eager" connects at startup, directTools: true registers every tool individually, and toolPrefix: "none" keeps the published names (jev_verify, ...), so the tools sit in the model's initial tool list, callable like any builtin.
Claude Desktop (~/Library/Application Support/Claude/claude_desktop_config.json) uses the same shape with the key itself in env. Pythinker (~/.pythinker-code/mcp.json) uses it without env, unless the Pythinker desktop app shares the file and needs the key there.
Codex CLI and the ChatGPT app share ~/.codex/config.toml:
[mcp_servers.jev]
command = "<uvx>"
args = ["--from", "<spec>", "jev-judge-mcp"]
env_vars = ["TYPESAFE_API_KEY"]
When the ChatGPT app shares that file, it also needs the key itself in an [mcp_servers.jev.env] table with TYPESAFE_API_KEY = "<key>".
OpenCode (~/.config/opencode/opencode.json):
{
"mcp": {
"jev": {
"type": "local",
"command": ["<uvx>", "--from", "<spec>", "jev-judge-mcp"],
"environment": {"TYPESAFE_API_KEY": "{env:TYPESAFE_API_KEY}"}
}
}
}
What to use it for
Ask your agent in plain words. It picks the tool, or you can name it.
| You want to | Tool | You get |
|---|---|---|
| Check that the agent's "done" matches the diff and the test log | jev_gate |
one ship decision over the patch and each completion claim |
| Check claims in a summary or PR description against the sources | jev_verify |
verified, contradicted, or unsupported for each claim |
| Screen a fetched web page for prompt injection before reading it | jev_screen |
pass, review, block, or skip |
| Review a patch against the request | jev_review |
correctness, spec match, test gaps, blast radius |
| Spot drift between docs and code, or a changelog and a diff | jev_compare |
same fact, contradiction, or different facts |
| Find the file or note that answers a question | jev_find |
the best match, plus whether anything matches at all |
| Rank search hits or grep results | jev_rerank |
a relevance score for every candidate, sorted |
| Route tickets or label many items at once | jev_classify |
one class per item from your catalog |
| Pick one option, or decide whether to keep waiting on a slow command | jev_decide |
your option, or ask_user / investigate / none |
| Grade severity or risk on your own scale | jev_score |
a position on your 2 to 10 levels, with the distribution; threshold it in code, since positions between levels are weakly calibrated |
| Pull a version, date, or price out of a document | jev_extract |
a value copied from a match of your regex, or null |
For example, "use jev_verify to check your summary against the changelog" returns one row per claim:
{
"claim": "The setup command accepts the API key as a command-line argument.",
"verdict": "contradicted",
"probabilities": { "supports": 0, "contradicts": 1, "says_nothing": 0 },
"confidence": 1,
"action": "auto",
"supporting_evidence": "setup.py"
}
Jev sees only what the agent passes in the call, so the agent has to include the evidence. docs/skills/jev-mcp/SKILL.md is a skill you can give your agent: it covers which tool fits which step and what to do with each action. Allow rules for Claude Code are printed by doctor, and opt-in setups for Claude Code, Codex, and Pi are in the harness samples.
Configuration
The server reads environment variables only. It does not load a .env file.
| Variable | Default | What it does |
|---|---|---|
TYPESAFE_API_KEY |
unset | TypeSafe key; takes priority over the stored key |
JEV_MCP_KEY_FILE |
~/.config/jev-mcp/key |
where setup stores the key and the server reads it |
JEV_PROVIDER |
auto |
auto takes the first provider with credentials: typesafe, openrouter, cloudflare, compatible. The reference's vercel provider (AI_GATEWAY_API_KEY) is not supported |
JEV_MCP_MODEL |
jev-latest |
Jev model to ask |
JEV_MCP_CACHE |
off | replay identical requests from disk at no API cost; leave it off when answers must be fresh, and delete the directory to clear it |
JEV_MCP_CACHE_DIR |
~/.cache/jev-mcp |
where the cache lives |
JEV_MCP_TRANSPORT |
stdio |
streamable-http is experimental and binds JEV_MCP_HTTP_HOST:JEV_MCP_HTTP_PORT, default 127.0.0.1:8000 |
JEV_MCP_LOG_LEVEL |
INFO |
logs go to stderr |
Architecture
One diagram covers the whole server: the tool-call loop from tools/call to the returned action text, the fail-closed answer path, and the local CLI commands around it (install, setup, hook gate, doctor) with the stored key file and the optional response cache. Open docs/architecture.html for the interactive version (guided views, dark mode, node search, relationship tracing).
Measured results
Two paid studies, both descriptive, with small samples and no significance test.
On 150 questions with Pi (opencode-go/deepseek-v4.1-flash), forcing a Jev call added 10.4 s median wall time per task. Letting the agent choose left Jev uncalled on all 150. Jev itself answered in 465 ms median over 157 calls. Accuracy was not measured. Details: evals/reports/bench150.md.
| arm | median wall s | p95 | called Jev | agent spend |
|---|---|---|---|---|
| A direct | 3.06 | 6.17 | 0/150 | $0.0928 |
| B automatic | 2.91 | 8.09 | 0/150 | $0.0955 |
| C forced | 13.95 | 28.67 | 150/150 | $0.2904 |
The agent outcome study ran on 2026-09-23 with jev-1.13.0: three tasks, three repeats per arm, with and without Jev. Both agents solved the same pairs either way and picked the right decision on every run. Both were slower with Jev. One Pi pair is excluded because its with-Jev run never called Jev. Details, raw records, and the chart script: docs/evals/.
| agent | solved without / with Jev | median time to correct, without / with | extra wall time with Jev (paired median) | spend |
|---|---|---|---|---|
Claude Code (claude-sonnet-5) |
6/9 / 6/9 | 14.6 s / 18.3 s | +4.6 s | $1.4953 |
Pi (ds4/glm-5.3-flash, local) |
6/8 / 6/8 | 49.4 s / 127.8 s | +85.8 s | $0.0006 |
About this project
This is a Python rewrite of the TypeScript @jkudish/jev-mcp 0.5.0. The ten reference tools match it on the wire, checked by recorded parity fixtures; jev_score is an addition. Vocabulary is in docs/CONTEXT.md, decisions in docs/adr/, and security notes in SECURITY.md. Windows is not supported; the server exits at startup on a non-POSIX platform.
uv sync --all-extras
make ci # lint, types, unit, property, policy coverage, contract, parity, security, build, smoke
make eval # offline scorer checks
make eval-live, make security-live, and JEV_AB_LIVE=1 make ab call paid services and stay off CI. Contribution notes are in CONTRIBUTING.md.
MIT license.
Установить Jev Judge в Claude Desktop, Claude Code, Cursor
unyly install jev-judgeСтавит в Claude Desktop, Claude Code, Cursor и VS Code — сам разбирается с npx, uvx и сборкой из исходников.
Впервые? Поставь CLI: curl -fsSL https://unyly.org/install | sh
Или настроить вручную
Выполни в терминале:
claude mcp add jev-judge -- uvx jev-judge-mcpПошаговые гайды: как установить Jev Judge
FAQ
Jev Judge MCP бесплатный?
Да, Jev Judge MCP бесплатный — установка в пару кликов через Unyly без оплаты.
Нужен ли API-ключ для Jev Judge?
Нет, Jev Judge работает без API-ключей и переменных окружения.
Jev Judge — hosted или self-hosted?
Self-hosted: сервер запускается локально на твоей машине командой из раздела установки.
Как установить Jev Judge в Claude Desktop, Claude Code или Cursor?
Открой Jev Judge на unyly.org, выбери вкладку своего клиента (Claude Desktop, Claude Code, Cursor) и нажми Install — конфиг сгенерируется автоматически, без правки JSON.
Похожие MCP
GitHub
PRs, issues, code search, CI status
автор: GitHubFilesystem
Secure file operations with configurable access controls.
Memory
Knowledge graph-based persistent memory system.
Template MCP Server
A CLI tool to create a new Model Context Protocol server project with TypeScript support, dual transport options, and an extensible structure
автор: mcpdotdirectAmap Maps Mcp Server
MCP server for using the AMap Maps API
автор: duxiaohuiSupabase
Database, auth and storage
автор: SupabaseEverything
Reference / test server with prompts, resources, and tools.
Git
Tools to read, search, and manipulate Git repositories.
Sequential Thinking
Dynamic and reflective problem-solving through thought sequences.
Time
Time and timezone conversion capabilities.
Compare Jev Judge with
Не уверен что выбрать?
Найди свой стек за 60 секунд
Автор?
Embed-бейдж для README
Похожее
Все в категории development
