llmtrim

Local proxy that compresses LLM API traffic so you pay less. Same answers, smaller bill.

_{−31% input · −74% output · −66% round-trip cost · 112 live A/B cases · ~5 ms/call · no model to load}

_{Proxy · CLI · MCP · library (Python · Ruby · Swift · Kotlin · JS/WASM)}

Install • Day to day • How it works • Works with • Claude Code • Numbers • Config • CLI & library

Get started

npm install -g @llmtrim/cli@latest && llmtrim setup
# open a new terminal, then keep working
llmtrim status

That's it. setup starts a local proxy, wires your shell, and (when Claude Code is present) turns on the status line, cold-cache guard, /sub, and cheaper /compact. You do not run a separate install for each of those.

Any tool that honors HTTPS_PROXY works (Claude Code, Codex, Cursor, Aider, your SDK). GitHub Copilot does not (certificate pinning). Full list →

# Linux / macOS
curl -fsSL https://raw.githubusercontent.com/fkiene/llmtrim/main/install.sh | sh

# Windows (PowerShell)
irm https://raw.githubusercontent.com/fkiene/llmtrim/main/install.ps1 | iex

# Package managers
brew install fkiene/tap/llmtrim
cargo binstall llmtrim
scoop install llmtrim
docker run -d -p 43117:43117 -v llmtrim-state:/data ghcr.io/fkiene/llmtrim

llmtrim ca
openssl x509 -in ~/.llmtrim/ca.pem -noout -text | grep -A3 "Name Constraints"

Day to day

llmtrim status     # savings + health  (aliases: monitor, gain)
llmtrim update     # new release, restart daemon, refresh integrations
llmtrim ensure     # match the recommended install state on this machine

After setup, update, or ensure, owned Claude Code pieces (status line, guard, /sub, compact defaults) stay in sync with the binary. You should not need statusline install or similar after an upgrade.

Time series: llmtrim status --daily · --weekly · --monthly · --json · --csv.

What it does

You run Claude Code, Codex, Cursor, or your own app. Each request carries a large blob: system prompt, tools, history, and raw tool output. You pay for that on every turn.

A lot of it is noise. A 200-line build log with two errors. Tool schemas resent fifty times. JSON with hundreds of near-identical rows. The model does not need the bulk to answer well.

llmtrim strips that noise on your machine before the request hits the provider. Your tool stays the same, the reply stays the same, the bill shrinks.

  before:  your tool ───── full request ─────▶  OpenAI / Anthropic / …
                    ◀──────── reply ──────────

  after:   your tool ──▶ llmtrim ──smaller──▶  OpenAI / Anthropic / …
                            (on your machine)
                    ◀──────── reply ──────────  (same answer)

[!IMPORTANT] Compression cannot raise your bill or break a request. Each step is re-measured with the provider's real tokenizer and undone if it does not save tokens. If the provider rejects the compressed body, the original is resent. Worst case is zero savings.

Everything runs locally. Nothing is sent to us.

In action

Real agent build log: 58 lines, two of them errors. Keep the errors, drop the rest.

Before (4,662 chars) → after (978 chars, −79%):

# before (noise + signal)
[2026-06-13T10:02:00Z] INFO  compiling module core::worker::task_0 (incremental)
… 28 more near-identical INFO lines …
[2026-06-13T10:02:31Z] ERROR src/worker/pool.rs:214: mismatched types: expected `usize`, found `i64`
… 25 more INFO lines …
[2026-06-13T10:03:01Z] ERROR src/net/conn.rs:88: cannot borrow `buf` as mutable more than once
[2026-06-13T10:03:02Z] INFO  build failed, 2 errors

# after (errors verbatim; INFO folded losslessly)
[{}] INFO compiling module core::worker::task_{} (incremental) [×30: (10:02:00Z..10:02:29Z step 1s; 0..29)]
[2026-06-13T10:02:31Z] ERROR src/worker/pool.rs:214: mismatched types: expected `usize`, found `i64`
[{}] INFO compiling module core::net::conn_{} (incremental) [×25: 10:02:32Z..10:02:56Z; 0..24]
[2026-06-13T10:03:01Z] ERROR src/net/conn.rs:88: cannot borrow `buf` as mutable more than once
[2026-06-13T10:03:02Z] INFO  build failed, 2 errors

Try any request body:

echo '{"model":"gpt-4o","messages":[...]}' | llmtrim compress --provider openai

Claude Code

When ~/.claude exists, setup, update, and ensure wire these. No separate install commands.

◆ Opus→gpt-5.6-terra   ▓▓▓▓▓░░░ 142k   ✂ 6.8%   ◔ 3h·24% · 4d·12%   ♻ 63% cached

Idle 6h 19m, 347k tokens of context. The prompt cache has expired, so the next turn
rewrites the whole context (about $3.47 before any work happens).

llmtrim compact models haiku sonnet   # setup already sets this by default
llmtrim compact status
llmtrim compact off

[compact]
models = ["haiku", "sonnet"]

llmtrim sub auth codex login    # or kimi / grok
llmtrim sub on codex            # or kimi / grok
llmtrim sub status
llmtrim sub mode fallback       # only when Anthropic fails
llmtrim sub chain codex,kimi,grok
llmtrim sub off

/sub on [optional:codex|kimi|grok]   # bare /sub on = last window provider or global sub
/sub off
/sub status

Use it as a CLI, MCP, or library

Same engine, no proxy required. No extra model calls; compress runs in-process.

echo '{"model":"gpt-4o","messages":[...]}' | llmtrim compress --provider openai > out.json
echo '{"model":"gpt-4o","messages":[...]}' | llmtrim send --provider openai

use llmtrim_core::{compress, ir::ProviderKind};
let out = compress(request_json, Some(ProviderKind::OpenAi))?;

import llmtrim
out = llmtrim.compress(request_json, llmtrim.Provider.OPEN_AI, "aggressive")

import { compress } from "@llmtrim/js";
const out = compress(requestJson, "openai", "aggressive");

llmtrim mcp install          # Claude Code
llmtrim mcp install --print  # paste into any client

{
  "mcpServers": {
    "llmtrim": { "command": "llmtrim", "args": ["mcp"] }
  }
}

Works with

Any tool that honors HTTPS_PROXY and an env-provided CA:

No proxy: any MCP client can call llmtrim as tools (llmtrim mcp install), or use the CLI / library.

Providers come from the llm_providers registry (OpenAI, Anthropic, Google, DeepSeek, Mistral, xAI, Moonshot, Zhipu, Qwen, OpenRouter, …) and update with it. Non-LLM connections pass through untouched.

Configuration

Default is fine for most traffic. auto inspects each request and picks compressors by shape (tools → agent, code → code, long Q&A → rag, else aggressive).

Override with LLMTRIM_PRESET=<name> or preset = "<name>" in $XDG_CONFIG_HOME/llmtrim/config.toml:

Claude Code options (compact models, subscription reroute) are under Claude Code.

export LLMTRIM_UPSTREAM_PROXY=http://host:port
# or with auth: http://user:pass@host:port  (redacted in logs)

The numbers

Every case is sent twice, once original and once compressed, then both answers are scored and billed at real rates. Cost and quality are measured together, not estimated, across 112 cases:

The token cuts are model-independent (−31% input, −74% output). The dollar saving tracks the model's output-to-input price ratio: −66% here, projecting to −57% at GPT-4o rates and −59% at Claude Sonnet rates. The proxy compresses only the new-content surface and never rewrites the cache-controlled prefix, so your prompt-cache discount survives.

make -C crates/llmtrim-cli/bench data
(cd crates/llmtrim-cli && cargo run -q --features live -- bench quality \
   --corpus bench/data/squad2.jsonl --preset rag \
   --model qwen/qwen3-next-80b-a3b-instruct --route "" --n 20)

Methodology, per-corpus frontier, and confidence intervals: crates/llmtrim-cli/bench/README.md. Reproduce it:

make -C crates/llmtrim-cli/bench data   # pull real corpora (gsm8k, humaneval, dolly, hotpotqa, …)
(cd crates/llmtrim-cli && cargo run -q --features live -- bench suite)  # live A/B across all corpora (needs OPENROUTER_API_KEY)
(cd crates/llmtrim-cli/bench/scripts && PYTHONPATH=. python3 -m benchkit.tools.chart)  # regenerate the chart + table

How it compares

Each tool compresses one slice of the request. llmtrim compresses input and output, leaves the cached prefix untouched to keep the prompt cache stable, and scores on whether the answer survives the cut, not on tokens removed. Both axes below use the o200k_base encoder and reproduce from this repo.

Input

Input reduction (deterministic) next to answer quality from a live A/B. Quality is the drop vs llmtrim at each tool's compared setting (✅ held, a statistical tie; ❌ significantly lower), so a big reduction with a ❌ means the tool bought tokens by losing answers:

Overhead is the median per-call compress time (Python wall-clock, not like-for-like CPU): Headroom and leanctx run ML on CPU here (faster on a GPU) and pay a one-time model load on top (~3 s and ~4 s); llmtrim is Rust and entroly is lexical, so neither does.

• auto is the quality-gated default; aggressive accepts lossy cuts where the gate holds.
• Headroom drops to 0% with its ML disabled (its routers no-op on prose).
• leanctx and entroly are lossy with no quality gate; entroly has no low-reduction mode.

Headroom ties at matched reduction (24-25%, n=30, not significant) but its longer answers hit the model's output-token limit and get truncated 12 times to llmtrim's 2, the output inflation behind its higher cost. leanctx (measured at 26%) and entroly (at 69%, its mildest) score significantly lower than llmtrim (n=20), and fall further at their headline reductions (vs-leanctx, vs-entroly).

Output

Output reduction by asking for terser responses, on a paid live call over 9 coding prompts:

The cost is the 949-token system prompt caveman resends on every request (right column); llmtrim's is 19 for nearly the same cut. Both still net-save here, so caveman's deeper cut comes out ahead only when the output it removes is worth more than the 949 tokens it adds back (vs-caveman artifact).

The tools stack: RTK shrinks CLI output, then llmtrim compresses the tool schemas on top. Full head-to-heads: crates/llmtrim-cli/bench/README.md.

Known limits

These are surfaced by the same A/B that proves the savings:

• Anthropic / Gemini token counts are approximate. There's no public exact tokenizer, so a BPE proxy is used and flagged in status. OpenAI is exact.
• Output savings aren't measured live. The proxy compresses input; an output saving needs the A/B counterfactual, which only the offline benchmark runs. status "saved" is input-side.
• The default is quality-gated, not lossless. Lossy stages run only where the eval shows quality holds. Want a byte-faithful round-trip? Use the safe preset.
• "Lossless" is input-side, not response restoration. A lossless stage preserves the information the model reads (a folded log run, a TOON-encoded array, an abbreviation legend the model decodes in-prompt), and the token gate reverts any input cut that doesn't pay off. The engine does not transform the model's response back to an original form.

Acknowledgments

Every compressor is a deterministic implementation of published research: the ideas are theirs, the engineering and the token gate are ours.

Found a problem?

llmtrim doctor          # diagnose
llmtrim doctor --fix     # diagnose + apply repairs
llmtrim ensure          # same repair path

Each failing check names its fix. If a request was mangled, set LLMTRIM_CAPTURE_DIR and open an issue with the before/after pair.

If llmtrim saved you money, a ⭐ helps others find it.

Star history

_{Licensed under MPL-2.0. Use llmtrim freely in your stack, including commercially, with no source-disclosure obligation for your own code; the file-level copyleft applies only to modifications you make to llmtrim's own source files. Contributions via DCO sign-off.}