loading…
Search for a command to run...
loading…
CLI-first API testing tool with MCP server. Tests are .tarn.yaml; failures come back as structured JSON for AI agents to branch on. Tools: tarn_run, tarn_valida
CLI-first API testing tool with MCP server. Tests are .tarn.yaml; failures come back as structured JSON for AI agents to branch on. Tools: tarn_run, tarn_validate, tarn_fix_plan, tarn_inspect, tarn_rerun_failed.
Tarn
API tests an AI agent can write, run, and debug.
Docs · Get started · MCP integration · Agent loop demo
Tarn is a CLI-first API testing tool written in Rust. Tests are .tarn.yaml files. Output is structured JSON with categorized failures and remediation hints, so an agent — Claude Code, Cursor, Windsurf, opencode — can write a test, run it, read what broke, and fix it without scraping logs.
# tests/health.tarn.yaml
name: Health check
steps:
- name: GET /health
request:
method: GET
url: "{{ env.base_url }}/health"
assert:
status: 200
$ tarn run
TARN Running tests/health.tarn.yaml
● Health check
✓ GET /health (4ms)
Results: 1 passed (15ms)
When something breaks, --format json returns the same run as machine-readable data with failure_category, error_code, and the offending request/response. The tarn-mcp companion exposes a tarn_fix_plan tool that turns that report into actionable suggestions an agent can apply directly.
tarn-mcp exposes list, validate, run, and fix_plan as structured tools for Claude Code, opencode, Cursor, and Windsurf..tarn.yaml and ships.curl | sh install, no runtime, drops into any CI image.# macOS / Linux
curl -fsSL https://raw.githubusercontent.com/NazarKalytiuk/tarn/main/install.sh | sh
# from source
cargo install --git https://github.com/NazarKalytiuk/tarn.git --bin tarn
Pre-built binaries for macOS (Intel + Apple Silicon), Linux (amd64 + arm64), and Windows (amd64 zip) are on the releases page, each with a tarn-checksums.txt for SHA-256 verification and a generated tarn.rb Homebrew formula artifact. The installer also lays down tarn-mcp and tarn-lsp when present in the archive. Set TARN_INSTALL_DIR to install elsewhere. Container path: ghcr.io/<owner>/tarn:<tag> from the release workflow. Manual verification works with shasum -a 256 -c tarn-checksums.txt.
The 60-second path:
tarn init # scaffold tests/ + tarn.env.yaml + advanced templates
# edit tarn.env.yaml so base_url points at your API
tarn run # runs every .tarn.yaml under tests/
Layer on the flags you actually need:
tarn run --format json --json-mode compact # structured output for agents and CI
tarn run --env staging # use a named environment
tarn run --only-failed # quiet down a noisy run
tarn run --watch # rerun on file changes
tarn run --parallel # run files in parallel
tarn list --tag smoke # what would run, without running
tarn fmt --check # canonical YAML, CI-gateable
Default to the failures-first loop — it keeps agents and humans off the megabyte-scale full report until they actually need it:
tarn validate <path> # syntax/config before running
tarn run <path> # writes .tarn/runs/<run_id>/
tarn failures # root-cause groups; cascades collapsed
tarn inspect last FILE::TEST::STEP # full context for ONE failure
# patch tests or application code
tarn rerun --failed # replay only failing (file, test) pairs
tarn diff prev last # confirm fixed / new / persistent
tarn failures groups by root-cause fingerprint and collapses skipped_due_to_failed_capture cascades into their upstream entry — one failing step with five downstream skips surfaces as one entry with cascades: 5, not six. tarn inspect supports run-id aliases last / latest / @latest / prev and drills into one record via FILE[::TEST[::STEP]]. tarn rerun --failed stamps rerun_source onto the new report. tarn diff prev last buckets failure fingerprints into new / fixed / persistent so you can confirm a patch without re-reading the full report.
Reach for .tarn/runs/<run_id>/report.json only when failures + inspect cannot answer the question. See plugin/skills/tarn-api-testing/SKILL.md (Failures-First Loop) and docs/TROUBLESHOOTING.md for the canonical agent-facing guidance, including a worked example of a mutation endpoint whose response shape changed from {"uuid": "..."} to {"request": {"uuid": "..."}} and the $.uuid → $.request.uuid fix.
A fully local demo with no external network dependency:
PORT=3000 cargo run -p demo-server &
cargo run -p tarn -- run examples/demo-server/hello-world.tarn.yaml
More local scenarios — redirects, cookies, forms, error responses, authenticated CRUD — live in examples/demo-server/.
Full guides, CLI reference, AI workflow walkthroughs, and editor setup live on the docs site:
https://nazarkalytiuk.github.io/tarn/
In-repo docs to start with:
tarn-lsp Language Server (LSP 3.17) for Neovim, Helix, Zed, and other compatible clientsThe reference sections below mirror what's on the docs site — useful when reading on GitHub directly.
Test files use .tarn.yaml and can be organized in any directory structure.
name: Health check
steps:
- name: GET /health
request:
method: GET
url: "http://localhost:3000/health"
assert:
status: 200
request.method accepts standard verbs and custom tokens such as PURGE or PROPFIND.
version: "1"
name: "User CRUD Operations"
description: "Tests complete user lifecycle"
tags: [crud, users, smoke]
env:
base_url: "http://localhost:3000/api/v1"
defaults:
headers:
Content-Type: "application/json"
timeout: 5000
retries: 1
tests:
create_and_verify:
description: "Create a user, then verify it exists"
tags: [smoke]
steps:
- name: Create user
request:
method: POST
url: "{{ env.base_url }}/users"
body:
name: "Jane Doe"
email: "jane.{{ $random_hex(6) }}@example.com"
capture:
user_id: "$.id"
assert:
status: 201
body:
"$.name": "Jane Doe"
"$.id": { type: string, not_empty: true }
- name: Verify user
request:
method: GET
url: "{{ env.base_url }}/users/{{ capture.user_id }}"
assert:
status: 200
body:
"$.id": "{{ capture.user_id }}"
setup runs once before all tests. teardown runs after all tests even if tests fail.
name: "CRUD with auth"
setup:
- name: Login
request:
method: POST
url: "{{ env.base_url }}/auth/login"
body:
email: "{{ env.admin_email }}"
password: "{{ env.admin_password }}"
capture:
auth_token: "$.token"
teardown:
- name: Cleanup
request:
method: POST
url: "{{ env.base_url }}/test/cleanup"
tests:
my_test:
steps:
- name: Authenticated request
request:
method: GET
url: "{{ env.base_url }}/users"
headers:
Authorization: "Bearer {{ capture.auth_token }}"
assert:
status: 200
assert:
status: 200 # exact match
status: "2xx" # any 2xx status
status: # set of allowed codes
in: [200, 201, 204]
status: # range
gte: 400
lt: 500
All body assertions use JSONPath expressions.
Equality:
body:
"$.name": "Alice" # string
"$.age": 30 # number
"$.active": true # boolean
"$.deletedAt": null # null
"$.field": { eq: "value" } # explicit
"$.field": { not_eq: "bad" } # inequality
Numeric comparisons:
body:
"$.age": { gt: 18, lt: 100 }
"$.count": { gte: 1, lte: 50 }
String assertions:
body:
"$.email": { contains: "@example.com" }
"$.id": { starts_with: "usr_", matches: "^usr_[a-z0-9]+$" }
"$.name": { not_empty: true }
"$.notes": { empty: true }
"$.code": { length: 6 }
"$.msg": { not_contains: "error" }
Format assertions:
body:
"$.request_id": { is_uuid: true }
"$.created_at": { is_date: true }
"$.client_ip": { is_ipv4: true }
"$.server_ip": { is_ipv6: true }
Integrity assertions:
body:
"$": { bytes: 15 } # raw response body length
"$.payload": { sha256: "2cf24dba5fb0a30e..." } # matched value digest
"$.legacy": { md5: "5d41402abc4b2a76..." }
Type checks:
body:
"$.name": { type: string }
"$.tags": { type: array, length_gt: 0 }
"$.meta": { type: object }
Existence:
body:
"$.id": { exists: true }
"$.internal": { exists: false }
Combined (AND logic):
body:
"$.id": { type: string, not_empty: true, starts_with: "usr_" }
assert:
headers:
content-type: "application/json" # exact match
content-type: contains "application/json" # substring
x-request-id: matches "^[a-f0-9-]{36}$" # regex
Header names are case-insensitive.
assert:
duration: "< 500ms"
duration: "<= 1s"
assert:
redirect:
url: "https://api.example.com/health"
count: 2
redirect.url checks the final response URL after following redirects. redirect.count checks how many redirects were actually followed.
| Priority | Source | Example |
|---|---|---|
| 1 (highest) | CLI --var |
--var base_url=http://staging |
| 2 | Shell env ${VAR} |
password: "${ADMIN_PASSWORD}" |
| 3 | tarn.env.local.yaml |
(gitignored, for secrets) |
| 4 | tarn.env.{name}.yaml |
--env staging loads this |
| 5 | tarn.env.yaml |
default env file |
| 6 (lowest) | Inline env: block |
in the test file itself |
Capture values from responses to use in subsequent steps. Captured values preserve their original JSON types (numbers stay numbers, booleans stay booleans).
steps:
- name: Login
request:
method: POST
url: "{{ env.base_url }}/auth/login"
body:
email: "[email protected]"
password: "password123"
capture:
token: "$.token" # JSONPath capture from body
user_id: "$.user.id" # nested path
- name: Use token
request:
method: GET
url: "{{ env.base_url }}/users"
headers:
Authorization: "Bearer {{ capture.token }}"
Header capture — capture values from response headers with optional regex:
capture:
session_token:
header: "set-cookie"
regex: "session_token=([^;]+)"
request_id:
header: "x-request-id"
Cookie capture — capture a response cookie by name from Set-Cookie:
capture:
session_cookie:
cookie: "session"
Status capture — capture the HTTP status code as a number:
capture:
status_code:
status: true
Final URL capture — capture the final response URL after redirects:
capture:
final_url:
url: true
JSONPath with regex — extract a sub-match from a body field:
capture:
user_id:
jsonpath: "$.message"
regex: "ID: (\\w+)"
Whole-body regex — extract from the full response body string:
capture:
body_word:
body: true
regex: "plain (text)"
Transform-lite in interpolation — reshape captured arrays and collections without dropping into Lua:
request:
form:
first_tag: "{{ capture.tags | first }}"
last_tag: "{{ capture.tags | last }}"
tag_count: "{{ capture.tags | count }}"
joined_tags: "{{ capture.tags | join('|') }}"
words: "{{ capture.message | split(' ') | count }}"
normalized: "{{ capture.message | replace(' response', '') }}"
status_code: "{{ capture.status_text | to_int }}"
payload: "{{ capture.user | to_string }}"
first and last expect arrays. count works on arrays, objects, and strings. join(...) joins array items after converting each item to its string form. split(...) and replace(..., ...) operate on strings. to_int parses integer strings, and to_string stringifies any captured value.
# UUIDs
"{{ $uuid }}" # UUID v4 (alias for $uuid_v4)
"{{ $uuid_v4 }}" # random UUID v4
"{{ $uuid_v7 }}" # time-ordered UUID v7 (Unix-ms prefix)
# Random primitives
"{{ $random_hex(8) }}" # 8-char hex string
"{{ $random_int(1, 100) }}" # random integer in range
# Wall-clock
"{{ $timestamp }}" # unix timestamp
"{{ $now_iso }}" # ISO 8601 datetime
# Faker (EN locale)
"{{ $email }}" # random email
"{{ $first_name }}" "{{ $last_name }}" "{{ $name }}" "{{ $username }}"
"{{ $phone }}" # random phone number
"{{ $word }}" "{{ $words(3) }}" "{{ $sentence }}" "{{ $slug }}"
"{{ $alpha(8) }}" # n lowercase letters
"{{ $alnum(8) }}" # n lowercase alphanumerics
"{{ $choice(red, green, blue) }}"
"{{ $bool }}" # "true" or "false"
"{{ $ipv4 }}" "{{ $ipv6 }}"
Reproducible runs. Set TARN_FAKER_SEED=<u64> (or faker.seed: <u64> in tarn.config.yaml) to freeze every RNG-backed built-in for the process. Wall-clock values ($timestamp, $now_iso, the timestamp prefix of $uuid_v7) stay real-time.
body:
"$.id": { is_uuid: true } # any UUID version
"$.legacy_id": { is_uuid_v4: true } # must be random v4
"$.event_id": { is_uuid_v7: true } # must be time-ordered v7
Tarn automatically captures Set-Cookie headers and sends stored cookies on subsequent requests. This is enabled by default.
name: Auth flow
steps:
- name: Login
request:
method: POST
url: "{{ env.base_url }}/auth/login"
body:
email: "[email protected]"
password: "secret"
# Set-Cookie from response is automatically stored
assert:
status: 200
- name: Access protected resource
request:
method: GET
url: "{{ env.base_url }}/profile"
# Cookie header is automatically sent
assert:
status: 200
Disable automatic cookies per file:
cookies: "off"
Or reset the default jar between named tests in a file so IDE subset runs and flaky suites never see session state from a prior test. Setup and teardown still share the file-level jar. Named jars (multi-user scenarios) are untouched.
cookies: "per-test"
The --cookie-jar-per-test CLI flag forces per-test isolation regardless of the file's declared mode (except when the file sets cookies: "off", which always wins).
Tarn supports first-class bearer and basic auth helpers, while keeping explicit Authorization headers as the escape hatch:
request:
auth:
bearer: "{{ env.token }}"
headers:
X-API-Key: "{{ env.api_key }}"
Basic auth:
request:
auth:
basic:
username: "{{ env.username }}"
password: "{{ env.password }}"
You can also set defaults.auth once per file. If headers.Authorization is already present, Tarn leaves it unchanged.
Use cookies: false on a step to bypass the cookie jar entirely. No cookies are sent and no Set-Cookie headers are captured:
steps:
- name: Login
request:
method: POST
url: "{{ env.base_url }}/auth/login"
body:
email: "[email protected]"
password: "secret"
assert:
status: 200
- name: Test unauthenticated access
cookies: false
request:
method: GET
url: "{{ env.base_url }}/profile"
assert:
status: 401
For multi-user scenarios, use named jars to maintain separate cookie sessions:
steps:
- name: Login as admin
cookies: "admin"
request:
method: POST
url: "{{ env.base_url }}/auth/login"
body:
email: "[email protected]"
password: "secret"
- name: Login as viewer
cookies: "viewer"
request:
method: POST
url: "{{ env.base_url }}/auth/login"
body:
email: "[email protected]"
password: "secret"
- name: Admin can manage users
cookies: "admin"
request:
method: GET
url: "{{ env.base_url }}/admin/users"
assert:
status: 200
- name: Viewer cannot manage users
cookies: "viewer"
request:
method: GET
url: "{{ env.base_url }}/admin/users"
assert:
status: 403
Each named jar is independent — cookies captured in "admin" are never sent with "viewer" requests. Steps without a cookies: field (or with cookies: true) use the default jar.
Use tarn run --cookie-jar .tarn-cookies.json to preload jars from disk and write back the updated state after the run. The file stores named jars too, so multi-user sessions can survive across runs.
--cookie-jar currently works only with sequential execution. Combine it with --parallel only after jar sharing becomes deterministic.
When the cookie jar sends cookies automatically, frameworks with CSRF protection (e.g., Better Auth) may reject requests that lack an Origin header. Add it to defaults to fix:
defaults:
headers:
Content-Type: "application/json"
Origin: "http://localhost:3000"
If your app derives the expected origin from the request URL, set Origin to match env.base_url:
defaults:
headers:
Origin: "{{ env.base_url }}"
Send application/x-www-form-urlencoded payloads with form::
steps:
- name: Login form
request:
method: POST
url: "{{ env.base_url }}/auth/login"
form:
email: "[email protected]"
password: "{{ env.password }}"
assert:
status: 200
Tarn URL-encodes the fields and auto-sets Content-Type: application/x-www-form-urlencoded unless you override it explicitly.
Note:
formcannot be combined withbody,graphql, ormultiparton the same step.
Send multipart form data for file uploads using the multipart: field:
steps:
- name: Upload photo
request:
method: POST
url: "{{ env.base_url }}/api/photos"
headers:
Authorization: "Bearer {{ capture.token }}"
multipart:
fields:
- name: "title"
value: "My Photo"
- name: "description"
value: "A test upload"
files:
- name: "photo"
path: "./fixtures/test.jpg"
content_type: "image/jpeg"
- name: "thumbnail"
path: "./fixtures/thumb.png"
filename: "custom-name.png"
assert:
status: 201
Note:
multipartcannot be combined withbody,form, orgraphqlon the same step.
Reuse shared step sequences across test files with include: directives:
name: User tests
setup:
- include: ./shared/auth-setup.tarn.yaml
steps:
- name: Get users
request:
method: GET
url: "{{ env.base_url }}/users"
assert:
status: 200
The included file's setup and steps are inlined at the include point. Includes work in setup, teardown, steps, and tests.*.steps. Circular includes are detected and rejected.
Includes also support lightweight parametrization and deep overrides for reusable step packs:
steps:
- include: ./shared/user-pack.tarn.yaml
with:
tenant: "acme"
user_id: 42
override:
request:
headers:
X-Tenant: "acme"
Inside the included file, use {{ params.tenant }} and {{ params.user_id }} placeholders.
Native GraphQL support with the graphql: block. Automatically sets Content-Type: application/json and constructs the standard GraphQL JSON body.
steps:
- name: Get user
request:
method: POST
url: "{{ env.base_url }}/graphql"
graphql:
query: |
query GetUser($id: ID!) {
user(id: $id) {
id
name
email
}
}
variables:
id: "{{ capture.user_id }}"
operation_name: "GetUser"
assert:
status: 200
body:
"$.data.user.name": "Alice"
"$.errors": { exists: false }
Re-execute a step until a condition is met. Useful for async workflows where you need to wait for a state change.
steps:
- name: Create export
request:
method: POST
url: "{{ env.base_url }}/exports"
capture:
export_id: "$.id"
assert:
status: 202
- name: Wait for completion
request:
method: GET
url: "{{ env.base_url }}/exports/{{ capture.export_id }}"
poll:
until:
body:
"$.status": "completed"
interval: "2s"
max_attempts: 15
assert:
status: 200
body:
"$.status": "completed"
poll.until uses the same assertion syntax. The step re-executes every interval until the until condition passes or max_attempts is reached.
For logic that goes beyond declarative assertions, use inline Lua scripts. Scripts run after the HTTP response is received and have access to response and captures.
steps:
- name: Validate complex logic
request:
method: GET
url: "{{ env.base_url }}/users"
script: |
-- Access response
assert(response.status == 200, "Expected 200")
-- Work with the response body (Lua table)
local users = response.body.users
assert(#users > 0, "Expected at least one user")
-- Cross-field validation
for _, user in ipairs(users) do
assert(user.email:find("@"), "Invalid email for " .. user.name)
end
-- Set captures for subsequent steps
captures.first_user_id = users[1].id
assert:
status: 200
Available in Lua:
response.status — HTTP status coderesponse.headers — response headers tableresponse.body — response body as Lua table (auto-parsed from JSON)captures — read/write captures tableassert(condition, message) — assertion (collected, not thrown)json.decode(string) — parse a JSON string into a Lua tablejson.encode(value) — serialize a Lua value to a JSON stringtarn run [PATH] [OPTIONS] Run test files
tarn bench <PATH> [OPTIONS] Benchmark a step
tarn validate [PATH] [--format] Validate YAML (--format human|json)
tarn fmt [PATH] [--check] Normalize Tarn YAML
tarn list List all tests
tarn summary <PATH|-> Re-render a prior JSON report (llm/compact)
tarn import-hurl <PATH> Convert common-case Hurl files to Tarn
tarn init Scaffold a new project
tarn update Update to the latest version
tarn update --check Check for updates without installing
tarn completions <SHELL> Generate shell completions
tarn run Options| Flag | Description |
|---|---|
--format <FORMAT> |
Repeatable. Supports human, json, junit, tap, html, curl, curl-all, compact, llm, or FORMAT=PATH. When omitted, tarn picks human on a TTY and llm when stdout is piped. |
--json-mode <MODE> |
For JSON outputs: verbose (default) or compact |
--tag <TAGS> |
Filter by tag (comma-separated, AND logic) |
--select <FILE[::TEST[::STEP]]> |
Narrow execution to specific files, tests, or steps (repeatable; ANDs with --tag) |
--var <KEY=VALUE> |
Override env variables (repeatable) |
--env <NAME> |
Load tarn.env.{name}.yaml |
-v, --verbose |
Print full request/response for every step in the streaming progress |
--verbose-responses |
Include response body, headers, and captures in the report for every step (not just failed ones). Applies to json, html, compact, and llm. |
--max-body <BYTES> |
Cap the response body size embedded when --verbose-responses or step-level debug: true is active (default 8192). Larger bodies are truncated with a "...<truncated: N bytes>" marker. |
--only-failed |
Show only failed tests and steps (summary counts stay accurate). --only-fails is accepted as an alias. |
--no-progress |
Disable streaming progress output; print the final report in one batch |
--ndjson |
Stream machine-readable NDJSON events to stdout (for editor integrations, MCP, structured CI) |
--dry-run |
Show interpolated requests without sending |
-w, --watch |
Re-run on file changes |
--parallel |
Run test files in parallel (see Parallel Execution) |
-j, --jobs <N> |
Number of parallel workers (default: CPU count) |
--no-parallel-warning |
Suppress the --parallel isolation warning (use in CI after the suite is audited) |
tarn run # all tests
tarn run tests/auth.tarn.yaml # specific file
tarn run --tag smoke # filter by tag
tarn run --env staging # staging env
tarn run --var base_url=http://localhost:8080 # override var
tarn run --format json # JSON for LLM/CI
tarn run --format json --json-mode compact # smaller JSON for automation loops
tarn run --format html # HTML dashboard
tarn run --format curl # failed requests as curl
tarn run --format curl-all=reports/replay.sh # full suite replay script
tarn run --format human --format json=reports/run.json --format junit=reports/junit.xml
tarn run --format human,json=reports/run.json # comma-separated also works
tarn run --watch # re-run on changes
tarn run --parallel --jobs 4 # parallel execution
tarn run -v # verbose
tarn run --dry-run # preview only
tarn run --only-failed # hide passing tests, show failures only
tarn run --no-progress # disable streaming, batch dump at end
tarn run --only-failed --format json # CI-friendly: only failed items in JSON
tarn run --select tests/users.tarn.yaml::login # run just the "login" test in one file
tarn run --select tests/users.tarn.yaml::login::2 # run just step index 2 of login
tarn run --select "a.tarn.yaml::login" --select "b.tarn.yaml::checkout" # union across files
tarn fmt tests/ # rewrite a directory in place
tarn fmt tests/auth.tarn.yaml --check # CI-style formatting check
--format llm)--format llm emits a grep-friendly summary line followed by only the
failed steps, each expanded with request, response, and the assertion
that failed. It strips ANSI colors automatically when stdout is piped
and omits boxed/colored headers entirely.
tarn run tests/ # emits llm format when piped (auto-selected)
tarn run tests/ --format llm # force llm format explicitly
tarn summary .tarn/last-run.json # re-summarize a prior run as llm
tarn summary reads a prior JSON report (.tarn/last-run.json is
written after every run, or whatever you produced with tarn run --format json) and re-renders it as the llm format without re-running
the tests. Use - to stream from stdin:
tarn run tests/ --format json > run.json
tarn summary run.json # render run.json as llm
cat run.json | tarn summary - # same, piped
tarn summary run.json --format compact # render as compact instead
The sibling --format compact format is a shorter human-ish variant
for quick console scanning — one line per file, inline expansion of
failed tests, and a trailing HTTP 500: 3 | JSONPath mismatch: 18
tally of failure categories. Both formats strip colors in non-TTY
output.
tarn validate --format json)tarn validate --format json emits a machine-readable report so editors and CI can surface parse errors inline. The schema:
{
"files": [
{
"file": "tests/users.tarn.yaml",
"valid": false,
"errors": [
{ "message": "found unexpected end of stream ...", "line": 14, "column": 7 }
]
}
]
}
line and column are populated for YAML syntax errors (derived from serde_yaml's error location).message only when the underlying error does not carry a location.0 when every file is valid, 2 otherwise. The human format (--format human, the default) is unchanged.tarn env --json)tarn env --json prints the project's named environments in a stable schema so editors can populate pickers and previews:
{
"project_root": "/path/to/project",
"default_env_file": "tarn.env.yaml",
"environments": [
{
"name": "staging",
"source_file": "tarn.env.staging.yaml",
"vars": {
"base_url": "https://staging.example.com",
"api_token": "***"
}
}
]
}
Inline vars from tarn.config.yaml are redacted when the key matches redaction.env (case-insensitive), so tarn env --json never prints literal secrets. Environments are sorted alphabetically by name.
tarn run --parallel dispatches test files across multiple rayon workers. The parallelism unit is the file: all setup, teardown, captures, and cookie jars stay file-scoped, but files may execute concurrently. That's unsafe when tests share mutable state (DB rows, singletons, filesystem fixtures, rate-limited upstreams). Tarn ships three isolation primitives to close the gap:
serial_only: true on a TestFile (top-level) or on an individual named test under tests: pins the file onto a single worker that runs sequentially after every parallel bucket completes. A single serial_only test escalates its whole file to the serial bucket so per-file isolation (setup/teardown, cookie jars) stays intact.group: "postgres" on a TestFile buckets files by resource name. Files sharing a group run on the same worker (serialized within the group), while different groups run in parallel. Use this to serialize "all the postgres tests" without giving up concurrency across unrelated resources.parallel_opt_in: true in tarn.config.yaml silences the startup warning once the suite has been audited. While this flag is absent (or set to false), running tarn run --parallel emits a one-line stderr warning: warning: --parallel enabled without parallel_opt_in: true in tarn.config.yaml. Tests without serial_only may share state. Pass --no-parallel-warning on the CLI to suppress it for one-off CI runs.# tarn.config.yaml
parallel: true
parallel_opt_in: true # opts in once the suite has been audited
# any .tarn.yaml that shares DB state
name: Users CRUD
serial_only: true
# or bucket by resource so postgres tests serialize while S3 tests run in parallel
name: Postgres integration
group: postgres
By default tarn run streams per-test output as each test finishes instead of dumping everything at the end. The behaviour adapts to how stdout is used:
--parallel) — each file is printed atomically when it completes, so output from concurrently running files never interleaves.human — streaming writes directly to stdout and the final emit prints only the summary line (no duplication).json, junit, tap, html, curl) — progress streams to stderr so stdout stays pure and parseable.Pass --no-progress to disable streaming entirely and restore the old "batch at end" behaviour (useful for CI logs that already capture per-line timestamps).
--ndjson)tarn run --ndjson streams machine-readable events to stdout, one JSON object per line. Designed for editor integrations (live Test Explorer updates), MCP clients, and CI pipelines that want structured progress without post-processing the final report.
Event types, in order:
file_started — a test file has begun runningstep_finished — one step finished (with phase: "setup" | "test" | "teardown"). On failure, also carries failure_category, error_code, and assertion_failures[]test_finished — a named test finished, with per-step countsfile_finished — a file finished, with its own summarydone — emitted once at the very end, carrying the aggregated summary for the whole run--ndjson composes with file-bound --format targets, so you can stream live progress and write a final report at the same time:
# Stream NDJSON to stdout, final JSON report to disk
tarn run --ndjson --format json=reports/run.json | jq '.event'
# Pure NDJSON (default human output is silently dropped on stdout)
tarn run --ndjson
--ndjson collides with any other structured format writing to stdout (e.g. --format json). Route the other format to a file, or pick one of the two streams.
In parallel mode (--parallel), each file's event stream is emitted atomically on file_finished so events from concurrently running files never interleave.
--only-failed works with both streaming and batch modes: passing tests and steps are omitted everywhere, but the final summary still reports total passed/failed counts.
| Code | Meaning |
|---|---|
0 |
All tests passed |
1 |
One or more tests failed |
2 |
Configuration/parse error |
3 |
Runtime error (network, timeout, script) |
You can emit multiple formats in one run. Keep at most one bare non-HTML format for stdout and send the rest to files:
tarn run \
--format human \
--format json=reports/run.json \
--format junit=reports/junit.xml \
--format html=reports/run.html \
--format curl=reports/failures.sh \
--format curl-all=reports/replay.sh
--format json)Structured JSON with versioned schema. Key design:
schema_version: 1 for forward compatibilityfailure_category on failures: assertion_failed, response_shape_mismatch, connection_error, timeout, parse_error, capture_error, unresolved_template, skipped_due_to_failed_capture, skipped_due_to_fail_fasterror_code and remediation_hints are included on failed steps for automation-friendly diagnosticsresponse_status and response_summary on all executed steps (passed and failed) — AI agents can see what a passed step returnedcaptures_set on steps listing which capture variables were set; captures map on test groups showing all resolved values--json-mode compact keeps the same top-level schema but drops passed assertion details and truncates response bodies to ~200 charsredaction:request is present for failed executed steps; response is omitted for connection/setup failures where no response existsSchema files:
schemas/v1/testfile.jsonschemas/v1/report.json{
"schema_version": 1,
"summary": { "status": "FAILED", "steps": { "total": 5, "passed": 4, "failed": 1 } },
"files": [{
"tests": [{
"captures": { "user_id": "usr_123", "token": "abc" },
"steps": [{
"name": "Create user",
"status": "PASSED",
"response_status": 201,
"response_summary": "201 Created: Object{3 keys}",
"captures_set": ["user_id"]
}, {
"name": "Update user",
"status": "FAILED",
"response_status": 400,
"response_summary": "400 Bad Request: name required",
"failure_category": "assertion_failed",
"error_code": "assertion_mismatch",
"remediation_hints": ["..."],
"assertions": {
"failures": [{ "assertion": "status", "expected": "201", "actual": "400", "message": "..." }]
},
"request": { "method": "POST", "url": "..." },
"response": { "status": 400, "body": { "error": "name required" } }
}]
}]
}]
}
--format curl, --format curl-all)curl exports only failed executed requests. curl-all exports every executed request in run order, including setup and teardown.
tarn run --format human --format curl=reports/failures.sh
tarn run --format curl-all=reports/replay.sh
Also supports: Human (colored terminal), JUnit XML, TAP, HTML (self-contained dashboard).
Example:
redaction:
headers:
- authorization
- x-session-token
env:
- api_token
captures:
- session_token
replacement: "[redacted]"
Reuses your existing test files for benchmarking.
tarn bench tests/health.tarn.yaml -n 1000 -c 50
tarn bench tests/health.tarn.yaml -n 500 -c 25 --ramp-up 5s
tarn bench tests/health.tarn.yaml --format json # for CI thresholds
tarn bench tests/health.tarn.yaml --format csv --export json=reports/bench.json
tarn bench tests/health.tarn.yaml --fail-under-rps 200 --fail-above-p95-ms 80
TARN BENCH GET http://localhost:3000/health — 200 requests, 20 concurrent
Requests: 200 total, 200 ok, 0 failed (0.0%)
Throughput: 3125.0 req/s
Latency:
min 1ms
p50 2ms
p95 43ms
p99 45ms
max 45ms
Tarn includes an MCP (Model Context Protocol) server for direct integration with AI coding tools.
The simplest approach is a project-level .mcp.json in the repo root (works with Claude Code and other MCP-compatible tools):
{
"mcpServers": {
"tarn": {
"command": "tarn-mcp",
"args": []
}
}
}
Alternatively, add to your Claude Code project settings (.claude/settings.json):
{
"mcpServers": {
"tarn": {
"command": "tarn-mcp",
"args": []
}
}
}
For Cursor, add to .cursor/mcp.json:
{
"mcpServers": {
"tarn": {
"command": "tarn-mcp"
}
}
}
| Tool | Description |
|---|---|
tarn_run |
Run tests, returns structured JSON results |
tarn_validate |
Validate YAML syntax without executing |
tarn_list |
List all tests and their steps |
tarn_fix_plan |
Analyze a Tarn JSON report and return prioritized next actions |
The MCP server lets your AI agent write .tarn.yaml tests, execute them, parse structured results, and iterate — all without leaving the editor.
Typical agent loop:
tarn_list to discover tests and stepstarn_validate after generating YAMLtarn_run to get structured failurestarn_fix_plan to turn the latest report into machine-friendly next stepsfailure_category, error_code, assertions.failures, and optional request/responsePASSEDSee docs/MCP_WORKFLOW.md, docs/AI_WORKFLOW_DEMO.md, and docs/CONFORMANCE.md.
Tarn ships two Claude Code plugins from a single marketplace. They solve different problems and can be installed independently or together:
tarn (top-level plugin/) — bundles the tarn-mcp MCP server and the tarn-api-testing skill. Gives your agent structured API testing capabilities: tarn_run, tarn_validate, tarn_list, tarn_fix_plan.tarn-lsp (editors/claude-code/tarn-lsp-plugin/) — registers the tarn-lsp language server with Claude Code's LSP plugin system so you get full .tarn.yaml language intelligence (diagnostics, hover, completion, code lens, code actions, quick fix, rename, go-to-definition, and the JSONPath evaluator) while editing in Claude Code.Both plugins live in the same marketplace (the repo root .claude-plugin/marketplace.json). Register it once, then install either or both:
# 1. Register the marketplace (once)
claude plugin marketplace add NazarKalytiuk/tarn
# 2a. Install the MCP + skill plugin
claude plugin install tarn@tarn
# 2b. Install the LSP plugin (project scope — see caveat below)
claude plugin install tarn-lsp@tarn --scope project
After installing tarn, Claude Code can write, run, and debug .tarn.yaml tests directly via the bundled MCP server and skill. See MCP Server and Claude Code Skill for what each component provides.
tarn — MCP + skill pluginIf you prefer manual configuration, add the MCP server to a project-level .mcp.json in the repo root:
{
"mcpServers": {
"tarn": {
"command": "tarn-mcp",
"args": []
}
}
}
This is equivalent to configuring the MCP server in .claude/settings.json but is portable across editors and tools that support MCP.
The tarn plugin configuration lives in .claude-plugin/:
plugin.json — name, version, description, author, and repository URLmarketplace.json — marketplace listing with owner info and the plugin registry (both tarn and tarn-lsp)tarn-lsp — language server pluginSeparate install from the MCP plugin above (same marketplace though). This one registers tarn-lsp for .tarn.yaml / .yaml / .yml via Claude Code's LSP plugin system so every feature documented in docs/TARN_LSP.md is available while you edit in Claude Code.
Prerequisites:
tarn-lsp binary available on $PATH — install with cargo install --path tarn-lsp from this repo, or symlink a workspace build (ln -s $(pwd)/target/release/tarn-lsp /usr/local/bin/tarn-lsp)Install (from inside a Claude Code session):
/plugin marketplace add NazarKalytiuk/tarn
/plugin install tarn-lsp@tarn --scope project
/reload-plugins
Already registered the marketplace for the tarn plugin? Skip the add line — both plugins share the same marketplace now. Substitute /absolute/path/to/repo for NazarKalytiuk/tarn if you want to install from a local checkout instead.
Compound-extension caveat: Claude Code's LSP plugin system only supports simple file extensions, so the tarn-lsp plugin claims all .yaml and .yml files in any project it is installed in (not just .tarn.yaml). Always install with --scope project in Tarn-focused repos only — do not install it globally if you also edit unrelated YAML in Claude Code.
See editors/claude-code/tarn-lsp-plugin/README.md for the full spec, troubleshooting, and the list of supported LSP features.
opencode supports Tarn through config only — there is no plugin installer or marketplace, so integration is three files checked into your repo:
your-repo/
├── opencode.jsonc # MCP + LSP registration
└── .opencode/skills/tarn-api-testing/ # agent-visible skill
└── SKILL.md
This repo ships exactly this layout at opencode.jsonc and .opencode/skills/tarn-api-testing/ (the skill is a symlink to the canonical plugin/skills/tarn-api-testing/). Clone the repo, install tarn-mcp and tarn-lsp on $PATH, run opencode inside — MCP tools, .tarn.yaml diagnostics/hover/completion, and the tarn-api-testing skill light up immediately.
To mirror the setup in your own repo, copy the snippet from editors/opencode/opencode.example.jsonc into your own opencode.jsonc:
{
"$schema": "https://opencode.ai/config.json",
"mcp": {
"tarn": { "type": "local", "command": ["tarn-mcp"], "enabled": true }
},
"lsp": {
"tarn": { "command": ["tarn-lsp"], "extensions": [".yaml", ".yml"] }
}
}
Compound-extension caveat: opencode's LSP matcher uses path.parse(file).ext, so the tarn LSP entry claims every .yaml / .yml file in the workspace (not just .tarn.yaml) — the same limitation as Claude Code. Keep this in project-level opencode.jsonc, not your global config.
See editors/opencode/README.md for prerequisites, troubleshooting, and the full skill-install flow.
The skills/tarn-api-testing/ directory contains a Claude Code skill that teaches AI agents how to write, run, debug, and iterate on Tarn tests. The skill is automatically loaded when an agent encounters API testing tasks.
What the skill provides:
Reference docs in skills/tarn-api-testing/references/:
| File | Contents |
|---|---|
yaml-format.md |
Complete .tarn.yaml schema with all properties |
assertion-reference.md |
Every assertion operator with examples |
json-output.md |
Structured JSON report schema and diagnosis algorithm |
mcp-integration.md |
MCP server setup and tool reference |
The skill triggers on keywords like "API test", "tarn", ".tarn.yaml", "test this endpoint", "smoke test", and "integration test for API".
See docs/TROUBLESHOOTING.md for the full guide, including the NestJS-style route ordering trap that Tarn flags automatically.
Common cases:
connection_error: server is down, wrong host/port, DNS issue, TLS/connect failuretimeout: step timed out before receiving a complete responseassertion_failed: request succeeded, but status/header/body/duration check failedcapture_error: the step passed assertions, but extraction failed afterwardparse_error: invalid YAML, invalid JSONPath, or invalid config surfaceAgent diagnosis loop:
tarn validate first for syntax/config errorstarn run --format jsonfailure_category before reading the message textstatus assertion carries hints, follow the first hint before second-guessing the testresponse exists, inspect it before editing assertionsrequest.url still contains {{ ... }}, fix env/capture interpolation before retryingNon-JSON bodies:
body: { "$": "plain text response" } to assert the whole root string when neededTarn does not aim for full Hurl parity. The main intentionally unclosed gaps are:
- uses: NazarKalytiuk/tarn@v1
with:
path: tests/
format: junit
env: staging
Inputs:
| Input | Default | Description |
|---|---|---|
path |
tests |
Test file or directory |
format |
human |
Output format |
env |
— | Environment name |
tag |
— | Tag filter |
version |
latest |
Tarn version |
vars |
— | Variables (newline-separated KEY=VALUE) |
tarn.config.yaml (optional)test_dir: "tests"
env_file: "tarn.env.yaml"
timeout: 10000
retries: 0
parallel: false
parallel_opt_in: false # set to `true` once tests have been audited for cross-file state sharing; silences the --parallel warning
defaults:
connect_timeout: 1000
follow_redirects: true
redaction:
headers: ["authorization", "cookie"]
environments:
staging:
env_file: "env/staging.yaml"
vars:
base_url: "https://staging.example.com"
Behavior:
test_dir sets the default discovery directory for tarn run, tarn validate, and tarn listenv_file changes the root env file name; Tarn also checks .{name} and .local variantsdefaults acts as project-wide request policy for headers/auth/timeouts/retries/redirects/delayredaction provides a project-wide default report sanitization policyenvironments makes named --env profiles first-class and powers tarn envparallel: true makes parallel file execution the default for tarn runparallel_opt_in: true acknowledges the isolation tradeoff (see Parallel Execution) and silences the --parallel warning; pair it with serial_only: and group: markers on files that share mutable statedefaults:
headers:
Content-Type: "application/json"
timeout: 5000
retries: 1
delay: "100ms" # default delay before each request
retries: 3 # retry up to 3 times on failure (exponential backoff)
timeout: 30000 # 30 seconds for this step
delay: "2s" # wait before executing
Mark an individual step so the report always records its response body,
response headers, and captures — even when the step passes. Equivalent
to running with --verbose-responses but scoped to a single step:
- name: fetch user
debug: true # keep response in the report for this step
request:
method: GET
url: "{{ env.base_url }}/users/42"
assert:
status: 200
The global --verbose-responses flag plus --max-body <BYTES> give
the same behavior for every step in the run; debug: true is a
targeted override for one-off debugging. Bodies exceeding the
--max-body cap (8 KiB by default) are truncated with a
"...<truncated: N bytes>" marker.
Add to the top of your .tarn.yaml files for IDE autocompletion:
# yaml-language-server: $schema=https://raw.githubusercontent.com/NazarKalytiuk/tarn/main/schemas/v1/testfile.json
name: My test
steps: ...
The schema is bundled at schemas/v1/testfile.json in the repository.
The structured report schema is bundled at schemas/v1/report.json.
A full-featured Tarn extension lives in editors/vscode and is published from tagged releases to both the VS Marketplace (nazarkalytiuk.tarn-vscode) and Open VSX via .github/workflows/vscode-extension-release.yml. Current version: 0.6.1.
.tarn.yaml files are indexed into a file → test → step tree. Run and Dry Run profiles, cancellable runs, and live streaming via tarn run --ndjson keep the UI in sync with long runs.failure_category, and error_code pulled straight from Tarn's JSON report.tarn stdout/stderr for each run.tarn.defaultEnvironment setting for the initial pick.*.tarn.yaml and *.tarn.yml.tarn-report.json via the redhat.vscode-yaml extension dependency.tarn.experimentalLspClient setting spawns the tarn-lsp server alongside the extension's in-process providers. This is Phase V scaffolding; no feature has migrated to the LSP path yet, so leave it disabled unless you are testing the handoff.tarn are disabled.The extension exports a TarnExtensionApi for other extensions to consume:
const tarn = vscode.extensions
.getExtension('nazarkalytiuk.tarn-vscode')
?.exports as TarnExtensionApi | undefined;
See editors/vscode/docs/API.md for the surface and stability guarantees.
A Zed extension lives in editors/zed and is published to the zed-industries/extensions registry under the id tarn. The extension wraps the same tarn-lsp binary used by the VS Code extension — installing from Zed's Extensions panel auto-downloads the matching tarn-lsp release on first activation.
Coverage:
.tarn.yaml / .tarn.yml, backed by tree-sitter-yaml.tarn-lsp language intelligence: diagnostics, completion, hover, code actions, code lens, formatting, symbols, rename, references.tarn-test, tarn-step, tarn-capture, tarn-poll, tarn-form, tarn-graphql, tarn-multipart, tarn-lifecycle, tarn-include).tarn: run file, tarn: dry-run file, tarn: validate file, plus whole-workspace variants. Accessible from the task picker or the gutter runnable at the top of each file.lsp.tarn-lsp.settings in Zed's settings.json, forwarded to tarn-lsp as workspace/configuration.Zed has no custom UI surface for extensions, so the VS Code-only features (Test Explorer tree, environment picker, run-history panel, HTML report viewer, walkthrough) are not ported. Users who need them stay on VS Code; Zed users rely on LSP-driven feedback and the task runner.
See editors/zed/README.md for install and configuration.
tarn completions bash > /etc/bash_completion.d/tarn
tarn completions zsh > ~/.zsh/completions/_tarn
tarn completions fish > ~/.config/fish/completions/tarn.fish
git clone https://github.com/NazarKalytiuk/tarn.git
cd tarn
cargo build # build
cargo test --all # test suite
cargo clippy # lint
cargo fmt # format
bash scripts/ci/smoke.sh # release-path smoke test
# Run demo server + examples
PORT=3333 cargo run -p demo-server &
cargo run -p tarn -- run examples/ --var base_url=http://localhost:3333
See docs/RELEASE_VERIFICATION.md for the broader release-candidate checklist, including watch-mode and installer verification.
Pipeline: parse YAML → resolve env → interpolate → execute HTTP → assert → report
| Module | Role |
|---|---|
model.rs |
Serde structs for .tarn.yaml |
parser.rs |
YAML loading + validation |
env.rs |
6-layer env resolution |
interpolation.rs |
{{ }} template engine |
runner.rs |
Orchestrator (setup → tests → teardown) |
http.rs |
HTTP client (reqwest) |
capture.rs |
JSONPath + header extraction |
cookie.rs |
Automatic cookie jar |
config.rs |
tarn.config.yaml parsing |
builtin.rs |
Built-in functions ($uuid, $uuid_v7, $email, $name, $timestamp, etc.) |
faker.rs |
Seedable RNG source for built-ins (TARN_FAKER_SEED / faker.seed) |
update.rs |
Self-update mechanism |
assert/ |
Status, body, headers, duration |
report/ |
Human, JSON, JUnit, TAP, HTML |
scripting.rs |
Lua scripting engine (mlua) |
watch.rs |
File watcher (notify) |
bench.rs |
Performance testing (async) |
MIT
Добавь это в claude_desktop_config.json и перезапусти Claude Desktop.
{
"mcpServers": {
"tarn-mcp": {
"command": "npx",
"args": []
}
}
}