Hydra ACI

---

The Mount Test

Every design decision in this repo is judged against one scenario:

You point Hydra at a URL it has never seen. Seconds later, your agent is calling typed tools against that site: search, filter, submit, whatever it actually exposes. No server was built. No config was hand-written. No human sat in a browser translating the UI.

This is Hydra's release-defining target interface — what a passing run is designed to look like:

[!IMPORTANT] Hydra is ready for developer testing, not production deployment. The canonical runtime, structured-first planner, browser-probe promotion path, MCP surfaces, H8 persistence, and autonomous challenge-routing foundation are implemented and tested. Broad live-site evidence, actual-client compatibility runs, complete OAuth/login coverage, and release-candidate gates remain active work. The session above illustrates the real interface, not a recorded benchmark. Read the honest status, architecture, and compatibility policy before relying on Hydra for consequential automation.

Try the public alpha

pip install hydra-aci                     # core install
hydra install-browser                     # Chromium for the browser-as-probe fallback
export ANTHROPIC_API_KEY=sk-ant-...        # or OPENAI_API_KEY / GEMINI_API_KEY (for goal-driven runs)

# One goal in — the AI does the rest, autonomously:
hydra run --goal "find the price of the first product" https://example.com

# Or drive the typed tools yourself (one-shot, warm cache makes repeats instant):
hydra tools list https://example.com
hydra call https://example.com search --arguments '{"q":"widgets"}'

Autonomous goal run hydra run --goal "…" <url>

Connect a harness (MCP) hydra-connect verify codex

Reuse installed MCP apps hydra-connect mcp-apps

The AI runs the whole task itself — filling forms, calling APIs, navigating — and only pauses to ask you at a financial/destructive finalization or an MFA challenge (--yes auto-approves for unattended use). Testing against localhost? Set HYDRA_EGRESS_ALLOW_HOSTS=127.0.0.1 (private IPs are blocked by default for SSRF safety).

That is the whole product. Everything below is how it survives contact with the real web.

---

Why Hydra Exists

The web was built as a presentation system for people, not as one coherent interface for software agents. A human sees a product page, a search box, and a checkout flow. An agent sees the same operation scattered across HTML, framework hydration, XHR calls, undocumented endpoints, visual controls, cookies, and authentication challenges. The capability is already there; its machine-readable meaning is fragmented.

The industry has built powerful ways to reach individual fragments, but none is a universal agentic web layer on its own:

• Browser-use, Stagehand, and Playwright-style agents give models broad reach, but make the human presentation layer the machine protocol. Even when they inspect the DOM instead of pixels, the core loop is still observe UI, reason, act, and observe again. That adds repeated model latency, couples plans to presentation, and generally preserves any reusable result as a page action or selector rather than a cross-surface capability contract.
• MCP gives agents the right interface - typed tools and resources - but it is opt-in infrastructure, not automatic web coverage. Someone must publish or select a server, audit it, install it, configure transport, bind credentials, and repeat that setup across clients. Most unchanged websites expose no MCP server at all.
• Firecrawl, Crawl4AI, and Scrapling-style systems make web content much easier to read, but reading is only one slice of the web. Authenticated sessions, state transitions, side effects, challenge handling, and action verification remain separate problems.
• Lightpanda, Chromium, Camoufox, Patchright, and remote browsers can execute the web with different speed and compatibility profiles. They are runtimes, not a shared capability schema, planner, credential boundary, memory, or harness interface.

These are not bad tools. They are strong specialists with different contracts. The fundamental mismatch appears when an agent is expected to discover, configure, reason across, and remember all of those contracts by itself.

The missing primitive is a translation layer: the web should not need to be rebuilt for agents, and an agent should not have to relearn every website through controls designed for eyes and hands.

Current path	What it gets right	The exact failure mode	Hydra's response
OpenAPI, GraphQL, feeds, manifests	Native semantics and efficient direct calls	Coverage is opt-in and fragmented; discovery, auth binding, schema repair, and harness wiring are still integration work	Discover and normalize those surfaces first, then bind them into the same mounted catalog
MCP applications	The cleanest agent-facing tool contract where one exists	A human still finds, installs, trusts, configures, authenticates, and repeats setup across clients; most destinations do not publish MCP	Import trusted existing MCP apps, while Hydra itself serves unchanged sites as dynamic MCP tools
Browser-use, Stagehand, and Playwright-style agents	Can reach interfaces with no public API	The model must reason through a presentation layer designed for eyes and hands; observations and actions add latency and failure surface, while cached UI actions remain tied to page structure instead of becoming cross-surface capability contracts	Use the browser as a governed probe or final action substrate, then promote stable network and semantic behavior into typed tools
Firecrawl, Crawl4AI, Scrapling-style data tools	Strong crawl, rendering, markdown, and structured extraction	Reading is not the same as performing authenticated, stateful, side-effecting work; action, session, and verifier semantics live elsewhere	Put extraction and action behind one auth/session/capability system with provenance and verification
Lightpanda, Chromium, Camoufox, Patchright, remote browsers	Provide different speed, compatibility, and resistance profiles	A runtime is an execution engine, not a universal schema, planner, memory, or harness interface	Select the runtime per route while keeping the capability contract independent of the engine
Hand-built site integrations	Can be deterministic and excellent	Reliability is purchased with per-site engineering and permanent maintenance	Compile from surfaces the site already ships and preserve stable IDs across refresh when behavior is unchanged

Hydra began with a different architectural question: what if the agent side could translate the web that already exists? Hydra is a client-side web capability compiler. It combines the specialist layers behind one deterministic runtime and turns whatever a destination exposes into the things models already understand: typed tool calls, JSON Schema, structured data, and Markdown.

1. Discover existing machine surfaces first: MCP, OpenAPI, GraphQL, manifests, feeds, framework state, structured data, forms, and live application routes.
2. Bind sessions and credentials through H6 without placing secrets in prompts, tool schemas, traces, or memory.
3. Probe with a governed browser only when the structured surface is incomplete, observing network behavior and application state instead of defaulting to long click/type loops.
4. Compile every useful route into one canonical Web Capability IR with stable identity, typed inputs, auth requirements, side-effect class, confidence, verifier, fallback chain, and redaction policy.
5. Serve the same live capability catalog through MCP, SDK, CLI, and model-native adapters, so the agent uses one language regardless of the site or underlying runtime.
6. Verify and remember successful capabilities, workflows, and drift fingerprints, allowing later mounts to reuse proven behavior and recompile when the site changes.

That combination is Hydra's distinction. It can consume an existing MCP app when one is available, compile an unchanged website when one is not, and fall back to browser execution without making raw UI manipulation the agent's primary language. The destination does not need to amend itself.

[!NOTE] Hydra's earlier “5% directly usable / 95% dark” framing — what earlier drafts called "the 95% problem" — is a north-star hypothesis, not a measured statistic. The project will publish a coverage percentage only after a defensible site-task sampling study exists.

Hydra takes the inverse-MCP path. Instead of waiting for every destination to become agent-native, Hydra makes existing destinations agent-usable from the client side. It discovers what is already there, compiles it into canonical capabilities, and serves those capabilities through MCP, SDK, CLI, and provider adapters.

from hydra_aci.sdk import HydraSDK

sdk = HydraSDK()
session = await sdk.mount("https://any-site.com")
print(session["capabilities"])
await sdk.call_tool(session["session_id"], "search", {"q": "widgets"})

---

How It Works

The finished architecture runs every session through one deterministic finite-state kernel and is structured-first by construction. The browser is a probe or a final execution substrate, never the model's default language. The status section states where the current implementation still falls short of this invariant.

you ──▶ hydra mount https://any-site.com
        │
        ├─ discover   H1 scout: OpenAPI · GraphQL · llms.txt · MCP apps · __NEXT_DATA__ · live XHR
        ├─ compile    H4 schema engine: everything → one canonical Web Capability IR, structured-first
        ├─ serve      ▶ typed tools · JSON Schema · markdown — MCP · SDK · OpenAI · Anthropic · Gemini
        └─ remember   H8 archive: verified capabilities + drift fingerprints → the second mount is instant
                        call_tool("search", {"q": "widgets"})   # as if the site shipped MCP all along

Everything a site exposes becomes one stable record in the canonical Web Capability IR: an id, an input schema, a source type, an auth requirement, a side-effect class, a verifier summary, a fallback chain, and a redaction policy. A route planner always prefers the highest-signal surface (mcp > openapi > graphql > llms.txt > embedded data > observed XHR > forms > browser action), so an agent calls a real API before it ever considers clicking a button.

The invariants

Five rules keep Hydra coherent. They are documented, test-enforced, and non-negotiable in review:

1. Structured-first by construction. The browser is a probe, never the pilot. A probe exists to compile durable direct tools, then get out of the way.
2. One IR, no dialects. Whatever the surface — OpenAPI, GraphQL, an observed XHR, a form — it becomes the same canonical record, consumed identically by every adapter.
3. Secrets never meet the model. H6 holds credentials as opaque handles and types values straight into the page; they never enter LLM context, traces, or archives.
4. No capability without a verifier. A tool is not trusted because it parsed; it is trusted because it ran and its verifier agreed.
5. Determinism or it didn't happen. One FSM kernel, deterministic traces, replay. Head boundaries stay sealed.

The Eight Heads

The name is literal. Eight specialized Heads run inside an isolated browser enclave, each designed to be best-in-class at one job, composed by the kernel — a design target the project tracks against category leaders, not a claim that every Head has already closed that gap.

Head-by-head technical ownership

Head	Name	Responsibility
H1	Protocol Scout	API-first discovery: OpenAPI/Swagger, AI-plugin manifests, llms.txt, GraphQL introspection, MCP configs, embedded app data (__NEXT_DATA__, Nuxt, Remix, Apollo/Relay, JSON-LD), and sitemaps before touching the DOM
H2	AXTree Compiler	Live DOM into an LLM-optimized accessibility tree, readable prose Markdown, and stable fingerprints
H3	Network Sniffer	Intercepts XHR/fetch, infers typed API schemas, merges samples across observations, and redacts secrets
H4	Schema Engine	Generates typed JIT tools with token-aware budgeting and goal-relevance ranking, then emits the canonical IR
H5	Camouflage	Human-paced input, coherent session fingerprints, and pluggable stealth backends
H6	Security Vault	AES-256-GCM, scoped secrets, and direct injection so values never enter LLM context, traces, or archives
H7	Challenge Router	Detects Turnstile, reCAPTCHA, hCaptcha, and MFA, then routes to autonomous resolution or a necessary human boundary
H8	Web Memory	Site profiles, drift fingerprints, auth-flow shapes, verified workflows, stable-ID remapping, and deterministic replay

Why "Hydra"? Because the web keeps decapitating automations: selectors rot, markup shifts, APIs move behind new auth. Hydra is built so capabilities grow back — H8 keeps drift fingerprints and verified workflows, so a site redesign is designed to trigger a re-compile, not a rewrite.

Full architecture: docs/architecture.md · Capability strategy: docs/capability-compiler.md

---

⚡ Quickstart

The whole product, copy-paste tested, no clone required:

pip install hydra-aci          # or: pipx install hydra-aci (isolated, zero-venv-to-manage)
hydra install-browser          # Chromium for the browser-as-probe fallback (same as `playwright install chromium`)

export ANTHROPIC_API_KEY=sk-...   # or OPENAI_API_KEY / GEMINI_API_KEY / HYDRA_LLM_API_KEY
                                   # structured mount/call/tools work with no key configured at all;
                                   # only goal-driven `hydra run` needs one

hydra tools list https://example.com
hydra call https://example.com <tool_name> --arguments '{"q":"widgets"}'
hydra run --goal "find the price of the first product" https://example.com

Reaching non-public targets. Private, loopback, and cloud-metadata addresses are blocked by default — even localhost — as an SSRF guard. To test against a local or internal service, opt in explicitly:

export HYDRA_EGRESS_ALLOW_HOSTS=127.0.0.1,localhost

hydra run is autonomous end-to-end: it mounts, plans, executes, and verifies with no human in the loop, pausing only for a financial/destructive confirmation or an MFA challenge. Pass --yes to auto-approve those confirmations for unattended runs (MFA always still needs a human).

Source checkout / local development

git clone https://github.com/Suraj1235/Hydra-aci.git
cd Hydra-aci
pip install -e .
hydra install-browser

Zero-install (uvx) — no clone, no install, ephemeral:

uvx --from hydra-aci hydra-setup --auto

--auto is an alias for --all (configure every supported harness Hydra can detect); hydra-connect setup --auto and hydra setup --auto accept it too.

Optional vector search for the Knowledge Archive:

pip install -e .[vector]

Python SDK

import asyncio
from hydra_aci.sdk import HydraSDK

async def main():
    sdk = HydraSDK()

    # Mount a site, get a live typed tool catalog
    session = await sdk.mount("https://example.com", goal="find product prices")
    for cap in session["capabilities"]:
        print(cap["name"], "<-", cap["source_type"])

    # Or run a one-shot structured extraction
    data = await sdk.extract(
        "https://example.com/products",
        schema={"type": "array"},
        prompt="every product name and price",
    )
    print(data)

asyncio.run(main())

The SDK surface: mount, compile, probe, call_tool, refresh, run, scrape, crawl, extract, map, diff, replay, and a memory facade.

Drop Hydra into your coding agent with no manual config editing:

hydra-connect doctor                    # or: python -m hydra_aci.connect doctor
hydra-setup --all --no-verify           # writes Codex, Claude Code, Cursor, Gemini CLI, OpenCode, etc.
hydra-connect verify codex --session-smoke

Harness path	Command
🩺 Public readiness check	hydra-connect doctor
🚀 Install everywhere Hydra can detect	hydra-setup --all --no-verify
🟣 Claude Code only	hydra-connect install claude-code && hydra-connect verify claude-code
🟢 Codex only	hydra-connect install codex && hydra-connect verify codex --session-smoke
🔌 Existing MCP apps	hydra-connect mcp-apps
🧹 Remove from a harness	hydra-connect uninstall claude-code

🔧 hydra-connect, in depth — single-harness setup, JSON doctor reports, authorization, uninstall

For a single harness, run hydra-connect setup claude-code or hydra-connect install claude-code && hydra-connect verify claude-code. hydra-connect doctor --json reports package health, MCP stdio readiness, configured harnesses, authorization posture, remediation, and next steps. Add --authorize when you want the installed MCP server to start with advanced browser/credential capabilities enabled. During public testing, hydra-connect uninstall claude-code removes Hydra from that harness config; pass --config when you want to target an explicit file.

🔌 Bring the MCP apps you already have — Hydra imports them and serves them beside compiled web tools

On mount, Hydra scans common local MCP config files and repo manifests at .hydra/mcp-apps.json or mcp-apps.json, skips its own hydra entry, verifies upstream stdio MCP servers with tools/list, converts their tools into the same canonical IR, and exposes them beside website-compiled tools. Use ${ENV_VAR} placeholders in repo manifests so each tester supplies secrets through their shell, not through committed config. hydra-connect mcp-apps shows which apps are available and which env vars are missing. If a task is better served by an existing GitHub, database, filesystem, or SaaS MCP app, Hydra can call that app; if no MCP app exists, Hydra synthesizes the web capability itself. Secret-like environment values stay local and are redacted from capability metadata.

Or run the MCP server directly. Hydra is the MCP server. The destination site never needs one.

python -m hydra_aci.mcp_server              # stdio
python -m hydra_aci.mcp_http_server         # local HTTP + SSE at /mcp

{ "mcpServers": { "hydra": { "command": "python", "args": ["-m", "hydra_aci.mcp_server"] } } }

---

Works With Every Harness

Because every adapter consumes the same canonical capability IR — surfaced as typed tool calls, JSON Schema, or clean markdown, whichever your stack speaks — one mounted site behaves identically everywhere it is used.

Coding agents (via hydra-connect or MCP):

Setup recipes for each live in docs/integrations/.

Native SDK adapters for direct model-provider users:

Provider	Import	Status
OpenAI	hydra_aci.adapters.openai	✅ first-class
Anthropic	hydra_aci.adapters.anthropic	✅ first-class
Google Gemini	hydra_aci.adapters.gemini	✅ first-class
AWS Bedrock	hydra_aci.adapters.bedrock	✅ first-class

Plus MCP, LangChain, CrewAI, AutoGen, and OpenClaw: 12 adapters, all consuming the same IR through a shared from_capabilities() contract, never re-hand-wrapped per harness. Three further adapters (Hermes, KimiClaw, AntiGravity) exist but are experimental and unverified — do not rely on them in production.

---

How Hydra Compares

Hydra is not a new browser engine, scraper, or reasoning model. It is the layer above them: compile an unchanged destination into typed capabilities, route each call through the best available substrate, and keep the client-facing contract stable. That is its architectural advantage. Performance leadership remains an evidence question.

Capability	Hydra today	Browser agent	Crawl / extract service	Browser runtime
One canonical typed catalog from API, MCP, embedded state, XHR, DOM, and replay	Implemented	UI action vocabulary	Extraction schema	No semantic catalog
Structured route before browser acquisition	Implemented	Usually browser-first	Structured/read-first	Caller decides
Browser observation promoted into a reusable direct capability	Implemented on the verified probe path; broader live corpus pending	Usually remains an action trace	Outside core focus	Outside core focus
Existing local/remote MCP apps in the same catalog	Implemented broker and trust path; registry-scale discovery still maturing	Client-specific	Outside core focus	Outside core focus
Shared auth boundary for direct calls and browser work	Partial: H6 handles and session reuse exist; broad OAuth/login conformance remains	Framework-specific	Service-specific	Caller-owned
Challenge-aware stealth fallback	Implemented provider/router/H5 path; public live-site corpus pending	Tool-dependent	Outside core focus	Engine-dependent
Verifier-backed memory and stable capability IDs	Implemented foundation and warm-mount proof; drift evidence still expanding	Trace/memory varies	Cache varies	Outside core focus
Same capabilities through MCP, SDK, CLI, and provider adapters	Implemented contract; actual-client compatibility matrix pending	Framework-specific	API client-specific	Caller-owned
Pluggable Chromium, Patchright, Camoufox, Lightpanda, CDP, and remote paths	Implemented selection surface; installation and site compatibility vary	Often one browser stack	Managed by vendor	Is the substrate

The comparison is intentionally asymmetric: browser-use, Firecrawl, Crawl4AI, Playwright, and Lightpanda are category specialists. Hydra is designed to use or route around specialists without making the agent learn a different interface for each one. The repository does not yet claim verified category leadership. The reproducible methodology lives in docs/benchmarks/methodology.md; only repeated, isolated, verifier-backed runs may turn this architecture claim into a performance claim.

---

Status & Honest Numbers

Hydra is developer alpha (0.2.0). The ambition is large and stated plainly above. The status below is stated just as plainly, because a claim without evidence is worth nothing.

• ✅ 2,688 tests passing (8 skipped, 0 failing) with 81.90% branch coverage at the latest committed runtime checkpoint (pytest -n auto -q + pytest tests -q --cov=hydra_aci --cov-branch, re-measured 2026-07-13 — this snapshot drifts as the suite grows and is refreshed at each release checkpoint, not continuously).
• ✅ Built-artifact proof: clean, non-editable wheel installs pass doctor, MCP handshake, and every console-script smoke test on Linux, macOS, and Windows in GitHub Actions.
• ✅ Real and exercised today: canonical immutable capability descriptors, one executable registry, one orchestrator for SDK compile/mount/execute/run/call/refresh/navigation paths, policy-aware structured-first planning, MCP stdio and HTTP transports, adapter formatting, encrypted vault primitives, browser backends, and H8 persistence foundations.
• 🚧 Partially implemented: direct canonical H1–H4 candidate emission and deduplication, a bounded active browser-probe loop, live H8 workflow/auth learning, and safe automatic upstream MCP reuse.
• 🧪 Current public benchmark pointer: a non-competitive dry run that validates the benchmark plumbing. It is not evidence of Hydra's task success, web coverage, or superiority over another product.
• 🎯 Evidence target: publish verified site-task coverage only after isolated repeated runs against named baselines. No percentage of the worldwide web is asserted today.

Public implementation boundaries are summarized above and in docs/architecture.md. Historical design slices live in docs/plans/; release evidence lives in docs/release-evidence/.

---

Responsible Use

Hydra is infrastructure for authorized agentic automation: your own apps, internal tooling, consented user automation, QA and accessibility testing, and research on systems you may access.

Defaults are conservative: robots.txt is respected and requests are rate-limited. Advanced capabilities (automated challenge solving, anti-bot fingerprint evasion, raw evaluate_js, credential injection) require a single, one-time, informed authorization granted at install or config time (hydra-authorize, --authorize on the MCP server, or HydraSDK.authorize()), persisted locally and audit-logged.

Do not use Hydra to access systems you lack permission for, to circumvent access controls, or in violation of applicable law. See AUP.md and SECURITY.md.

---

Contributing

Contributions are welcome. Read CONTRIBUTING.md first. The invariants that keep Hydra coherent (head boundaries, structured-first routing, secrets never leaving H6, determinism and replay) are documented and enforced by tests. Please keep them green.

Credits

Hydra is built in the open, and three open-source projects shaped its thinking more than any others:

• browser-use — showed how far an LLM-driven browser agent can go, and where the ceiling is. Hydra's "browser as probe, never the pilot" stance is a direct answer to lessons browser-use taught the whole field.
• Firecrawl — set the bar for developer-friendly scrape / crawl / extract APIs. Hydra's scrape, crawl, extract, and map verbs deliberately speak the dialect Firecrawl made familiar.
• Model Context Protocol — the open standard that made "typed tools for agents" a shared language. Hydra serves MCP natively, and exists to extend that promise to the sites that never built a server.

License

Hydra's current source and distributions are licensed under the GNU Affero General Public License v3.0 only (AGPL-3.0-only). See LICENSE for the complete terms, including its remote-network provisions. Previously distributed versions remain available under the license terms that applied when they were received. The license text controls if this summary differs.

Author

Suraj Kuncham — GitHub @Suraj1235 · LinkedIn