WET Web Extended Toolkit

Distribution & CLI: the repo, CLI and Python module are wet. The PyPI distribution is n24q02m-wet (install pip install n24q02m-wet / uvx n24q02m-wet). The MCP server remains available: run wet with no subcommand (bare = MCP passthrough); the legacy aliases wet and wet-mcp remain available.

mcp-name: io.github.n24q02m/wet

Open-source MCP server for AI agents: web search, content extraction, and library docs.

Phase	Status	Scope
Phase 1	Shipped	web-core ScrapingAgent migration, smart chunks output, search polish, media slim
Phase 2	Shipped	Context7-level docs search: library index (Tier 1 + Tier 2), version-aware queries with token cap, project lock (Cabinets)
Phase 3	Shipped	extract.agent multi-step research with cited synthesis, extract.interact click/fill/submit via patchright (optional session persistence), docs_004_chunk_summaries migration, media.analyze removed (v2.0.0)

Current release: v3.x. media(action="analyze") was removed in the v2.0.0 BREAKING release. Use imagine-mcp's understand action for vision/audio/video analysis. See docs/migration.md for the upgrade recipe.

CI codecov PyPI License: Apache-2.0

Python SearXNG MCP semantic-release Renovate

Sister projects from n24q02m (click to expand)

Project	Tagline	Tag
crg	Knowledge graph for token-efficient code reviews -- semantic search and call-...	MCP
better-drive	2-way Google Drive sync with .driveignore filter — rclone engine, Windows tray	Tooling
better-email-mcp	IMAP/SMTP email for AI agents -- read, send, organize folders, and manage att...	MCP
better-godot-mcp	Composite MCP server for Godot Engine -- 17 composite tools for AI-assisted g...	MCP
better-notion-mcp	Markdown-first Notion for AI agents -- pages, databases, blocks, and comments...	MCP
better-semantic-release	Drop-in python-semantic-release fork with built-in release-safety guards (orp...	Tooling
better-telegram-mcp	Telegram for AI agents -- messages, chats, media, and contacts across both bo...	MCP
better-workspace-mcp	Google Workspace MCP server (Docs/Drive/Calendar/Gmail/Sheets/Slides/Tasks/Ch...	MCP
claude-plugins	Claude Code plugin marketplace for the n24q02m MCP servers -- install web sea...	Marketplace
imagine-mcp	Image and video understanding + generation for AI agents -- across Gemini, Op...	MCP
jules-task-archiver	Chrome Extension for bulk operations on Jules tasks via batchexecute API -- a...	Tooling
mcp-core	Shared foundation for building MCP servers -- Streamable HTTP transport, OAut...	MCP
mnemo	Persistent AI memory with hybrid search and embedded sync. Open, free, unlimi...	MCP
fastretrieval	Multi-model embedding and reranking runtime via ONNX and GGUF	Library
skret	Secrets without the server.	CLI
tacet	A self-distilling neuro-symbolic cascade that amortises LLM cost across knowl...	Tooling
web-core	Shared web infrastructure package for search, scraping, HTTP security, and st...	Library
wet	Open-source MCP server for AI agents: web search, content extraction, and lib...	MCP

Table of contents

• Features
• Status
• Quick install
• Self-host usage
• Configuration
• Documentation
• Tools
• CLI
• Comparison
• Security
• Build from Source
• Smithery
• Trust Model
• License

Features

• Web Search -- Embedded SearXNG metasearch (Google, Bing, DuckDuckGo, Brave) with query expansion, TTL cache (1 h general / 5 min time-sensitive), standardized citation format, and 200-token snippet cap. Optional cloud search backends (Tavily, Brave, Exa, Kagi, OpenRouter, Firecrawl) as a fallback chain via SEARCH_BACKENDS
• Academic Research -- Search Google Scholar, Semantic Scholar, arXiv, PubMed, CrossRef, BASE
• Library Docs -- Auto-discover and index documentation with FTS5 hybrid search, HyDE-enhanced retrieval, and version-specific docs
• Content Extract -- 5-strategy escalation chain via n24q02m-web-core ScrapingAgent (basic_http -> tls_spoof -> render backends from BROWSER_BACKENDS (native / browserless / cf-browser-rendering) -> optional key-gated captcha), markitdown bridge for low-tier HTML/MD fallback, smart chunks structured output (clean text + markdown + JSON-LD + code blocks + metadata), batch processing (up to 50 URLs), deep crawling, site mapping
• Local File Conversion -- Convert PDF, DOCX, XLSX, CSV, HTML, EPUB, PPTX to Markdown
• Media -- List + download images / videos / audio files. analyze was removed in v2.0.0 -- use imagine-mcp.understand for vision/audio inference
• Anti-bot -- Stealth strategies bypass Cloudflare, Medium, LinkedIn, Twitter
• Zero Config -- Built-in local reference embedding + reranking through fastretrieval, no API keys needed. Optional cloud models configured per task via the [models.embed|rerank|chat|jev_score] cells in ~/.wet/config.toml (OpenRouter default; one OPENROUTER_API_KEY serves every cell)
• Sync -- Cross-machine sync of indexed docs via Google Drive (OAuth Device Code, no browser redirect)

Quick install

# Method 1 (default): plugin install via Claude Code
/plugin marketplace add n24q02m/claude-plugins
/plugin install wet@n24q02m-plugins

# Method 2 (CLI): direct uvx invocation
claude mcp add wet -- uvx n24q02m-wet

# Method 3 (source-built container for HTTP / multi-device / OAuth)
docker build --target http -t wet:local .
docker run -d --name wet-http -p 8084:8080 \
  -v wet-data:/data -e PUBLIC_URL=https://wet.example.com \
  wet:local

# Method 4 (remote): point a client at an HTTP deployment
claude mcp add --transport http wet https://<your-host>/mcp

Install matrix (stdio unless noted; see the Setup page for full steps):

Client	Install
Claude Code (plugin)	/plugin marketplace add n24q02m/claude-plugins then /plugin install wet@n24q02m-plugins
Claude Code (stdio)	claude mcp add wet -- uvx n24q02m-wet
Codex	register stdio command uvx n24q02m-wet under mcp_servers in ~/.codex/config.toml
Gemini CLI	add the mcpServers JSON below to ~/.gemini/settings.json
Cursor / Windsurf	add the mcpServers JSON below via the client's MCP settings (mcp.json)
Any client (HTTP self-host)	point the client at https://<your-host>/mcp (Streamable HTTP, OAuth-gated)

Public OCI image publication is discontinued. Existing historical registry tags remain untouched; new container deployments build from source or use the Cloudflare-managed registry.

The HTTP endpoint speaks Streamable HTTP and is OAuth-gated -- your client is prompted to authenticate in the browser on first connect (no API key to paste). Stand one up via Method 3.

Full setup matrices live at the canonical docs site mcp.n24q02m.com/servers/wet/setup/ and the paste-to-agent snippets at claude-plugins/plugins/wet/setup-with-agent.md (per Spec F single source of truth).

Self-host usage

Two supported ways to run the always-listening HTTP server. In both, the MCP endpoint is http://127.0.0.1:8000/mcp (Streamable HTTP) and the config root is ~/.wet/ (config.toml, docs.db, subs/). Never run the server as a spawned subprocess of a client — register the endpoint in your client instead.

Dev (uv, no-auth loopback)

git clone https://github.com/n24q02m/wet && cd wet
uv run wet config init            # writes ~/.wet/config.toml (default: auth = "no-auth")
uv run wet                        # serves http://127.0.0.1:8000/mcp

no-auth refuses non-loopback binds — the server is localhost-only until you switch to token auth.

Always-on (docker)

# 1. Mint a token and its scrypt hash (hull-core canonical; the hash command
#    never echoes the token itself)
openssl rand -hex 32                 # the token — give it to clients, keep it secret
uv run wet token hash <token>        # paste the output as token_hash

# 2. Instance config from the example
cp docker-config/config.example.toml docker-config/config.toml
#    edit [server] token_hash and the [models.*] cells (see below)

# 3. Data continuity (optional): carry an existing instance over by copying
#    docs.db and subs/ into the wet-home named volume BEFORE first start:
docker volume create wet-wet-home && \
  docker cp ~/.wet/docs.db wet-wet-home:/docs.db && \
  docker cp ~/.wet/subs wet-wet-home:/subs  # adjust: files land as root, chown 999:999

docker compose up -d --build

Compose publishes 127.0.0.1:${WET_PORT:-8000} → container 8000 (loopback only) and mounts docker-config/config.toml read-only at /home/appuser/.wet/config.toml. Persistent data (docs.db, subs/) lives in the wet-home named volume; the download/cache dir in wet-data. Editing the config takes effect on docker compose restart.

Consumers

# CLI (same token, JSON-RPC over HTTP not needed — use the installed CLI):
uvx n24q02m-wet --help                  # or `wet <tool>` for direct tool calls

# MCP client (Claude Code):
claude mcp add --transport http wet http://127.0.0.1:8000/mcp \
  --header "Authorization: Bearer <token>"

# Raw probe:
curl -s -o /dev/null -w '%{http_code}\n' http://127.0.0.1:8000/mcp \
  -X POST -H 'Content-Type: application/json' \
  -d '{"jsonrpc":"2.0","id":1,"method":"initialize","params":{}}'            # 401
curl -s -o /dev/null -w '%{http_code}\n' http://127.0.0.1:8000/mcp \
  -X POST -H 'Authorization: Bearer <token>' -H 'Content-Type: application/json' \
  -H 'Accept: application/json, text/event-stream' \
  -d '{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":"2025-03-26","capabilities":{},"clientInfo":{"name":"curl","version":"0"}}}'  # 200

Config: local vs cloud models

The [models.embed|rerank|chat|jev_score] cells in ~/.wet/config.toml are per-task (each its own base_url + api_key + model, OpenAI-spec). The example ships OpenRouter examples; any OpenAI-compatible endpoint works — cloud, or a local server such as Ollama (base_url = "http://host.docker.internal:11434/v1" from the container). Keys in the file are host-only material; alternatively leave api_key = "" and inject at start via HULL_EMBED_API_KEY / HULL_RERANK_API_KEY / HULL_CHAT_API_KEY / HULL_JEV_SCORE_API_KEY.

Configuration

wet runs zero-config out of the box: web search uses an embedded local SearXNG, and embedding/reranking fall back to the bundled local ONNX models through fastretrieval when no cloud keys are set. For higher-quality results, configure the per-task provider cells in ~/.wet/config.toml. Operational settings are plain environment variables (no app prefix) -- in HTTP self-host mode configure them via environment variables or ~/.wet/config.toml.

Provider cells -- [models.embed|rerank|chat|jev_score] each take their own base_url + api_key + model (OpenAI-spec). OpenRouter is the pre-wired default for every cell; any OpenAI-compatible endpoint works. Cell keys may be left empty in the file and injected at start via HULL_EMBED_API_KEY / HULL_RERANK_API_KEY / HULL_CHAT_API_KEY / HULL_JEV_SCORE_API_KEY. When a cell keeps the OpenRouter default base_url, OPENROUTER_API_KEY in the environment serves it as a last-resort credential — one OpenRouter key can power every task. Empty embed/rerank cells fall back to the local ONNX models; an unkeyed chat cell disables LLM features (e.g. extract(action="agent")).

FASTRETRIEVAL_CACHE_PATH controls the local model cache.

Search backends -- SEARCH_BACKENDS is an ordered runtime fallback chain: searxng (default, local or external via SEARXNG_URL), keyed tavily / brave / exa / kagi / openrouter, optional-key firecrawl, and credential-free duckduckgo / startpage. Keyed providers use TAVILY_API_KEY, BRAVE_API_KEY, EXA_API_KEY, KAGI_API_KEY, or OPENROUTER_API_KEY (comma-separate for rotation). The openrouter backend runs the query through a chat completion with the openrouter:web_search server tool and maps the returned url_citation annotations onto the shared result shape; OPENROUTER_MODEL (default a free-tier model), OPENROUTER_BASE_URL, and OPENROUTER_SEARCH_ENGINE override its behavior. Firecrawl attempts a keyless request when FIRECRAWL_API_KEY is absent; rejection or a DuckDuckGo/Startpage bot challenge advances the chain.

Result reranking uses the configured rerank cell ([models.rerank] — cloud via OpenRouter by default, or local ONNX when unkeyed); the same chain serves web search, research, similar-page search, agent search, and docs discovery/indexing fallbacks. SearXNG retains its science-category filter for research; other providers use their own search capabilities.

Browser render backends -- BROWSER_BACKENDS (CSV, escalation chain) picks the headless render leg of extract: native (in-process chromium, the zero-config default), browserless (self-host render service -- set BROWSERLESS_URL + BROWSERLESS_TOKEN), and cf-browser-rendering (Cloudflare Browser Rendering -- set CF_ACCOUNT_ID + CF_BROWSER_RENDERING_TOKEN). Empty chain falls back to native. Set CAPSOLVER_API_KEY to append an optional, key-gated CAPTCHA tier as the last escalation step.

Robots policy -- set RESPECT_ROBOTS_TXT=true to enforce robots.txt across both the extract strategy chain and the Crawl4AI-backed crawl, sitemap, and list_media actions. The default is false to preserve existing deployment behaviour; configure this process-level policy explicitly when the operator requires robots enforcement.

Invisible tier + seeded identity (opt-in) -- append invisible to BROWSER_BACKENDS to add a stealth-Firefox engine tier as the last escalation step (n24q02m-hull[invisible] extra: pip install "n24q02m-wet[invisible]"). One coherent browser identity spans the whole chain: seed it with WET_IDENTITY_SEED (or let wet derive once and persist it under ~/.wet/subs/<sub>/identity.json) and install the wet[identity] extra. Without the extras the chain keeps legacy behaviour (warn-once). STEALTHFOX_BINARY points at a patched Firefox build to skip the engine download; IDENTITY_PROFILE_DIR (default ~/.wet/subs/<sub>/profiles) keeps persistent browser profiles per namespace.

Disable local fallbacks -- opt out of the heavy in-process local fallbacks per capability (e.g. on a slim container that renders/searches/embeds via cloud backends only): DISABLE_LOCAL_BROWSER, DISABLE_LOCAL_SEARCH, DISABLE_LOCAL_EMBED, DISABLE_LOCAL_RERANK.

HTTP self-host -- MCP_TRANSPORT=http, PUBLIC_URL=<your-domain>. Configure token authentication via token_hash in config.toml or Streamable HTTP with OAuth.

Example stdio config (one OpenRouter key powers every provider cell):

{
  "mcpServers": {
    "wet": {
      "command": "uvx",
      "args": ["n24q02m-wet"],
      "env": {
        "OPENROUTER_API_KEY": "sk-or-xxx"
      }
    }
  }
}

To use a different endpoint or model per task, edit the [models.*] cells in ~/.wet/config.toml, or inject HULL_EMBED_API_KEY / HULL_RERANK_API_KEY / HULL_CHAT_API_KEY / HULL_JEV_SCORE_API_KEY at start instead of storing keys in the file.

Status

Stable architecture with two transports: stdio (default, local) and HTTP (self-host, OAuth-gated). No daemon-bridge layer and no auto-spawn from stdio. The media.analyze action was removed in the v2.0.0 BREAKING release -- see docs/migration.md for the upgrade recipe. Current release line: v3.x.

Documentation

Full docs at mcp.n24q02m.com/servers/wet/setup/:

• Setup -- install methods for Claude Code, Codex, Gemini CLI, Cursor, Windsurf, mcp.json
• Modes overview -- stdio / http
• Multi-user setup -- per-JWT-sub credential model

In-repo references (Spec F single source of truth: setup docs live in claude-plugins/plugins/wet/):

• docs/ARCHITECTURE.md -- web-core ScrapingAgent integration, strategy chain, storage layout, LLM provider dispatch
• docs/BENCHMARKS.md -- v1.x baseline coverage / latency placeholders + tier-1 fixture metrics

Install with AI agent -- paste this to your AI coding agent:

Install MCP server wet following the steps at https://raw.githubusercontent.com/n24q02m/claude-plugins/main/plugins/wet/setup-with-agent.md

Tools

5 MCP tools (3 domain + config + help). The legacy setup tool merged into config action dispatch.

Tool	Description
search	Web (SearXNG metasearch), news, images, academic research (Scholar / arXiv / PubMed / CrossRef / Semantic Scholar / BASE), library docs (HyDE + FTS5), find similar pages. Includes docs_resolve (library name -> ranked id), docs_query (version-aware + topic + 5000-token cap), docs_lock_project (Cabinets project pin via pyproject / package.json / go.mod / Cargo.toml manifest detection).
extract	URL -> smart chunks dict (clean_text + markdown + structured_data + code_blocks + metadata) via web-core 5-strategy chain. Batch processing (up to 50 URLs), deep crawling, site mapping, local file conversion (PDF/DOCX/XLSX/PPTX/EPUB), structured extraction (JSON Schema)
media	list (discover URLs from gallery pages), download (SSRF-safe). analyze was removed in v2.0.0 -- use imagine-mcp.understand instead
config	status, set, cache_clear, docs_reindex, warmup, setup_sync, setup_status, setup_skip, setup_reset, setup_complete
help	Per-tool documentation: search, extract, media, config

Media boundary: For vision / audio understanding (image captioning, OCR, audio transcription, video summarization), use imagine-mcp. media.analyze was removed in wet v2.0.0 -- use imagine-mcp.understand instead.

CLI

The package installs three console scripts: n24q02m-wet (primary package command), wet (short alias), and wet-mcp (legacy alias kept so existing uvx wet-mcp configs keep working). A bare invocation (or any leading-dash flag) starts the server; a leading positional argument is dispatched as a subcommand.

uvx --from n24q02m-wet wet warmup   # try a subcommand without a persistent install

wet                             # start the server over stdio (default transport)
wet --http                      # start the server over Streamable HTTP (self-host mode)

wet auth google                 # authorize the Google credential provider for Drive sync
wet logout                      # clear the local Google Drive sync token
wet warmup                      # pre-download local models + run auto-setup (SearXNG, browser) to avoid first-run delays
wet docs reindex <library>      # drop the cached docs index for <library>; the next docs search re-indexes it

auth google accepts an optional bring-your-own OAuth client via --client-id and --client-secret (single-user / local machine only; the token is written to the local store). Each subcommand prints a JSON result and exits.

Capability	wet	Brave Search	Tavily	Firecrawl	Context7
Web search	Yes (SearXNG aggregation)	Yes	Yes	No	No
Extract URL	Yes (5-strategy chain)	No	Yes (basic)	Yes	No
Media list / download	Yes	No	No	No	No
Library docs search	Yes (Tier 1 curated + Tier 2 on-demand, version-aware, Cabinets)	No	No	No	Yes
Academic research	Yes (6 providers)	No	No	No	No
Self-hostable	Yes	No	No	No	Yes
Free tier	Yes (open source)	Limited	Limited	Limited	Yes

Security

• SSRF prevention -- URL validation on crawl targets
• Graceful fallbacks -- Cloud → Local embedding, multi-tier crawling
• Error sanitization -- No credentials in error messages
• File conversion sandboxing -- Defaults to the working directory and temp dir; override with CONVERT_ALLOWED_DIRS

Build from Source

git clone https://github.com/n24q02m/wet.git
cd wet
uv sync
uv run wet

Smithery

wet ships a smithery.yaml so it can be installed and run through Smithery. The manifest declares a stdio start command (uvx --python 3.13 n24q02m-wet) with an empty config schema -- no config is required to start, and providers and credentials are configured at runtime via the server's own config flow (see Configuration).

Trust Model

This plugin implements TC-Local (machine-bound, single trust principal). See mcp-core trust model for full classification.

Mode	Storage	Encryption	Who can read your data?
stdio (default)	~/.wet/config.json	AES-GCM, machine-bound key	Only your OS user (file perm 0600)
HTTP self-host	Same as stdio	Same	Only you (admin = user)

License

Apache-2.0 -- See LICENSE.