ShantyCrawl MCP

Firecrawl with a chill token footprint. 🦥🔥

Drop-in replacement for firecrawl-mcp that loads 6 tools instead of 28 on session start — saving ~18,000 tokens before you've even sent your first message.

---

_{Why · Quick Start · Configuration · Tool Architecture · How It Works · Local Dev · Contributing · License}

 

Why ShantyCrawl MCP

Most full-featured MCP servers register every tool on every session — including the ones you'll rarely touch — with full JSON schemas re-sent on each tool-list fetch. ShantyCrawl loads 6 core tools up front and lazy-loads the rest on demand, which measured out to ~18,000 fewer tokens on a fresh session in our testing:

	firecrawl-mcp (official)	shantycrawl-mcp
Tools on session start	28	6
Initial schema footprint	Full schemas for all 28 tools	Minimal schemas for 6 tools
Advanced tools (research, monitoring, agent, etc.)	Always loaded	Loaded on demand via tool_enable
Measured token cost, fresh session	Baseline	~18k tokens lighter
Runtime dependencies	—	Zero third-party wrappers, native fetch

Is This For You?

• ✅ Good fit — you use Firecrawl mostly for scrape/crawl/search, run long or multi-tool agent sessions, or are tight on context budget.
• ⚪ Less critical — you're already using tool_enable-style dynamic loading elsewhere, or you live almost entirely in the monitoring/research toolset and rarely call the core 6 — in that case the savings are smaller, since you'll be loading most of the schema anyway.

 

🚀 Quick Start

Run instantly via npx — no installation required:

FIRECRAWL_API_URL=http://localhost:3002 npx shantycrawl-mcp

Environment variables:

Variable	Required	Default	Description
FIRECRAWL_API_URL	No	http://localhost:3002	URL of your Firecrawl instance (self-hosted or cloud)
FIRECRAWL_API_KEY	Only for Firecrawl Cloud	—	API key, required if FIRECRAWL_API_URL points to a hosted/cloud instance

Requirements: Node.js 18+ and a reachable Firecrawl instance (self-hosted or cloud).

 

⚙️ Configuration

Claude Desktop

Add this to your claude_desktop_config.json:

{
  "mcpServers": {
    "shantycrawl": {
      "command": "npx",
      "args": ["shantycrawl-mcp"],
      "env": {
        "FIRECRAWL_API_URL": "http://localhost:3002"
      }
    }
  }
}

OpenCode

Add this to your workspace settings:

{
  "mcp": {
    "shantycrawl": {
      "type": "local",
      "command": ["npx", "shantycrawl-mcp"],
      "environment": {
        "FIRECRAWL_API_URL": "http://localhost:3002"
      }
    }
  }
}

 

🛠️ Tool Architecture

Always Available (6 Core Tools)

Tool	Description
scrape	Extract clean markdown from any URL
crawl	Crawl a target website recursively
search	Execute web searches via Firecrawl
tool_enable	Dynamically inject an advanced tool into the active session
tool_disable	Unload an advanced tool to free up context
check_crawl_status	Check the status of an async crawl job

Lazy-Loaded Tools (22 Advanced)

Activate any tool instantly during a session — e.g. tool_enable map unblocks the mapping capability.

Category	Tools
Discovery	map, extract, parse
AI Agent	agent, agent_status
Browser Interaction	interact, interact_stop
Research & Academic	research_search_papers, research_inspect_paper, research_read_paper, research_related_papers, research_search_github
Monitoring	monitor_create, monitor_check, monitor_checks, monitor_delete, monitor_get, monitor_list, monitor_run, monitor_update
Feedback	search_feedback, feedback

 

🧠 How Dynamic Loading Works

1. The client sees a minimal 6-tool schema on connect.
2. When the AI needs an advanced capability, it calls tool_enable <tool_name>.
3. The server fires a notifications/tools/list_changed event, and the client's available tools update instantly — no reconnect required.
4. Call tool_disable <tool_name> once you're done to free the context back up.

⚠️ Known Issues

Some agents don't call tool_enable on their own. A compelling tool description isn't always enough — if another skill or system prompt has already taught the agent a different path to the same outcome (e.g. calling the Firecrawl API directly via curl/raw HTTP), it can follow that instead of discovering tool_enable. This shows up most with coding agents that have built-in or third-party Firecrawl skills already loaded.

Fix: add an explicit instruction to your agent's system prompt or AGENTS.md/CLAUDE.md:

This project uses the shantycrawl-mcp server. Advanced tools (map, extract, parse,
agent, interact, research_*, monitor_*) are not available until activated with
tool_enable("<tool_name>"). Always use the MCP server's tools directly — never call
the Firecrawl API via curl, raw HTTP, or other bash-based workarounds.

We also ship a ready-made SKILL.md that encodes this protocol — drop it into your skills directory if your agent supports skill loading, and pair it with the snippet above for the strongest guarantee.

 

🏗️ Local Development

git clone https://github.com/schlemperdev/shantycrawl-mcp.git
cd shantycrawl-mcp
npm install
npm run build
npm start

🤝 Contributing

Issues and PRs are welcome. If you're adding a new lazy-loaded tool, please keep its schema as lean as the core 6 — that's the whole point of this project.

📄 License

MIT © schlemperdev