Wafle Scraper

FreeNot checked

Enables AI agents to scrape web pages safely with incognito browsing, rate limiting, and ethical CAPTCHA handling.

About

Enables AI agents to scrape web pages safely with incognito browsing, rate limiting, and ethical CAPTCHA handling.

README

██╗    ██╗  █████╗  ███████╗██╗     ███████╗
██║    ██║ ██╔══██╗ ██╔════╝██║     ██╔════╝
██║ █╗ ██║ ███████║ █████╗  ██║     █████╗
██║███╗██║ ██╔══██║ ██╔══╝  ██║     ██╔══╝
╚███╔███╔╝ ██║  ██║ ██║     ███████╗███████║
 ╚══╝╚══╝  ╚═╝  ╚═╝ ╚═╝     ╚══════╝╚══════╝

Wafle-Scraper — Universal MCP Server for Safe Web Scraping

Wafle-Scraper is an MCP server that lets AI agents (OpenCode, Claude, Cursor) extract data from the web safely and responsibly.

Safety First

Rule	Enforcement
No private data	Every browser session is incognito — no cookies, no localStorage, no saved passwords
Only what you ask	The scraper never mines extra data beyond your explicit request
No localhost	Internal/private IPs are blocked by default
Rate limited	Minimum 2 seconds between requests — never floods servers
CAPTCHA = human only	No automated CAPTCHA solving. If one appears, you solve it interactively
User-Agent rotation	Each request looks like a real browser

Quick Start

pip install wafle-scraper
playwright install chromium
wafle-scraper

Configuration

OpenCode / Claude Desktop / Cursor

{
  "mcpServers": {
    "wafle-scraper": {
      "command": "wafle-scraper",
      "description": "Web scraping & browser automation — incognito, audited, safe"
    }
  }
}

CLI Options

wafle-scraper                    # MCP stdio mode (default for agents)
wafle-scraper --http --port 8000 # HTTP SSE mode
wafle-scraper --version          # Show version

MCP Tools

Tool	Description	Safety
`scrape_url`	Extract text from a static URL (requests + BeautifulSoup)	✅ Read-only, no JS
`scrape_browser`	Navigate a page in isolated incognito browser and extract text	✅ Incognito, no cookies
`scrape_reddit`	Fetch public posts from a subreddit via official API	✅ API, no scraping
`extract_emails`	Find email addresses on a public page	✅ Only what you ask
`browser_interact`	Click, type, scroll, extract, screenshot in live browser	✅ You control the actions

Browser Backend (Playwright)

Incognito always: storage_state=None, fresh context per session
No permissions: No camera, mic, location access
Anti-detection: Rotating UA, viewport, locale, timezone
Natural delays: Human-like timing between actions
Gradual scroll: Loads lazy content naturally

CAPTCHA Handling

Wafle-Scraper does NOT solve CAPTCHAs automatically. When a CAPTCHA is detected:

The scraper pauses
Prompts you to open the URL in your browser
You solve the CAPTCHA manually
Type done and the scraper continues

This is the only ethical and reliable approach without paid services.

Security

Blocked: localhost, 127.0.0.1, private IPs, file://, chrome://
Rate limiting (configurable, default 2s min interval)
Scope enforcement — only processes what you explicitly request
User-Agent rotation
Browser isolation — Playwright contexts are fully sandboxed

Requirements

Python 3.10+
Playwright with Chromium installed (playwright install chromium)
Windows, macOS, Linux

Dependencies

Standalone — no other MCP servers required.

Known Issues

Issue	Detail	Status
CDP health check unstable	`_cdp_health_check()` creates a throwaway context just to eval `1+1`. Masks real CDP issues instead of diagnosing them.	🟡 Medium
CAPTCHA handler is interactive only	CAPTCHA solver relies on stdin — doesn't work in MCP server mode (no stdin available in stdio transport).	🔴 High
`_safe_goto` only 2 retries	No exponential backoff, no proxy fallback. If 2 attempts fail, the whole scrape fails.	🟡 Medium
Error strings instead of exceptions	`browser_navigate`, `browser_click`, etc. return `"Error: ..."` strings instead of raising. Consumers must parse strings.	🟡 Medium
Single browser context	Only 1 context at a time. Concurrent `scrape_browser` calls cause race conditions.	🔴 High
Reddit HTML fallback is silent	If JSON API fails, falls back to `old.reddit.com` without logging the fallback path.	🟢 Low

Installation from Source

git clone https://github.com/creandoaldia/wafle-scraper.git
cd wafle-scraper
pip install -e .
playwright install chromium

License

MIT

Why "Wafle-Scraper"?

Part of the WAFLE ecosystem (Web AI Framework for Language Ecosystems). Wafle-Scraper gives WAFLE agents the ability to read the live web — safely, transparently, and under your control.

from github.com/creandoaldia/wafle-scraper

Install Wafle Scraper in Claude Desktop, Claude Code & Cursor

Recommended · one command, every IDE

unyly install wafle-scraper

Installs into Claude Desktop, Claude Code, Cursor & VS Code — handles npx, uvx and build-from-source repos for you.

First time? Get the CLI: curl -fsSL https://unyly.org/install | sh

Or configure manually

Run in your terminal:

claude mcp add wafle-scraper -- uvx wafle-scraper

FAQ

Is Wafle Scraper MCP free?

Yes, Wafle Scraper MCP is free — one-click install via Unyly at no cost.

Does Wafle Scraper need an API key?

No, Wafle Scraper runs without API keys or environment variables.

Is Wafle Scraper hosted or self-hosted?

A hosted option is available: Unyly runs the server in the cloud, no local setup required.

How do I install Wafle Scraper in Claude Desktop, Claude Code or Cursor?

Open Wafle Scraper on unyly.org, pick your client tab (Claude Desktop, Claude Code, Cursor) and press Install — the config is generated automatically, no JSON editing.

Related MCPs

Playwright

Browser automation, scraping, screenshots

by Microsoft

Puppeteer

Browser automation and web scraping.

by modelcontextprotocol

opentabs-dev/opentabs

Plugin-based MCP server + Chrome extension that gives AI agents access to web applications through the user's authenticated browser session. 100+ plugins with a

by opentabs-dev

robhunter/agentdeals

1,500+ developer infrastructure deals, free tiers, and startup programs across 54 categories. Search deals, compare vendors, plan stacks, and track pricing chan

by robhunter

Compare Wafle Scraper with

Wafle ScrapervsPlaywright Wafle ScrapervsPuppeteer Wafle Scrapervsopentabs-dev/opentabs Wafle Scrapervsrobhunter/agentdeals

Not sure what to pick?

Find your stack in 60 seconds

Author?

Embed badge for your README

Browse similar

All browse MCPs