Weblens

Web scraping and content extraction MCP server for AI agents. Renders any URL — including JavaScript-heavy SPAs — with headless Chromium via Playwright, extracts readable content with Mozilla Readability, captures the page's navigation links, downloads images locally, and returns a clean markdown file. Works with Claude, Claude Code, Cursor, Copilot, VS Code, Codex, and any MCP-compatible client.

npx -y weblens-mcp — runs as a local stdio MCP server. Playwright downloads its own Chromium on install, so there's nothing else to set up.

Key Features

• Single tool — one fetch_page call does everything: render, extract, download, return
• Bundled browser — Chromium is auto-installed by Playwright; no system Chrome required
• Markdown output — returns a local .md file path with images embedded as local paths
• Article extraction — uses Mozilla Readability for clean content
• Navigation links — captures the site's menu/sidebar/header links (same-host, deduped)
• Asset download — page images are downloaded to a local tmp directory automatically
• Auto cleanup — downloaded files are purged after 6 hours

Requirements

• Node.js 20 or newer

That's it. On install, Playwright downloads a matching Chromium build automatically (~90 MB, cached globally and reused across projects). No system Chrome needed.

Browser

You normally don't need to do anything — Chromium is downloaded on npm install / first npx. If the automatic download was skipped (offline, firewall, or --ignore-scripts), you have two options:

# 1. Install the bundled browser manually with the Playwright CLI
npx playwright install chromium

# 2. Or point WebLens at a Chrome/Chromium you already have
CHROMIUM_PATH="/Applications/Google Chrome.app/Contents/MacOS/Google Chrome"

If CHROMIUM_PATH is unset and the bundled browser is missing, WebLens also falls back to any Chrome/Chromium/Edge found in standard OS install locations.

Getting started

Standard config works in most MCP clients (no environment variables needed — Chromium is bundled):

{
  "mcpServers": {
    "weblens": {
      "command": "npx",
      "args": ["-y", "weblens-mcp"]
    }
  }
}

Add an "env": { "CHROMIUM_PATH": "..." } block only to use a specific browser instead of the bundled Chromium.

claude mcp add weblens -- npx -y weblens-mcp

{
  "mcpServers": {
    "weblens": {
      "command": "npx",
      "args": ["-y", "weblens-mcp"]
    }
  }
}

[mcp_servers.weblens]
command = "npx"
args = ["-y", "weblens-mcp"]

# Optional — only to override the bundled Chromium:
# [mcp_servers.weblens.env]
# CHROMIUM_PATH = "/Applications/Google Chrome.app/Contents/MacOS/Google Chrome"

code --add-mcp '{"name":"weblens","command":"npx","args":["-y","weblens-mcp"]}'

Configuration

Browser resolution order: CHROMIUM_PATH (if set) → Playwright's bundled Chromium → a system Chrome/Chromium/Edge found in standard OS install locations (/Applications/... on macOS, C:\Program Files\... on Windows, /usr/bin/... on Linux).

Tool

fetch_page

Fetch and render a web page. Returns the absolute path to a local markdown file containing the page content with downloaded images embedded as local file paths.

Parameters:

Returns: Absolute path to a .md file in the system temp directory.

Example response:

/var/folders/lp/.../T/weblens-mcp/327c3fda87ce286848a574982ddd0b7c7487f816.md

Generated markdown format:

# Page Title

Source: https://example.com/article

> Article excerpt or description

Article body text content...

## Navigation

- [Docs](https://example.com/docs)
- [Pricing](https://example.com/pricing)
- [Blog](https://example.com/blog)

## Images

![alt text](/var/folders/lp/.../T/weblens-mcp/abc123.png)
![another image](/var/folders/lp/.../T/weblens-mcp/def456.jpg)

Behavior:

• Renders the page with Playwright (headless Chromium)
• Blocks media and font requests for faster loading
• Extracts article content using Mozilla Readability when possible
• Captures navigation links from nav/header/aside/menu regions (same-host, deduped, up to 50)
• Downloads page images (skips icons smaller than 50x50px)
• Writes markdown with local image paths to the system temp dir (<os-tmp>/weblens-mcp/)
• Files older than 6 hours are automatically cleaned up

Local development

npm install
npm run build
node dist/index.js

How it works

URL
 └→ Playwright renders page (headless Chromium)
     └→ Extract title, text, HTML, images, navigation links from DOM
         └→ Mozilla Readability extracts clean article content
             └→ Download images to <os-tmp>/weblens-mcp/
                 └→ Compose markdown with local image paths
                     └→ Write .md file, return path

Tmp directory

Downloaded assets and markdown files are stored in your system temp directory under weblens-mcp/ (e.g. /tmp/weblens-mcp/ on Linux, /var/folders/.../T/weblens-mcp/ on macOS). Cleanup runs automatically:

• On every fetch_page call (throttled to every 5 minutes)
• Files older than 6 hours (by mtime) are deleted
• No external cron or scheduler needed

Docker

{
  "mcpServers": {
    "weblens": {
      "command": "docker",
      "args": [
        "run", "-i", "--rm", "--init",
        "-v", "/tmp/weblens-mcp:/tmp/weblens-mcp",
        "weblens-mcp"
      ]
    }
  }
}

License

ISC