Webx Server

Python License Stars MCP GitHub

A Model Context Protocol (MCP) server for web content extraction — fetch clean text, extract links, query by CSS selector, and search the web.

Built for AI agents. Works with Hermes Agent, Claude Code, Cursor, and any MCP-compatible client.

✨ Features

Tool	Description
fetch_clean_text	Fetch a URL and extract clean readable text (up to 50K chars)
extract_links	Extract all links with anchor text, optionally filtered by domain
extract_by_selector	Extract content by CSS selector (text, HTML, or attribute)
search_web	Search the web via DuckDuckGo HTML search

🚀 Quick Start

# Install from GitHub
pip install git+https://github.com/ceeyang-ai/webx-mcp-server.git

# Run as MCP server
webx-mcp-server

🔌 Usage with AI Agents

Hermes Agent

Add to ~/.hermes/config.yaml:

mcp_servers:
  webx:
    command: "webx-mcp-server"

Restart → use mcp_webx_fetch_clean_text, mcp_webx_extract_links, etc.

Claude Code / Cursor / Any MCP Client

Add to your MCP config:

{
  "mcpServers": {
    "webx": {
      "command": "webx-mcp-server"
    }
  }
}

📖 Examples

Fetch clean text

result = fetch_clean_text(
    url="https://en.wikipedia.org/wiki/Web_scraping",
    max_chars=5000
)

Extract links

result = extract_links(
    url="https://example.com",
    filter_domain=True,
    max_links=20
)

Search the web

result = search_web(
    query="latest AI research papers",
    max_results=10
)

🛠 Requirements

• Python 3.10+
• requests ≥ 2.28
• beautifulsoup4 ≥ 4.11
• lxml ≥ 4.9
• mcp ≥ 1.0

👨‍💻 Development

git clone https://github.com/ceeyang-ai/webx-mcp-server.git
cd webx-mcp-server
pip install -e .
webx-mcp-server  # Start MCP server

📄 License

MIT — free for personal and commercial use.