Web Fetch Server

A Model Context Protocol (MCP) server that enables LLMs to fetch and process web page contents. This server provides tools for retrieving raw HTML, extracting clean text, and converting web pages to markdown format.

Features

• fetch_webpage: Retrieves raw HTML content from any URL
• fetch_webpage_text: Extracts clean text content without HTML tags
• fetch_webpage_markdown: Converts web pages to markdown format

Installation

1. Install dependencies:

npm install

2. Build the project:

npm run build

Usage with LM Studio

To use this MCP server with LM Studio, you need to configure it in your MCP settings.

Configuration

Add the following to your LM Studio MCP configuration file (typically located at ~/.lmstudio/mcp-settings.json or similar):

{
  "mcpServers": {
    "web-fetch": {
      "command": "node",
      "args": ["D:\\Projects\\web search mcp\\dist\\index.js"]
    }
  }
}

Adjust the path to match your actual installation directory.

Alternative: Global Installation

You can also install the server globally or use npx:

npm install -g .

Then configure LM Studio to use:

{
  "mcpServers": {
    "web-fetch": {
      "command": "web-fetch-mcp"
    }
  }
}

Available Tools

1. fetch_webpage

Fetches the raw HTML content of a webpage.

Input:

• url (string, required): The URL of the webpage to fetch

Example:

{
  "url": "https://example.com"
}

2. fetch_webpage_text

Fetches a webpage and extracts clean text content without HTML tags, scripts, or styles.

Input:

• url (string, required): The URL of the webpage to fetch

Example:

{
  "url": "https://example.com"
}

3. fetch_webpage_markdown

Fetches a webpage and converts it to markdown format, preserving structure like headings, links, and lists.

Input:

• url (string, required): The URL of the webpage to fetch

Example:

{
  "url": "https://example.com"
}

Development

Build

npm run build

Watch mode (auto-rebuild on changes)

npm run dev

Run directly

npm start

Technical Details

• Protocol: Model Context Protocol (MCP) via stdio transport
• HTTP Client: Axios with 30-second timeout and automatic redirect handling
• HTML Parsing: Cheerio for efficient HTML parsing
• Markdown Conversion: Turndown for high-quality HTML to Markdown conversion
• User Agent: Mimics modern browser to avoid bot detection

Error Handling

The server includes comprehensive error handling for:

• Network failures
• Invalid URLs
• Timeout errors
• HTTP errors (404, 500, etc.)

Errors are returned in a structured format with descriptive messages.

Security Notes

• The server fetches content from any URL provided
• Be cautious when fetching content from untrusted sources
• Consider implementing URL allowlists for production use
• The server respects redirects (max 5)

License

MIT