Universal Docs Server

An MCP server that gives AI assistants access to live, searchable documentation for any programming library. No more hallucinated APIs.

Field	Value
Author	Kudakwashe Chikumbu
Date	30 June 2026
Stack	Python, FastMCP, httpx, BeautifulSoup
Transport	STDIO + HTTP
Version	1.0

---

Table of Contents

1. Problem Statement
2. Solution Overview
3. Architecture
4. MCP Tools (API Surface)
5. Data Sources & Scraping Strategy
6. Caching Strategy
7. Project Structure
8. Tech Stack
9. Configuration
10. Error Handling
11. Deployment
12. Future Roadmap

---

01 — Problem Statement

AI coding assistants (Claude, Copilot, Cursor) hallucinate library APIs because their training data has a cutoff date. When a developer asks about a recently released function, updated parameter, or deprecated method, the AI confidently provides wrong information.

Current workarounds are all bad:

• Manual copy-paste: Developers paste documentation into the chat context, wasting tokens and time.
• RAG pipelines: Complex infrastructure (vector DBs, embeddings, chunking) for what should be a simple lookup.
• Trust the AI: Developers accept hallucinated APIs, ship bugs, then spend 30 minutes debugging a function signature that doesn't exist.

⚠️ The cost: According to community data, developers waste 15-30 minutes per hallucinated API error. With AI usage at 3-5+ sessions per day, that's potentially 1-2 hours of wasted debugging per developer per week.

---

02 — Solution Overview

Universal Docs MCP is a server that sits between the AI client and live documentation sources. When an AI needs to reference a library's API, it calls our MCP tools instead of relying on stale training data.

Core Value Proposition

• Live data: Always returns current documentation, not training-cutoff snapshots.
• Universal: Works with any library from PyPI, npm, crates.io, Go pkg, or any docs site.
• Structured: Returns clean, parsed function signatures, parameters, return types, and examples (not raw HTML).
• Cached: Doesn't re-scrape on every request. Intelligent TTL-based caching.
• Client-agnostic: Works with Claude Desktop, VS Code Copilot, Cursor, or any MCP client.

What This Is Not

• Not a vector database or RAG system (no embeddings, no similarity search)
• Not a documentation hosting platform
• Not a replacement for reading docs (it's a lookup tool, not a tutorial engine)

---

03 — Architecture

┌─────────────────────────────────────────────────────────────────┐
│                        MCP CLIENTS                               │
│   Claude Desktop  │  VS Code Copilot  │  Cursor  │  Custom      │
└────────┬──────────┴─────────┬─────────┴────┬─────┴──────────────┘
         │                    │              │
         │         STDIO or HTTP (SSE)       │
         │                    │              │
┌────────▼────────────────────▼──────────────▼────────────────────┐
│                   UNIVERSAL DOCS MCP SERVER                       │
│                                                                   │
│  ┌──────────────┐  ┌──────────────┐  ┌────────────────────────┐ │
│  │  Tool Router │  │  Query Parser│  │  Response Formatter    │ │
│  └──────┬───────┘  └──────┬───────┘  └────────────┬───────────┘ │
│         │                  │                       │             │
│  ┌──────▼──────────────────▼───────────────────────▼───────────┐ │
│  │                    DOCS ENGINE                               │ │
│  │                                                              │ │
│  │  ┌─────────────┐  ┌──────────────┐  ┌───────────────────┐  │ │
│  │  │  Scraper    │  │  Parser      │  │  Cache (SQLite)   │  │ │
│  │  │  (httpx +   │  │  (BS4 +      │  │  TTL: 6 hours     │  │ │
│  │  │   async)    │  │   custom)    │  │  LRU: 1000 items  │  │ │
│  │  └──────┬──────┘  └──────┬───────┘  └───────────────────┘  │ │
│  │         │                 │                                  │ │
│  └─────────┼─────────────────┼──────────────────────────────────┘ │
│            │                 │                                    │
└────────────┼─────────────────┼────────────────────────────────────┘
             │                 │
┌────────────▼─────────────────▼────────────────────────────────────┐
│                     DOCUMENTATION SOURCES                          │
│                                                                    │
│  PyPI/ReadTheDocs  │  npm/MDN  │  Docs Sites  │  GitHub READMEs  │
└────────────────────┴───────────┴──────────────┴──────────────────┘

Component Responsibilities

Component	Responsibility	Key Decisions
Tool Router	Maps incoming MCP tool calls to the correct handler	Stateless, pure function dispatch
Query Parser	Normalizes library names, versions, function paths	Handles aliases (e.g. "bs4" → "beautifulsoup4")
Docs Engine	Orchestrates fetch → parse → cache → return	Async pipeline, fail-fast on unknown libraries
Scraper	Fetches raw HTML from documentation sites	httpx async, respects robots.txt, rate-limited
Parser	Extracts structured data from HTML docs	Per-source parsing strategies (registry pattern)
Cache	Stores parsed docs, avoids redundant scraping	SQLite for persistence, in-memory LRU for hot path
Response Formatter	Shapes output for minimal token usage by AI	Concise, structured, no HTML in output

---

04 — MCP Tools (API Surface)

The server exposes 5 core tools. Each is designed to answer a specific class of developer question with minimal token waste.

Tool 1: search_docs

Full-text search across a library's documentation. The "I know it exists but can't remember the name" tool.

Schema:

@mcp.tool()
async def search_docs(
    library: str,
    query: str,
    version: str = "latest",
    max_results: int = 5
) -> list[dict]:
    """
    Search documentation for a library by keyword.

    Args:
        library: Package name (e.g. "fastapi", "pandas", "react")
        query: Search term (e.g. "async middleware", "groupby aggregate")
        version: Specific version or "latest" (default)
        max_results: Maximum results to return (1-10)

    Returns:
        List of matching doc entries with name, signature,
        short description, and source URL.
    """

Example Call:

search_docs(library="fastapi", query="dependency injection", max_results=3)

Example Response:

[
  {
    "name": "Depends",
    "module": "fastapi.params",
    "signature": "Depends(dependency, *, use_cache=True)",
    "description": "Declare a FastAPI dependency. The dependency callable will be called with the same parameters as the endpoint.",
    "url": "https://fastapi.tiangolo.com/tutorial/dependencies/"
  }
]

---

Tool 2: get_function_reference

Precise lookup of a specific function, class, or method. Returns full signature, all parameters with types and defaults, return type, and usage examples.

Schema:

@mcp.tool()
async def get_function_reference(
    library: str,
    function_path: str,
    version: str = "latest",
    include_examples: bool = True
) -> dict:
    """
    Get complete reference for a specific function or class.

    Args:
        library: Package name
        function_path: Dotted path (e.g. "DataFrame.merge", "Router.get")
        version: Specific version or "latest"
        include_examples: Whether to include code examples

    Returns:
        Full function reference: signature, params, return type,
        description, examples, deprecation warnings.
    """

Example Response:

{
  "name": "merge",
  "module": "pandas.DataFrame",
  "signature": "DataFrame.merge(right, how='inner', on=None, left_on=None, right_on=None, left_index=False, right_index=False, sort=False, suffixes=('_x', '_y'), copy=None, indicator=False, validate=None)",
  "parameters": [
    {"name": "right", "type": "DataFrame | Series", "required": true, "description": "Object to merge with"},
    {"name": "how", "type": "str", "default": "'inner'", "description": "Type of merge: 'left', 'right', 'outer', 'inner', 'cross'"}
  ],
  "returns": {"type": "DataFrame", "description": "A DataFrame of the two merged objects"},
  "examples": ["df1.merge(df2, on='key', how='left')"],
  "deprecated": false,
  "added_in_version": "0.19.0",
  "url": "https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.merge.html"
}

---

Tool 3: get_module_overview

High-level overview of a module or subpackage. Returns all public classes, functions, and constants with one-line descriptions.

Schema:

@mcp.tool()
async def get_module_overview(
    library: str,
    module_path: str = "",
    version: str = "latest"
) -> dict:
    """
    List all public symbols in a module with brief descriptions.

    Args:
        library: Package name
        module_path: Submodule path (empty = top-level package)
        version: Specific version or "latest"

    Returns:
        Module overview with classes, functions, constants listed.
    """

---

Tool 4: get_changelog

What changed between versions? Critical for migration and debugging "this worked yesterday" situations.

Schema:

@mcp.tool()
async def get_changelog(
    library: str,
    from_version: str = None,
    to_version: str = "latest",
    breaking_only: bool = False
) -> dict:
    """
    Get changelog entries between two versions.

    Args:
        library: Package name
        from_version: Starting version (None = last 3 releases)
        to_version: Ending version (default: latest)
        breaking_only: Only show breaking changes

    Returns:
        Changelog with version, date, breaking changes,
        new features, bug fixes, and deprecations.
    """

---

Tool 5: get_install_info

Quick package metadata: latest version, install command, dependencies, Python/Node version requirements.

Schema:

@mcp.tool()
async def get_install_info(
    library: str,
    ecosystem: str = "auto"
) -> dict:
    """
    Get installation info and package metadata.

    Args:
        library: Package name
        ecosystem: "pypi", "npm", "cargo", "go", or "auto" (detect)

    Returns:
        Install command, latest version, license, dependencies,
        runtime requirements, and repository URL.
    """

---

05 — Data Sources & Scraping Strategy

Documentation lives in different places depending on the ecosystem. The server uses a registry of "source adapters" that know how to fetch and parse docs from each source type.

Source Priority (per ecosystem)

Ecosystem	Primary Source	Fallback	Metadata
Python	ReadTheDocs / official docs site	PyPI description, GitHub README	PyPI JSON API
JavaScript	Official docs site / MDN	npm README, GitHub README	npm registry API
Rust	docs.rs	GitHub README	crates.io API
Go	pkg.go.dev	GitHub README	pkg.go.dev API

Source Adapter Pattern

class BaseSourceAdapter:
    """Base class for documentation source adapters."""

    async def can_handle(self, library: str) -> bool:
        """Return True if this adapter can fetch docs for this library."""
        ...

    async def fetch_search(self, library: str, query: str) -> list[dict]:
        """Search documentation and return structured results."""
        ...

    async def fetch_reference(self, library: str, path: str) -> dict:
        """Fetch a specific function/class reference."""
        ...

    async def fetch_changelog(self, library: str) -> list[dict]:
        """Fetch version changelog."""
        ...


class PyPIAdapter(BaseSourceAdapter):
    """Handles Python packages via PyPI + ReadTheDocs."""
    ...

class NpmAdapter(BaseSourceAdapter):
    """Handles JavaScript packages via npm + official docs."""
    ...

class DocsRsAdapter(BaseSourceAdapter):
    """Handles Rust crates via docs.rs."""
    ...

Scraping Rules

• Respect robots.txt on all domains
• Rate limit: max 2 requests/second per domain
• Set a descriptive User-Agent: UniversalDocsMCP/1.0 (docs-lookup; +github.com/you/repo)
• Prefer JSON APIs where available (PyPI, npm, crates.io) over scraping
• Cache aggressively to minimize requests
• Graceful degradation: if scraping fails, return metadata-only response with URL

⚠️ Important: Start with PyPI + ReadTheDocs support only. Add npm, Rust, Go adapters incrementally. A working Python docs server is more valuable than a broken multi-ecosystem one.

---

06 — Caching Strategy

Documentation doesn't change every minute. A smart caching layer reduces network calls by 90%+ while keeping data reasonably fresh.

Two-Layer Cache

Layer	Storage	TTL	Capacity	Purpose
L1: Hot	In-memory (dict)	30 minutes	500 entries (LRU)	Instant response for repeated queries in same session
L2: Warm	SQLite (disk)	6 hours	10,000 entries	Persist across server restarts, serve stale if source is down

Cache Key Structure

# Format: {ecosystem}:{library}:{version}:{tool}:{query_hash}
# Examples:
"pypi:fastapi:0.115.0:reference:sha256_abc123"
"pypi:pandas:latest:search:sha256_def456"
"npm:react:19.0.0:changelog:sha256_ghi789"

Cache Invalidation Rules

• Version-pinned queries: Long TTL (24 hours). Docs for v2.1.0 won't change.
• "latest" queries: Short TTL (6 hours). Re-check what "latest" means.
• Search queries: Medium TTL (6 hours). New content might match.
• Install info: Short TTL (1 hour). Versions release frequently.
• Stale-while-revalidate: If source is unreachable, serve stale cache entry with a warning flag.

SQLite Schema

CREATE TABLE docs_cache (
    cache_key TEXT PRIMARY KEY,
    data TEXT NOT NULL,           -- JSON blob
    created_at INTEGER NOT NULL,  -- Unix timestamp
    expires_at INTEGER NOT NULL,  -- Unix timestamp
    source_url TEXT,              -- Where it was fetched from
    hit_count INTEGER DEFAULT 0   -- For LRU eviction
);

CREATE INDEX idx_expires ON docs_cache(expires_at);
CREATE INDEX idx_hits ON docs_cache(hit_count);

---

07 — Project Structure

universal-docs-mcp/
├── pyproject.toml              # Project config, dependencies
├── README.md
├── .env.example                # Environment template
├── Dockerfile
├── src/
│   ├── __init__.py
│   ├── server.py               # MCP server entry point & tool definitions
│   ├── config.py               # Settings, env vars, defaults
│   ├── engine/
│   │   ├── __init__.py
│   │   ├── docs_engine.py      # Orchestrator: fetch → parse → cache → return
│   │   ├── query_parser.py     # Normalize library names, resolve aliases
│   │   └── formatter.py        # Shape responses for minimal tokens
│   ├── adapters/
│   │   ├── __init__.py
│   │   ├── base.py             # BaseSourceAdapter ABC
│   │   ├── pypi.py             # Python/PyPI/ReadTheDocs adapter
│   │   ├── npm.py              # JavaScript/npm adapter
│   │   └── github.py           # GitHub README fallback adapter
│   ├── cache/
│   │   ├── __init__.py
│   │   ├── memory_cache.py     # L1 in-memory LRU
│   │   └── sqlite_cache.py     # L2 persistent SQLite
│   └── utils/
│       ├── __init__.py
│       ├── http_client.py      # Shared httpx client with rate limiting
│       └── html_parser.py      # HTML → structured data utilities
├── tests/
│   ├── test_search.py
│   ├── test_reference.py
│   ├── test_cache.py
│   └── test_adapters.py
└── .vscode/
    └── mcp.json                # Local MCP config for testing

---

08 — Tech Stack

Layer	Technology	Why
MCP Framework	fastmcp >= 3.4	Decorator-based, handles protocol plumbing
HTTP Client	httpx[http2]	Async, HTTP/2, connection pooling, timeout control
HTML Parsing	beautifulsoup4 + lxml	Fast, forgiving HTML parsing
Cache (L2)	aiosqlite	Async SQLite, zero infrastructure, file-based
Config	pydantic-settings	Typed config from env vars with validation
Rate Limiting	asyncio.Semaphore + custom	Per-domain rate limits, no external dependency
Testing	pytest + pytest-asyncio	Async test support, fixtures for mocked responses
HTTP Server (optional)	uvicorn + fastapi	Only needed for HTTP transport deployment
Package Manager	uv	Fast dependency resolution, lockfile support

Install Command

uv add fastmcp httpx[http2] beautifulsoup4 lxml aiosqlite pydantic-settings

Dev Dependencies

uv add --dev pytest pytest-asyncio respx ruff

---

09 — Configuration

.env.example

# Cache
CACHE_DIR=./data                     # SQLite file location
CACHE_TTL_HOURS=6                    # Default TTL for cached docs
CACHE_MAX_MEMORY_ENTRIES=500         # L1 in-memory cache size

# Network
REQUEST_TIMEOUT_SECONDS=10           # Per-request timeout
MAX_CONCURRENT_REQUESTS=5            # Global concurrency limit
RATE_LIMIT_PER_DOMAIN=2              # Requests per second per domain
USER_AGENT="UniversalDocsMCP/1.0"

# Server
TRANSPORT=stdio                      # "stdio" or "http"
HTTP_HOST=0.0.0.0                    # Only for HTTP transport
HTTP_PORT=8000                       # Only for HTTP transport

# Optional: GitHub token for higher rate limits on README fetches
GITHUB_TOKEN=                        # ghp_xxxxxxxxxxxx

src/config.py

from pydantic_settings import BaseSettings

class Settings(BaseSettings):
    # Cache
    cache_dir: str = "./data"
    cache_ttl_hours: int = 6
    cache_max_memory_entries: int = 500

    # Network
    request_timeout_seconds: int = 10
    max_concurrent_requests: int = 5
    rate_limit_per_domain: int = 2
    user_agent: str = "UniversalDocsMCP/1.0"

    # Server
    transport: str = "stdio"
    http_host: str = "0.0.0.0"
    http_port: int = 8000

    # Optional
    github_token: str | None = None

    class Config:
        env_file = ".env"

settings = Settings()

---

10 — Error Handling

The server must never crash or return raw exceptions to the AI client. Every failure mode has a graceful response.

Failure	Response to Client	Internal Action
Library not found	{"error": "not_found", "message": "Library 'xyz' not found in PyPI or npm", "suggestion": "Did you mean 'xyzlib'?"}	Check aliases, suggest similar names
Function path invalid	{"error": "invalid_path", "message": "No function 'Foo.baz' in library X", "available": ["Foo.bar", "Foo.bat"]}	Fuzzy match available symbols
Source unreachable (timeout)	Return stale cache if available, else: {"error": "source_unavailable", "cached_at": "...", "data": ...}	Log, increment failure counter, use stale cache
Rate limited by source	Return stale cache or retry-after message	Back off, respect Retry-After header
Parsing failed (HTML changed)	{"error": "parse_error", "message": "Docs structure changed, returning raw URL", "url": "..."}	Log for adapter maintenance, return URL as fallback
Invalid parameters	FastMCP handles via Pydantic validation automatically	N/A

ℹ️ Design Principle: Always return something useful. A URL to the docs page is better than an empty error. Stale data with a warning is better than no data.

---

11 — Deployment

Local Development (STDIO)

For testing with VS Code Copilot or Claude Desktop:

.vscode/mcp.json:

{
  "servers": {
    "UniversalDocs": {
      "command": "python",
      "args": ["-m", "src.server"]
    }
  }
}

Claude Desktop config (%APPDATA%\Claude\claude_desktop_config.json):

{
  "mcpServers": {
    "UniversalDocs": {
      "command": "C:/path/to/.venv/Scripts/python.exe",
      "args": ["-m", "src.server"]
    }
  }
}

Remote Deployment (HTTP via Docker)

Dockerfile:

FROM python:3.12-slim

WORKDIR /app
COPY pyproject.toml uv.lock ./
RUN pip install uv && uv sync --frozen

COPY src/ ./src/
COPY .env .env

ENV TRANSPORT=http
EXPOSE 8000

CMD ["python", "-m", "src.server"]

docker-compose.yml:

services:
  universal-docs-mcp:
    build: .
    ports:
      - "8000:8000"
    volumes:
      - ./data:/app/data    # Persist SQLite cache
    environment:
      - TRANSPORT=http
      - GITHUB_TOKEN=${GITHUB_TOKEN}
    restart: unless-stopped

Production Checklist

• Enable HTTPS (nginx reverse proxy or cloud load balancer)
• Add authentication if exposing publicly (API key header or OAuth)
• Set up health check endpoint at /health
• Monitor cache hit rates (target: >80%)
• Log scraping failures for adapter maintenance
• Set memory limits on container (SQLite + LRU shouldn't exceed 512MB)

---

12 — Future Roadmap

Phase 1: MVP (Week 1-2)

• Python/PyPI adapter only
• search_docs and get_function_reference tools
• SQLite cache
• STDIO transport
• Working in VS Code Copilot

Phase 2: Expand (Week 3-4)

• Add get_changelog and get_install_info tools
• npm/JavaScript adapter
• GitHub README fallback adapter
• HTTP transport option
• Claude Desktop support confirmed

Phase 3: Production (Week 5-6)

• Docker deployment
• Rust (docs.rs) and Go (pkg.go.dev) adapters
• Library alias database (bs4 → beautifulsoup4, etc.)
• Fuzzy search for typos in function names
• Cache analytics dashboard

Phase 4: Community (Week 7+)

• Publish to MCP server registry
• Custom docs source configuration (point at any docs site)
• Plugin system for community-contributed adapters
• Version diff tool ("what changed between FastAPI 0.109 and 0.115?")
• Offline mode (pre-cache top 100 libraries)

✅ Start here: Phase 1 is your first PR. Get search_docs working for one Python library (try FastAPI, its docs are well-structured HTML). Everything else builds on top.