Faq Rag

A RAG prototype that answers natural-language questions from FAQ documents using vector search and LLM generation, exposed as an MCP tool.

What We Built

A three-stage RAG pipeline:

1. Ingest — FAQ markdown files are chunked (~200 chars), embedded with VoyageAI, and stored in MongoDB
2. Retrieve — User questions are embedded and matched against stored chunks using MongoDB $vectorSearch (cosine similarity)
3. Generate — Top matching chunks are passed as context to an LLM, which generates a cited answer

The whole thing is wrapped as an MCP tool (ask_faq) so any MCP-compatible client can call it directly.

Architecture

Component	Choice	Why
Embeddings	VoyageAI voyage-3-lite (512d)	Purpose-built for retrieval; outperforms OpenAI ada-002 on search benchmarks
Vector Store	MongoDB $vectorSearch	Persistent, scalable, production-realistic — vs in-memory numpy which loses data on restart
LLM	OpenAI gpt-4o-mini	Cost-efficient, fast, plenty capable for FAQ Q&A
MCP	stdio transport	Standard for local MCP tools

How It Works

Question → VoyageAI embed → MongoDB $vectorSearch → Top-K chunks → OpenAI generate → Cited answer

• Chunking: Fixed ~200 character splits. Simple and predictable for a small corpus.
• Retrieval: Cosine similarity via MongoDB vector search index (HNSW). Returns top 4 chunks by default.
• Generation: System prompt enforces grounded answers — no hallucination, must cite source filenames, infers intent (e.g. "locked out" → password reset).
• Lazy client init: API clients connect on first query, not at server startup — so the MCP server registers tools cleanly before any API calls.

How to Run

1. Configure environment

cp .env.example .env

Set VOYAGE_API_KEY, OPENAI_API_KEY, and MONGODB_URI. That’s all that’s required.

2. Ingest the FAQ corpus

uv run ingest.py

3. Test via CLI

uv run rag_core.py

4. Run as MCP tool in Cursor

Add to .cursor/mcp.json:

{
  "mcpServers": {
    "faq-rag": {
      "type": "stdio",
      "command": "uv",
      "args": ["run", "python", "${workspaceFolder}/mcp_server.py"],
      "envFile": "${workspaceFolder}/.env"
    }
  }
}

Example Questions

These show the system understands intent, not just keywords:

Question	What it tests
"How do I reset my password?"	Direct keyword match (faq_auth.md)
"I'm locked out of my account"	Semantic inference — no "password" or "reset" in query
"Can I take 3 weeks off in a row?"	Retrieves the 2-week approval rule from PTO policy
"When do my shares kick in?"	Maps "shares" → equity vesting schedule
"I want to use one login for everything"	Maps to SSO without mentioning it
"What do new employees need to know?"	Cross-document retrieval from multiple FAQ files

Deviations from Starter Skeleton

The starter used OpenAI embeddings + in-memory numpy for cosine similarity. We replaced both:

• VoyageAI instead of OpenAI embeddings — Voyage models are purpose-built for retrieval and rank higher on search benchmarks (MTEB). Using a separate embedding provider also decouples retrieval quality from the LLM choice.
• MongoDB instead of numpy — A real vector database with persistence, indexing (HNSW), and $vectorSearch aggregation. Data survives restarts, and the same approach scales to millions of documents without code changes.
• Lazy client initialization — API clients connect on first tool call, not at import. This lets the MCP server start and register tools cleanly.
• Kept everything else simple — no LangChain, no caching layers, no retry logic. Clean Python with direct API calls.

Files

ingest.py        # Build the index: read faqs/ → chunk → embed → store in MongoDB → ensure vector index
rag_core.py      # Query path only: embed question → vector search → generate answer (no ingestion)
mcp_server.py    # MCP server (exposes ask_faq, calls rag_core)
faqs/            # FAQ markdown corpus