Vault Semantic

A local semantic search MCP server for a Markdown vault. Intended as a sidecar for OpenClaw or other MCP-enabled agents.

What it does

• Indexes markdown files under a vault root (inbox, projects, decisions, entities, memory, sessions, templates)
• Chunks content by headings, with paragraph subdivision for large sections
• Embeds chunks using OpenAI text-embedding-3-small (stored as JSON in SQLite for v1)
• Search combines FTS5 keyword search with cosine-similarity semantic search and folder-based ranking
• Related notes finds notes similar to a given path
• File watcher keeps the index in sync as files change
• MCP tools expose everything to agents over stdio

Important: The vault markdown files are the source of truth. The SQLite database is a derived search index only. If the DB is lost, it can be rebuilt with a full reindex.

Architecture

• Vault root → scan .md files → parse frontmatter, chunk by headings
• Chunks → OpenAI embeddings → stored in SQLite with FTS5 for full-text
• Search → hybrid FTS + semantic → folder boost (memory/entities/decisions > projects/sessions > inbox)
• MCP → stdio transport → tools call search/get/recent/related/reindex/status

Setup

1. Requirements: Node.js 20+

2. Install:

   npm install
   

3. Configure: Copy .env.example to .env:

   cp .env.example .env
   

   Set OPENAI_API_KEY and adjust paths:

   • VAULT_ROOT – vault directory (default ./data/vault)
   • SQLITE_PATH – index DB (default ./data/index/vault.db)

4. Run:

   npm run dev   # development with watch
   npm run build && npm start   # production
   

Environment variables

Variable	Default	Description
OPENAI_API_KEY	(required)	OpenAI API key for embeddings
VAULT_ROOT	./data/vault	Root directory of the markdown vault
SQLITE_PATH	./data/index/vault.db	Path to SQLite index database
EMBEDDING_MODEL	text-embedding-3-small	OpenAI embedding model
TOP_K_DEFAULT	8	Default number of search results

MCP usage

Configure your MCP client (e.g. OpenClaw) to run this server via stdio:

{
  "mcpServers": {
    "vault": {
      "command": "node",
      "args": ["/path/to/vault-semantic-mcp/dist/index.js"],
      "env": {
        "OPENAI_API_KEY": "...",
        "VAULT_ROOT": "/path/to/vault",
        "SQLITE_PATH": "/path/to/index/vault.db"
      }
    }
  }
}

Or with tsx for development:

{
  "mcpServers": {
    "vault": {
      "command": "npx",
      "args": ["tsx", "/path/to/vault-semantic-mcp/src/index.ts"],
      "env": { ... }
    }
  }
}

Tools

Tool	Args	Description
vault_search	query, folders?, topK?	Hybrid search over the vault
vault_get	path	Get full markdown for a file
vault_recent	folder?, topK?	Recently indexed documents
vault_related	path, topK?	Notes related to a given path
vault_reindex	path?	Reindex one path or whole vault
vault_status	–	Vault root, counts, watcher state

Local validation (test harness)

Before wiring into OpenClaw, validate indexing and retrieval locally:

Seed vault

The repo includes sample notes in data/vault/ across projects, decisions, entities, memory, sessions, and inbox. Add or edit markdown files as needed.

Run test harness

# Reindex and run all evaluation queries (uses OPENAI_API_KEY)
npm run test:search

# Skip reindex, reuse existing index (faster for iterating on queries)
SKIP_REINDEX=1 npm run test:search

The harness runs the same hybridSearch used by vault_search, so results reflect real MCP behavior.

Evaluation queries

Query ID	Purpose
exact_keyword_sqlite	FTS exact term match
exact_keyword_chunking	FTS on common term
semantic_memory_routing	Semantic: "how should the agent store durable knowledge"
embedding_cost_strategy	Semantic: "why OpenAI instead of local embeddings"
file_watcher_reindex	Semantic: "what happens when vault files are edited"
related_notes_openclaw	Semantic: "notes related to semantic search sidecar"
decision_log_architecture	Semantic: "where did we decide vault files as source of truth"
fts_keyword_mcp	FTS exact term

What to inspect

• Exact matches (SQLite, MCP, chunking): FTS should surface those notes
• Semantic matches: Different phrasing should find the right notes (e.g. "durable knowledge" → memory/durable-knowledge-storage)
• Deduplication: At most one chunk per document in results
• Folder ranking: memory/entities/decisions should rank higher than inbox for similar content
• Snippets: Chunk text should be readable and relevant

Reindex only

npm run reindex
VERBOSE=1 npm run reindex   # log each file

Embeddings

• v1 uses OpenAI text-embedding-3-small and stores vectors as JSON in SQLite.
• No sqlite-vec or vector extensions. Future versions may add Ollama support.

License

MIT