Vault Search

English · 繁體中文

Three tools that turn your Obsidian vault into a semantically searchable, AI-chattable knowledge base — running entirely on your own machine. No cloud indexing, no notes leaving your computer for search, no per-query API bill for retrieval.

Built by a physician to study from a vault of thousands of clinical notes, but domain-agnostic: point it at any vault of markdown notes (research, law, engineering, a personal wiki) and it works.

Two ways to use it (one shared index)

This repo ships two front-ends over the same local LanceDB index + retrieval core — use either or both:

• vault-search-obsidian — the Obsidian plugin (Search / Related Notes / Chat in the sidebar), talking to a small local API server. → Quickstart
• vault-search-mcp — an MCP server exposing the same search as tools (vault_search, vault_similar, vault_stats, textbook_search) to Claude Code or any MCP client. No Obsidian required. → Quickstart

Both reuse the same server/ core (indexer · scoring · Personalized-PageRank · knowledge graph), so you index once and query from wherever you like.

The three-piece set (三套組)

All three live in the Obsidian sidebar. The same index also exposes an MCP server, so coding agents like Claude Code can search your vault as a tool.

Screenshots

🔍 Vault Search	🔗 Related Notes

💬 Vault Chat	👁 Context preview

Why it exists

Built-in Obsidian search is lexical — it finds the words you typed. But you rarely remember the exact words in a note you wrote months ago. Semantic search finds notes by meaning: search "how to manage a flaccid bladder after spinal injury" and it finds your note titled Neurogenic Bladder even if those exact words never appear.

The design priority is local-first and private:

• Embeddings run locally via Ollama (bge-m3, bilingual, CPU-friendly). Your note text is never sent anywhere to be indexed or searched.
• Vault-mode chat is fully local and free — a local LLM answers using retrieved notes, zero tokens billed.
• Hybrid/Free chat optionally shells out to the claude CLI so you can use a Claude subscription you already pay for, instead of wiring up (and paying per-token for) a raw API key.

How it works

                    ┌─────────────────────────────────────────┐
   Obsidian  ◄────► │  Plugin (sidebar): Search · Related · Chat│
                    └───────────────────┬─────────────────────┘
                                        │ HTTP (localhost:3789)
                                        ▼
                    ┌─────────────────────────────────────────┐
                    │  api_server.py  (FastAPI)                 │
                    │   /search  /similar  /chat  /reindex      │
                    └──────┬───────────────────────┬───────────┘
                           │                        │
                  embed query                 RAG context
                           ▼                        ▼
        ┌────────────────────────┐     ┌──────────────────────────┐
        │ Ollama  (bge-m3)        │     │ Vault mode → local LLM    │
        │ → vector               │     │ Hybrid/Free → `claude` CLI │
        └───────────┬────────────┘     └──────────────────────────┘
                    ▼
        ┌────────────────────────┐
        │ LanceDB vector index   │  ◄── indexer.py  (chunks notes by heading,
        │ (~/.vault-search)      │       embeds, stores; incremental updates)
        └────────────────────────┘

   Same index + scoring is also served to coding agents via mcp_server.py (MCP).

• indexer.py walks your vault, splits each note into chunks by heading, embeds them with bge-m3, and stores them in a local LanceDB table. Re-runs are incremental (content-hash cache).
• scoring.py reranks raw vector hits with optional path weighting (trust some folders more), recency (newer notes rank higher), and relation boosts.
• ppr.py ranks notes by Personalized PageRank over the [[wiki-link]] graph — a HippoRAG / LinearRAG-style random-walk-with-restart. It powers Related Notes and multi-hop query expansion (seeding the walk on the top semantic hits), surfacing relevant notes that pure vector similarity misses. On the author's vault this lifted sparse-bridge recall@10 from ~60% to ~84% versus naive 1-hop / shared-link expansion.
• api_server.py serves the plugin: semantic search, find-similar, single-note reindex, and chat (with retrieval-augmented context).
• mcp_server.py exposes the same search to MCP clients (e.g. Claude Code) as vault_search, vault_similar, vault_stats.
• Optional add-ons: a second long-form reference corpus (textbook_indexer.py, parent-child chunking) and a knowledge graph of [[wikilinks]] + entities (graph_builder.py).

Prerequisites

Pull the embedding model once:

ollama pull bge-m3
# optional, for zero-cost local Vault-mode chat:
ollama pull gemma2:9b

Installation

Step 0 — shared core (do this once, for either front-end)

git clone https://github.com/drpwchen/vault-search.git
cd vault-search
pip install -r requirements.txt

cp env.example .env          # then edit .env — set VAULT_PATH to your vault

cd server
python indexer.py            # build the index (incremental on later runs)

ollama pull bge-m3 first if you haven't (see Prerequisites). Now pick one or both front-ends below.

vault-search-obsidian — the Obsidian plugin

Start the local API server, then install the plugin:

python server/api_server.py                       # serves http://localhost:3789
cp -r plugin "<YOUR_VAULT>/.obsidian/plugins/vault-search-plugin"

In Obsidian: Settings → Community plugins → enable "Vault Semantic Search". Open its settings and confirm API Server URL is http://localhost:3789. Three ribbon icons appear: 🔍 Search, 💬 Chat, 🔗 Related Notes.

The plugin ships a data.json.example. Obsidian writes its real data.json on first run; never commit a data.json that contains an API key.

vault-search-mcp — use it from Claude Code (or any MCP client)

No Obsidian, no API server — just register the MCP server against the index you built in Step 0:

claude mcp add vault-search -- python "/abs/path/to/server/mcp_server.py"

Now Claude Code (or any MCP client) can call vault_search, vault_similar, vault_stats, and — if you indexed a reference corpus — textbook_search, all backed by the same local index, Personalized-PageRank expansion, and reranking.

Configuration

Everything is driven by environment variables (see env.example for the full annotated list). The only required one is VAULT_PATH. Highlights:

Optional add-ons

export VAULT_SEARCH_TEXTBOOK_PATH=/path/to/reference-md
ollama pull qwen3-embedding:0.6b
cd server && python textbook_indexer.py

cd server
python graph_builder.py            # parse [[wikilinks]] into an adjacency graph
python graph_builder.py --ner      # optional: scispaCy/NER entity extraction

Adapting it to your setup (alternatives)

This project was built around an always-on PC with a GPU + a Claude subscription, but none of that is required. Pick the row that matches you:

Want a cloud embedder instead of Ollama? The embedding call is isolated in indexer.py / api_server.py (client.embed(...)); swap it for any provider and keep the same LanceDB pipeline.

Security notes

• The API server supports an API key (X-API-Key). Set VAULT_API_KEY (env) or ~/.vault-search/api_key.txt on the server and the matching key in plugin settings. Always set one if the server is reachable beyond localhost.
• .env, plugin/data.json, and the whole ~/.vault-search/ data dir are git-ignored. Never commit secrets or your index.
• Hybrid/Free chat runs the claude CLI as a subprocess with a deny-list of tools (no file write, no shell) — it can only read the context you pass it and call read-only literature tools.

Repository layout

server/      indexer · scoring · ppr · api_server · mcp_server   (core three-piece set)
             textbook_indexer · graph_builder              (optional add-ons)
             config.py                                      (all settings, env-driven)
plugin/      main.js · manifest.json · styles.css           (Obsidian plugin)
examples/    source_boost.example.json
docs/images/ screenshots
env.example  copy to .env

Author

Built by Dr. P.W. Chen — a physician who needed to actually find things in his own notes. 🌐 drpwchen.com · 🐙 github.com/drpwchen

License

MIT — see LICENSE. Contributions and issues welcome.

🧋 Support

If this tool saves you time, you can buy me a boba — it keeps the servers running 🧡

🌱 Start here if you're new to AI agents ／ AI agent 新手起點

This tool is one piece of my personal AI workflow. If you want to learn how to use AI agents like Claude Code from zero (no programming background needed), I wrote a beginner series (in Traditional Chinese):

這個工具是我個人 AI 工作流的一部分。想從零開始學怎麼用 Claude Code 這類 AI agent（不需要程式背景），可以從我的入門系列開始：

1. 從零開始：安裝、看懂 GitHub、跑起你的第一個工具
2. 怎麼跟 AI agent 講話：心法、元技能與規則檔
3. 自動化流程不是設計出來的，是長出來的

Full map of my tools and posts ／ 所有工具與文章的全貌 → drpwchen.com/map