SearchPaper

中文文档

SearchPaper is a literature-first infrastructure layer for agent-assisted research. It searches open scholarly literature, normalizes identifiers, merges duplicate records conservatively, preserves provenance, and exports traceable results for downstream agents or human review.

SearchPaper is not a full research agent, systematic-review engine, evidence evaluator, PDF parser, or Zotero replacement. It provides structured literature records and provenance; interpretation stays with the user or a downstream research workflow.

🔎 At a Glance

Area	What SearchPaper provides
📚 Sources	PubMed, Crossref, OpenAlex, Europe PMC, arXiv, bioRxiv, medRxiv, Semantic Scholar, Unpaywall, OpenCitations, PubTator3
🧬 Biomedical workflows	MeSH lookup, PubMed related articles, DOI/PMID/PMCID conversion, PubTator3 literature signals
🔗 Identifiers	DOI, PMID, PMCID, arXiv, OpenAlex, Semantic Scholar IDs
🧾 Provenance	Source records, merge reasons, conflicts, warnings, trace fields
🧠 Agent access	CLI plus MCP tools for OpenCode, Codex, Claude, and other MCP clients
📦 Exports	JSON, JSONL, BibTeX, RIS, NBIB, ENW, Markdown, Zotero CSV, Obsidian notes, APA, MLA
🔓 API keys	Core workflows are no-key usable; optional keys improve rate limits and coverage

✨ What It Does

• Multi-source literature search across open scholarly APIs.
• Identifier resolution for common scholarly IDs.
• Conservative duplicate merge with provenance and conflict preservation.
• Local SQLite cache for speed and reproducibility.
• Full-text readiness via OA links, Europe PMC XML anchors, and full-text availability fields.
• Structured literature signals from PubTator3 with explicit interpretation warnings.
• CLI-first workflows plus a small, stable MCP tool surface.
• Agent integration helpers for OpenCode, Codex, and Claude.

⚡ Quick Start

git clone <repo-url> ~/Publish/PaperSearch
cd ~/Publish/PaperSearch
uv sync --extra dev
uv run pytest -q

Task	Command
Search literature	uv run searchpaper search "HCC TACE PD-1" --json
Resolve a DOI	uv run searchpaper resolve --doi 10.1038/s41586-021-03819-2 --json
Check source health	uv run searchpaper sources status --json
Run diagnostics	uv run searchpaper doctor
Start MCP server	uv run searchpaper serve --mcp

For an editable local install:

uv pip install -e .

After installation, the package provides the searchpaper CLI.

⚙️ Configuration

SearchPaper reads ~/.config/searchpaper/config.json. Environment variables override file values.

{
  "semantic_scholar_api_key": "YOUR_KEY",
  "crossref_email": "[email protected]",
  "openalex_email": "[email protected]",
  "ncbi_api_key": "YOUR_NCBI_KEY",
  "cache_path": "~/.cache/searchpaper/cache.db"
}

Variable	Maps to	Description
SEARCHPAPER_CACHE	cache_path	SQLite cache file
SEARCHPAPER_CONFIG	config_path	Config file path
SEMANTIC_SCHOLAR_API_KEY	semantic_scholar_api_key	Optional Semantic Scholar key
CROSSREF_EMAIL	crossref_email	Crossref polite-pool email
OPENALEX_EMAIL	openalex_email	OpenAlex polite-pool email
NCBI_API_KEY	ncbi_api_key	Optional NCBI key

🧭 Core CLI Workflows

Workflow	Command
Plan source routing	searchpaper plan "query" --profile auto --json
Search literature	searchpaper search "query" --json
Get work by PMID	searchpaper get --pmid 12345 --json
Resolve identifier	searchpaper resolve --doi 10.xxx --json
Expand from a seed work	searchpaper expand --doi 10.xxx --by citations,references --json
Build context pack	searchpaper context --doi 10.xxx --json
Fetch full-text anchors	searchpaper get --pmcid PMC8371605 --sources europepmc --anchors --json
Probe sources	searchpaper sources preflight --sources pubmed,arxiv --json
Explain source behavior	searchpaper sources explain pubmed
Diagnose source health	searchpaper sources diagnose --json
Save project trace	searchpaper search "query" --json --save
Export saved trace	searchpaper project export <trace_id> --format bibtex
Lookup MeSH	searchpaper pubmed mesh "term" --json
Find related PubMed articles	searchpaper pubmed related 12345 --json
Convert PubMed IDs	searchpaper pubmed convert-ids 10.xxx --json
Batch process inputs	searchpaper batch queries.jsonl --format jsonl --output results.jsonl
Verify references	searchpaper references verify refs.txt --output-format markdown
Export results	searchpaper export results.json --format bibtex
Inspect capabilities	searchpaper capabilities --json

🧩 MCP and Agent Integration

Start the default stdio MCP server:

searchpaper serve --mcp

Start a remote HTTP/SSE MCP server with searchpaper[remote] installed:

searchpaper serve --mcp --transport http --host 127.0.0.1 --port 8000 --path /mcp

Tool	Purpose
search_literature	Multi-source literature search
plan_literature_search	Source-routing plan without a live search
get_work	Fetch one work by identifier
resolve_identifiers	Resolve and merge multiple identifiers
get_source_status	Source health and optional catalog
lookup_mesh	PubMed MeSH descriptor lookup
find_related_articles	PubMed related article IDs
convert_identifiers	DOI/PMID/PMCID conversion
search_literature_signals	Biomedical entity/relation signals
expand_from_work	Citations, references, and related papers
export_records	Export structured records

Agent	Install command	Output
OpenCode	searchpaper install --target opencode	MCP config plus literature-search skill
Codex	searchpaper install --target codex	MCP snippet plus SearchPaper instructions
Claude	searchpaper install --target claude	MCP snippet plus SearchPaper instructions
All	searchpaper install --target all	Scaffolds all supported integrations

Review generated files and enable the searchpaper MCP server in your agent settings.

📖 Full-Text Readiness

SearchPaper reports full-text availability without requiring PDF parsing. Records may include full_text_availability, open_access_pdf, and canonical URLs such as doi, pubmed, pmc, oa_url, and oa_pdf_url.

searchpaper resolve --doi 10.1038/s41586-021-03819-2 --json
searchpaper get --pmcid PMC8371605 --sources europepmc --anchors --json
searchpaper context --doi 10.1038/s41586-021-03819-2 --json
searchpaper get --doi 10.1038/s41586-021-03819-2 --download-pdf paper.pdf --json
searchpaper get --pmcid PMC8371605 --download-xml paper.xml --json

PDF parsing remains optional and out of scope.

🛡️ Failure Modes and Signal Rules

Topic	Behavior
Source degradation	Public APIs can fail; optional sources warn and continue
Required sources	--require pubmed or --source-mode pubmed:required returns exit code 1 on failure
Preferred sources	source:preferred emits louder warnings but does not fail the full search
Biomedical signals	PubTator3/co-mention signals are not proof of causality
Evidence evaluation	Out of scope; SearchPaper returns records, provenance, and signals

🗂️ Cache, Privacy, and Project Mode

• Default cache: ~/.cache/searchpaper/cache.db
• Bypass cache: searchpaper search "query" --no-cache --json
• Project mode:

searchpaper init
searchpaper search "HCC TACE PD-1" --json --save
searchpaper project traces
searchpaper project export <trace_id> --format bibtex

The cache may contain raw API responses and query identifiers. Store it accordingly.

🧪 Development

uv sync --extra dev
uv run pytest -q
uv run ruff check .
uv run ruff format .

CI should rely on fixture-backed tests. Live public APIs are useful for manual checks but should not be required for routine validation.

📜 License

MIT