Cqp

An MCP server that gives AI agents direct access to CWB/CQP corpora. Built for agent-driven corpus linguistics -- let any MCP-compatible AI agent query, count, and analyze annotated text corpora autonomously.

Available Corpora

Corpus	Tokens	Time Span	Genres	POS Tagset
CLMET (Corpus of Late Modern English Texts 3.1)	~40M (~34M non-punct)	1710--1920 (3 x 70-year periods)	Narrative fiction, Non-fiction, Drama, Letters, Treatise, Other	Penn Treebank (pos + class)
GUTENBERGTEST2 (Project Gutenberg English)	~5M	Multiple centuries	13 categories (Mythology, Religion, Politics, Poetry, etc.)	spaCy (pos + tag, plus dependency & NER)

Quick Start

1. Get a token -- email [email protected] or open an issue.
2. Add to your MCP client config (e.g. .mcp.json):

{
  "mcpServers": {
    "cqp-corpus": {
      "type": "sse",
      "url": "https://corpus.zisu.edu.cn/mcp/sse",
      "headers": {
        "Authorization": "Bearer YOUR_TOKEN_HERE"
      }
    }
  }
}

3. Start querying. No local installation needed.

Tools

Tool	Purpose
list_corpora	List available corpora
corpus_info	Get corpus schema -- field names, metadata, POS tagset, query examples
cqp_query	Concordance search (KWIC), with metadata filtering and pagination
cqp_frequency	Frequency counts, cross-tabulation by metadata, normalization (pmw)
cqp_collocation	Collocations with MI, t-score, Dice, or log-likelihood
define_subcorpus	Save a metadata filter as a named subcorpus for reuse
list_subcorpora	List defined subcorpora

Research Workflow

A recommended sequence for corpus research with an AI agent:

1. Discover and understand the corpus

list_corpora()
corpus_info(corpus="CLMET")

Always call corpus_info first -- it tells you the exact field names, metadata attributes, POS tagset, and query syntax for each corpus.

2. Explore with concordance

cqp_query(
    query='[word="terribly"] [pos="JJ"]',
    corpus="CLMET",
    max_results=10,
    show_attributes="pos lemma"
)

Start small (max_results=5-10) to see what the data looks like before scaling up.

3. Count and compare

cqp_frequency(
    query='[word="very"] [pos="JJ"]',
    corpus="CLMET",
    count_by="word",
    anchor="matchend",
    group_by="match text_period",
    normalize=true
)

Use group_by for distributional analysis across time periods, genres, or authors. Set normalize=true when comparing across sub-corpora of different sizes.

4. Find collocates

cqp_collocation(
    query='[lemma="make"]',
    corpus="CLMET",
    statistic="t-score",
    window=5
)

• t-score: frequent, typical collocates
• MI: exclusive, characteristic collocates
• Dice: balanced measure
• log-likelihood: statistical significance test

5. Define subcorpora for repeated filters

define_subcorpus(
    name="drama-early",
    corpus="CLMET",
    metadata_filter='match.text_genre="Drama" & match.text_period="1710-1780"',
    description="Drama texts from 1710-1780"
)

# Then use @name in any query:
cqp_frequency(query='[lemma="shall"]', corpus="CLMET", metadata_filter="@drama-early")

CQP Syntax Quick Reference

Token-level queries

[word="running"]              -- exact word form
[lemma="run"]                 -- by lemma
[pos="JJ"]                    -- by POS tag
[word="very"] [pos="JJ"]      -- sequence: "very" + adjective
[lemma="go"] []{0,3} [pos="NN"]  -- "go" ... noun, with 0-3 intervening tokens
"shall" | "will"              -- alternatives

Metadata constraints

-- In query constraints (::), use DOT:
[word="very"] :: match.text_period="1710-1780"
[word="shall"] :: match.text_genre="Drama"

-- In group_by, use SPACE:
group_by="match text_period"
group_by="match text_genre"

CLMET metadata

Attribute	Values
text_period	1710-1780, 1780-1850, 1850-1920
text_genre	Narrative fiction, Narrative non-fiction, Drama, LET, Treatise, Other
text_year	publication year
text_author	author name
text_gender	author gender

GUTENBERGTEST2 metadata

Attribute	Values
text_period	ancient, medieval, early_modern, 18C, early_19C, late_19C, early_20C, late_20C
text_category	Poetry, Politics, Religion_Spirituality, etc.
text_author	author name
s_sentiment	sentence-level sentiment
s_clause_type	sentence clause type

Example: Diachronic Intensifier Study

Here's a real research workflow studying how English intensifiers changed over 1710--1920 using CLMET:

# 1. How did "terribly" frequency change over time?
cqp_frequency(
    query='[word="terribly"]',
    corpus="CLMET",
    group_by="match text_period",
    normalize=true
)

# 2. Did "terribly + ADJ" (intensifier use) increase relative to "terribly + VERB" (manner adverb)?
cqp_frequency(query='[word="terribly"] [pos="JJ"]', corpus="CLMET",
              group_by="match text_period", normalize=true)
cqp_frequency(query='[word="terribly"] [class="VERB"]', corpus="CLMET",
              group_by="match text_period", normalize=true)

# 3. What adjectives did "terribly" collocate with in each period?
cqp_collocation(query='[word="terribly"] [pos="JJ"]', corpus="CLMET",
                statistic="t-score", window=1,
                metadata_filter='match.text_period="1710-1780"')
cqp_collocation(query='[word="terribly"] [pos="JJ"]', corpus="CLMET",
                statistic="t-score", window=1,
                metadata_filter='match.text_period="1850-1920"')

# 4. Concordance examples for qualitative analysis
cqp_query(query='[word="terribly"] [pos="JJ"]', corpus="CLMET",
          metadata_filter='match.text_period="1850-1920"', max_results=20)

See examples/ for complete runnable scripts.

Self-Hosted Deployment

If you want to run your own instance with your own CWB corpora:

Prerequisites

• Python 3.11+, uv
• CWB with cqp in PATH
• One or more CWB-indexed corpora

Run

uv sync

# STDIO mode (local, single-user)
python cqp_server.py

# SSE mode (remote, multi-user)
MCP_TRANSPORT=sse MCP_AUTH_TOKEN=$(python -c "import secrets; print(secrets.token_urlsafe(32))") python cqp_server.py

Environment variables

Variable	Default	Description
CQP_REGISTRY	/data/cqpweb/data/registry	CWB registry directory
CQP_CORPUS	GUTENBERGTEST2	Default corpus
MCP_TRANSPORT	stdio	stdio or sse
FASTMCP_HOST	0.0.0.0	Listen address (SSE)
FASTMCP_PORT	8000	Listen port (SSE)
MCP_AUTH_TOKEN	(empty)	Bearer token (SSE)
MCP_AUTH_TOKENS	(empty)	Multi-token JSON {"token": "user_id"}

systemd

cp deploy/cqp-mcp.service /etc/systemd/system/
cp deploy/env.example /opt/cqp-mcp-server/.env  # edit values
systemctl enable --now cqp-mcp

About

This project is developed at Zhejiang International Studies University (ZISU). It is part of ongoing research into agent-driven corpus linguistics, exploring how AI agents can autonomously conduct corpus-based language research.

Citation

If you use this project in your research, please cite:

Jia Yu, Weiwei Yu, Pengfei Xiao, Fukun Xing. Agent-Driven Corpus Linguistics: A Framework for Autonomous Linguistic Discovery. arXiv:2604.07189, 2026. [Paper]

@article{yu2026agent,
  title={Agent-Driven Corpus Linguistics: A Framework for Autonomous Linguistic Discovery},
  author={Yu, Jia and Yu, Weiwei and Xiao, Pengfei and Xing, Fukun},
  journal={arXiv preprint arXiv:2604.07189},
  year={2026},
  doi={10.48550/arXiv.2604.07189}
}

License

MIT