Skill Context Manager

Context-aware skill selection for AI agents — Solves the "too many skills" problem.

Reduces skill context tokens by 85-98%, improves skill selection accuracy, and learns from feedback.

Python 3.11+ SQLite FTS5 MCP Version Tests

---

The Problem

Problem	Impact	Root Cause
Users install 50-100+ skills	30K-60K tokens consumed pre-conversation	Static tool injection doesn't scale
Agent loses direction	Accuracy drops from 95% → **<50%** (Anthropic eval, >50 tools)	"Lost in the Middle" + metadata overload
Forgets skills after 20-30 messages	Re-reads everything every turn	No session memory
Similar skills indistinguishable	Can't decide which to pick	Keyword search is insufficient
Wrong skill selected	Wasted tokens + cost from retries	No feedback loop

Research References

• SkillRouter (Zheng et al., 2026): 91.7% of cross-encoder attention goes to skill body, only 1.0% to description — metadata alone is insufficient. (arXiv:2603.22455)
• Anthropic Tool Search: BM25-based deferred loading, 85% token reduction. (Engineering blog)
• Anthropic eval data (via Hermes Agent): Opus 4 accuracy drops to ~49% without tool search; Tool Search restores it to ~74%.

Solution

SCM is a proxy layer between the agent and the skill directory. Instead of loading all skills into context, SCM performs:

1. Two-Stage Retrieval (SkillRouter architecture) — Retrieve → Rerank
2. Session Memory — Remembers which skills were used, boosts them when relevant
3. Feedback Loop — Bayesian weight updates from success/failure data
4. Single Shared Database — Eliminates cross-DB bugs
5. Graceful Degradation — Works at every dependency level

Token Savings

Scenario	Before	After	Savings
100 skills metadata loaded	~30K tokens	~300 tokens	99%
50 MCP tools loaded	~72K tokens	~8.7K tokens	88%
Session tracking (50 messages)	Skills forgotten	100% recall	N/A
Query latency (77 skills)	—	~7ms (BM25)	Instant

Installation

Requirements

• Python 3.11+
• uv (Astral) — auto-installed if missing

Install

Prerequisites: Python 3.11+ and uv (auto-installed if missing).

Three install levels — pick one:

# Level 1: BM25 only (fast, 0 AI deps, works everywhere)
uv tool install git+https://github.com/Mavis2103/skill-context-manager

# Level 2: + Embedding search with all-MiniLM-L6-v2 (recommended)
uv tool install 'git+https://github.com/Mavis2103/skill-context-manager[light]'

# Level 3: + Full reranker (cross-encoder) for best accuracy
uv tool install 'git+https://github.com/Mavis2103/skill-context-manager[full]'

Note: If using zsh, quote the URL ('...') to prevent [light] being interpreted as a glob pattern.

Then verify and set up:

scm --version
scm mcp setup --all                 # MCP config for all 13 agent platforms
scm index                           # auto-detect skill dirs
# Or point at a specific directory:
scm index --dir ~/.hermes/skills/

Update

# Reinstall from GitHub to get latest (keeps same extra: light/full if any)
uv tool install --reinstall 'git+https://github.com/Mavis2103/skill-context-manager[light]'

Uninstall

scm mcp setup --force-all --uninstall   # clean MCP configs first
uv tool uninstall scm                   # remove tool + venv
rm -rf ~/.scm                           # remove database

Dev Install (for contributors)

git clone https://github.com/Mavis2103/skill-context-manager.git
cd skill-context-manager
uv venv && source .venv/bin/activate
uv pip install -e .                   # Level 1: BM25 only
# or: uv pip install -e ".[light]"    # Level 2: + embedding (recommended)
# or: uv pip install -e ".[full]"     # Level 3: + reranker

Features

0. Skill Indexing

Index your skill files so SCM can search them. Safe by default — never accidentally scans noise directories.

# Auto-detect — finds all agent skill dirs installed on this system
# (~/.agents/skills/, ~/.hermes/skills/, ~/.claude/skills/, ...)
scm index
# 📂 Scanning ~/.hermes/skills/ for skills...
# ✅ Indexed 47 skill files

# Or point at any directory (safe — skips .git, node_modules, .venv, __pycache__,
# dist, all hidden dirs, and more)
scm index --dir ~/my-skills/

# Scan full home safely — still skips the same noise dirs
scm index --dir ~/

# Progress shown automatically for large scans:
#    ... scanned 100/150
#    ✅ Indexed 150 skill files

1. Semantic Skill Retrieval

Find skills using hybrid BM25 + embedding, zero-dependency fallback.

# BM25 (FTS5) — stdlib only, fast, precise for keywords
scm query "kubernetes deploy helm" --method bm25

# Embedding — semantic search (requires sentence-transformers)
scm query "orchestrate container cluster management" --method embedding

# Hybrid (default) — best of both worlds
scm query "deploy app to production" --method hybrid

2. Session Tracking

Remembers which skills were used in a session — no more forgetting:

scm session start --id "chat-abc-123"
scm session use --skill k8s-deploy --query "deploy"
scm session use --skill docker-build --query "build image"

# Export context for the agent — only ~30 tokens
scm session context --id "chat-abc-123"

# Output:
# {
#   "active_skills": ["k8s-deploy", "docker-build"],
#   "context_size_tokens": 42,
#   "matching_skills": [...]
# }

3. Feedback Loop — Self-Learning

SCM improves over time:

# Record feedback
scm feedback record --query "deploy app" --skill k8s-deploy --success true --rating 5
scm feedback record --query "deploy app" --skill helm --success false

# View statistics
scm feedback stats
# 📊 Feedback Statistics
#    Total feedback:     47
#    Success rate:       87%
#    Query patterns:     12
#    Skills with data:   8
#    Top skills by success rate:
#      • k8s-deploy: 15/16 (94%)
#      • docker-build: 8/10 (80%)

4. Metadata Optimization

Compress descriptions to save tokens:

# Preview
scm optimize --dir ~/.hermes/skills/ --dry-run
# 📊 Potential savings:
#    Before: 1,847 meta tokens
#    After:  1,240 meta tokens
#    Saved:  607 tokens per load (33%)

# Apply
scm optimize --dir ~/.hermes/skills/ --no-dry-run

5. Usage Analytics

scm insights
# 📈 Usage Insights (last 30 days)
#    Total queries:     142
#    Tokens saved:      ~28,400
#    Retrieval methods: bm25: 89, hybrid: 42, embedding: 11
#    Top skills used:
#      • k8s-deploy: 23 times
#      • pytest-run: 18 times
#      • docker-build: 15 times

scm stats
# 📊 Skill Index Statistics
#    Total skills:     24
#    Categories:       5
#    Metadata tokens:  1,847
#    Body tokens:      12,430

Architecture

User Request
    │
    ▼
┌─────────────────────────────────────────────────┐
│ 1. Query Analysis                               │
│    - Extract key terms                          │
│    - Embed query (if embedding enabled)         │
└─────────────────────┬───────────────────────────┘
                      │
                      ▼
┌─────────────────────────────────────────────────┐
│ 2. Stage 1: Retrieval (top 20)                   │
│    ┌──────────┐   ┌──────────┐   ┌──────────┐   │
│    │  BM25    │ + │Embedding │ = │  Hybrid  │   │
│    │ (FTS5)   │   │ (cosine) │   │ (0.3+0.7)│   │
│    └──────────┘   └──────────┘   └──────────┘   │
└─────────────────────┬───────────────────────────┘
                      │
                      ▼
┌─────────────────────────────────────────────────┐
│ 3. Context Injection                             │
│    + Session boost (recently used +0.5)          │
│    + Feedback weights (Bayesian prior)           │
└─────────────────────┬───────────────────────────┘
                      │
                      ▼
┌─────────────────────────────────────────────────┐
│ 4. Stage 2: Rerank (top 5)                       │
│    Cross-encoder: query × skill body             │
│    "cross-encoder/ms-marco-MiniLM-L6-v2"         │
│    ~50ms on CPU for 20 candidates                │
└─────────────────────┬───────────────────────────┘
                      │
                      ▼
┌─────────────────────────────────────────────────┐
│ 5. Output                                        │
│    - Top 5 skill names + descriptions (~300 t)   │
│    - Session context (~30 tokens)                │
│    - Agent loads only the 1 skill body it needs  │
└─────────────────────────────────────────────────┘

Token Flow

Without SCM:
  Session start: load 50 skills metadata = 50 × 60 tokens = 3,000 tokens
  Agent picks 1, but ALL 50 stay in context
  Session grows → agent forgets → re-load all: +3,000 tokens
  Total waste: ~6,000+ tokens per session

With SCM:
  Session start: active skills only = 3 × 15 tokens = 45 tokens
  Query → top 5 metadata = 5 × 40 tokens = 200 tokens
  Session tracker: ~30 tokens
  Total: ~275 tokens per query
  Savings: 85-98%

MCP Server

SCM runs as an MCP server with 11 tools, compatible with any MCP-compatible agent.

Multi-Agent Setup Registry

SCM v0.5.0 ships with a single-command setup for 13 agent platforms. Instead of manually configuring each agent's MCP settings, run:

# Configure for ALL supported agents at once
scm mcp setup --all

# Or pick specific agents
scm mcp setup --claude-code --cursor --windsurf --hermes

# Remove SCM config from all agents
scm mcp setup --all --uninstall

# List all supported platforms with their config paths
scm mcp setup --list

Supported platforms:

Agent	Flag	Config Path
Claude Code	--claude-code	~/.claude.json
Claude Desktop	--claude-desktop	~/.config/Claude/claude_desktop_config.json
Cursor	--cursor	~/.cursor/mcp.json
Windsurf	--windsurf	~/.codeium/windsurf/mcp_config.json
Cline	--cline	VS Code globalStorage/cline_mcp_settings.json
Gemini CLI	--gemini	~/.gemini/settings.json
VS Code (Copilot)	--vscode	VS Code User/mcp.json
Zed	--zed	~/.config/zed/settings.json
Codex CLI	--codex	~/.codex/config.toml
Goose	--goose	~/.config/goose/config.yaml
Continue.dev	--continue	~/.continue/config.yaml
OpenCode	--opencode	~/.config/opencode/opencode.json
Hermes Agent	--hermes	~/.hermes/config.yaml

Each platform gets the correct config format automatically:

• JSON mcpServers — Claude Code, Desktop, Cursor, Windsurf, Cline, Gemini
• JSON servers (type: stdio) — VS Code
• JSON context_servers — Zed
• JSON mcp (type: local) — OpenCode
• YAML mcp_servers — Hermes
• YAML extensions — Goose
• YAML mcpServers (list) — Continue.dev
• TOML [mcp_servers.scm] — Codex CLI

Verify Configuration

# Check which agents have SCM configured
scm mcp status

# Output (example):
# SCM MCP Status
#   ✅ Claude Code: Configured
#   ✅ Cursor: Configured
#   ○ Windsurf: Config exists, not configured
#   · Zed: Config not found

Quick Start

# Auto-configure for all agents (idempotent)
scm mcp setup --all

# Check configuration status
scm mcp status

# Start server in stdio mode (default)
python3 -m scm.mcp_server

# Start server in HTTP/SSE mode
python3 -m scm.mcp_server --http --port 8321

Available Tools

Tool	Layer	Description
skill_query	Retrieve	Find the most relevant skills for a task
skill_index	Index	Index skills from a directory
skill_stats	Index	Get database statistics
skill_session_start	Session	Start a tracking session
skill_session_use	Session	Record skill usage
skill_session_context	Session	Export session context (~30 tokens)
skill_session_end	Session	End a session
skill_optimize	Optimize	Compress metadata to save tokens
skill_feedback	Feedback	Record usage feedback
skill_feedback_stats	Feedback	View feedback statistics
skill_insights	Analytics	Usage analytics dashboard

Per-Agent Config Formats (Reference)

Each agent uses a unique config format. The scm mcp setup command handles all 13 platforms automatically — these examples illustrate the variety of formats in use:

Hermes Agent (~/.hermes/config.yaml):

mcp_servers:
  scm:
    command: python3
    args: ["-m", "scm.mcp_server"]
    allowed_tools:
      - skill_query
      - skill_session_start
      - skill_session_use
      - skill_session_context
      - skill_session_end
      - skill_feedback
      - skill_feedback_stats
      - skill_stats
      - skill_insights

Test connection:

hermes mcp test scm
# ✓ Connected (738ms)
# ✓ Tools discovered: 11

After that, Hermes Agent automatically discovers and can call the MCP tools.

OpenCode (~/.config/opencode/opencode.json):

{
  "mcp": {
    "scm": {
      "type": "local",
      "command": ["python3", "-m", "scm.mcp_server"],
      "enabled": true
    }
  }
}

Claude Code (~/.claude.json):

{
  "mcpServers": {
    "scm": {
      "command": "python3",
      "args": ["-m", "scm.mcp_server"]
    }
  }
}

VS Code (Copilot) (~/.config/Code/User/mcp.json):

{
  "servers": {
    "scm": {
      "type": "stdio",
      "command": "python3",
      "args": ["-m", "scm.mcp_server"]
    }
  }
}

Codex CLI (~/.codex/config.toml):

[mcp_servers.scm]
command = "python3"
args = ["-m", "scm.mcp_server"]

Remote Mode (HTTP/SSE)

# Start server
python3 -m scm.mcp_server --http --port 8321

# Client config
{
  "mcpServers": {
    "scm": {
      "url": "http://localhost:8321/sse"
    }
  }
}

Agent Skill Template (for Hermes Agent skills)

Create a skill-router/SKILL.md:

---
name: skill-router
description: Select and load the most relevant agent skills using semantic search
---

When a skill needs to be selected for a task, use:
  scm query "<user_task>" --top 3 --format json
Then load the SKILL.md body of the top-matching skill.

Graceful Degradation

Dependencies	Features Available
Lightweight core (mcp + PyYAML)	BM25 (FTS5) + Session tracking + Feedback + MCP server
+ sentence-transformers	Semantic embedding search
+ transformers + torch	Cross-encoder reranking
+ feedback data	Self-improving Bayesian weights

The core has no heavy/ML dependencies — only mcp (the MCP SDK) and PyYAML. Retrieval runs on Python's stdlib sqlite3 FTS5, so BM25 search, session tracking, and feedback all work without any AI models. The embedding and cross-encoder models are entirely optional.

Comparison with Alternatives

Solution	Progressive Discovery	Semantic Search	Session Memory	Feedback Loop	Token Cost	Light Core
Claude Code Skills	✅ Load on-demand	❌ Keyword	❌ No	❌ No	~500 tokens	❌
MCP Tool Search	✅ Deferred load	✅ BM25	❌ No	❌ No	~500 tokens	❌
SkillRouter (arXiv)	❌ All at once	✅ Cross-encoder	❌ No	✅ Yes	Training needed	❌ GPU
Hermes Skills	✅ Metadata only	❌ Keyword	❌ No	❌ No	~3K tokens	✅
Lunar MCPX	✅ Tool groups	✅ Custom	❌ No	❌ No	~8.7K tokens	❌
✨ SCM (This)	✅ Metadata only	✅ BM25 + Embedding + Cross-encoder	✅ Full session tracking	✅ Bayesian	~275 tokens	✅

Project Structure

skill-context-manager/
├── src/scm/
│   ├── __init__.py          # Version + schema init
│   ├── cli.py               # CLI interface (argparse, 9 subcommands)
│   ├── db.py                # Shared database connection (single DB, WAL)
│   ├── indexer.py           # Skill indexing engine (FTS5)
│   ├── retriever.py         # BM25 + embedding retrieval
│   ├── reranker.py          # Cross-encoder reranking
│   ├── session.py           # Session state tracker
│   ├── optimizer.py         # Skill metadata optimizer
│   ├── feedback.py          # Feedback collection + Bayesian learning
│   ├── tracker.py           # Usage analytics
│   ├── models.py            # Data models (Skill, QueryResult, SessionState, FeedbackRecord)
│   └── mcp_server.py        # MCP server (11 tools)
│   └── mcp_setup.py         # Multi-agent MCP setup registry (13 platforms)
├── tests/
│   ├── test_models.py       # 14 tests — data models + YAML parsing + unquoted colon
│   ├── test_indexer.py      # 19 tests — index/reindex/skip/detect/progress/WAL
│   ├── test_retriever.py    # 10 tests — BM25/embedding/RRF/hybrid/session/graph boost/empty
│   ├── test_adaptive.py     # 20 tests — elbow detection/DBSCAN clustering/diverse/adaptive query
│   ├── test_graph.py        # 7 tests — graph edges/PPR/graph boost
│   ├── test_session_feedback.py  # 21 tests — session lifecycle + feedback
│   ├── test_optimizer.py    # 9 tests — compression/expansion/info-leak
│   ├── test_tracker.py      # 8 tests — recording/insights/daily-trend
│   ├── test_reranker.py     # 6 tests — fallback/empty/top-k/custom model
│   ├── test_mcp_setup.py    # 26 tests — multi-agent registry (13 platforms)
│   └── test_regression.py   # 24 tests — bug regression coverage
├── scripts/
│   ├── benchmark.sh         # Performance benchmark
│   └── demo.sh              # Interactive demo
├── configs/
│   └── default.yaml         # Default configuration
├── docs/
│   ├── ARCHITECTURE.md      # Detailed architecture docs
│   └── MCP-INTEGRATION.md   # MCP integration guide
├── pyproject.toml
├── LICENSE
└── README.md

Storage

Single SQLite database (~/.scm/db/scm.db) with WAL mode:

Table	Purpose
skills + skills_fts (FTS5)	Skill index + full-text search
sessions + session_skills	Session tracking
feedback + skill_weights + query_patterns	Feedback & learning
usage_events + daily_stats	Usage analytics

Workflow Example

1. Agent receives a new task

User: "Deploy app to production"

Agent internally calls:
  → skill_query(query="deploy app to production", top_k=3)
  → Returns: [kubernetes-deploy (0.92), docker-build (0.78), monitoring (0.45)]
  → Agent loads kubernetes-deploy SKILL.md, executes deploy steps
  → skill_session_use(session_id="...", skill_name="kubernetes-deploy", success=true)

2. Agent needs context injection

Agent generates system prompt block:
  "Session active skills: [kubernetes-deploy]
   Related skills: [docker-build, helm-chart]
   Estimated context: 15 tokens"

3. Agent encounters a similar task later

# This query doesn't need to scan all skills again.
# Session tracker knows kubernetes-deploy was used and boosts it.
# Saves 50-200 tokens per query.
scm session context --id "..." --query "scale deployment"

Development

Run Tests

# All 168 tests
uv run pytest -v

# Specific module
uv run pytest tests/test_indexer.py -v

# Just regression tests
uv run pytest tests/test_regression.py -v

# Coverage (optional)
uv run pytest --cov=src/scm/ tests/

Supported Skill Formats

• SKILL.md with YAML frontmatter (Hermes Agent, Claude Code)
• Plain text files (directory name = skill name)

Database Migration

# SCM auto-migrates schema on version changes (CREATE TABLE IF NOT EXISTS)
# No manual migration needed

Roadmap

• Research & Architecture (SkillRouter, Anthropic, MCP scalability)
• Core indexing engine (FTS5 + BM25)
• Semantic retrieval (embedding + hybrid)
• Session tracker with persistence
• Metadata optimizer (compress + expand)
• Cross-encoder reranker (evaluated, skipped — RAM 1.2GB/1.5GB needed)
• Feedback loop with Bayesian weights
• Usage analytics and insights
• MCP Server (11 tools)
• Multi-agent MCP setup (13 platforms: Claude Code, Claude Desktop, Cursor, Windsurf, Cline, Gemini, VS Code, Zed, Codex CLI, Goose, Continue.dev, OpenCode, Hermes Agent)
• Single shared DB (eliminates cross-DB bugs)
• 77 tests across all modules
• 101 tests + 16 bug fixes (v0.2.1)
• Agent auto-detection (v0.4.0 — --all = detected only, --force-all = all 13)
• Package rename + install dir (v0.5.0 — package → scm, dirs → ~/.scm/)
• Index safety + auto-detect (v0.6.0 — skip hidden/noise dirs, scm index auto-detects agent skill dirs, progress indicator)
• GUI dashboard
• Multi-agent session sharing

References

1. SkillRouter: Retrieve-and-Rerank Skill Selection for LLM Agents at Scale — Zheng et al., 2026 (arXiv preprint). arXiv:2603.22455
2. Advanced Tool Use & Tool Search — Anthropic Engineering Blog. Link
3. MCP Tool Scalability Problem — Jenova AI. Link
4. Skills Over MCPs: Context-Efficient Agent Capabilities — Agentic Engineer. Link
5. Beyond the Prompt: Agent Skills as Dynamic Context Management — Dev.to. Link

License

MIT — Copyright (c) 2026 Mavis2103

Changelog

See CHANGELOG.md for version history. Current: v0.6.2.