🎯 At a glance

📊 Component status

⚡ Quick Start

# 1. Clone and install
git clone https://github.com/harshthakur6293/threat-research-mcp
cd threat-research-mcp
pip install -e ".[dev]"

# 2. Verify — 129 tests should pass (5 skipped: optional deps)
python -m pytest -q

# 3. Start the MCP server (your client connects via stdio)
python -m threat_research_mcp

PyPI / uvx distribution coming. Publish workflow is merged; first release pending PyPI Trusted Publisher setup. Local install is the supported path today.

{
  "mcpServers": {
    "threat-research-mcp": {
      "command": "python",
      "args": ["-m", "threat_research_mcp"],
      "cwd": "/path/to/threat-research-mcp"
    }
  }
}

{
  "cline.mcpServers": {
    "threat-research-mcp": {
      "command": "python",
      "args": ["-m", "threat_research_mcp"],
      "cwd": "${workspaceFolder}/../threat-research-mcp"
    }
  }
}

{
  "mcpServers": {
    "threat-research-mcp": {
      "command": "python",
      "args": ["-m", "threat_research_mcp"],
      "cwd": "/path/to/threat-research-mcp"
    }
  }
}

🎬 Live Example

Real run against the Google Mandiant UNC6692 Snow Flurries report (Teams phishing → browser extension backdoor → credential theft):

Input — paste into Claude:

Source: https://cloud.google.com/blog/topics/threat-intelligence/unc6692
UNC6692 Microsoft Teams phishing impersonating IT helpdesk. AutoHotkey
scripts from S3. SNOWBELT browser extension C2 over WebSocket AES-GCM.
SNOWGLAZE Python tunneler wss://sad4w7h913-b4a57f9c36eb.herokuapp.com:443/ws
SHA256 7f1d71e1e079f3244a69205588d504ed830d4c473747bb1b5c520634cc5a2477
lsass credential dump pass-the-hash lateral movement. Black Basta ransomware.

Pipeline output:

Source quality auto-detected: vendor_blog (0.75)
IOCs extracted: 4  (1 domain · 3 SHA256 hashes)

ATT&CK Techniques: 14 above threshold
  T1566.004  [MEDIUM 0.59]  Spearphishing via Service
               evidence: teams phishing, teams lure, impersonating it
  T1071      [MEDIUM 0.56]  Application Layer Protocol
               evidence: c2, command and control
  T1003.001  [LOW    0.54]  LSASS Memory
               evidence: lsass, credential dump
  T1550.002  [LOW    0.43]  Pass the Hash
  T1486      [LOW    0.47]  Data Encrypted for Impact
  ... 9 more

Sigma rules generated: 14  (1 curated · 13 with SigmaHQ/Elastic links)
Hunt hypotheses: 8

What you get back:

• IOC table with confidence scores and MALICIOUS / UNKNOWN / VICTIM labels
• ATT&CK technique cards with keyword evidence and links to attack.mitre.org
• Hunt queries (SPL · KQL · EQL · SQL) per technique × log source
• Curated Sigma YAML for the 3 supported techniques; community search links for the rest
• ATT&CK Navigator JSON — drag into attack.mitre.org/navigator
• Self-contained HTML report with D3 force graph, heatmap, and hunt cards

ℹ️ The confidence model is intentionally conservative — even high-quality vendor_blog sources often land in LOW / MEDIUM. Thresholds and weights are tunable in playbook/confidence_weights.yaml. See Limitations for details.

🧭 Coverage Matrix

Legend:  ✅ hand-written / curated  ·  🟡 playbook query (hand-written, embedded in hunt playbook)  ·  🔗 community link only (no_curated_rule)  ·  ❌ not covered

Bottom line:

• 3 full hand-written Sigma rules (T1059.001, T1003.001, T1071.001)
• 37 techniques with embedded SPL/KQL/Elastic/SQL queries in the hunt playbook
• 119 unique techniques in the keyword index (mapping only, no hand-written queries)
• All other techniques fall through to no_curated_rule with community search links

This matrix is manually maintained. If it drifts from reality, please open an issue.

🔬 How It Works

The pipeline

run_pipeline_tool chains all stages automatically from a single text input:

flowchart TD
    A["📄 Raw Intel Text<br/>report · blog · IR note · paste"] --> B
    B["🔎 extract_iocs<br/>IPv4 · domains · hashes · emails<br/>confidence-scored · context-labelled<br/>RFC1918 / benign CDN auto-filtered"]
    B --> C["🗺️ map_ttp<br/>284-keyword ATT&CK index<br/>evidence-based confidence score<br/>source quality auto-detected<br/>IOC corroboration bonus"]
    C --> D["🎯 hunt_for_techniques<br/>SPL · KQL · EQL · SQL queries<br/>per technique × log source"]
    C --> E["📋 sigma_bundle<br/>3 curated rules<br/>community links for the rest<br/>no fake rules generated"]
    C --> F["🗃️ navigator_layer<br/>ATT&CK Navigator JSON<br/>drag into attack.mitre.org"]
    C --> G["📊 generate_threat_report<br/>self-contained HTML<br/>D3 force graph · heatmap<br/>hunt cards · Sigma cards"]

Each stage is also callable individually.

Confidence scoring

Every detected technique gets a score built from four dimensions:

⚠️ Calibration is conservative. On many real-world vendor reports, most techniques will land in LOW / MEDIUM. Tune thresholds and dimension weights in playbook/confidence_weights.yaml for your environment.

Source quality is auto-detected from URLs in the pasted text — CISA, Microsoft, Google/Mandiant, NCSC, and ISAC domains are all recognized automatically.

✨ STIX Enrichment (optional)

After run_pipeline_tool maps technique IDs, pass them to enrich_techniques_stix to pull authoritative detail directly from the official MITRE CTI STIX bundle:

T1059.001 — PowerShell
platforms:      Windows
data_sources:   Process: Process Creation
                Script: Script Execution
                Command: Command Execution
detection:      MITRE official guidance string
threat_groups:  APT32, Lazarus Group, Kimsuky, OilRig (+more)

One-time setup:

pip install "threat-research-mcp[attack]"      # installs mitreattack-python
python scripts/download_attack_stix.py          # downloads enterprise-attack.json (~50 MB)

Run stix_status to check if enrichment is active. The core pipeline works without this — STIX enrichment degrades gracefully.

📖 What Each Stage Produces

{
  "ips":     [{"value": "185.220.101.47",     "confidence": 0.92, "label": "MALICIOUS"}],
  "domains": [{"value": "cdn.apple-cdn.org",  "confidence": 0.85, "label": "MALICIOUS"}],
  "hashes":  [{"value": "a3f8c2d1...",        "confidence": 0.78, "label": "HASH"}],
  "emails":  [{"value": "[email protected]","confidence": 0.71, "label": "MALICIOUS"}],
  "filtered_fps": [{"value": "192.168.1.1",   "reason": "RFC1918"}]
}

{
  "techniques": [
    {
      "id": "T1059.002",
      "name": "AppleScript",
      "tactic": "execution",
      "evidence": ["osascript", "applescript"],
      "confidence": 0.82,
      "confidence_label": "HIGH",
      "url": "https://attack.mitre.org/techniques/T1059/002/"
    }
  ],
  "suppressed": [...],
  "source_quality": "vendor_blog"
}

{
  "hypothesis": "Attacker invoked osascript to execute in-memory payload",
  "technique_id": "T1059.002",
  "log_source_key": "edr_macos",
  "queries": {
    "spl":     "index=edr source=macos process_name=osascript ...",
    "kql":     "DeviceProcessEvents | where FileName =~ 'osascript' ...",
    "elastic": "process.name: osascript AND ...",
    "sql":     "SELECT * FROM process_events WHERE process_name = 'osascript' ..."
  }
}

{"technique_id": "T1059.001", "status": "curated",         "rule_yaml": "title: PowerShell Download Cradle ..."}
{"technique_id": "T1190",     "status": "no_curated_rule", "fallback": {"sigmahq_search": "...", "elastic_rules": "..."}}

🗂️ Tool Catalog

49 registered MCP tools across 12 categories. Full inventory in server.py.

🚀 Primary Workflow

🌐 STIX Enrichment (optional — requires setup)

📋 Sigma and Detection Drafts

🔬 IOC Enrichment (optional — requires API keys)

📥 Intake and Parsing

🔗 MISP Integration (optional — requires MISP_URL + MISP_KEY)

📁 Campaign Tracking (JSON file-based — not a TIP)

🗄️ Storage and Search

🧬 Local ATT&CK Database (optional — requires python scripts/build_attack_db.py)

🔧 Playbook Files

Everything tunable lives in playbook/ — no Python changes needed:

Add a new keyword — edit keywords.yaml and restart:

entries:
  - keyword: "your-new-keyword"
    tactic: initial-access
    technique_id: T1190
    technique_name: Exploit Public-Facing Application

⚠️ Limitations

Generated outputs are analyst drafts — review and tune before deploying anything to production.

Coverage

Confidence model

Known issues

These are tracked as roadmap items — see Contributing if you want to take one on.

Operational gaps

Setup requirements

🛣️ Roadmap

Honest priorities, in order:

1. Recalibrate the confidence model so high-quality vendor advisories land MEDIUM, not LOW
2. Hand-author 10–15 more curated Sigma rules targeting top-10 enterprise techniques + macOS / cloud gaps
3. Fix score_sigma coverage metric and score_technique_sigma envelope bug
4. Stable rule UUIDs via uuid5(namespace, technique_id) for change management
5. Platform-aware Sigma dispatch so macOS reports don't receive Windows rules
6. Domain extractor: loosen TLD heuristic for multi-dot domains (*.trycloudflare.com etc.)
7. Eval harness with 10+ public reports + expected outputs as pytest fixtures
8. Environment profile config (configs/my-soc.yaml) for SOC filtering
9. Native SIEM artifact output (savedsearches.conf, ARM, detection-rules TOML)
10. PyPI publish + uvx threat-research-mcp support

If one of these matters to you, open an issue or PR — see Contributing.

🛠️ Development

pip install -e ".[dev]"

# Full CI check — identical to GitHub Actions
python -m ruff check .
python -m ruff format --check .
python -m pytest -q --cov=src/threat_research_mcp --cov-fail-under=65
python -m bandit -c pyproject.toml -r src
python -m pip_audit --cache-dir .pip-audit-cache

129 passed, 5 skipped
coverage: 65%   ruff: pass   bandit: pass   pip-audit: pass

📁 Repository Layout

src/threat_research_mcp/
  server.py                   MCP tool registration (49 tools)
  tools/
    run_pipeline.py           end-to-end pipeline orchestrator
    extract_iocs.py           context-aware IOC extraction
    map_attack.py             ATT&CK mapping (loads from keywords.yaml)
    attack_enrichment.py      MITRE STIX enrichment via mitreattack-python
    generate_html_report.py   D3.js self-contained HTML report
    generate_sigma.py         curated Sigma rules wrapper (3 curated)
    generate_ioc_sigma.py     IOC blocklist Sigma bundle
    generate_detections.py    KQL / SPL / EQL / SQL / YARA helpers
    generate_hunt_hypothesis.py hunt playbook (37 techniques)
    navigator_export.py       ATT&CK Navigator layer export
    score_sigma.py            Sigma quality scoring (see Known Issues)
    attack_lookup.py          optional local ATT&CK SQLite lookup
    campaign_tracker.py       JSON campaign state
    misp_bridge.py            MISP integration

playbook/
  keywords.yaml               ATT&CK keyword index — 284 entries, single source of truth
  confidence_weights.yaml     confidence model + scoring thresholds
  ioc_context_patterns.yaml   IOC context scoring patterns
  atomic_tests.yaml           Atomic Red Team mapping
  siems/                      SIEM field profiles (Splunk · Sentinel · Elastic · QRadar)

scripts/
  build_attack_db.py          build local ATT&CK SQLite from MITRE STIX
  download_attack_stix.py     download enterprise-attack.json from github.com/mitre/cti

demo/
  sapphire_sleet_*            pre-generated DPRK/BlueNoroff macOS detection package

evals/
  run_test.py                 pipeline regression test against real vendor reports

🤝 Contributing

Contributions are welcome — all PRs require CI to pass and owner review before merging.

Highest-impact areas (matched to the Roadmap):

1. Curated Sigma rules — add to src/threat_research_mcp/tools/generate_sigma.py. Priority gaps: T1190 T1059.003 T1105 T1098.004 T1136.001 T1219 T1496 T1543.001 (macOS) T1548.006 (macOS)

2. New ATT&CK keyword mappings — edit playbook/keywords.yaml, no Python needed:

- keyword: "your-keyword"
  tactic: execution
  technique_id: T1059.001
  technique_name: PowerShell

3. Eval cases — add threat reports with expected IOC counts and technique IDs to evals/. Format matches evals/run_test.py.

4. SIEM profiles — extend playbook/siems/ with field mappings for additional log sources.

5. Known issue fixes — the Limitations section enumerates each with severity. Start there for a clear-scoped first PR.

Workflow:

git clone https://github.com/harshthakur6293/threat-research-mcp
pip install -e ".[dev]"
git checkout -b feat/your-change

# Make changes, then verify
python -m pytest -q
python -m ruff check . && python -m ruff format --check .

git push -u origin feat/your-change
# Open a pull request — CI must pass and owner must approve before merge

Please open an issue before starting large changes.

📄 License

MIT © Harsh Thakur