SIEMen

Security AI data layer on Cloudflare Workers. Purpose-built for AI security agents — SIEMen stores scan findings in a relational database, indexes them as semantic vectors for similarity search, maintains per-engagement agent memory, and exposes everything through both a REST API and a built-in MCP server.

When a security scanner finishes a run, SIEMen becomes the single place where findings accumulate, get enriched, get triaged, and eventually get exported into a SOC report — all without spinning up any additional infrastructure.

---

How It Fits Into a Security Pipeline

SIC scan output (JSON)
  └─▶ sic_siemen_bridge.py --scan <file> --engagement-name "Client"
        ├─▶ POST /v1/engagements       — create or resume the engagement record
        └─▶ POST /v1/findings/batch    — transform and ingest findings (100/batch)

SIEMen data layer
  ├─▶ D1 (SQLite)    — structured storage: engagements, findings, memory, cache log
  ├─▶ Vectorize      — BGE-768 embeddings for semantic similarity search
  └─▶ Workers KV     — fast scratch cache for in-flight agent state

Report and handoff
  └─▶ GET /v1/engagements/:id/report  — severity rollup, status rollup, memories, cache stats
        └─▶ soc-reporter-mcp          — maps findings into P0–P3 buckets → SOC handoff HTML

---

Stack

Layer	Technology	Why This Choice
Runtime	Cloudflare Workers	Edge-native, globally distributed, zero cold starts — findings are stored and queried at the edge without a separate server
Framework	Hono v4 + TypeScript	Lightweight routing with zero Node.js dependencies, required for the edge runtime
Database	Cloudflare D1 (SQLite)	Relational data (engagements, findings, memory) with full SQL — no separate database server to manage
Vector Search	Cloudflare Vectorize	768-dimension BGE embeddings for semantic similarity — lets agents find "SQL injection" findings even when the text uses different phrasing
KV Cache	Workers KV	Sub-millisecond reads for scratch state and triage cache values
Embeddings	Workers AI (BGE-768)	Runs inside Cloudflare — no external embedding API call. OpenRouter fallback on failure
Rate Limiting	Workers Rate Limiting API	Cross-isolate, durable limits — works correctly even when requests hit different edge locations
Auth	Web Crypto SHA-256	Timing-safe Bearer token verification without the Node.js crypto module (not available at the edge)
MCP	JSON-RPC 2.0	11 tools on POST /mcp — same handlers as REST, just a different transport

---

What It Does

Fast KV Scratch Cache

Per-engagement key/value store for in-flight agent notes, intermediate results, and shared state across agent steps. Namespaced to prevent cross-engagement leakage. Optional TTL (max 24h).

Why it exists: AI agents running multi-step workflows need a place to write intermediate state that isn't yet final enough to be a finding. KV is the scratchpad.

Semantic Vector Search

Embed and store security findings, CVEs, and controls with finding_store. Retrieve closest matches by semantic similarity with finding_search. Namespace-scoped to engagement — an agent searching for "SQL injection" only surfaces findings from its own engagement.

Why it exists: Traditional keyword search misses findings that describe the same vulnerability with different words. Semantic search finds them regardless of phrasing.

Per-Engagement Agent Memory

Agents store notes with engagement_remember and recall them by recency or vector similarity with engagement_recall. Supports sub-scoping by session_id for multi-step workflows.

Why it exists: Without memory, each agent step starts from scratch. Memory allows an agent to build context across multiple steps within the same engagement.

Semantic LLM Triage Cache

semantic_triage routes security prompts through a two-level cache before hitting an LLM:

1. Exact hash match — identical prompts reuse prior analysis instantly
2. Cosine similarity check — prompts above the configured threshold (default 0.92) return a cached response without an LLM call
3. Cache miss — calls the configured LLM gateway, writes result to KV + Vectorize for future hits

Why it exists: When agents analyze 50 findings across 10 engagements, many prompts are structurally identical. The cache eliminates redundant LLM calls.

Finding Lifecycle

Findings transition through status states (open → accepted → remediated → false_positive) via PATCH /v1/findings/:id. Batch ingest via POST /v1/findings/batch for pipeline ingestion from scanners.

Why it exists: A raw list of findings has no operational value unless you can track which ones have been acted on. Status tracking makes the data usable by a SOC team.

Engagement Reports

GET /v1/engagements/:id/report exports all findings with cursor pagination, severity rollup, status rollup, memories, and cache stats in a single response — ready for SOC pipeline consumption.

Why it exists: The SOC report generator needs a single structured snapshot of the engagement. This endpoint provides it without requiring multiple round-trips.

---

MCP Tools

Tool	Description
engagement_open	Create or register a new pentest engagement
engagement_list	List all engagements (non-deleted, newest first)
engagement_report	Full export: findings, rollups, memories, cache stats
sec_cache_get	Read from the fast KV scratch store
sec_cache_set	Write to the fast KV scratch store (optional TTL)
finding_store	Embed and store a finding, CVE, or control
finding_search	Semantic search over findings scoped to an engagement
engagement_remember	Store agent memory for an engagement
engagement_recall	Recall memories by recency or vector similarity
semantic_triage	Semantic-cache-backed LLM triage
cache_stats	Get cache hit/miss stats for an engagement

---

REST Routes

All /v1/* routes require Authorization: Bearer <SIEMEN_API_KEY>.

Method	Path	Description
GET	/health	Health check (unauthenticated)
GET	/v1/engagements	List all engagements
POST	/v1/engagements	Create or upsert an engagement
GET	/v1/engagements/:id/report	Full export for SOC handoff
PATCH	/v1/engagements/:id	Update engagement status, name, or client
POST	/v1/findings	Embed and store a single finding
POST	/v1/findings/batch	Batch ingest (up to 100 per request)
POST	/v1/findings/search	Semantic search over findings
PATCH	/v1/findings/:id	Update finding severity, body, tags, or status
DELETE	/v1/findings/:id	Soft-delete a finding
GET	/v1/stats?engagement_id=	Cache hit/miss stats and severity rollup
GET	/v1/kv/:ns/:key	KV scratch read
PUT	/v1/kv/:ns/:key	KV scratch write
POST	/v1/memory	Store agent memory
GET	/v1/memory?engagement_id=	Recall agent memories
POST	/v1/triage	Semantic triage with LLM cache
POST	/mcp	MCP JSON-RPC endpoint (all 11 tools)

---

D1 Schema

Apply migrations in order with wrangler d1 execute siemen-db --remote --file migrations/<file>.sql:

Table	Purpose
engagements	Engagement registry — id, name, client, status, timestamps
memory	Agent memory per engagement — content, tags, vector_id, session_id
findings	Security findings, CVEs, controls — kind, title, body, severity, asset, external_id, vector_id, tags, finding_status, deleted_at
semantic_cache_log	Triage cache outcome log — prompt_hash, outcome, similarity, model, tokens

---

SIC Integration

sic_siemen_bridge.py is the official SIC → SIEMen bridge. It reads a SIC scan output file, transforms each finding into the SIEMen schema, and batch-posts them to the API. Every field is mapped from multiple possible source names so the bridge handles output from different SIC scanner tools without configuration.

Field Mapping

SIEMen field	SIC source fields (tried in order)
title	name, vulnerabilityName, Title, template-id, checkID
body	description, info.description, details (capped at 2000 chars)
severity	severity, info.severity → normalized (none/unknown → info)
kind	cve if CVE-YYYY-NNNNN in title, control if Checkov ID, else finding
external_id	CVE ID extracted from title/template-id — used for deduplication
asset	host, url, target, affected_component, matched-at
tags	[scanner, category] (up to 5, deduplicated)

CLI

python sic_siemen_bridge.py \
    --scan ./_runs/scan-20260101.json \
    --engagement-name "Example Corp Pentest" \
    [--client "Example Corp"] \
    [--engagement-id "existing-id"] \
    [--url https://your-worker.workers.dev] \
    [--dry-run]

Library

from sic_siemen_bridge import SIEMenClient

client = SIEMenClient()  # reads SIEMEN_URL + SIEMEN_API_KEY from env
eid = client.open_engagement("My Pentest", client="Acme Corp")
result = client.push_findings(eid, sic_findings)
# { stored, duplicates, errors, total_pushed }

report = client.get_report(eid)
# { findings, severity_rollup, status_rollup, memories, cache_stats }

---

Deploy

# 1. Create infrastructure
wrangler vectorize create siemen-vectors --dimensions=768 --metric=cosine
wrangler d1 create siemen-db
wrangler kv namespace create siemen-sc-cache
wrangler kv namespace create siemen-kv

# 2. Update wrangler.toml with the IDs returned above

# 3. Apply D1 migrations in order
wrangler d1 execute siemen-db --remote --file migrations/migration-001-engagements.sql
wrangler d1 execute siemen-db --remote --file migrations/migration-002-memory.sql
wrangler d1 execute siemen-db --remote --file migrations/migration-003-findings.sql
wrangler d1 execute siemen-db --remote --file migrations/migration-004-semantic-cache-log.sql
wrangler d1 execute siemen-db --remote --file migrations/migration-005-roi.sql
wrangler d1 execute siemen-db --remote --file migrations/migration-006-finding-status.sql

# 4. Set secrets
wrangler secret put SIEMEN_API_KEY
wrangler secret put OPENROUTER_API_KEY
wrangler secret put LLM_GATEWAY_KEY

# 5. Deploy
npm run deploy

---

MCP Configuration

Add to Claude Desktop or Claude Code MCP config:

{
  "mcpServers": {
    "siemen": {
      "url": "https://your-worker.workers.dev/mcp",
      "headers": {
        "Authorization": "Bearer ${SIEMEN_API_KEY}"
      },
      "alwaysAllow": [
        "engagement_open",
        "engagement_list",
        "sec_cache_get",
        "finding_search",
        "engagement_recall",
        "cache_stats"
      ]
    }
  }
}

Set SIEMEN_API_KEY in your shell environment. Write tools (finding_store, sec_cache_set, engagement_remember, semantic_triage) require explicit approval by default — intentional, because they modify stored data.

---

Security

• All /v1/* and /mcp routes require a Bearer token verified with Web Crypto SHA-256. The Web Crypto API is used instead of Node.js crypto because Node.js APIs are not available in the Cloudflare Workers edge runtime.
• Timing-safe comparison prevents timing attacks where an attacker could infer token length or prefix by measuring response time.
• Findings are namespace-scoped to engagement_id in Vectorize — a semantic search for one engagement cannot surface findings from another.
• Secrets are managed via wrangler secret put only and never appear in source code or wrangler.toml.
• Rate limiting is durable across edge instances — counters are shared, so distributing requests across data centers does not bypass the limit.

See SECURITY.md for vulnerability reporting.

---

License

MIT