Red Team Blue Team Agent Fabric

PyPI version Python 3.10+ Apache 2.0 License Tests OWASP Agentic T1-T17 ClawScan Static Analysis VirusTotal

Even if an agent is properly authenticated and authorized, can it still be manipulated into unsafe or policy-violating behavior?

Evidence before coverage

Every claim in this project is bounded by the E1-E5 Evidence Class Taxonomy: observation, runtime characterization, enforcement, persistence/replay resistance, and isolation. A second axis, I0-I2, states who produced the oracle: self-authored, independently reimplemented, or an independent sensor the target does not control. Strength and independence are different properties, so both are cited. A result is not promoted beyond what its retained artifact and execution record demonstrate. Author-performed mappings and test runs are not independent certification.

The AIUC-1 Evidence Field Guide is a plain-language companion that applies this same taxonomy. It adds one distinction the ladder above does not encode: whether evidence is mapped (a documented requirement relationship), executed (a recorded run against a stated target and pinned revision), or independently reviewed (assessed by a qualified outside party). Those describe the status of evidence and are orthogonal to E1-E5, which describes its strength. A mapping alone is E1-level material regardless of how many requirements it covers. The taxonomy in this repository is canonical; the field guide is hosted outside it and is not version-pinned.

603 executable security tests across 44 modules (verified 2026-08-02 via scripts/count_tests.py). MCP + A2A + L402 + x402 wire-protocol testing, plus UCP/ACP merchant-journey, AP2 mandate-chain, Fireblocks x402 hardening, Visa TAP / Mastercard Agentic Token funding-instrument, and denial-of-settlement finality conformance across the full agentic-payments stack. Decision-layer attack scenarios. One pip install away.

OWASP Agentic AI v1.1 Threat Coverage Report — commit-pinned mapping from the full T1–T17 taxonomy to executable tests: 13 direct, 4 partial, 0 not evidenced, across 96 mapped tests and 66 named OWASP scenarios. Mitigation-control validation is tracked separately from threat coverage (11 validated, 10 partial, 1 guidance-only), and every gap, evidence class and reproduction command is in the report. (T1–T15 submission view · canonical mapping)

Adapted from OWASP Agentic AI — Threats and Mitigations v1.1 under CC BY-SA 4.0. A test-capability report — not a certification, conformance claim, or OWASP endorsement. The adjudication is author-performed and is not independent review.

$ agent-security test mcp --url http://localhost:8080/mcp
Running MCP Protocol Security Tests v4.13.1...
 MCP-001: Tool List Integrity Check [PASS] (0.234s)
 MCP-002: Tool Registration via Call Injection [PASS] (0.412s)
 MCP-003: Capability Escalation via Initialize [FAIL] (0.156s)
...
Results: 8/10 passed (80% pass rate) - see report.json

Illustrative output. A target the harness cannot reach, or that answers without servicing the request, reports INCONCLUSIVE — never PASS. See v4.13.1 for why that distinction is enforced rather than assumed.

Quick Start

pip install agent-security-harness

# If 'agent-security' is not found, add ~/.local/bin to your PATH:
export PATH="$HOME/.local/bin:$PATH"

# See it work immediately — no server needed:
agent-security test mcp --simulate

# Then test your real MCP server:
agent-security test mcp --url http://localhost:8080/mcp

# Test an x402 payment endpoint
agent-security test x402 --url https://your-x402-endpoint.com

# Human-oversight surface: reviewer exposure (T10) and agent-to-human
# manipulation (T15). An unreachable target reports INCONCLUSIVE, never PASS.
agent-security test hitl --url http://localhost:8080

# Which tests back a given OWASP threat, scenario or mitigation control
python scripts/owasp_agentic_select.py --threat T16
python scripts/owasp_agentic_select.py --control P5-REA-001

See docs/QUICKSTART.md for mock server setup, rate limiting, MCP server mode, and CI/CD integration.

---

Three Layers of Agent Decision Security

Layer	What it covers	Example focus
Protocol Integrity	Prevent spoofing, replay, downgrade, diversion, and malformed protocol behavior	MCP, A2A, L402, x402 wire-level tests
Operational Governance	Validate session state, capability boundaries, platform actions, trust chains, and execution context	capability escalation, facilitator trust, provenance, session security
Decision Governance	Test whether an agent should act at all under its authority, confidence, scope, and policy constraints	autonomy scoring, scope creep, return-channel poisoning, normalization-of-deviance
Human Oversight	Test whether the human review layer can be saturated, starved, blinded, or turned against its own principal	approval flooding, risk starvation, stripped decision context, agent→human manipulation (OWASP T10/T15)

---

How This Differs From Other Projects

Capability	Snyk Agent Scan (2.9K stars)	Cisco MCP Scanner (1.0K stars)	NVIDIA Garak (8.7K stars)	This framework
What it does	Scans agent/MCP configs for tool poisoning and skill security	YARA + LLM-as-judge for malicious tools	LLM model vulnerability testing	Active protocol exploitation + decision governance
Approach	Static analysis	Static + LLM classification	Model-layer probing	Wire-protocol adversarial testing
MCP coverage	Tool descriptions, config files	Tool descriptions, YARA rules	-	46 tests: protocol (32) + supply-chain (4) + tool-poisoning repro (10), real JSON-RPC 2.0 attacks
A2A coverage	-	-	-	13 tests
L402/x402 coverage	-	-	-	85 tests (L402 33 + x402 52)
Merchant journey (UCP/ACP)	-	-	-	12 tests: agent-profile + cross-merchant cart + delegated checkout
Funding instrument (Visa TAP / MC Agentic Tokens)	-	-	-	12 tests: holder/merchant/amount/velocity scope + dynamic cryptogram + revocation
Settlement finality (denial-of-settlement)	-	-	-	8 tests: release-before-finality, reorg revocation, withheld-settlement liveness
Payment authz/hardening	-	-	-	AP2 mandate (17) + Fireblocks x402 (17)
Enterprise platforms	-	-	-	25 cloud + 58 enterprise (core 31 + extended 27)
Human oversight (T10/T15)	-	-	-	8 tests: reviewer exposure + agent→human manipulation
APT simulation	-	-	-	GTG-1002 (17 tests)
Jailbreak/over-refusal	-	-	Yes	50 tests (25 + 25 FPR)
AIUC-1 certification	-	-	-	Maps to 19 of 20 testable requirements (2026-Q1/Q2 set; Q3 delta)
OWASP Agentic v1.1	-	-	-	T1–T17 commit-pinned: 13 direct, 4 partial, 0 not evidenced
Research backing	-	Cisco blog	Papers	7 DOIs + 3 NIST submissions (self-authored; see Research)
MCP server mode	-	-	-	Yes - invoke from any AI agent
Statistical testing	-	-	-	Wilson CIs, multi-trial
Total tests	Config checks	YARA rules	Model probes	603 active tests across 44 modules

Star counts verified 2026-08-02 via the GitHub API. Invariant Labs' mcp-scan now redirects to snyk/agent-scan and is listed once rather than as two separate projects. Competitor rows describe what each tool does by design; a dash means the capability is outside its stated scope, not a defect.

Use both. Scan with Snyk Agent Scan or Cisco MCP Scanner for static analysis. Test with this framework for active exploitation. They're complementary layers.

---

Research

Seven public preprints and notes deposited on Zenodo (not represented as peer-reviewed publications) and three NIST submissions underpin the methodology. Every DOI below was re-verified on 2026-08-02 by content negotiation against doi.org — title and authorship confirmed:

Publication	DOI
Constitutional Self-Governance for Autonomous AI Agents — 12 governance mechanisms, 77 days production data, 56 agents	10.5281/zenodo.19162104
Detecting Normalization of Deviance in Multi-Agent Systems — First empirical demonstration that automated harnesses detect behavioral drift	10.5281/zenodo.19195516
Decision Load Index (DLI): A Quantitative Framework for Agent Autonomy Risk — Measuring cognitive burden of AI agent oversight	10.5281/zenodo.18217577
Beyond Identity Governance: A Protocol-Level Security Testing Framework for Multi-Agent Systems	10.5281/zenodo.19343034
Community-Driven Security for AI Agents: Evolution of an Adversarial Test Corpus	10.5281/zenodo.19343108
Claim-Level Negative Testing for Agent-Governance Evidence — Receipt-claim decomposition; the RCL-001..011 receipt-verification module in this harness	10.5281/zenodo.21418701
Signing Is Not Authorization: Claim-Level Negative Vectors for Agent-Payment Receipts — payment-authority application of the receipt-claim decomposition; RCL-001..011 under adversarial payment receipts	10.5281/zenodo.21535452

On citation counts. These records carry internal citation lineage across later work in this portfolio. An OpenAlex cited_by audit on 2026-08-02 found 30 citation edges and 0 qualifying independent citations — every edge is a self-citation, collapsing to nine duplicate/version records across three title families. Nothing here should be read as third-party validation or scholarly adoption. The one external check this project has received is an independent reproduction, listed under Used By.

Correction (2026-08-02). Two entries previously in this table cited DOIs belonging to other researchers — 10.5281/zenodo.15105866 (a MALDI mass-spectrometry dataset by Ranes et al.) and 10.5281/zenodo.15106553 (an e-learning article by Toshtemirov). They were attributed here to "Normalization of Deviance in Autonomous Agent Systems" and "Cognitive Style Governance for Multi-Agent Deployments". No Zenodo record under those titles by this author was located, so both rows were removed rather than re-pointed. The surviving Detecting Normalization of Deviance in Multi-Agent Systems record (19195516) is unaffected and was already listed separately.

---

Related Projects

Constitutional Governance (WHY layer)

The constitutional-agent package provides the governance gates and hard constraints that complement this test harness. Six gates, 12 hard constraints, amendment protocol — enforced in code, not YAML policy files. pip install constitutional-agent.

---

Documentation

Resource	Link
Expanded Quick Start	docs/QUICKSTART.md
Full Test Inventory (603 tests)	docs/TEST-INVENTORY.md
OWASP Agentic v1.1 Coverage (T1–T17)	docs/OWASP-AGENTIC-V1.1-COVERAGE.md
Canonical coverage mapping (source of truth)	docs/coverage/owasp-agentic-v1.1.yaml
Release history & known gaps	ROADMAP.md · CHANGELOG.md
E1-E5 Evidence Class Taxonomy (canonical)	docs/EVIDENCE-CLASS-TAXONOMY.md
AIUC-1 Evidence Field Guide (external, not version-pinned)	msaleme.github.io/aiuc1-readiness
AIUC-1 Crosswalk	docs/AIUC1-CROSSWALK.md
Advanced Capabilities	docs/ADVANCED.md
MCP Server	docs/mcp-server.md
CI/CD GitHub Action	docs/github-action.md
Payment Attack Taxonomy	docs/PAYMENT-ATTACK-TAXONOMY.md
Decision Governance Checklist	docs/DECISION-GOVERNANCE-CHECKLIST.md
Decision Governance Benchmark Leaderboard	benchmarks/LEADERBOARD.md
Related Work	docs/RELATED-WORK.md
Comparison (detailed)	docs/COMPARISON.md
Privacy & Telemetry	docs/PRIVACY.md

---

Roadmap

Current: v4.13.1 (2026-08-02). Recent shipped work — v4.5 skill supply chain and governance modification · v4.6–v4.9 payment-stack depth (AP2 mandate chain, UCP/ACP merchant journey, card-network agentic tokens, settlement finality, Fireblocks x402) · v4.10 benchmark integrity · v4.11–v4.12 decision-governance corpus currency and provenance repair · v4.13 OWASP Agentic v1.1 T1–T17 coverage mapping and the human-in-the-loop harness · v4.13.1 a correctness fix to that harness (see CHANGELOG.md).

Next — Standards & Evidence. Reproducible settlement-time payment evidence, a methodology paper, and the attestation/evidence schema submitted to a standards venue. Coverage breadth and test count are explicitly not goals. Full detail and the anti-goals in ROADMAP.md.

---

Used By

Who	Use Case
FransDevelopment / Open Agent Trust Registry	OATR SDK v1.2.0 test fixtures (X4-021 through X4-030) -- Ed25519 attestation verification

Independent reproduction

The only external check this project has received. @VrtxOmega wrote a separate Node verifier for the portable receipt-claim oracle fixtures and ran it against a pinned commit and fixture hash, matching all 11 RCL results including both acceptance controls — so it did not pass by rejecting everything.

The exchange also produced two corrections to this repository: signature_algorithm now names the actual encoding and states plainly that it is not RFC 8785 JCS, and coverage_gaps now declares that freshness is exercised only in the stale direction. Both are in fixtures/rcl/.

It is a reproduction of a pinned artifact by one external party, submitted as a report and explicitly not as a contribution, endorsement, certification, or adoption. It is not a substitute for independent review of the harness as a whole, which this project still does not have.

Using the harness? Open a PR to add yourself, or tag us in your project.

---

Contributing

See CONTRIBUTING.md for guidelines, SECURITY_POLICY.md for security policy, and CONTRIBUTION_REVIEW_CHECKLIST.md for the PR checklist.

Citation

If you cite this work in research:

Saleme, M. K. (2026). Agent Security Harness — multi-protocol agent security testing framework. ORCID: 0009-0003-6736-1900. https://github.com/msaleme/red-team-blue-team-agent-fabric

Related Zenodo preprints are listed in full under Research; that table is the single source of truth for DOIs in this repository. Do not cite a DOI for this work that does not appear there.

---

License

Apache License 2.0 -- see LICENSE.