RankedLM Server

AI security and governance is quickly becoming one of the most important problems in tech ~ and most people assume it's an IT problem, a legal department problem, or a big company's problem. All three are wrong.

AI governance is the checkpoint every agent action has to pass through ~ it is the set of controls that decide what an agent can access, what it can do, and who's accountable when something goes wrong.

RankedLM Governance is a system built to answer that question: it controls exactly what an AI agent is allowed to do, for how long, and proves ~ with a real, checkable record ~ what it actually did.

What

RankedLM Governance is an access-control and accountability layer for AI agents that query your systems through an MCP server. Built using the Model Context Protocol ~ the open standard for how AI agents connect to and call tools ~ it decides what each agent can do, shuts down access the moment something looks wrong, and keeps a permanent, checkable record of everything that happened.

Concretely, every agent gets a badge (an API key) that proves its identity. That badge gets exchanged for a temporary pass (a JWT) the agent actually uses while working, so the badge itself never has to be handed around. For short, one-off jobs, an agent can request an even smaller, weaker pass (a task-scoped credential) that only works for a few minutes and can never do more than the badge it came from.

Every single action an agent takes gets checked against a rulebook (RBAC/ABAC) before it's allowed through, and every action ~ allowed or refused ~ gets written down permanently, searchable later by who did what, when, and whether it was allowed or denied.

If a badge or a pass is ever compromised, or an agent starts misbehaving, any of them can be shut off immediately ~ even mid-task ~ and the system is actively watching for patterns that look like something going wrong, not just recording history after the fact.

Why

Imagine an AI agent that can approve payments, access customer data, or update records on your behalf. Now imagine that agent has a bug, or its credentials leak, or it starts doing something it was never meant to do.

Who notices? How fast? And once you know, how do you actually stop it ~ without shutting down every other agent that's working correctly?

Most teams building AI agents today have no real answer to these questions. The agent either has full access or none at all. Nobody's watching what it actually does in between. And if something goes wrong, there's no real record to prove what happened, who did it, or when it started.

RankedLM Governance exists because "trust the agent" isn't a security model. Every agent needs its own scoped, revocable, provable identity ~ the same way every employee at a real company has a badge that can be turned off the moment it needs to be, and a record of everywhere that badge was used.

How

The system is built around five ideas, each solving one part of the problem named above.

1. Identity ~ API keys and JWTs Every agent is issued an API key when it's set up ~ a long-lived credential tied to that specific agent, and nothing else. The agent doesn't use this key directly for every action; instead, it exchanges the key for a JWT (a signed, temporary token) each time it starts working. This means the powerful, long-lived credential rarely has to be passed around ~ only the temporary token does.

2. Narrower access for short jobs ~ task-scoped credentials For a single batch of work, an agent can request a task-scoped credential: a JWT that's deliberately smaller and shorter-lived than its parent key. It can never hold more permissions than the key it came from, it dies automatically after a short, configurable window (15 minutes by default) whether or not the work is finished, and it can't be used to create another one ~ so there's no way to chain these into indefinite access.

3. Deciding what's allowed ~ RBAC and ABAC Every tool call is checked against two layers of rules before it's allowed to run. RBAC (role-based access control) checks whether the caller's role holds the required permission level for that action. ABAC (attribute-based access control) adds conditions on top ~ for certain sensitive actions, requiring a recently issued token, so a stolen but aging credential can't be used to escalate access.

4. Shutting things off ~ revocation Access can be revoked two ways: killing an entire API key (which immediately invalidates every live token tied to it), or killing one specific token by itself, leaving its parent key untouched. Both take effect on the very next request ~ there's no waiting for something to expire naturally. Revoking your own access is always allowed, even with an aging credential, because someone who suspects their own credential is compromised needs to be able to act immediately.

5. Proving what happened ~ audit search, reporting, and alerting Every action, allowed or denied, is permanently logged. That log can be searched by agent, by tool, by outcome, or by time range. It can also be aggregated into reports ~ for example, "how many actions did each agent take this week, and how many were denied." And a background watcher checks on a short interval for a specific warning sign ~ repeated denied attempts from the same agent in a short window ~ and raises an alert soon after it happens, rather than waiting for someone to notice.

flowchart TD
    Agent("🤖 AI agent")
    Agent --> Key("API key<br/><small>issued once, per agent</small>")
    Key --> JWT("JWT<br/><small>minted fresh, per session</small>")
    JWT --> Gate{"RBAC / ABAC gate<br/><small>scope and conditions, checked live</small>"}

    Gate -->|allowed| Pass("✅ Allowed")
    Gate -->|denied| Fail("❌ Denied")

    Pass --> Log[("Audit log<br/><small>every outcome, permanently recorded</small>")]
    Fail --> Log

    Log --> Search("🔍 Search")
    Log --> Report("📊 Report")
    Log --> Alert("🚨 Alert")

    classDef agent fill:#B4B2A9,stroke:#5F5E5A,color:#2C2C2A
    classDef cred fill:#AFA9EC,stroke:#534AB7,color:#26215C
    classDef gate fill:#5DCAA5,stroke:#0F6E56,color:#04342C
    classDef pass fill:#97C459,stroke:#3B6D11,color:#173404
    classDef fail fill:#F09595,stroke:#A32D2D,color:#501313
    classDef log fill:#F0997B,stroke:#993C1D,color:#4A1B0C
    classDef obs fill:#ED93B1,stroke:#993556,color:#4B1528

    class Agent agent
    class Key,JWT cred
    class Gate gate
    class Pass pass
    class Fail fail
    class Log log
    class Search,Report,Alert obs

This diagram shows the core request flow. Task-scoped credentials and revocation ~ two of this system's most tested features ~ are covered in the diagram below, and in full detail in the Tool Reference and How sections above.

flowchart TD
    Key("API key<br/><small>long-lived, per agent</small>")
    Key --> Session("JWT<br/><small>one working session</small>")
    Session --> Task("Task-scoped credential<br/><small>narrower, expires in minutes</small>")
    Task -.->|"cannot mint another"| Task

    Admin("🛡️ Admin") -->|revoke a key| Key
    Owner("Agent owner") -->|revoke their own| Task
    Admin -->|revoke anyone's| Task

    Key -->|revoked| Dead1["❌ Every live token tied to this key dies"]
    Task -->|revoked| Dead2["❌ This one token dies, key unaffected"]

    classDef cred fill:#AFA9EC,stroke:#534AB7,color:#26215C
    classDef people fill:#B4B2A9,stroke:#5F5E5A,color:#2C2C2A
    classDef dead fill:#F09595,stroke:#A32D2D,color:#501313

    class Key,Session,Task cred
    class Admin,Owner people
    class Dead1,Dead2 dead

Tool Reference

12 tools are registered. All are enforced by an import-time check that fails the app on startup if any tool is missing a scope or condition entry ~ so this list can't silently drift from what's actually running.

Tool	What it does	Who can call it
validate_permissions	Checks whether your own token holds a given scope. Note: accepts user_id/tool_name parameters that are currently ignored ~ it never checks anyone else's permissions, only the caller's	Any authenticated caller
request_tool_access	Files a request for a higher scope. Grants nothing by itself ~ a human admin must separately approve it	Self only
audit_access_history	Reads one agent's activity history	Self, or any agent's if admin
resolve_access_request	Approves or denies a pending access request. Approving does not grant access ~ real permissions still come from the agent's key	Admin only; cannot approve your own request
revoke_access	Kills an entire API key, or one specific token, immediately	Self (your own credential) or admin (anyone's)
check_token_status	Checks whether a credential is still valid	Self by default; admin can check any
issue_task_credential	Mints a short-lived, narrower credential for one batch of work	Any agent with at least read access; requires a freshly issued token
revoke_task_credential	Kills one specific task credential	The credential's owner, or an admin
check_task_credential_status	Checks whether a task credential is still usable	The credential's owner, or an admin
search_audit_log	Searches the full activity log, filterable by agent, tool, outcome, or time	Self-scoped for agents; admins can search anyone or everyone
audit_report	Aggregates activity into counts, grouped by agent, tool, or outcome	Admin only

Deprecated: log_tool_call remains registered for backward compatibility but no longer writes anything ~ every tool call is now logged automatically. Calling it succeeds but has no effect.

"Self" means the caller's own verified identity must match the target's owner ~ never something the caller can simply assert.

Setup Instructions

1. Prerequisites Docker and Docker Compose. Nothing else ~ no local Python, no local Postgres.

2. Environment file

cp .env.example .env

Two things must be changed before this is safe to run:

• DATABASE_URL ~ ships with a literal placeholder password. Generate a real one now: openssl rand -hex 20. You'll use this same value in step 4.
• JWT_SECRET ~ ships with a default that technically works, meaning anyone with this repo could mint valid tokens until you change it.

Everything else in .env.example has a safe default.

3. Bring up the stack ~ order matters

Don't run docker compose up -d directly ~ the app will start before its database role exists and exit immediately.

# 1. Database first
docker compose up -d postgres
docker compose ps    # wait for "healthy"

# 2. Apply migrations manually, in order ~ there is no migration runner
docker compose exec -T postgres psql -U rankedlm -d rankedlm \
  -v ON_ERROR_STOP=1 < migrations/002_task_credentials.sql

# migration 003 requires the password you generated in step 2 ~
# there's no default, on purpose, so a working credential never ends up in this repo
docker compose exec -T postgres psql -U rankedlm -d rankedlm \
  -v ON_ERROR_STOP=1 -v app_password='<your generated password>' \
  < migrations/003_least_privilege_app_role.sql

docker compose exec -T postgres psql -U rankedlm -d rankedlm \
  -v ON_ERROR_STOP=1 < migrations/004_alerts.sql

# 3. Now the app
docker compose up -d --build app

4. Confirm it's working

curl -s localhost:8000/health
# {"status":"ok"}

This confirms the app is running, but not that it can reach the database. For that, request a real token using the one seed credential a clean install has:

curl -s -X POST localhost:8000/auth/token \
  -H 'Content-Type: application/json' \
  -d '{"api_key":"rankedlm-test-key-001"}'

A token coming back confirms the whole chain ~ app, database, and the least-privilege role ~ is working end to end.

5. Things a first-time setup will run into

• A misleading error if you skip ahead: starting the app before migration 003 produces password authentication failed, the same message you'd get from a genuine typo. If you see this, check the migration ran before you check the password.
• No admin key exists by default. The one seed credential can only read and write ~ it can't approve requests, revoke access, or pull reports. Creating an admin key currently requires a direct database insert (see Known Limitations).
• docker compose down -v wipes the database, including migrations 002-004. They'll need to be re-applied. A plain docker compose down (without -v) keeps the database intact.

Claude Desktop Config

Add to claude_desktop_config.json:

{
  "mcpServers": {
    "rankedlm": {
      "url": "http://localhost:8000/mcp",
      "headers": {
        "Authorization": "Bearer <your-jwt-here>"
      }
    }
  }
}

Get a JWT first:

curl -X POST http://localhost:8000/auth/token \
  -H "Content-Type: application/json" \
  -d '{"api_key": "rankedlm-test-key-001"}'

This token is not permanent, and the config above won't tell you when it stops working. The Authorization header is a static paste of a credential that expires on its own ~ one hour by default. Once it does, every call from Claude Desktop starts being refused until you request a fresh token and paste the new one in. The same thing happens immediately, without waiting for expiry, if that token or its parent key is revoked ~ revocation takes effect on the very next request, so a config that worked a moment ago can stop working mid-session. Both cases look identical from the client side: if calls suddenly start failing, get a new token before assuming anything else is wrong.

The Demo Agent (agent_demo/)

agent_demo/demo_agent.py walks a single agent through the full request path and prints each step ~ exchanging the seed API key for a JWT, initializing an MCP session, then calling tools over the wire. agent_demo/test_manual.py sits alongside it as a hand-editable smoke test, with marked constants you change in place to provoke failure cases.

Be aware of what it is. This demo predates the security remediation work described in this README. It exercises 4 of the 12 tools that now exist, and one of those ~ log_tool_call ~ is now a documented no-op. The demo still runs green, but it passes for a reason unrelated to what its own comments claim: its # EXPECTED: log_id, immutable: true line is satisfied by values the automatic audit wrapper substitutes on the way out, not by the tool writing anything. Nothing in it touches task credentials, revocation, ABAC conditions, audit search, reporting, or alerting.

Its real remaining value is the transport. It is the only thing in this repo that exercises the real HTTP path end to end against a live Docker stack ~ the auth endpoint, the JWT middleware, and the MCP session handshake, over actual requests. The automated suite under tests/ is entirely in-process and never makes an HTTP call, so nothing there would catch a break in that path.

Updating agent_demo/ to cover the newer tools ~ and to stop presenting log_tool_call as if it still writes ~ would be a good follow-up. It has not been done yet.

Known Limitations

This system is built and live-tested, but it's honest about what's still open:

• Creating an admin key requires a direct database insert. There's no tool for issuing the first admin credential ~ by design, the app's own database role deliberately can't create new API keys, so this has to be done once, manually, against Postgres directly.
• The database's own bootstrap role is still a superuser. The app itself runs with narrow, minimal permissions ~ but the role used to run migrations still has full database power. Fully closing this would mean demoting that bootstrap role, which is separate, larger work.
• Postgres' default trust authentication is still in place for local connections. Something already running inside the database container could connect as any role, including the superuser, with no password. Only exploitable by something already inside that container.
• Cross-agent audit reads don't record which agent's data was viewed, only that a read happened. An admin searching another agent's history is fully logged as an action, but not with enough detail to say exactly whose data was pulled.
• The alert watcher checks every 30 seconds, not instantly. A burst of denied attempts is caught within moments, not the exact instant it happens.
• Alerts are recorded, not sent anywhere yet. They're written to a database table for now ~ no Slack, email, or notification exists yet (planned for v2).

A complete, current list of every known issue ~ including minor ones ~ is kept in errors-rankedlm-gov.md, updated as part of this project's normal workflow.

V2 Roadmap

• Sending alerts somewhere real. Right now alerts are written to a table. Next: a Slack or email notification the moment one fires.
• A tool for reading and acknowledging alerts. Currently checked with a manual database query.
• Delegation between agents. One agent temporarily handing part of its own access to another. Paused for now ~ this system currently runs with a single agent, so there's no real case to design against yet.
• A tool for issuing the first admin key. Right now this requires a manual database insert; a proper tool would remove that one remaining manual step.