Annotate

Draw on a live page, and your coding agent reads what you drew.

An MCP server that opens a real browser and injects a comment/pencil overlay into every page it loads — your app, a staging deploy, someone else's site. You mark things up like a PDF annotation layer; the agent gets back your note plus the DOM elements it lands on.

It is project-agnostic by construction. Nothing is installed into the app under review: no component, no route, no dev-server dependency, no framework assumption.

Why it can be agnostic

Two Playwright primitives:

• addInitScript injects src/overlay.js into every page and frame before any page script runs, for the life of the browser context. Navigation, reloads, and cross-origin hops all keep the overlay.
• exposeBinding gives that injected script a real function into Node, so marks round-trip to the server instead of being trapped in page memory.

State lives in Node, not the page. The overlay is a pure view: on load it asks the server for the marks, on every change it pushes them back. That inversion is what makes a mark left on /dashboard still exist after you navigate to /settings, reload, or close the tab — none of which localStorage would survive cleanly across origins.

Quick start

Clone it anywhere — it does not have to live inside the project you are reviewing:

git clone <this-repo> annotate-mcp
cd annotate-mcp
npm install          # builds automatically via `prepare`; pnpm and bun work too

Then point your MCP client at dist/index.js. For Claude Code, add it to the project you want to annotate — .mcp.json in that project's root:

{
  "mcpServers": {
    "annotate": {
      "command": "node",
      "args": ["/absolute/path/to/annotate-mcp/dist/index.js"],
      "timeout": 28800000
    }
  }
}

Restart the session, accept the "new MCP server" prompt, and ask the agent to open the browser. Full per-client instructions, including one-line install and where marks are stored: docs/INSTALL.md.

Uses your installed Google Chrome by default (familiar browser, no 150MB download), falling back to Playwright's bundled Chromium. The profile is persistent, so logins survive between sessions — you need to be signed in to review the flows worth reviewing.

Using it

Ask the agent to open the browser, then mark things up. Toolbar starts bottom-right; drag it anywhere by the grip on its left edge and it stays there, per browser.

grip	move the toolbar — the panel opens away from whichever edge you park it on
comment	click anywhere to pin a note
pencil	drag to draw; the note box opens automatically, so the text is linked to that drawing with no tool swap
erase	click a single pin or line to delete just that one
clear all	press twice to confirm
3+7	review panel — unsent marks here + elsewhere; everything ticked by default, untick to exclude, then Send

Esc backs out one level at a time: open note → unlinking → linking mode → active tool → panel. Without it you can get stuck in comment mode unable to click the page underneath.

Working with existing marks

• Click a pin or a drawing to reopen its note — edit the text, or delete it from there. Clicking never stacks a second comment on top of the first, and it works whichever tool is in hand, including the pencil you just drew with.
• Drag a pin to move it. It re-resolves what it points at on drop, so a moved pin cites the element it landed on and not the one it left.
• Link (chain icon in the note box) makes one comment cover several places: press it, then click another mark to pull it in, click empty page to drop a new linked pin, or draw to add a linked drawing. Comments and drawings link to each other freely — a group can be any mix. Linked marks share one text (editing any of them rewrites all of them) and are joined on screen by a dashed thread. Press Done or Esc to leave linking mode.
• Unlink — the eraser in the link banner. It appears once a group exists, and clicking a member takes it out while leaving the mark, its text, and the rest of the group alone. It erases links and nothing else; the toolbar eraser is the one that deletes marks. Take the second-to-last member out and the group dissolves on its own, since one mark is not a link.

Sent vs pending

The panel has two tabs. Send moves a mark from To send into Sent, where it is dimmed and cannot be sent again — a queue that never shrinks stops telling you what is outstanding. The ink stays on the page as a record; the eye toggle in the Sent tab takes it off the page entirely when the archive starts covering the app you are still reviewing.

Editing a sent mark's text is the one way back: rewriting it makes it new information, so it rejoins the queue.

Reflow

Marks are anchored to the element they land on, as a fraction of that element's box. Resize the window and the ink follows its component — pins are repositioned, drawings are translated without being stretched, and the stored coordinates are rewritten so the agent, the screenshot crop, and the screen never disagree.

If the element is not rendered at the new size (a nav that collapsed, a card dropped from the layout), its mark is deliberately left where it was rather than dragged somewhere arbitrary, and drawn hollow/dashed so it reads as "this is about something not on screen" instead of silently labelling whatever slid underneath.

Tools

tool	what it does
annotate_open	launch/focus the browser, optionally navigate
annotate_wait	block until you press Send, then return that batch — the main hand-off
annotate_list	all marks as a briefing, filterable by status/path
annotate_inbox	re-read the most recent sent batch
annotate_screenshot	screenshot with ink visible, toolbar hidden; markId crops around one mark
annotate_resolve	mark items done — they dim on the page so you can see what was handled
annotate_clear	delete marks
annotate_close	close the browser (marks are kept on disk)

The hand-off, and the three states of Send

An MCP server answers calls; it cannot start a turn in an agent's session. So a send only reaches the agent on its own while something is parked in annotate_wait.

annotate_wait parks for an hour by default (max 8h). Parking is free — a blocked call spends no tokens — and on clients that background long calls it doesn't block the agent either: Claude Code moves any MCP call still running after two minutes into a background task, so it keeps working and the batch arrives as a task notification the moment you press Send. Long parks are kept alive with progress pings, because a client will abort an stdio call that has been silent too long — 30 minutes, in Claude Code's case.

Two ceilings are worth knowing:

timeout in .mcp.json	hard wall-clock per call. Set to 8h here. Also floors the idle timeout
MCP_TOOL_TIMEOUT	the same limit as an env var; defaults to ~28h when unset

Nothing sent is ever lost. A send with nobody parked is held, and the next park collects it immediately instead of waiting for a fresh one — so the gap between an agent taking one batch and parking again costs a delay, not your feedback.

That leaves three states, and the panel says which one you are in — but only when it is not the obvious one:

state	what Send does	what the panel says
live — an agent is parked	starts work immediately	nothing
busy — it took a batch and hasn't come back	queued; collected when it parks again	"The agent is working on your last batch"
asleep — nothing listening	held on disk	"The agent is asleep — send its session any message"

No copy anywhere names a model or vendor — the panel says "the agent", so the tool reads the same whichever assistant is on the other end of the MCP connection. Falling asleep while you are working raises a one-off notice, since that is the moment the button quietly changes meaning. Loading a page with nothing listening does not — that is ordinary, and the line under Send already covers it.

What a mark carries

The note is what gets acted on; the DOM capture is how it gets found in source.

### 1. ✏️ drawing
> currency should come first

- where: stroke over 30,369 → 160×30 (document coords)
- viewport: 782×495 @1x — tablet/laptop
- elements under the drawing (most specific first):
  - testid=`price-badge` · name="€25.00"
    - selector: `[data-testid="price-badge"]`
    - box: 40,379 140×38

A pin records the element clicked plus three ancestors. A stroke hit-tests a grid across its bounding box, so "I circled these three cards" survives as structured data rather than only as pixels — that is what makes a drawing actionable instead of merely suggestive.

Deliberately not captured: a file:line source mapping. React 19 dropped _debugSource from fibers and Next 16 is on SWC, so that needs a build-plugin dependency in every project — which would destroy the agnosticism. testid plus visible text finds the component in one grep.

Output

Written to <cwd>/.annotations/ (override with ANNOTATE_DIR):

• store.json — canonical state, reloaded on restart
• annotations.md — every mark, rewritten on change
• inbox.md — only the batch you last sent. The separation is the point: one file is everything ever marked, the other is what you're asking about right now.

Environment

var	default
ANNOTATE_DIR	<cwd>/.annotations	where marks are stored
ANNOTATE_PROFILE	<ANNOTATE_DIR>/chrome-profile	persistent browser profile
ANNOTATE_CHANNEL	chrome	Playwright channel; unset-able to force bundled Chromium
ANNOTATE_HEADLESS	0	1 only for the smoke test — headed is the point

Tests

pnpm build && pnpm test

test/smoke.mjs drives the real overlay in a real browser with real clicks and drags. It covers what typechecking cannot: injection ordering, the Node↔page binding, stroke hit-testing, mark survival across a full page load, selective send, the eraser, reopening and editing a mark, pin dragging, linking and unlinking across kinds, linked notes staying in step, ink surviving the note box, the sent/pending split, and marks following their element through a reflow.

Known limits

• One browser. You annotate in the window this server opens, not your everyday Chrome.
• No interaction tools. The server navigates (annotate_open) and screenshots, but exposes no click/type/hover — a human drives this window. If you want the agent to drive the app too, that is the Playwright MCP's job, or add the tools here; the Playwright page is already in hand.
• Iframes are skipped. The overlay mounts only in the top frame — otherwise a page with three iframes gets four toolbars.
• Re-anchoring resolves a selector. Short CSS paths can match more than one element, so a mark on a generic div > button may re-anchor to the wrong sibling after a reflow. Marks on anything carrying a testid or id are exact.
• Only pins can be dragged. A drawing's position follows its element on reflow, but there is no handle to move the ink by hand — erase and redraw.

Contributing

Bug reports: yes, please. Especially "the overlay broke on this site" — the injected script has to survive whatever CSS and JS the host page throws at it, and the failures worth knowing about are on pages I will never think to test.

Pull requests: not by default. This is a small tool with a specific shape, and most of it is one injected script that cannot import anything, cannot ship a stylesheet, and has to work inside a stranger's cascade — constraints that are invisible until you have been bitten by them. Fork it and make it yours; that is what the licence is for. If you have something you think genuinely belongs upstream, open an issue first and we can talk.

License

MIT © Nikolaos Psycharis