Browsertrace

An unpacked Chrome MV3 extension that exposes screenshots, trusted mouse/keyboard input, and click-to-fetch/XHR tracing through the @sunwu51/chrome-mcp-sdk WebSocket bridge for sunwu51/mcp-center.

The tracer does not add an OPID header or otherwise modify page requests. It records the operation context locally in chrome.storage.local and deletes a tab's records when the tab closes or its main document navigates. The extension requests unlimitedStorage because captured request/response bodies can exceed Chrome's default extension storage quota.

Build

npm install
npm run build

Then open chrome://extensions, enable Developer mode, choose Load unpacked, and select the generated dist directory. Refresh pages that were already open before installing or reloading the extension.

By default the extension connects to:

ws://localhost:3000/ws/browsertrace

Click the extension action to change the URL or disable the bridge. MCP Center exposes the tools with its normal server prefix, for example browsertrace_screenshot. This switch controls only the MCP Center WebSocket connection; direct calls from other extensions remain enabled through the same Chrome MCP SDK tool registry.

Direct calls from another extension

BrowserTrace also exposes the same JSON-RPC 2.0 MCP surface directly to other installed Chrome extensions. The receiving manifest uses externally_connectable.ids: ["*"] with matches: [], so extension callers are allowed but ordinary web pages are not. A caller only needs the BrowserTrace extension ID shown on chrome://extensions:

const BROWSERTRACE_ID = "<browsertrace-extension-id>";

chrome.runtime.sendMessage(
  BROWSERTRACE_ID,
  { jsonrpc: "2.0", id: 1, method: "tools/list" },
  (response) => console.log(response.result.tools)
);

chrome.runtime.sendMessage(
  BROWSERTRACE_ID,
  {
    jsonrpc: "2.0",
    id: 2,
    method: "tools/call",
    params: {
      name: "screenshot",
      arguments: { tabId: 123, includeImage: true }
    }
  },
  (response) => console.log(response)
);

Both transports support initialize, tools/list, tools/call, and ping. Direct extension calls can receive inline MCP image content, snapshots, normal JSON results, and upload screenshots to MCP Center. Restrict externally_connectable.ids to known extension IDs if the local Chrome profile contains untrusted extensions.

Typical agent loop

1. Call tab_open and keep its returned tabId. New tabs open in the background by default, and navigation waits for load by default. Set waitUntil to none, domcontentloaded, or network_idle when a different boundary is needed.
2. Call screenshot. It returns a viewport image and a compact accessibility snapshot with element UIDs.
3. Call cua_action with an element uid, or use mouse_click with viewport coordinates. Keep the returned opid.
4. Wait for the page operation to settle.
5. Call operation_get_requests with that opid.
6. If OPID results are empty or incomplete, call network_list_requests with the same tabId, time window, and a URL/method filter. This CDP-level list does not depend on Zone-style context propagation. For normal API traffic, use "resourceTypes": ["XHR", "Fetch"] to hide Font, Script, Stylesheet, Image, and other static-resource noise. Use "resourceType": "Document" when checking native form submissions or navigation.
7. Call request_get_details for bodies and complete page-tracer request details.

Every input CUA action generates an OPID when omitted. Supply opid directly on the action only when a caller needs a stable custom identifier.

tab_open defaults to active: false, sets autoDiscardable: false, and puts created tabs in the BrowserTrace MCP tab group. Supply groupName to use a different group in that Chrome window. The debugger is attached immediately, so its indicator is present before coordinates are measured.

popup_wait waits for a new tab opened by a managed tab and adopts only that tab into the current MCP session; it does not enumerate or expose unrelated Chrome tabs. Use it after an action that opens a popup or target=_blank link.

JavaScript dialogs are exposed through dialog_get, dialog_accept, and dialog_dismiss. A pending dialog blocks the page until it is handled.

form_action provides targeted fill, select, check, and uncheck operations. It requires a snapshot uid or portable locator, and dispatches the normal input and change events for value changes.

Managed tabs also support tab_goto, tab_back, tab_forward, and tab_reload; each waits for load by default and accepts the same waitUntil and timeoutMs options as tab_open.

download_list, download_wait, and download_cancel expose only downloads whose URL origin matches a currently managed tab. Cookie operations are limited to the current HTTP(S) origin of a managed tab. storage_get, storage_set, storage_remove, and storage_clear operate on that page's localStorage or sessionStorage; they do not access extension storage or other origins.

Screenshots use CSS-pixel output by default, so their PNG dimensions align with CSS-coordinate bounds and CUA coordinates even when the device pixel ratio is greater than 1. Set cssPixels: false to request native device pixels. viewport_get reports the current CSS viewport and scale. viewport_set can override width, height, deviceScaleFactor, and mobile emulation; pass clear: true to remove the override.

All screenshots use CDP Page.captureScreenshot; they do not activate the tab or focus its window. Set fullPage: true to capture the entire rendered page. The virtual cursor is hidden during capture by default. Set hideCursor: false to include it in viewport, full-page, element, or workflow screenshots. If a tab was manually discarded despite autoDiscardable: false, screenshot and snapshot operations restore it with a background reload first.

Semantic snapshots and element screenshots

screenshot always returns a compact accessibility tree. It is not a complete HTML DOM dump: it focuses on semantic content such as buttons, links, inputs, headings, menus, and element state. Nodes that resolve to DOM elements have stable UIDs for the lifetime of that document. When a stable DOM attribute is available, the line also includes a portable locator with ordered CSS/XPath strategies; use that locator for a macro rather than the document-local UID.

Screenshots are uploaded to MCP Center temporary storage by default, and the MCP result returns filePath as an absolute filename on the MCP Center machine. includeImage controls only whether the PNG is also returned as MCP image content. This avoids passing base64 through a shell command when an agent needs a local file:

{
  "tabId": 123,
  "includeImage": false,
  "saveToFile": true,
  "name": "sign-in"
}

The upload URL is derived from the configured WebSocket URL, for example ws://localhost:3000/ws/browsertrace becomes http://localhost:3000/fs/upload. Files are sent as binary multipart/form-data in the file field. Set both includeImage: false and saveToFile: false for a semantic-only snapshot with no PNG capture. UID and CUA screenshot actions use the same behavior.

Example snapshot output:

RootWebArea "Sign in" uid=mabc_1
  textbox "Username" uid=mabc_4 required=true
  textbox "Password" uid=mabc_5 required=true
  button "Sign in" uid=mabc_6

Pass a UID to scroll to an element, capture only its box, and return only its semantic subtree:

{ "tabId": 123, "uid": "mabc_6", "includeImage": true, "saveToFile": true }

Capture the full rendered page without activating it:

{ "tabId": 123, "fullPage": true, "includeImage": true }

UIDs are backed by the current document loader and CDP backend DOM node IDs. They are reused while the same DOM node survives, and become invalid after navigation or when a framework replaces the node. Capture a new snapshot after an invalid-UID error. Canvas contents and DOM nodes excluded from the accessibility tree still require coordinate CUA.

Human-like actions

cua_action accepts one action object. Supported action types are move, hover, click, double_click, right_click, mouse_down, mouse_up, drag, scroll, type, key_press, key_down, key_up, select_all, clear, wait, wait_for, and screenshot.

Action parameters vary by type:

Action	Required parameters	Optional target / notes
click, double_click, right_click, move, hover, mouse_down, mouse_up	uid or locator, or both x and y	Coordinates are the visual fallback.
drag	fromUid and toUid, or fromX, fromY, toX, and toY	steps and durationMs control interpolation.
type	text	Supply uid or locator to focus that control first; otherwise text goes to the currently focused control.
key_press, key_down, key_up	key	Supply uid or locator to focus that control first.
select_all, clear	None	Supply uid or locator to focus that control first; otherwise they use the current focus.
scroll	deltaX and/or deltaY	uid / locator or x / y optionally chooses the wheel position.
wait	None	durationMs defaults to 250 ms.
wait_for	state	Element states require uid or locator; network_idle requires neither.
screenshot	None	uid captures that element; otherwise it captures the viewport or fullPage.

Mouse actions accept uid instead of x/y. drag accepts fromUid and toUid instead of coordinate pairs. Text and keyboard actions may also include uid; the element is clicked to focus it before input. Coordinate fields remain available as a visual fallback:

{
  "tabId": 123,
  "action": { "type": "click", "uid": "mabc_6", "opid": "op_login" }
}

Portable locators can be passed instead of a UID. Strategies are tried in order and must resolve to exactly one visible element (or specify nth):

{
  "tabId": 123,
  "action": {
    "type": "click",
    "locator": { "strategies": [{ "kind": "css", "value": "[data-testid='sign-in']" }] }
  }
}

Use wait_for rather than fixed sleeps when a later step depends on page state. It supports present, visible, hidden, absent, and CDP-backed network_idle (WebSocket and EventSource are ignored):

{ "tabId": 123, "action": { "type": "wait_for", "state": "visible", "locator": { "strategies": [{ "kind": "css", "value": ".results" }] }, "timeoutMs": 10000 } }

Workflows and macros

Use workflow_run for every multi-step operation, including a simple linear sequence. It also supports bounded control flow: a step is { "do": <CUA action> }, { "waitFor": <wait_for fields> }, { "if": { "when": ..., "then": [...], "else": [...] } }, or { "while": { "when": ..., "maxIterations": 20, "steps": [...] } }. Conditions use the same element-state or network_idle fields as wait_for.

{ "tabId": 123, "workflow": { "steps": [
  { "do": { "type": "click", "x": 300, "y": 220 } },
  { "do": { "type": "type", "text": "sunwu" } },
  { "do": { "type": "key_press", "key": "Enter" } }
] } }

Search workflow with visual checkpoints

Capture the state before and after entering a search, click a previously observed button UID, wait for the resulting requests to settle, and capture the final result:

{
  "tabId": 123,
  "workflow": {
    "steps": [
      { "do": { "type": "screenshot", "output": "file", "name": "search-before" } },
      {
        "do": {
          "type": "type",
          "locator": { "strategies": [{ "kind": "css", "value": "input[type='search']" }] },
          "text": "search today's tech news"
        }
      },
      { "do": { "type": "screenshot", "output": "file", "name": "search-entered" } },
      { "do": { "type": "click", "uid": "abc" } },
      { "waitFor": { "state": "network_idle", "idleMs": 500, "timeoutMs": 10000 } },
      { "do": { "type": "screenshot", "output": "file", "name": "search-results" } }
    ]
  }
}

Each screenshot step returns exactly one image output: output: "file" returns steps[].filePath, while output: "base64" returns steps[].image. Use snapshot when only accessibility text/UIDs are needed. Targeted actions return steps[].target, including the actual click point and, for UID or locator targets, the element bounds. A Remotion or HyperFrames composition can use the checkpoint images plus this per-step target data to animate cursor movement/clicks and produce an operation walkthrough.

Every completed workflow returns a short-lived runId. Call macro_export with that ID to return a portable, versioned macro configuration. Export only uses the locator captured while the action ran: UIDs, OPIDs, and coordinates are deliberately omitted. It does not save browser state; pass the returned macro.workflow directly to workflow_run to replay it later.

Script, console, and tab tools

• evaluate_script evaluates a JavaScript expression through CDP, awaits Promises by default, and returns a JSON-serializable value.
• console_list returns captured console.* calls, uncaught exceptions, and browser log entries. Use level and limit to filter it.
• console_clear clears both stored entries and the page console.
• tab_close closes a specified tab, or the active tab when omitted. Pass groupName instead to close every tab in all same-named groups across Chrome windows, allowing an agent to clean up all tabs it opened:

{ "groupName": "BrowserTrace MCP" }

For task-scoped cleanup, supply a stable ownerId to every tab_open call, then call session_finish with the same ID. It detaches the debugger and closes every tab the task opened:

{ "ownerId": "agent-task-abc" }

For a form field addressed by uid, type replaces the field's current value by default (including browser-prefilled credentials). Set append: true only when text should be appended deliberately.

Current scope and limitations

• OPID traces page-world fetch and XMLHttpRequest but does not correlate browser/CDP request IDs. network_list_requests separately captures CDP Network events, including navigation and resource traffic, as a fallback. WebSocket frames and WebTransport activity are not currently listed.
• A lightweight page tracer propagates operation context through interaction listeners, Promise callbacks, timers, animation/idle callbacks, and queueMicrotask without installing Zone.js or replacing the page's global Promise implementation. Native async continuations or code that deliberately escapes these patched callbacks may appear as a background request with opid: null.
• Request and response text is capped at 64 KiB in stored records. Binary bodies are reported as unavailable.
• chrome.debugger displays Chrome's debugging indicator and conflicts with a DevTools/CDP debugger attached to the same tab.
• Restricted Chrome pages cannot be injected or controlled.
• The current MCP Center WebSocket bridge does not authenticate registering clients. Keep it local until bridge authentication is added.

Development test page

Serve this repository with any local static server and open test-page/index.html. Its button makes two delayed fetch requests, which should appear under the single OPID returned by mouse_click.