Screenshot Vision

When reviewing a PR locally, you need to see if the UI looks right. Normally that means Claude takes a screenshot — which burns vision tokens and sends your screen contents to Anthropic. This MCP server routes screenshots through a local Ollama vision model instead: zero token cost, nothing leaves your machine.

Tool	When to use
analyze_screenshot	Public or local URLs — headless browser, no session needed
capture_window	Any app window on screen — sees your real logged-in session
locate_element	Native macOS apps only — returns click coordinates without Claude seeing the image

Chrome automation: for clicking elements in Chrome, prefer javascript_tool + getBoundingClientRect — it's exact and requires no vision at all. See Clicking in Chrome.

---

Requirements

• macOS (window capture uses screencapture and AppleScript)
• Node.js 22+
• Ollama installed (the server auto-starts it if it isn't running)
• A multimodal Ollama model — default is gemma4:e4b:

  ollama pull gemma4:e4b
  

---

Installation

git clone [email protected]:AVS845/screenshot-vision-mcp.git
cd screenshot-vision-mcp
npm install
npx playwright install chromium
npm run build

---

Claude Code configuration

Add this to your ~/.claude/settings.json under mcpServers:

{
  "mcpServers": {
    "screenshot-vision": {
      "command": "node",
      "args": ["/absolute/path/to/screenshot-vision-mcp/dist/index.js"]
    }
  }
}

Replace /absolute/path/to/ with the actual path. Then restart Claude Code.

Two optional env vars let you configure the server without touching call sites:

Variable	Default	Description
OLLAMA_URL	http://localhost:11434	Ollama host
OLLAMA_MODEL	gemma4:e4b	Default vision model for all tools

---

Tools

analyze_screenshot

Takes a screenshot of a URL in a headless Playwright browser and analyzes it with Ollama.

analyze_screenshot(
  url: "http://localhost:3004/dashboard",
  question: "Does the revenue chart render correctly? Are there any layout issues?"
)

Parameter	Type	Default	Description
url	string	—	URL to screenshot
question	string	—	What to analyze
model	string	gemma4:e4b	Ollama vision model
viewport_width	number	1280	Viewport width in px
viewport_height	number	800	Viewport height in px
wait_ms	number	2000	Wait after page load (ms)
full_page	boolean	false	Capture full page height, not just the visible viewport
max_slices	number	8	Max slices when full_page is true (1–20)

When full_page: true, the page is sliced into viewport_height-tall segments and sent as multiple images. This matters because Gemma 4's image token budget is ~280 tokens — a single tall screenshot gets crushed into noise, while properly-proportioned slices each get full detail.

---

capture_window

Captures an app window currently on screen and analyzes it. Use this when the page requires a login — it sees your real browser session.

capture_window(
  app_name: "Google Chrome",
  question: "Is the form validation error displaying correctly under the email field?"
)

Parameter	Type	Default	Description
app_name	string	—	Exact macOS app name, e.g. "Google Chrome"
question	string	—	What to analyze
model	string	gemma4:e4b	Ollama vision model
window_index	number	1	Which window (1 = frontmost)
scale	number	1	Upscale factor (2–3 helps with small text)
crop	object	—	Crop to a sub-region before analysis

The crop parameter uses fractional values (0–1). To inspect just the bottom half of the window:

{ "x": 0, "y": 0.5, "width": 1, "height": 0.5 }

---

locate_element

Finds a UI element in a native macOS app window and returns its click coordinates. Designed for apps where DOM access isn't available — Terminal, Figma, Xcode, etc.

Not recommended for Chrome. Chrome's DOM gives exact coordinates with no vision model involved. See Clicking in Chrome.

Parameter	Type	Default	Description
app_name	string	—	Exact macOS app name, e.g. "Figma", "Terminal"
element_description	string	—	Natural language description of the element
model	string	gemma4:e4b	Ollama vision model
window_index	number	1	Which window (1 = frontmost)
viewport_bounds	object	—	Screen-coordinate bounds of the region to capture

Returns { x, y, coordinate_mode, clamped }. clamped: true means the model returned out-of-range coordinates that were corrected — treat as low confidence.

Accuracy note: gemma4:e4b has a ~280-token image budget, which limits spatial precision. Elements in the center of the screen locate reliably; elements near edges may have 50–150px errors. Use a larger model (e.g. gemma4:26b) if precision matters, at the cost of slower inference.

---

Clicking in Chrome

For Chrome browser automation, skip locate_element entirely. Use javascript_tool to get exact coordinates from the DOM:

// In javascript_tool — returns perfect viewport coordinates, no vision needed
const el = document.querySelector('button.submit');
const r = el.getBoundingClientRect();
JSON.stringify({ x: Math.round(r.left + r.width/2), y: Math.round(r.top + r.height/2) });

Only fall back to locate_element for elements that genuinely can't be DOM-queried (canvas content, rendered images, visually-composed widgets without CSS selectors). When you do, pass viewport_bounds so the returned coordinates are viewport-relative and work directly with the Chrome computer tool.

Measure viewport bounds immediately before the call — the Chrome automation InfoBar shifts innerHeight, causing ~40px errors with stale bounds:

// javascript_tool — run right before locate_element
JSON.stringify({
  screenX: window.screenX,
  screenY: window.screenY,
  outerHeight: window.outerHeight,
  innerWidth: window.innerWidth,
  innerHeight: window.innerHeight,
})

locate_element(
  app_name: "Google Chrome",
  element_description: "the close button on the confirmation modal",
  viewport_bounds: {
    x: screenX,
    y: screenY + outerHeight - innerHeight,
    width: innerWidth,
    height: innerHeight,
  }
)
→ { x: 891, y: 267, coordinate_mode: "viewport" }

---

How it works

1. analyze_screenshot launches a headless Chromium browser via Playwright, navigates to the URL, waits for JS to settle, and captures a PNG.
2. capture_window uses AppleScript to get the window bounds, screencapture -R to grab exactly that region, and optionally sips to crop and scale.
3. locate_element captures the specified region, asks Ollama to return element coordinates as JSON fractions (0–1), then converts to pixel coordinates.
4. All tools base64-encode the PNG and POST it to Ollama's /api/generate endpoint.
5. If Ollama isn't running, the server spawns ollama serve and waits up to 30 seconds for it to become ready.

---

Rebuilding after changes

npm run build

Then restart Claude Code to reload the server.