Llm Vision Mcp

A TypeScript MCP server that gives text-only LLMs image understanding through StepFun vision models.

This is useful when the primary model, such as GLM5.2 or DeepSeek V4, does not support image input. The model can call these MCP tools, receive text or structured visual analysis, then continue reasoning with the result.

Tools

• analyze_image: general image understanding
• extract_text_from_image: OCR for screenshots, logs, documents, code, and UI text
• diagnose_error_screenshot: error screenshot and stack trace diagnosis
• understand_technical_diagram: architecture, flowchart, UML, ER, sequence, and network diagrams
• analyze_data_visualization: charts, tables, dashboards, and metrics screenshots
• ui_to_artifact: UI screenshot to implementation notes or design specs
• ui_diff_check: expected vs actual UI screenshot comparison

Setup

npm install
cp .env.example .env

Set STEPFUN_API_KEY in the MCP client environment. Injecting env vars through the MCP client config is usually the most explicit and reliable setup.

Required:

STEPFUN_API_KEY=your_stepfun_api_key

Optional:

# standard | step_plan
STEPFUN_API_MODE=standard
STEPFUN_BASE_URL=https://api.stepfun.com/v1
STEPFUN_VISION_MODEL=step-1o-turbo-vision
STEPFUN_DEFAULT_DETAIL=high
STEPFUN_TIMEOUT_MS=120000

Step Plan

Step Plan uses the same API key style but a different Base URL:

STEPFUN_API_MODE=step_plan

When STEPFUN_API_MODE=step_plan is set, defaults change to:

STEPFUN_BASE_URL=https://api.stepfun.com/step_plan/v1
STEPFUN_VISION_MODEL=step-3.7-flash

You can still override either value explicitly:

STEPFUN_API_MODE=step_plan
STEPFUN_BASE_URL=https://api.stepfun.com/step_plan/v1
STEPFUN_VISION_MODEL=step-3.7-flash

For backward compatibility, STEPFUN_USE_STEP_PLAN=true also enables Step Plan mode when STEPFUN_API_MODE is not set.

Run

npm run build
npm run start

Run With npx

After this package is published to npm:

npx -y llm-vision-mcp

The published package runs on Node.js and does not require Bun on the user's machine.

MCP Client Config

Example:

{
  "mcpServers": {
    "llm-vision-mcp": {
      "command": "node",
      "args": ["/Users/shaoyun/workdir/llm-vision-mcp/dist/index.js"],
      "env": {
        "STEPFUN_API_KEY": "your_stepfun_api_key",
        "STEPFUN_API_MODE": "step_plan",
        "STEPFUN_DEFAULT_DETAIL": "high"
      }
    }
  }
}

npm package example after publishing:

{
  "mcpServers": {
    "llm-vision-mcp": {
      "command": "npx",
      "args": ["-y", "llm-vision-mcp"],
      "env": {
        "STEPFUN_API_KEY": "your_stepfun_api_key",
        "STEPFUN_API_MODE": "step_plan",
        "STEPFUN_DEFAULT_DETAIL": "high"
      }
    }
  }
}

Image Inputs

Every single-image tool accepts:

{
  "image": "/absolute/path/to/screenshot.png",
  "question": "What does this error mean?",
  "detail": "high"
}

The image field supports:

• local file path
• file:// path
• http:// or https:// URL
• data:image/...;base64,... Data URL

ui_diff_check accepts two images:

{
  "expected_image": "/absolute/path/to/expected.png",
  "actual_image": "/absolute/path/to/actual.png",
  "question": "Focus on layout and missing buttons.",
  "detail": "high"
}

Notes

• Use detail: "high" for OCR, UI, diagrams, charts, and screenshots.
• Use detail: "low" for faster, cheaper coarse image understanding.
• StepFun supports JPG/JPEG, PNG, WebP, and static GIF image inputs.