Cvdlt

The repo is based on Ultralytics and Model Context procotol of Python SDK Related Links:

MCP Playground(client) - https://github.com/MRonaldo-gif/mcp-playground-local

Ultralytics - https://github.com/ultralytics/ultralytics

MCP of Python - https://github.com/modelcontextprotocol/python-sdk

Python server implementing Model Context Protocol (MCP) for image object detection, segmentation, and pose estimation operations.

Features

• Detect objects in images using YOLOv10
• Segment objects in images using YOLOv8
• Segment entire images using Ultralytics SAM
• Estimate human poses in images using YOLOv8
• Support for local and network image inputs
• MCP tool integration for client interactions
• Stdio and SSE transport protocols

Note: The server requires valid image paths or URLs and access to the following model files: yolov10b.pt (YOLOv10 detection), yolov8n-seg.pt (YOLOv8 segmentation), yolov8n-pose.pt (YOLOv8 pose estimation), and sam_b.pt (Ultralytics SAM).

TODO

• 3D Detection
• AIGC(GAN, Diffusion)
• Denso Estimation
• Deploy DL(Deep Learning) Models

QucikStart

Install Dependencies

uv sync
//如需要清华源
uv sync --index https://pypi.tuna.tsinghua.edu.cn/simple --extra-index-url https://pypi.org/simple

uv pip install -r requirements.txt -i https://pypi.tuna.tsinghua.edu.cn/simple

Start Server

1. stdio 模式：

   python server.py
   

   输出：

   使用 stdio 传输启动 MCP 服务器（YOLO）
   

2. SSE 模式：

   python server.py sse [端口号]
   

   示例：

   python server.py sse 8080
   

   输出：

   在端口 8080 上启动 MCP 服务器（YOLO），使用 SSE 传输
   

Moreover, users need to download the weights into the ./checkpoints directory. Downloads Links🔗：https://docs.ultralytics.com/models/yolov10/，https://docs.ultralytics.com/models/yolov8/，https://docs.ultralytics.com/models/sam-2/

├── checkpoints │ ├── sam_b.pt │ ├── yolov10b.pt │ ├── yolov8n-pose.pt │ └── yolov8n-seg.pt

API

Resources

• image://system: Image processing operations interface

Tools

• detect_objects
  • Detect objects in an image using YOLOv10
  • Input: image_url (string)
  • Supports local paths (file:// or relative) and network URLs (http:// or https://)
  • Returns JSON array of detected objects with bounding boxes, confidence scores, and class labels
  • Example output: [{"box": [x, y, w, h], "confidence": 0.9, "class": "person"}, ...]
• segment_objects
  • Segment objects in an image using YOLOv8
  • Input: image_url (string)
  • Supports local paths (file:// or relative) and network URLs (http:// or https://)
  • Returns JSON array of segmented objects with bounding boxes, confidence scores, and class labels
  • Example output: [{"box": [x, y, w, h], "confidence": 0.85, "class": "car"}, ...]
• segment_image
  • Segment entire image using Ultralytics SAM
  • Input: image_url (string)
  • Supports local paths (file:// or relative) and network URLs (http:// or https://)
  • Returns JSON array of segmented regions with bounding boxes, areas, and confidence scores
  • Example output: [{"bbox": [x, y, w, h], "area": 2500, "confidence": 0.95}, ...]
• estimate_pose
  • Estimate human poses in an image using YOLOv8
  • Input: image_url (string)
  • Supports local paths (file:// or relative) and network URLs (http:// or https://)
  • Returns JSON array of detected poses with keypoint coordinates and confidence scores
  • Example output: [{"keypoints": [[x1, y1], [x2, y2], ...], "confidence": [0.9, 0.8, ...]}, ...]

Usage with Claude Desktop

Add this to your claude_desktop_config.json:

Note: You can provide sandboxed directories to the server by mounting them to /projects. Adding the ro flag will make the directory readonly by the server.

SSE

{
  "mcpServers": {
    "server-with-yolo": {
      "url": "http://localhost:8080/sse"
    }
  }
}