ComputerMate

"Your AI's hands and eyes on any machine."
(The forbidden MCP of the AI era...)

Test Status npm version npm downloads License: MIT Platform

ComputerMate is a powerful Model Context Protocol (MCP) server that grants AI models the ability to interact with your computer just like a human would. From taking screenshots to clicking buttons and typing text, ComputerMate exposes your local machine (or a sandboxed Docker container) as a set of tools for LLMs.

---

✨ Features & Highlights

• 🖥️ Cross-Platform: Unified support for Linux, macOS, and Windows via a single native mode.
• 🐳 Docker-First: Run in a fully isolated Linux desktop environment with VNC access and state persistence.
• 🖱️ Virtual Cursor: Enable a visual mouse cursor in Playwright by setting VIRTUAL_CURSOR=true.
• 🌐 Web Automation: Integrated Playwright support for high-performance, browser-only computer use.
• 📐 Smart Scaling: Automatically scale screenshots and coordinates to fit LLM context limits via MAX_SCALING_DIMENSION.

---

🛠️ Available Tools

ComputerMate exposes the following tools to the LLM:

Tool	AI-Friendly Description
screenshot	Take a full screenshot of the current screen or browser viewport.
screenshot_region	Capture a specific rectangular area by providing two diagonal points.
click	Move pointer and click (left, middle, right supported).
double_click	Rapidly click twice at the given coordinates.
scroll	Scroll the window content at (x, y) by given amount.
type	Send keyboard text input to the active window.
keypress	Send key combinations (e.g. ["ctrl", "c"], ["alt", "tab"]).
move	Move the mouse pointer without clicking.
drag	Drag the mouse from start point along a path of coordinates.
wait	Pause execution for a set number of milliseconds.
get_dimensions	Retrieve the screen or viewport width and height.
get_environment	Returns the current platform (linux, macos, windows, browser).
goto	(Playwright only) Navigate to a specific URL.
back	(Playwright only) Go back in history.
forward	(Playwright only) Go forward in history.
get_current_url	(Playwright only) Retrieve the current active page URL.

---

🚀 Quick Start

📦 Using npx (Local)

Run the server directly without installing:

# For local OS interaction (macOS, Windows, Linux)
npx @one710/computermate native

# For Playwright (Browser only)
npx @one710/computermate playwright

🐳 Using Docker (Safe & Persistent)

The recommended way to use ComputerMate is via Docker. This provides a sandboxed environment and VNC access.

# Clone the repository
git clone https://github.com/one710/computermate.git
cd computermate

# Start the server with persistence
docker-compose up --build

• MCP Endpoint: http://localhost:3000
• VNC View: localhost:5900 (Password: one710)

---

🛠️ Prerequisites & Installation

🐧 Linux (Ubuntu / Debian Desktop)

sudo apt-get update
sudo apt-get install -y libxtst-dev libpng-dev xvfb xdotool

🍎 macOS

Ensure you have granted Accessibility permissions to your terminal or IDE (e.g., Cursor, VS Code, iTerm2) in System Settings > Privacy & Security > Accessibility.

🪟 Windows

No external binaries are required!

---

⚙️ Configuration

Environment Variables

Variable	Purpose	Default
COMPUTER_TYPE	(Docker / HTTP) Sets the computer backend: native or playwright.	native
MAX_SCALING_DIMENSION	Caps the max width or height of screenshots (e.g., 1024x768). Scales coordinates.	None
VIRTUAL_CURSOR	(Playwright) Shows a visual red dot where the "mouse" is.	false
HEADLESS	(Playwright) Runs the browser in headless mode.	false

---

🏗️ Local Development

To run from source:

# 1. Install dependencies
npm install

# 2. Build the project
npm run build

# 3. Run the server
node dist/index.js native

---

⚠️ Important Warnings

[!CAUTION] This tool grants significant control over your system. An LLM with access to ComputerMate can read your files, access your browser, and execute commands as the current user.

• Use with Caution: Do not run this on a machine with sensitive data or production access.
• Sandbox Recommended: Use the Docker environment for maximum safety.
• Monitor closely: Always keep an eye on what the AI is doing.

---

🛠️ MCP Configuration

Add this to your claude_desktop_config.json (or equivalent MCP client config):

{
  "mcpServers": {
    "computermate": {
      "command": "npx",
      "args": ["-y", "@one710/computermate", "native"],
      "env": {
        "MAX_SCALING_DIMENSION": "1024x768"
      }
    }
  }
}

🌉 Stdio-to-Docker Bridge (mcp-remote)

If your MCP client (like Cursor or older versions of Claude Desktop) only supports stdio transports, you can bridge it to the ComputerMate Docker container:

{
  "mcpServers": {
    "computermate": {
      "command": "npx",
      "args": [
        "-y",
        "mcp-remote@latest",
        "http://localhost:3000/mcp",
        "--transport",
        "http-only"
      ]
    }
  }
}

This command starts a local stdio server that transparently forwards all requests to the ComputerMate container running at http://localhost:3000.

---

📄 License

This project is licensed under the MIT License. See the LICENSE file for details.

---

Built with ❤️ by the One710 Softworks Team.