Macinput

Language: English | 中文

macinput is a macOS keyboard, mouse, and screenshot control tool for AI agents. This repository is now structured as an installable Python project and an MCP server so desktop agents can control a macOS GUI through standard MCP tool calls.

The project has two goals:

• Provide stable low-level macOS input and screenshot primitives.
• Provide an MCP server with a practical tool surface, runtime safety limits, and deployment guidance.

Features

• Mouse move, left click, right click, and double click
• Current mouse position lookup
• Key press, key down, key up, and modifier combinations
• Unicode text input
• Clipboard-backed paste input
• Full-screen screenshots with automatic cleanup
• MCP resources and prompt templates for agent guidance

Use cases

• Desktop AI agents controlling macOS applications
• UI automation prototypes
• Human-in-the-loop desktop workflows
• Screenshot-observe plus keyboard/mouse-act agent loops

Requirements

• macOS
• Python 3.10+
• The launching host app must have:
  • Accessibility permission
  • Screen Recording permission

Important: permissions apply to the program that launches the MCP server, not only to Python. If you launch through Claude Desktop, Terminal, iTerm2, Cursor, or VS Code, that host app must be granted permission.

Installation

uv

uv sync

pip

python -m pip install -e .

Run the MCP server

stdio is the recommended default for desktop AI clients:

macinput-mcp

If your MCP host requires HTTP transport:

macinput-mcp --transport streamable-http --host 127.0.0.1 --port 8000 --path /mcp

Example MCP client config

Generic stdio configuration:

{
  "mcpServers": {
    "macinput": {
      "command": "uv",
      "args": [
        "--directory",
        "/path/to/macinput",
        "run",
        "macinput-mcp"
      ]
    }
  }
}

If the package is already installed into the current environment:

{
  "mcpServers": {
    "macinput": {
      "command": "macinput-mcp",
      "args": []
    }
  }
}

Available tools

• get_server_settings
• get_mouse_position
• move_mouse
• click_mouse
• scroll_mouse
• press_keyboard_key
• keyboard_key_down
• keyboard_key_up
• type_text_input
• paste_text_input
• capture_screenshot
• cleanup_screenshot_file

Available resources and prompt

Resources:

• macinput://overview
• macinput://best-practices
• macinput://permissions

Prompt:

• ui_action_protocol(goal, current_context="")

These are part of the product surface, not decoration. They let hosts ship usage guidance together with the server instead of rewriting it in every system prompt.

Recommended usage

For users:

1. Prefer stdio for desktop agent integrations.
2. Verify macOS permissions before the first real run.
3. Prefer a dedicated macOS account, test machine, or VM for automation.
4. Keep screenshot TTLs short to reduce data residue.

For agents:

1. Capture a screenshot before acting.
2. Make one state-changing action at a time.
3. Capture a fresh screenshot after clicks, shortcuts, or text submission.
4. Do not reuse old coordinates after the UI changes.
5. Keep typed text short and task-specific.
6. Clean up screenshots when they are no longer needed.

Environment variables

• MACINPUT_DEFAULT_SCREENSHOT_TTL
  • Default screenshot cleanup timeout in seconds. Default: 30
• MACINPUT_MAX_SCREENSHOT_TTL
  • Maximum allowed screenshot retention in seconds. Default: 300
• MACINPUT_MAX_TYPING_LENGTH
  • Maximum characters per typing action. Default: 2000
• MACINPUT_MIN_ACTION_DELAY
  • Minimum delay after each tool action. Default: 0.05
• MACINPUT_DEFAULT_TYPING_INTERVAL
  • Default per-character typing interval. Default: 0.02

Use as a Python library

from macinput import click, move_to, press_key, type_text, capture_screen

move_to(400, 300)
click()
type_text("hello macOS")
press_key("a", modifiers=["command"])
path = capture_screen(cleanup_after=10)
print(path)

Development

Project layout

src/macinput/
  __init__.py
  __main__.py
  cli.py
  keyboard.py
  mouse.py
  screenshot.py
  server.py
  settings.py
docs/
  mcp-engineering.md
tests/

Local workflow

uv sync --extra dev
uv run pytest
uv run ruff check .

GitHub Actions

• CI workflow: .github/workflows/ci.yml
• Release workflow: .github/workflows/release.yml

The CI workflow runs lint and tests on push and pull_request. The release workflow builds distributions on workflow_dispatch and on version tags such as v0.1.0, then uploads artifacts and publishes to PyPI if your repository is configured for trusted publishing.

Project rules

• Keep low-level automation separate from the MCP server layer.
• Keep MCP tools small and stable.
• Prefer stdio by default.
• Keep state-changing actions explainable and composable.
• Document both user integration and developer maintenance workflows.

Documentation

• Engineering notes: docs/mcp-engineering.md
• User guide: docs/user-guide.md
• Agent best practices: docs/agent-best-practices.md
• Developer guide: docs/developer-guide.md

Limitations

• macOS only
• Requires a real GUI session
• CI can validate imports and configuration, but real machine validation is still necessary for UI injection
• paste_text_input uses the system clipboard and currently preserves/restores plain-text clipboard content only
• press_keyboard_key, keyboard_key_down, and keyboard_key_up accept either string keys like "3" or numeric digit inputs like 3
• Does not include OCR, UI element detection, or semantic window understanding

Recommended follow-up work

1. Add LICENSE.
2. Add release automation.
3. Add smoke tests on a real macOS runner.
4. Add an examples/ directory for common MCP hosts.
5. Add optional region screenshots, output directories, and audit logging.