mac-pilot-mcp

Self-learning macOS automation for AI agents

The only macOS MCP server that remembers. Save what works, skip what doesn't.

The Problem

There are 7+ macOS automation MCP servers. None of them remember anything. Every session starts from zero — same trial and error, same failures, same wasted tokens.

The Solution

Mac-Pilot is different. It learns from every interaction:

• Success? The pattern is auto-saved as app-specific knowledge
• Failure? The error is recorded so it won't repeat the same mistake
• Multi-step workflow? Save it as a recipe — replay it in one call next time

Ships with 21 built-in recipes so you're productive from the first run.

Quick Start

Install

npm install -g mac-pilot-mcp

Connect to your AI client

claude mcp add mac-pilot -- mac-pilot-mcp

{
  "mcpServers": {
    "mac-pilot": {
      "command": "mac-pilot-mcp"
    }
  }
}

{
  "mcpServers": {
    "mac-pilot": {
      "command": "npx",
      "args": ["-y", "mac-pilot-mcp"]
    }
  }
}

{
  "mcpServers": {
    "mac-pilot": {
      "command": "npx",
      "args": ["-y", "mac-pilot-mcp"]
    }
  }
}

{
  "mcpServers": {
    "mac-pilot": {
      "command": "npx",
      "args": ["-y", "mac-pilot-mcp"]
    }
  }
}

Grant Permissions

Mac-Pilot needs Accessibility access for UI automation:

1. System Settings → Privacy & Security → Accessibility
2. Toggle ON for your terminal app (Terminal, iTerm2, VS Code, Cursor, etc.)
3. Restart the terminal

How It Works

You: "Export the current Figma frame as PNG"

┌─ First time ─────────────────────────────────────────┐
│                                                       │
│  1. mac_recipe_search("Figma export") → No matches   │
│  2. mac_state() → Figma is frontmost                 │
│  3. mac_run(applescript) → File > Export > PNG        │
│  4. mac_recipe_save("export-figma-png", steps=[...]) │
│                                                       │
│  ✓ Worked! Pattern saved automatically.              │
└───────────────────────────────────────────────────────┘

┌─ Next time ───────────────────────────────────────────┐
│                                                       │
│  1. mac_recipe_search("Figma export")                │
│     → Found: export-figma-png (100% success rate)    │
│  2. mac_recipe_run("export-figma-png")               │
│     → Done instantly                                  │
│                                                       │
│  ⚡ 4 steps → 2 steps. No trial and error.           │
└───────────────────────────────────────────────────────┘

Tools

Built-in Recipes

21 recipes ship out of the box — no setup needed:

Examples

AppleScript

{ "actionType": "applescript", "script": "tell application \"Finder\" to get name of every window" }

JXA (JavaScript for Automation)

{ "actionType": "jxa", "script": "Application('Safari').documents[0].url()" }

Shell command

{ "actionType": "shell", "command": "ls -la ~/Desktop" }

Open an app or URL

{ "actionType": "open", "target": "Safari" }
{ "actionType": "open", "target": "https://github.com" }

Type text

{ "actionType": "type", "text": "Hello World" }

Keyboard shortcut

{ "actionType": "keypress", "text": "cmd+c" }
{ "actionType": "keypress", "text": "cmd+shift+4" }

Find UI elements

mac_find_ui { "app": "Safari", "role": "AXButton" }
mac_find_ui { "app": "Finder", "searchText": "Downloads" }

Electron apps (VSCode / Cursor / Slack / Discord) — macOS Accessibility exposes a thin tree for Electron, so Mac-Pilot can optionally fall back to Chrome DevTools Protocol when the app is launched with --remote-debugging-port=<PORT>. See docs/ELECTRON-SUPPORT.md.

mac_find_ui {
  "app": "Visual Studio Code",
  "searchText": "Run Test",
  "useElectronFallback": "auto"
}

Take a screenshot

mac_screenshot { "target": "screen", "scale": 0.3 }
mac_screenshot { "target": "window", "windowName": "Safari" }

Save a custom recipe

mac_recipe_save {
  "name": "open-project",
  "description": "Open VS Code at project directory",
  "steps": [
    { "actionType": "shell", "params": { "command": "code {{path}}" }, "description": "Open VS Code" }
  ],
  "parameters": [
    { "name": "path", "description": "Project directory path" }
  ],
  "tags": ["dev", "vscode"]
}

Security

Mac-Pilot blocks dangerous operations before they execute:

Architecture

~/.mac-pilot/pilot.db (SQLite, WAL mode)
├── action_log       — Every action executed, with timing + success/failure
├── action_log_fts   — Full-text search index over action history
├── recipes          — Saved automation sequences
├── recipes_fts      — Full-text search index over recipes
├── app_knowledge    — Per-app quirks, selectors, workarounds (auto-learned)
└── security_log     — Blocked command audit trail

Built-in recipes are auto-loaded on first run. Your custom recipes and learned knowledge persist across sessions.

Comparison

Troubleshooting

mac_run { "actionType": "open", "target": "AppName" }

mac_screenshot { "target": "screen", "scale": 0.3 }

mac_recipe_search { "query": "your keyword" }

mac_run { "actionType": "shell", "command": "your-command", "dryRun": true }

Requirements

• macOS (darwin only)
• Node.js >= 18
• Accessibility permission for UI automation

Contributing

Issues and PRs welcome at github.com/leesgit/mac-pilot-mcp.

git clone https://github.com/leesgit/mac-pilot-mcp.git
cd mac-pilot-mcp
npm install
npm run build
npm test        # 144 tests

License

MIT - Byeongchang Lee