Petp

中文 | English

License: MIT

Python RPA toolkit with 80+ processors orchestrating browser automation, AI/LLM (10 providers), databases, SSH, email, and HTTP tasks. Configurable pipelines with cron scheduling and loops. Runs as wxPython GUI, headless service, or Docker container. Built-in MCP Tool Server (Streamable-HTTP) for AI agent integration.

Pipeline  1:n  Execution
Execution 1:n  Task
Task      1:1  Processor

Links: Web Intro | Web App | Changelog

Quick Start

1. Install Python 3.14

Download from python.org. On Windows, check "Add Python to PATH".

2. Install wxPython (GUI only)

# macOS Apple Silicon
uv pip install wxPython-4.3.0a1XXXX-cp314-cp314-macosx_11_0_arm64.whl

# Windows 64-bit
uv pip install wxPython-4.3.0a1XXXX-cp314-cp314-win_amd64.whl

python PETP_background.py --run-execution OOTB_DOWNLOAD_LATEST_WXPYTHON_mac_arm

For Python 3.12/3.13: pip install wxPython

3. Install Dependencies

pip install -U uv
uv pip install -r requirements.txt        # Full (GUI)
# or: uv pip install -r requirements-nogui.txt   # Headless
# or: uv pip install -r requirements-docker.txt  # Docker

uv pip install -r requirements/core.txt -r requirements/ssh-sftp.txt -r requirements/http-client.txt

4. Run

python PETP.py          # GUI
python PETP_background.py   # Headless service (port 8866)

Screenshots

macOS

Windows

MCP Tool Server — integrate with Claude Code, Cursor, and other AI agents:

Features

🤖 AI Execution Generator

Highlight — Generate and modify PETP task flows through natural language conversation with LLM. Supports 10 LLM providers including Hyperspace, Anthropic, DeepSeek, Zhipu, Gemini, Ollama, and more.

Entry Points:

• Create Execution → "AI Generate" template
• Right-click taskGrid → "AI Assist" (modify existing) — pre-selects only the processors used in the current Execution to minimize token usage
• MCP Editor → "AI" button (auto-generate tool description)

Highlights:

• Multi-turn chat — ask questions, generate flows, modify tasks incrementally
• Processor browser — expandable TreeListCtrl with full documentation, search & filter
• Selective context — only checked Processors are sent to LLM (saves tokens)
• Connection caching — first-time validation, then instant reuse across sessions
• 10 LLM providers — minimal config: just set ai_provider in petpconfig.yaml
• 429 rate-limit defense — exponential backoff (2s / 4s / 8s), UI throttle ≥ 3s, per-call token accounting log

Token controls (new):

• ai_max_response_tokens — hard upper bound on model output length (default 8192)
• ai_max_request_tokens — local pre-flight cap on request size (default 60000); over-limit prompts are rejected before hitting the provider

AI-Powered MCP Tool Publishing:

• One-click generation of mcp_desc JSON for exposing Executions as MCP tools
• Auto-extracts input parameters from INITIAL_PARAMS and output keys from result tasks
• Generates AI-agent-friendly descriptions that help LLMs understand when to call the tool
• Smart merge — new fields are added without overwriting existing configuration
• Progress dialog with live status and preview before applying

AI Error Analysis & Auto-Fix:

• On execution failure, AI automatically analyzes the error with full context (failed task, surrounding tasks, traceback)
• Pinpoints root cause and suggests specific fixes
• One-click "Open AI Assist" pre-fills the diagnosis — continue fixing in multi-turn chat

Vision Model Support (Ollama):

• AI_LLM_QANDA accepts image_path parameter for multimodal prompts
• Works with Ollama vision models (gemma4, llava, moondream, etc.)
• Image path supports expressions — dynamically reference files from previous tasks in data_chain

Configuration (only ai_provider required, rest auto-fills from provider defaults):

application:
  ai_provider: zhipu              # or: deepseek, anthropic, hyperspace, gemini, ollama, etc.
  ai_model: ""                    # empty = provider default (e.g. GLM-5)
  ai_api_key: ""                  # empty = read from default env var (e.g. ZHIPU_ACCESS_KEY)
  ai_base_url: ""                 # empty = provider default URL
  ai_max_response_tokens: 8192
  ai_max_request_tokens: 60000

See Configuration Docs for full provider list and details.

Dynamic Function (_fn) & Expression Enhancements

All dynamic function parameters (_fn, _func, _func_body, lambda_*) receive the Processor instance as p, enabling full access to PETP utilities inside custom code:

# In _fn function body — p is the Processor instance
result = p.get_data("my_key")
p.populate_data("output", processed_value)
today = p.str_to_date("2026-05-11")

Available p methods in both expression and _fn contexts:

Edit Complex Value — Handy Tool button:

• Right-click any property → "Edit Complex Value" includes a Handy Tool button
• Automatically detects whether the parameter is an expression or _fn function body
• Expression context: snippets wrapped in {p.xxx()} for f-string evaluation
• Function body context: raw p.xxx() calls plus import statements

Running Modes

# Run one execution and exit
python PETP_background.py --run-execution ENDECODER --no-http

# Run execution with initial data
python PETP_background.py --run-execution MY_EXEC --init-data '{"key":"value"}' --no-http

# Run pipeline and exit
python PETP_background.py --run-pipeline DAILY_REPORT --no-http

# Run pipeline with initial data
python PETP_background.py --run-pipeline MY_PIPELINE --init-data '{"param":"value"}' --no-http

# Start HTTP/MCP service (default port 8866)
python PETP_background.py

# Custom port and auth token
python PETP_background.py --http-port 9090 --http-token my-secret-token

# Headless Selenium (Chrome without visible window)
python PETP_background.py --run-execution BROWSER_TASK --headless --no-http

# Override log level
python PETP_background.py --log-level DEBUG

# GUI-processor policy: skip (ignore) or abort (fail on GUI tasks)
python PETP_background.py --run-execution HAS_GUI_TASK --ui-policy skip --no-http
python PETP_background.py --run-execution HAS_GUI_TASK --ui-policy abort --no-http

# Stop a running background instance
python PETP_background.py --stop

Notes:

• --run-execution and --run-pipeline can be combined — both will run in sequence
• Pipeline reentrant protection: if the same pipeline is already running, a second call returns {"ok": false, "error": "Pipeline already running"}
• Cron-enabled pipelines (cronEnabled: true in YAML) auto-register as scheduled jobs instead of running immediately

Helper scripts for long-running sessions: see scripts/macos/start_petp.sh and scripts/windows/start_petp.ps1.

Portable Runtime

portable/ is a self-contained unit that runs a single Execution headlessly, decoupled from the GUI and HTTP layers. Copy portable/ into any Python project, or cf push it to Cloud Foundry.

from portable.petp_run import run
result = run("MY_EXECUTION", {"key": "value"})   # {"ok", "data", "error", "meta"}

Workflow: author the Execution in the desktop GUI → Ctrl+S → copy the YAML into portable/core/executions/ → python portable/petp_run.py NAME. The engine and YAML format are shared, so GUI-authored Executions run unchanged.

Chrome binaries (chrome-headless-shell + chromedriver, from Chrome for Testing) are placed under portable/webdriver/<system>/ and are not committed to git. On Cloud Foundry, manifest.yml + apt.yml (apt-buildpack) supply the required system .so libraries. See portable/README.md for full details.

MCP & AI Integration

PETP exposes executions as MCP tools via Streamable-HTTP on port 8866.

🔒 Auth is fail-closed. When http_request_token is unset in petpconfig.yaml, every protected endpoint (/petp/*, /mcp) returns 501 Not Configured. Set a token before exposing the server (e.g. via Tailscale Funnel). Send it as Authorization: Bearer <token> on every request.

🔒 Security hardening (Phase 2).

• CMD processor: defaults to shlex.split (no shell). Set shell="yes" only for trusted commands needing pipes/redirects.
• Dynamic _fn / lambda_* parameters: run in a sandbox with __import__, open, eval, exec, compile, getattr, hasattr removed. Whitelisted modules: re, json, datetime, math.
• Encrypted password salt: override the public default by setting env PETP_SALT or writing ~/.petp/secret (POSIX mode 0600). The default salt is logged with a WARNING — cryptocode ciphertext is not actually secret unless you set a custom salt.
• Path traversal guard (opt-in): set PETP_PATH_ALLOW_ROOTS=/path1:/path2 to confine all file IO processors (READ_*, WRITE_*, OPEN_FILE, FILE_DELETE, UNZIP) to a whitelist of root directories. Default off — preserves existing yaml using absolute paths.
• Request size limits: HTTP body capped at 4 MiB (PETP_MAX_BODY_BYTES); JSON-RPC batch arrays capped at 64 items (PETP_MAX_BATCH_ITEMS). Both return 413 / 400 with no body parsing.
• Log redaction (default on): values of sensitive keys (api_key, password, token, authorization, secret, ...) are masked as ***REDACTED*** in process start / [Type] input log lines. Disable with PETP_LOG_REDACT=off for ad-hoc debugging.

Performance (headless/Docker):

• Shared thread pool for concurrent tool calls (no per-request executor overhead)
• Static execution cache — zero filesystem I/O after startup (no stat/mtime checks)
• Processor class pre-loading on server start (eliminates cold-start latency)
• Real-time task-level SSE progress notifications during long-running tools/call
• Cached outputSchema parsing (same tool re-called without re-parsing mcp_desc JSON)

Built-in MCP Tools:

Claude Code / Cursor / any MCP client:

{
  "mcpServers": {
    "petp": {
      "type": "http",
      "url": "http://localhost:8866/mcp"
    }
  }
}

HTTP API (no MCP client needed):

# List tools
curl -H "Authorization: Bearer $TOKEN" http://localhost:8866/petp/tools

# Trigger execution
curl -X POST http://localhost:8866/petp/exec \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"action":"execution","params":{"execution":"MY_EXEC"},"wait_for_result":"true"}'

# Trigger pipeline (sync)
curl -X POST http://localhost:8866/petp/exec \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"action":"pipeline","params":{"pipeline":"MY_PIPELINE"},"wait_for_result":"true"}'

# Trigger pipeline (async — poll with /petp/result)
curl -X POST http://localhost:8866/petp/exec \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"action":"pipeline","params":{"pipeline":"MY_PIPELINE"},"wait_for_result":"false"}'

Build & Docker

# Standalone executable
python build/PETP_build.py   # → dist/PETP.app or dist/PETP.exe

# Docker — Background service (headless, port 8866)
./build/script/docker_build_bg.sh              # build + export tar
./build/script/docker_build_bg.sh --run        # build + start container
./build/script/docker_build_bg.sh --no-tar     # build only
./build/script/docker_build_bg.sh --push repo:tag  # push to registry
./build/script/docker_build_bg.sh --dirty      # use working dir (skip git archive)

# Docker — Web App (port 5555)
./build/script/docker_build_webapp.sh          # build + export tar
./build/script/docker_build_webapp.sh --run    # build + start container
./build/script/docker_build_webapp.sh --dirty  # use working dir (skip git archive)

Note: Build scripts default to git archive mode — only git-tracked and staged files are included in the Docker context. Use --dirty to include all working directory files (relies on .dockerignore).

Deploy to NAS

# Deploy BG image (scp + docker load + start container)
./build/script/deploy_bg_to_nas.sh
./build/script/deploy_bg_to_nas.sh --no-start   # only transfer + load
./build/script/deploy_bg_to_nas.sh --keep-tar   # don't delete remote tar

# Deploy Webapp image
./build/script/deploy_webapp_to_nas.sh

# Override NAS connection / port mapping
NAS_HOST=10.0.0.5 NAS_USER=myuser HOST_PORT=9090 ./build/script/deploy_webapp_to_nas.sh

Tailscale Funnel (run on NAS)

# Refresh funnel routes (reset + reconfigure)
sudo ./build/script/tailscale_funnel_refresh.sh

# Check current status only
./build/script/tailscale_funnel_refresh.sh --status

Routes configured:

• / → localhost:8088 (webapp)
• /mcp → localhost:8866 (petp background)

Project Structure

Troubleshooting

Documentation

Acknowledgements

• wxPython & wxGlade
• Selenium & ChromeDriver

Full Changelog →