Streamfog
БесплатноНе проверенAI-driven AR lens orchestrator for live OBS streams that enables control of Streamfog face filters, AR effects, and Vtuber avatars through MCP tools via the loc
Описание
AI-driven AR lens orchestrator for live OBS streams that enables control of Streamfog face filters, AR effects, and Vtuber avatars through MCP tools via the local Streamer.bot WebSocket bridge.
README
📖 Installation Guide — quick start, manual setup, and troubleshooting 📖 Docs — Configuration · Tools · Troubleshooting · Development 🤖 llms-full.txt — full LLM-readable documentation
AI-driven AR lens orchestrator for live OBS streams. Control Streamfog face filters, AR effects, and Vtuber avatars through MCP tools via the local Streamer.bot WebSocket bridge. Your AI assistant becomes a stream producer.
What is Streamfog?
Streamfog is a Windows desktop app (https://streamfog.com) that streamers use to put AR effects on their face during live broadcasts — face filters ("beauty smooth", "cyber helmet"), animated lenses, background effects, and Vtuber-style avatars. It takes your webcam feed, applies the effect in real time, and renders the result into OBS Studio through a browser source.
Two things matter for automation:
- Streamfog has no public API, CLI, or plugin SDK. You cannot script it directly, and it cannot tell you which lens is active. It is effectively a black box that only listens to Streamer.bot.
- Streamer.bot is the remote control. Streamer.bot
is a free automation tool for streamers. Streamfog registers itself with
Streamer.bot as an integration, and Streamer.bot exposes a local
WebSocket server that accepts
DoActioncommands by name (e.g. fire the actionSetLens_BeautySmooth).
This project is the bridge between the two: it gives an AI assistant (or
the dashboard, or any script) a clean tool interface — streamfog_set_lens,
streamfog_clear_effects, streamfog_toggle_avatar, streamfog_status,
streamfog_list_lenses — and translates each call into a Streamer.bot
DoAction dispatch. Without this server you would have to open Streamer.bot
and click actions manually; with it, your agent becomes a stream producer.
⚠️ Fire-and-forget: Streamer.bot does not report whether an action succeeded, and Streamfog cannot be queried. A tool returning "success" means dispatched, not applied — confirm visually on the stream.
| You might use this if… | You want your AI to switch AR lenses, toggle Vtuber avatars, or clear effects during live OBS broadcasts — controlled by Twitch chat events, channel points, or agentic automation. |
| What it connects to | Streamfog desktop app → Streamer.bot WebSocket → this MCP server |
| Ports | Backend 10994, Dashboard 10995 |
| Start | just bootstrap then start.ps1 |
How to use this server (3 steps)
- Install the two external apps (one-time): Streamfog + Streamer.bot, enable Streamfog's integration inside Streamer.bot, and enable Streamer.bot's WebSocket server on port 8080. Full walkthrough: docs/ONBOARDING.md.
- Map your lenses: create
lenses.jsonlinking human-readable lens names to the exact Streamer.bot action names that apply them (see Lens Map). - Start and drive it:
start.ps1boots the backend + dashboard; then either connect your MCP client (Claude Desktop / Cursor / opencode) over stdio orhttp://127.0.0.1:10994/mcp, click lenses on the dashboard, or call the REST API from scripts. All three surface the same bridge.
Architecture
Plain English: your AI talks to this server, this server talks to Streamer.bot, Streamer.bot tells Streamfog what to render, and OBS shows the result. Every hop is local.
┌─────────────┐ MCP ┌──────────────────┐ WebSocket ┌──────────────┐
│ LLM Agent │ ───────────────→ │ streamfog-mcp │ ────────────────→ │ Streamer.bot │
│ (Claude, │ ←─────────────── │ :10994 (FastMCP) │ ←──────────────── │ :8080 │
│ Gemini) │ stdio / /mcp │ :10995 (React) │ DoAction JSON │ │
└─────────────┘ └──────────────────┘ └──────┬────────┘
│ Native Hook
┌──────▼────────┐
│ Streamfog │
│ Desktop App │
└──────┬────────┘
│ Browser Source
┌──────▼────────┐
│ OBS Studio │
└───────────────┘
- MCP tools (for AI agents): served over stdio or the streamable HTTP
endpoint at
http://127.0.0.1:10994/mcp. - REST API (for scripts + dashboard):
/api/*on the same port, backed by the same bridge instance. - Dashboard (for humans): React SPA at
:10995— lens grid, status KPIs, chat, logs.
The MCP endpoint is served at http://127.0.0.1:10994/mcp (streamable HTTP)
in dual mode, alongside the REST API and dashboard.
Quick Start
uv sync
# Edit lenses.json with your Streamer.bot action names
# Set STREAMFOG_MCP_STREAMERBOT_TOKEN in .env if using auth
.\start.ps1
MCP-only via stdio (for Cursor, Claude Desktop):
uv run -m streamfog_mcp --stdio
Prerequisites
- Streamfog installed and running
- Streamer.bot installed and running
- Streamfog → Streamer.bot integration enabled in Streamfog's Integrations panel
- Streamer.bot WebSocket server enabled (Settings → WebSocket Server)
- Actions created in Streamer.bot (e.g.
SetLens_BeautySmooth,ClearEffects,ToggleAvatar) lenses.jsonpopulated with your action→lens mappings
See docs/ONBOARDING.md for the step-by-step walkthrough.
Configuration
| Variable | Default | Description |
|---|---|---|
STREAMFOG_MCP_STREAMERBOT_HOST |
127.0.0.1 |
Streamer.bot WebSocket host |
STREAMFOG_MCP_STREAMERBOT_PORT |
8080 |
Streamer.bot WebSocket port |
STREAMFOG_MCP_STREAMERBOT_TOKEN |
— | Streamer.bot auth token |
STREAMFOG_MCP_LENS_MAP_PATH |
lenses.json |
Path to lens→action mapping file |
STREAMFOG_MCP_PORT |
10994 |
Backend port |
STREAMFOG_MCP_LOG_LEVEL |
INFO |
Logging level |
Lens Map (lenses.json)
{
"beauty_smooth": "SetLens_BeautySmooth",
"cyber_helmet": "SetLens_CyberHelmet",
"vtuber_avatar": "SetLens_VTuberAvatar"
}
Keys are human-readable lens identifiers used in MCP tool calls. Values are the corresponding Streamer.bot action names. Keys starting with _ are ignored; unknown identifiers fall back to the action SetLens_{identifier}.
MCP Tools (5)
Lens Control
| Tool | Description |
|---|---|
streamfog_set_lens |
Activate a specific AR lens or face filter |
streamfog_clear_effects |
Strip all effects, return camera to baseline |
streamfog_toggle_avatar |
Toggle Vtuber-style avatar on/off |
Discovery — READ_ONLY
| Tool | Description |
|---|---|
streamfog_list_lenses |
List all configured lenses from lenses.json |
streamfog_status |
Bridge connection health + lens count |
REST API
| Endpoint | Method | Description |
|---|---|---|
/api/health |
GET | Liveness probe (status, version, uptime, tool count, bridge) |
/api/v1/status |
GET | Server + bridge health |
/api/v1/lenses |
GET | List all lenses |
/api/v1/lenses/set |
POST | Activate a lens ({"lens_identifier": "beauty_smooth"}) |
/api/v1/lenses/reload |
POST | Reload lens map from disk |
/api/v1/effects/clear |
POST | Clear all effects |
/api/v1/avatar/toggle |
POST | Toggle avatar |
/api/tools |
GET | Dynamic MCP tool list with schemas |
/api/capabilities |
GET | Capability discovery |
/api/skills |
GET | Skill list |
/api/skills/{name} |
GET | Skill content (markdown) |
/api/llm/discover |
GET | Local LLM provider probe (Ollama/LM Studio/vLLM) + GPU |
/api/logs |
GET | Ring-buffer log window |
/api/v1/diagnostics |
GET | Tool list, system info, errors |
Web Dashboard
Dark SOTA dashboard at :10995 (React + Vite + Tailwind + Zustand):
- Dashboard — hero + KPI cards (server, tools, bridge, uptime), lens grid, quick actions, exponential-backoff health
- Tools — dynamic tool list with schemas from
/api/tools - Chat — skill-first local LLM chat (Ollama/LM Studio/vLLM), 4 personalities, localStorage history, export
- Settings — backend health + LLM provider detection
- Help — architecture, env reference, troubleshooting
- Logs — live ring-buffer modal (Ctrl+L)
Keyboard: Ctrl+Scroll zoom, Ctrl+0 reset, Ctrl+L logs.
Project Structure
streamfog-mcp/
├── src/streamfog_mcp/
│ ├── _mcp.py FastMCP singleton + resources
│ ├── server.py FastAPI gateway (REST + /mcp mount)
│ ├── __main__.py CLI entry (--stdio / --serve)
│ ├── config.py Pydantic settings (STREAMFOG_MCP_ prefix)
│ ├── errors.py _error_response() with logger.exception
│ ├── logging_ring.py Ring buffer for /api/logs
│ ├── skills/streamfog/ SKILL.md (chat preprompt)
│ ├── tools/core_tools.py 5 @mcp.tool() decorators
│ └── services/streamerbot.py Streamer.bot WebSocket client
├── webapp/ Vite + React 19 + Tailwind + Zustand
├── native/ Tauri 2.0 desktop wrapper
├── lenses.json Lens → action mapping
├── pyproject.toml / uv.lock
├── start.ps1 / start.bat
├── justfile
└── tests/ 13 tests (units + endpoint + e2e)
Verification
just lint # ruff
just test # pytest (coverage-gated)
uv run pyright src/ # types
cd webapp && bun run check && bun run biome:ci
cd webapp && bunx playwright test # e2e
Known Limitations
- Streamfog does not expose a native CLI or local API — all control goes through Streamer.bot
- Lens activation is fire-and-forget (Streamer.bot does not report success/failure for actions)
- No lens preview or thumbnail retrieval (Streamfog desktop is a black box)
- Lumia/Crowd Control bridge path is documented but not yet implemented as an alternative transport
Установить Streamfog в Claude Desktop, Claude Code, Cursor
unyly install streamfog-mcpСтавит в Claude Desktop, Claude Code, Cursor и VS Code — сам разбирается с npx, uvx и сборкой из исходников.
Впервые? Поставь CLI: curl -fsSL https://unyly.org/install | sh
Или настроить вручную
Выполни в терминале:
claude mcp add streamfog-mcp -- uvx --from git+https://github.com/sandraschi/streamfog-mcp streamfog-mcpПошаговые гайды: как установить Streamfog
FAQ
Streamfog MCP бесплатный?
Да, Streamfog MCP бесплатный — установка в пару кликов через Unyly без оплаты.
Нужен ли API-ключ для Streamfog?
Нет, Streamfog работает без API-ключей и переменных окружения.
Streamfog — hosted или self-hosted?
Self-hosted: сервер запускается локально на твоей машине командой из раздела установки.
Как установить Streamfog в Claude Desktop, Claude Code или Cursor?
Открой Streamfog на unyly.org, выбери вкладку своего клиента (Claude Desktop, Claude Code, Cursor) и нажми Install — конфиг сгенерируется автоматически, без правки JSON.
Похожие MCP
Opencode Omniroute Plugin
OpenCode plugin for the OmniRoute AI Gateway. Drives dynamic model discovery, /connect auth flow, and multi-instance OmniRoute providers via the official @openc
автор: GitHub ActionsRoblox Studio
Enables AI coding tools to control Roblox Studio for workspace exploration, instance manipulation, and script management. It provides tools for playtesting, sce
автор: paralovAntigravity Drawio
MCP server that enables AI assistants to create, parse, render, and validate Draw.io diagrams programmatically.
автор: S-SUJAN-SFetch
Web content fetching and conversion for efficient LLM usage.
AWS KB Retrieval
Retrieval from AWS Knowledge Base using Bedrock Agent Runtime.
автор: modelcontextprotocolSpring AI MCP Server
Provides auto-configuration for setting up an MCP server in Spring Boot applications.
llm-analysis-assistant
A very streamlined mcp client that supports calling and monitoring stdio/sse/streamableHttp, and can also view request responses through the /logs page. It also
автор: xuzexin-hzMCP-Agent
A simple, composable framework to build agents using Model Context Protocol by [LastMile AI](https://www.lastmileai.dev)
автор: lastmile-aiSpring AI MCP Client
Provides auto-configuration for MCP client functionality in Spring Boot applications.
mcp.natoma.ai
A Hosted MCP Platform to discover, install, manage and deploy MCP servers by [Natoma Labs](https://www.natoma.ai)
Compare Streamfog with
Не уверен что выбрать?
Найди свой стек за 60 секунд
Автор?
Embed-бейдж для README
Похожее
Все в категории ai
