Avatar
БесплатноНе проверенVRM avatar management and animation server: VRChat OSC, VRoid/Blender pipeline, audio-driven digital humans. FastMCP + web studio.
Описание
VRM avatar management and animation server: VRChat OSC, VRoid/Blender pipeline, audio-driven digital humans. FastMCP + web studio.
README
📖 Installation Guide — quick start, manual setup, and troubleshooting
By FlowEngineer sandraschi
FastMCP 3.1.0+ compatible VRM avatar management and animation server with conversational capabilities, SEP-1577 sampling support, and VRChat OSC integration
License: MIT
Python Version
Code style: ruff
[](https://FastMCP 3.1.0readthedocs.io/)
VRChat OSC
Portmanteau Tools
Quick Start
git clone https://github.com/sandraschi/avatar-mcp
cd avatar-mcp
just
This opens an interactive dashboard showing all available commands. Run just bootstrap to install dependencies, then just serve or just dev to start.
Manual Setup
If you don't have just installed:
Features
Core Features
Portmanteau Tools Architecture: Consolidated 16 portmanteau tools with FastMCP 3.1.0 sampling capabilities
- Exposes only portmanteau tools to MCP (no raw core-tool list); bootstrap via
system_monitor(operation="initialize")
- Exposes only portmanteau tools to MCP (no raw core-tool list); bootstrap via
Creative Pipeline web UI at
/pipeline— Hub download, vroidstudio-mcp, blender-mcp, VTube staging, model_type detection- Docs: docs/CREATIVE_PIPELINE.md | MCD:
mcp-central-docs/docs/avatars/FLEET_VRM_PIPELINE.md - Reduces tool explosion from 28 individual tools to 16 unified interfaces
- Operation-based design with comprehensive multiline docstrings
- Clean, maintainable architecture with focused tool classes
- Docs: docs/CREATIVE_PIPELINE.md | MCD:
FastMCP 3.1.0 Compatible: Fully implements the MCP protocol with sampling capabilities (SEP-1577) for agentic workflows
Agentic Sampling Workflows: LLM-driven orchestration of complex avatar behaviors without manual sequencing
VRM 2.0 Support: Load and manage VRM 2.0 avatar models with real-time manipulation
VRChat OSC Integration: Seamless communication with VRChat for avatar control
Advanced Animation System: Play, blend, and manage animations with support for loops, varying speeds, and real-time recording
Advanced Bone Control: Precise control over avatar bones with local/world space transforms and hierarchical manipulation
Morph Target Control: Fine-grained control over blend shapes and morph targets with batch updates
Unity Desktop Integration: Full Unity desktop avatar control with window management
Chat System: Interactive chatbot functionality with session management
Secondary Features
- MCPB Packaging: Easy deployment and integration with MCP-compatible applications
- Type Annotated: Fully type-annotated code for better development experience
- Modular Design: Clean architecture with separate components for VRM loading, animation, and MCP server
- System Monitoring: Comprehensive system health and diagnostics
- Debug Tools: Built-in debugging and testing utilities
Portmanteau Tools Architecture
AvatarMCP uses a portmanteau tools architecture that consolidates related functionality into unified interfaces, leveraging FastMCP 3.1.0 sampling capabilities for agentic workflows.
Benefits
- 50% Tool Reduction: From 28 individual tools to 16 consolidated portmanteau tools with agentic sampling
- Better Organization: Related functionality logically grouped together
- Easier Maintenance: Single class per functional area
- Improved Discoverability: Clearer tool purposes and operations
- Standards Compliance: FastMCP 3.1.0 compliant with multiline docstrings
- Cleaner API: More intuitive tool structure with operation-based design
Architecture Overview
The server exposes only the 16 portmanteau tools. Bootstrap first with system_monitor({"operation": "initialize"}) (optional: models_dir); then use other portmanteau tools. Each portmanteau tool uses an operation-based design:
# Example: avatar_manager tool
await avatar_manager({
"operation": "load", # Operation type
"path": "models/avatar.vrm", # Operation-specific parameters
"make_active": True
})
await avatar_manager({
"operation": "list" # Different operation, same tool
})
Tool Categories
- Core Tools (6 implemented): Essential functionality
- Management Tools (5 implemented): System and resource management
- Advanced Tools (5 implemented): Specialized functionality
For detailed architecture documentation, see Portmanteau Tools Plan.
Documentation
Core Components
MCPServer: FastMCP 3.1.0-compatible server implementationVRChatOSC: OSC integration with VRChat for avatar controlVRMModel: VRM 2.0 model loading and managementAnimationController: Manage and play animations on avatarsMCPTools: MCP command handlers for avatar controlAvatarSamplingTool: Agentic workflow orchestration using FastMCP 3.1.0 sampling
Feature Guides
- Sampling Workflows Guide: Complete guide to agentic sampling workflows
- Workflow examples and practices
- Creative applications and use cases
- Technical implementation details
- Troubleshooting and optimization tips
Available Prompts
AvatarMCP supports natural language interaction through the following commands:
Avatar Management
- Load Avatar: "Load the VRM model from {path} as {avatar_name}"
- Unload Avatar: "Unload the avatar named {avatar_name}"
- List Avatars: "Show all loaded avatars"
- Set Active Avatar: "Set {avatar_name} as the active avatar"
Animation Control
- Play Animation: "Play {animation_name} on {avatar_name} at {speed}x speed"
- Stop Animation: "Stop current animation on {avatar_name}"
- Pause Animation: "Pause animation on {avatar_name}"
- Resume Animation: "Resume paused animation on {avatar_name}"
- Blend Animations: "Blend from {anim1} to {anim2} over {duration} seconds on {avatar_name}"
- Set Animation Parameter: "Set {parameter} to {value} on {avatar_name}'s animation"
Bone Control
- Rotate Bone: "Rotate {bone_name} to X:{x} Y:{y} Z:{z} on {avatar_name}"
- Reset Bone: "Reset {bone_name} rotation on {avatar_name}"
- List Bones: "Show all bones for {avatar_name}"
Morph/BlendShape Control
- Set Morph: "Set {morph_name} to {value} on {avatar_name}"
- Reset Morph: "Reset {morph_name} on {avatar_name}"
- List Morphs: "Show available morphs for {avatar_name}"
Export
- Export Avatar: "Export {avatar_name} to {file_path} as {format}"
- Take Screenshot: "Take a screenshot of {avatar_name} and save to {file_path}"
System
- Get System Info: "Show system information"
- Get Logs: "Show recent logs"
- Search Web: "Search the web for {query}"
- Query Knowledge Base: "Search knowledge base for {query}"
Advanced
- Send OSC Message: "Send OSC message {address} with {value} to {ip}:{port}"
- Run Script: "Run script {script_name} with parameters {params}"
3D Visualization & Controls
AvatarMCP features a 3D viewport system for avatar manipulation:
- Interactive 3D Viewport: Real-time rendering of your avatar with multiple camera angles
- View Controls:
- Rotate: Click and drag with left mouse button
- Pan: Right-click and drag
- Zoom: Mouse wheel or right-click + drag up/down
- Reset View: Double-click the viewport
- Visualization Modes:
- Solid: Full textured rendering
- Wireframe: See underlying mesh structure
- Shaded: Flat shading for better shape visualization
- X-Ray: See through the model for complex rigs
Advanced Avatar Controls
- Bone Control: Precise manipulation of individual bones with support for local and world space transformations
- Morph Target Control: Fine-grained control over blend shapes and morph targets with real-time preview
- Export Tools: Export avatars to various formats including FBX and Unity packages
For detailed documentation, see AVATAR_CONTROLS.md.
MCP Protocol Support
AvatarMCP implements the following portmanteau tools leveraging FastMCP 3.1.0 sampling capabilities:
Core Portmanteau Tools
avatar_manager: Comprehensive avatar lifecycle managementavatar_pipeline: Creative pipeline orchestrator (VRoid brute-force → Blender validate → VTube staging)animation_manager: Animation control, sequences, and layeringemotion_manager: Emotional states and personality profilesaudio_manager: Audio playback and synthesisbehavior_manager: AI behavior and adaptationchat_manager: Chat session managementsystem_monitor: System health and diagnostics
Advanced Portmanteau Tools
avatar_sampling: Agentic workflow orchestration (SEP-1577)collaboration_manager: Multi-user synchronizationcontent_manager: Content creation and publishinginteraction_manager: Tactical triggers and gesturesperformance_manager: Optimization and profilingunity_integration: Unity desktop avatar controlunity_window_manager: Unity window controlunity_config_manager: Unity configuration managementserver_controller: Server lifecycle management
Agentic Sampling Workflows (FastMCP 3.1.0)
AvatarMCP introduces agentic sampling workflows - a feature leveraging FastMCP 3.1.0's SEP-1577 "sampling with tools" specification. This enables LLMs to autonomously orchestrate complex avatar behaviors without manual step-by-step programming.
Key Benefits
- ** Intelligent Choreography**: AI creates seamless multi-step performances
- ** Efficiency Gains**: Reduces 10+ tool calls to single workflow requests
- ** Emotional Intelligence**: LLM understands timing, flow, and emotional context
- ** Complex Behaviors**: Sophisticated avatar interactions and storytelling
Sampling Workflow Examples
# Emotional performance
await avatar_agentic_workflow({
"workflow_prompt": "Express happiness with a smile and celebratory gesture",
"avatar_id": "companion_bot",
"available_operations": ["set_morph", "play_animation", "control_bone"],
"max_iterations": 3
})
# Complex dance choreography
await avatar_agentic_workflow({
"workflow_prompt": "Perform a 30-second joyful dance with emotional transitions",
"avatar_id": "dancer",
"available_operations": ["play_animation", "set_emotion", "blend_animations", "wait"],
"max_iterations": 10,
"context": {"duration": 30, "style": "joyful"}
})
# Interactive conversation response
await avatar_agentic_workflow({
"workflow_prompt": "React to surprising good news with excitement and gestures",
"avatar_id": "listener",
"available_operations": ["set_emotion", "control_bone", "send_osc"],
"max_iterations": 4
})
Available Operations for Sampling
The sampling tool can orchestrate these avatar operations:
load_avatar- Load VRM modelsplay_animation- Trigger animationsset_morph- Control facial expressionscontrol_bone- Manipulate bone positionssend_osc- Send OSC messages to VRChatset_emotion- Set emotional statescreate_sequence- Build animation sequencesblend_animations- Layer multiple animationsget_status- Check avatar statewait- Add timing delays
Technical Implementation
- SEP-1577 Compliance: Full implementation of sampling with tools specification
- Async Orchestration: Proper timing and sequencing of operations
- Context Awareness: Workflow parameters and environmental context
- Safety Limits: Configurable iteration limits (1-20) prevent infinite loops
- Error Recovery: Graceful handling of operation failures
Usage Examples
# Avatar management
await avatar_manager({
"operation": "load",
"path": "models/my_avatar.vrm",
"make_active": True
})
# Animation control
await animation_controller({
"operation": "play",
"name": "wave",
"loop": True,
"speed": 1.2
})
# OSC communication
await osc_communicator({
"operation": "send",
"address": "/avatar/parameters/Emotion",
"value": 0.8
})
# Unity integration
await unity_integration({
"operation": "set_expression",
"expression": "Happy",
"strength": 0.9
})
Sampling Workflow Examples
# Agentic emotional performance
await avatar_agentic_workflow({
"workflow_prompt": "Express genuine happiness with a warm smile and friendly gesture",
"avatar_id": "companion_bot",
"available_operations": ["set_morph", "play_animation", "control_bone"],
"max_iterations": 3
})
# Complex choreography with timing
await avatar_agentic_workflow({
"workflow_prompt": "Perform a 20-second celebratory dance with emotional peaks",
"avatar_id": "performer",
"available_operations": ["play_animation", "set_emotion", "blend_animations", "wait"],
"max_iterations": 8,
"context": {
"duration": 20,
"style": "celebratory",
"emotional_arc": ["joy", "excitement", "climax"]
}
})
# Interactive storytelling
await avatar_agentic_workflow({
"workflow_prompt": "React dramatically to an unexpected plot twist",
"avatar_id": "character",
"available_operations": ["set_emotion", "control_bone", "play_animation", "send_osc"],
"max_iterations": 6,
"context": {
"reaction_type": "shock",
"intensity": "high"
}
})
Getting Started with Sampling Workflows
- Load your avatar using
avatar_managerfirst - Choose appropriate operations for your workflow goal
- Write descriptive prompts that include timing and emotional context
- Start with simple workflows (3-5 iterations) and scale up
- Monitor results to refine prompts and operation selection
For complete MCP protocol documentation, see [FastMCP Documentation](https://FastMCP 3.1.0readthedocs.io/).
VRChat OSC Integration
See VRChat OSC Integration Guide for details on how to configure and use the OSC integration.
Installation
Prerequisites
- uv installed (RECOMMENDED)
- Python 3.12+
Quick Start
Run immediately via uvx:
uvx avatarmcp
Claude Desktop Integration
Add to your claude_desktop_config.json:
"mcpServers": {
"avatarmcp": {
"command": "uv",
"args": ["--directory", "D:/Dev/repos/avatar-mcp", "run", "avatarmcp"]
}
}
Prerequisites
- Python 3.9+
- pip (Python package manager)
- VRChat (for VRChat OSC integration - optional)
PyPI Package Install (RECOMMENDED)
Fastest Installation - Production Ready:
pip install avatarmcp
Claude Desktop Integration:
- Open Claude Desktop
- Settings MCP Servers
- Add new MCP server:
{ "mcpServers": { "avatarmcp": { "command": "avatarmcp" } } }
Web dashboard (web_sota)
Optional React dashboard (Vite, port 10792) with API proxy to the backend (10793). From repo root:
cd web_sota
npm install
npm run dev
Requires framer-motion and a valid APPS_CATALOG export from src/common/apps-catalog.ts. See CHANGELOG for recent web UI fixes.
Packaging & Distribution
This repository is SOTA 2026 compliant and uses the officially validated @anthropic-ai/mcpb workflow for distribution.
Pack Extension
To generate a .mcpb distribution bundle with complete source code and automated build exclusions:
# SOTA 2026 standard pack command
mcpb pack . dist/avatar-mcp.mcpb
Other MCP Clients (Cursor, Windsurf, etc.)
# Install from PyPI (recommended)
pip install avatarmcp
# OR install from source
git clone https://github.com/sandraschi/avatarmcp.git
cd avatarmcp
uv pip install -e .
Dependencies
Core dependencies (automatically installed):
python-osc: For VRChat OSC communicationpyvrm: For VRM model loading and manipulationnumpy: For animation mathfastmcp: MCP protocol implementation
Optional dependencies (for development and testing):
fastapi: For the optional REST APIuvicorn: ASGI server for the REST API
## Quick Start
### Running the Server
```bash
# Start the MCP server
python -m avatarmcp
The server will start and listen for MCP commands on stdin/stdout. You can interact with it using any MCP 2.12.0-compatible client.
Example MCP Commands
{
"jsonrpc": "2.0",
"id": 1,
"method": "avatar.load",
"params": {
"id": "my_avatar",
"path": "/path/to/avatar.vrm",
"scale": 1.0
}
}
{
"jsonrpc": "2.0",
"id": 2,
"method": "animation.play",
"params": {
"avatar_id": "my_avatar",
"animation": "wave_hand",
"loop": false
}
}
Optional REST API
For testing purposes, you can start the optional REST API server:
uvicorn avatarmcp.api:app --host 0.0.0.0 --port 8000
Note: The REST API is provided for testing and development purposes only and should not be used in production.
from avatarmcp import VRChatAvatarController
import asyncio
async def main():
# Create and initialize the controller
controller = VRChatAvatarController("path/to/your/model.vrm")
await controller.initialize()
try:
# Set a facial expression
await controller.osc_integrator.set_expression("happy", 1.0)
# Set a hand gesture
await controller.osc_integrator.set_gesture("left", "peace")
# Keep the application running
while True:
await asyncio.sleep(1)
except KeyboardInterrupt:
pass
finally:
await controller.close()
asyncio.run(main())
Running the Example
# Run the VRChat control example
python examples/vrchat_avatar_control.py --vrm path/to/your/model.vrm --demo
Configuration
OSC Settings
By default, AvatarMCP uses these OSC ports:
Receive Port: 9000 (for receiving parameters from VRChat)
Send Port: 9001 (for sending parameters to VRChat)
You can customize these in the AvatarOSCConfig:
from avatarmcp import AvatarOSCConfig
config = AvatarOSCConfig(
receive_port=9000,
send_port=9001,
server_ip="127.0.0.1"
)
Parameter Mappings
You can define custom parameter mappings for your avatar:
config.parameter_mappings = {
"BlendShape.A": "VRC_AA",
"BlendShape.E": "VRC_E",
# Add more mappings as needed
}
Integration with Other MCP Services
OSCMCP Integration
AvatarMCP can be integrated with OSCMCP for advanced routing and processing:
from fastmcp import FastMCP
from avatarmcp import AvatarOSCIntegrator
# Initialize MCP client
mcp = FastMCP("http://localhost:8000/mcp")
# Create and start the integrator
integrator = AvatarOSCIntegrator(mcp)
await integrator.start()
# Now you can control the avatar through OSCMCP
await integrator.set_expression("happy", 1.0)
API Reference
VRChatOSCServer
Core OSC server for VRChat communication.
Methods:
start(): Start the OSC serverstop(): Stop the OSC serverset_gesture(hand, gesture, strength=1.0): Set a hand gestureset_expression(expression, strength=1.0): Set a facial expressionset_viseme(viseme, strength=1.0): Set a viseme for lip sync
VRMLoader
Load and parse VRM 2.0 models.
Methods:
from_file(file_path): Load a VRM model from fileget_blend_shape_names(): Get all blend shape namesget_bone_names(): Get all bone namesget_material_names(): Get all material names
AvatarOSCIntegrator
High-level integration with MCP ecosystem.
Methods:
start(): Start the integratorstop(): Stop the integratorload_vrm(file_path): Load a VRM modelset_gesture(hand, gesture, strength=1.0): Set a hand gestureset_expression(expression, strength=1.0): Set a facial expression
Development
Setup
# Clone the repository
git clone https://github.com/yourusername/avatarmcp.git
cd avatarmcp
# Install development dependencies
pip install -e ".[dev]"
# Install pre-commit hooks
pre-commit install
Testing
# Install dev dependencies
uv sync --dev
# Run all tests
uv run pytest tests/ -v
# Run specific test suites
uv run pytest tests/test_portmanteau_tools.py -v
uv run pytest tests/test_server.py -v
uv run pytest tests/test_animation.py -v
The test suite includes:
- Portmanteau tool tests (14 tests): Covers all 16 portmanteau tools — system_monitor (init/shutdown/status), avatar_manager (load/list/set_active/unload/errors), animation_manager (play/stop/no-active), chat_manager (full lifecycle), artifact_manager (scan), tool registration.
- Server tests (5 tests): AvatarMCPServer lifecycle, tool instantiation, OSC helper, VRMManager integration.
- Animation tests (6 tests): AnimationKeyframe, AnimationClip, AnimationController (play, update, blend, stop).
- VRM loader tests (6 tests): File I/O, metadata extraction, bone loading, error handling.
- API tests (2 tests): FastAPI TestClient health, model listing.
- Integration tests (3 tests): Server start/stop, VRM scanning, OSC communication.
Status: 49/49 tests passing. Legacy tests rewritten for current architecture.
Code Style
# Lint Python (Ruff)
uv run ruff check .
# Lint frontend (Biome)
uv run --directory web_sota npx @biomejs/biome check .
# Fix both
uv run ruff check . --fix && uv run ruff format .
uv run --directory web_sota npx @biomejs/biome check --apply .
Code Style
# Format code
black .
# Sort imports
isort .
# Check code style
flake8
Contributing
Contributions are welcome! Please read our Contributing Guidelines for details.
🛡️ Industrial Quality Stack
This project adheres to SOTA 14.1 industrial standards for high-fidelity agentic orchestration:
- Python (Core): Ruff for linting and formatting. Zero-tolerance for
printstatements in core handlers (T201). - Webapp (UI): Biome for sub-millisecond linting. Strict
noConsoleLogenforcement. - Protocol Compliance: Hardened
stdout/stderrisolation to ensure crash-resistant JSON-RPC communication. - Automation: Justfile recipes for all fleet operations (
just lint,just fix,just test). - Security: Automated audits via
banditandsafety. - CI/CD: GitHub Actions — lint, type-check, test, build, release. Python 3.12/3.13 matrix.
- Quality Gates: 49 passing tests, Ruff + mypy + Bandit + Trivy in CI.
- Containers: Docker Compose with Prometheus, Loki, Grafana, Promtail for full observability.
License
This project is licensed under the MIT License - see the LICENSE file for details.
Credits
VRM Consortium for the VRM specification
VRChat for the social VR platform
The MCP community for building MCP tools
Made with by FlowEngineer sandraschi | GitHub
Webapp Dashboard
This MCP server includes a free, premium web interface for monitoring and control. By default, the web dashboard runs on port 10792. (Assigned ports: 10792 (Web dashboard frontend), 10793 (Web dashboard backend (API)))
To start the webapp:
- Navigate to the
webapp(orweb,frontend) directory. - Run
start.bat(Windows) or./start.ps1(PowerShell). - Open
http://localhost:10792in your browser.
Установка Avatar
У этого сервера нет опубликованного пакета — он собирается из исходников. Открой репозиторий и следуй инструкции в README.
▸ github.com/sandraschi/avatar-mcpFAQ
Avatar MCP бесплатный?
Да, Avatar MCP бесплатный — установка в пару кликов через Unyly без оплаты.
Нужен ли API-ключ для Avatar?
Нет, Avatar работает без API-ключей и переменных окружения.
Avatar — hosted или self-hosted?
Self-hosted: сервер запускается локально на твоей машине командой из раздела установки.
Как установить Avatar в Claude Desktop, Claude Code или Cursor?
Открой Avatar на unyly.org, выбери вкладку своего клиента (Claude Desktop, Claude Code, Cursor) и нажми Install — конфиг сгенерируется автоматически, без правки JSON.
Похожие MCP
ARA
Generate images, video and audio from any AI agent — one connector.
автор: ARAOmni Video
An MCP server that transforms LLM-enabled IDEs into professional video editors by pre-processing footage into text proxies, generating motion graphics via HTML/
автор: buildwithtazaYouTube
Transcripts, channel stats, search
автор: YouTubeEverArt
AI image generation using various models.
автор: modelcontextprotocolgpu-bridge/mcp-server
GPU inference API for LLMs, image and video generation, TTS, Whisper, embeddings, reranking and OCR, paid per use via x402 or an API key.
автор: gpu-bridgemerterbak/Grok-MCP
Access xAI's Grok API with agentic tool calling, image generation, vision and file support.
автор: merterbakSureScaleAI/openai-gpt-image-mcp
Image generation and editing with OpenAI's GPT image models.
автор: SureScaleAIYangLiangwei/PersonalizationMCP
Aggregate personal data from Steam, YouTube, Bilibili, Spotify, Reddit and other gaming, music, video and social platforms, with OAuth2 and token management.
автор: YangLiangweiAceDataCloud/MCPFlux
Flux image generation and editing from Black Forest Labs via the Ace Data Cloud API.
автор: AceDataCloudAceDataCloud/MCPNanoBanana
Nano Banana image generation and editing, including virtual try-on and product placement in realistic scenes.
автор: AceDataCloudCompare Avatar with
Не уверен что выбрать?
Найди свой стек за 60 секунд
Автор?
Embed-бейдж для README
Похожее
Все в категории media
