arXiv OpenReview

Updates

• 2025/11/14: OpenReview version is now available.
• 2025/08/14: arXiv version is now public.

Abstract

Text prompts enable intuitive content creation but may fall short in achieving high precision for intricate tasks; knob or slider controls offer precise adjustments at the cost of increased complexity. To address the gap between knobs and prompts, a new MCP (Model Context Protocol) server and a unique set of prompt design criteria are presented to enable exploring parametric OSC (OpenSoundControl) control by natural language prompts. Demonstrated by 15 practical QA examples with best practices and the generalized prompt templates, this study finds Claude integrated with the MCP2OSC server effective in generating OSC messages by natural language, interpreting, searching, and visualizing OSC messages, validating and debugging OSC messages, and managing OSC address patterns. MCP2OSC enhances human-machine collaboration by harnessing a LLM (Large Language Model) to handle intricate OSC development tasks. It empowers human creativity with an intuitive language interface featuring flexible precision controls: a promptbased OSC tool. This study provides a novel perspective on the creative MCP application at the network protocol level by utilizing LLM’s strength in directly processing and generating human-readable OSC messages. The results suggest its potential for a LLM-based universal control mechanism for multimedia devices.

Demo video

Watch the video Watch the video Watch the video Watch the video

Installation

1. Clone the repository

   git clone https://github.com/yyf/MCP2OSC.git
   cd MCP2OSC
   

2. Install dependencies

   npm install
   

3. Configure your LLM application Add to your Claude Desktop MCP configuration file — for example:

   • macOS: ~/Library/Application Support/Claude/claude_desktop_config.json

   Use the absolute path to mcp-server.js on your machine.

   {
      "mcpServers": {
         "mcp2osc": {
            "command": "node",
            "args": ["/Users/.../MCP2OSC/mcp-server.js"],
            "cwd": "/Users/.../MCP2OSC",
            "env": {
            "OSC_HOST": "127.0.0.1",
            "OSC_SEND_PORT": "9500",
            "OSC_RECEIVE_PORT": "9501", 
            "WEBSOCKET_PORT": "8765",
            "WEBSOCKET_OSC_PORT": "9500",
            "MAX_OSC_MESSAGES": "2000",
            "OSC_LOG_ROTATION": "true"
            }
         }
      }
   }
   

4. Start the LLM application, i.e. Claude desktop application

• Claude desktop application can be downloaded here: https://claude.ai/download
• Restart Claude if you make any change to the claude config json file
• Open the example MaxMSP or PureData patch and try the example prompts in Claude

5. Example prompts

• "What MCP tools are available"
• "Ping MaxMSP at port 9500 then check if there is any response"
• "Check received OSC messages"

6. Optional: start MCP + dashboard

   npm start
   

7. Creative application examples

• The examples folder include MaxMSP, PureData, and SuperCollider patches

Citation

@inproceedings{
fan2025mcposc,
title={{MCP}2{OSC}: Parametric Control by Natural Language},
author={Yuan-Yi Fan},
booktitle={The Thirty-ninth Annual Conference on Neural Information Processing Systems Creative AI Track: Humanity},
year={2025},
url={https://openreview.net/forum?id=XLdFf7Zarw}
}

❤️ Made with MCP2OSC 🎛️ - a selection of projects exploring the possibilities of MCP2OSC

Project	Author	Description
coralline	olivelogs	Coralline gives Claude the ability to make music in real-time with SuperCollider, effectively translating dialogue into music.