Music Perception

The ears of a DAW-control agent. An MCP server that turns an audio file into exact, reproducible facts a text LLM can act on — loudness, true peak, tempo, key, spectral balance, clipping.

 text-LLM brain (DeepSeek/…)  ── decides ──►  reaper-mcp.render_to_wav(...)  ──►  take.wav
        ▲                                                                            │
        └──────────────  facts (JSON)  ◄── music-perception-mcp.analyze_audio(take.wav)

The brain renders a WAV (e.g. via reaper-mcp's render_to_wav), calls a tool here to perceive it, then decides the next mixing action. This server is a 取数型 (data-fetch) MCP tool in prism-core terms: it returns context, it does not act on the DAW.

Speaks newline-delimited JSON-RPC 2.0 on stdin/stdout — the same protocol as reaper-mcp, so prism-core's mcp_client connects to it identically.

Scope: deterministic measurement only

This server measures. The numbers are exact and reproducible (same file → same answer), computed by signal-processing libraries, not by an AI model.

The deterministic tools (analyze_audio, measure_loudness) make no subjective judgement — no "muddy/harsh/sad". Those come from ONE clearly separated, non-deterministic tool, listen_subjective, backed by an audio LLM (Gemini). Exact numbers and opinions are kept apart on purpose — their trustworthiness and use differ. An empirical benchmark backs this split: deterministic MIR tracks controlled spectral defects perfectly (Spearman ρ≈1.0) while models don't (≤0.31); models judge mood/emotion decently (0.46–0.64 vs human DEAM ratings) while MIR is blind. See the music-agent design docs.

Tools

analyze_audio(path)

One-stop analysis. Returns:

Field	What you get	Library
loudness.integrated_lufs	Integrated loudness (ITU-R BS.1770 / EBU R128)	pyloudnorm
loudness.loudness_range_lu	Loudness range (dynamics), gated P95−P10 of short-term	pyloudnorm + numpy
loudness.true_peak_dbtp	True peak via 4× oversampling (catches inter-sample overs)	scipy
loudness.sample_peak_db	Raw sample peak	numpy
tempo.bpm	Estimated tempo	librosa
key.key / key.mode / key.confidence	Global musical key (Krumhansl-Schmuckler)	librosa
spectral.bands_db_rel	6-band energy balance (sub/bass/low-mid/mid/high-mid/high), relative dB	librosa
spectral.centroid_hz / rolloff_hz	Brightness measures	librosa
clipping	Digital full-scale clip count + first timestamps	numpy

measure_loudness(path)

Loudness block only (integrated LUFS, range, true peak, sample peak). Skips librosa, so it's fast — use it for quick master-bus checks against a target (e.g. −14 LUFS for streaming).

Both take an absolute path, e.g. one returned by reaper-mcp's render_to_wav. WAV is the expected input; any libsndfile-readable format works (FLAC/OGG/AIFF). MP3/M4A are not guaranteed — render to WAV first.

listen_subjective(path, question?) — the one non-deterministic tool

Holistic "listening" judgement via an audio LLM (Gemini): 0-100 muddy/harsh/sibilant/bright, valence/arousal in [-10,10], a mood word, timestamped issues, a one-line overall. Optional question focuses it ("is the vocal sibilant?"). Use it for mood / holistic feel; use analyze_audio for exact numbers.

Needs a key — set env before launching the server:

• GEMINI_API_KEY — required.
• GEMINI_BASE_URL — optional; set it to use an OpenAI-compatible relay (e.g. PackyCode https://www.packyapi.com/v1, or OpenRouter). Unset → Google's native Gemini API.
• GEMINI_MODEL — default gemini-2.5-flash (use gemini-2.5-flash-lite to save).

Without a key it returns {configured:false, error} and the deterministic tools keep working. Install a backend: pip install openai (relay) or google-genai (native). It downsamples to 16 kHz mono, ≤20 s, before sending.

Capabilities and boundaries

What this server is good for — and where each number stops being trustworthy. Read this before acting on a value.

Metric	Reliable for	Boundary / caveat
Integrated LUFS	Master/stem loudness vs a target; A/B before-after	Whole-file integrated; not a live/streaming meter
True peak (dBTP)	Catching inter-sample overs before a limiter ceiling	4× oversample (BS.1770 minimum); a hair below dedicated 8× meters but well within practical tolerance
Loudness range (LU)	Rough dynamics / over-compression check	EBU-style short-term implementation; treat as indicative, not certified
Tempo (BPM)	Steady electronic / pop / rock	Unreliable on rubato, free time, ambient, or no clear beat — returns 0.0 when it finds no beat (honest, not an error)
Key	Single-key tonal material	One global key only — misses modulations/key changes; weak on atonal/percussive/sparse audio; major-vs-minor can flip on ambiguous tonality. Use confidence
Spectral bands	Comparing a mix against a reference curve ("too much 2–6 kHz vs the reference")	Relative energy (dB vs total), not an absolute/calibrated spectrum; not loudness-weighted
Clipping	Detecting digital full-scale clipping	Full-scale only (≥0.999); soft/analog-style clipping and inter-sample overs are not here — those show up as a high true_peak_dbtp

Cross-cutting:

• Measurement vs opinion. The deterministic tools give exact numbers; listen_subjective gives the opinions ("muddy/harsh/sad") — separately, and non-deterministically.
• Garbage in, garbage out. Feed it the actual render. The numbers describe exactly the file you pass, including its sample rate and channel layout.
• One global answer per file for tempo/key. For per-section analysis, render that section (reaper-mcp render_to_wav with a time selection or region:N) and analyze it separately.

Setup

pip install -r requirements.txt          # numpy soundfile pyloudnorm librosa scipy
python server/test_server.py             # offline self-test on a synthetic WAV

Register with an MCP client (e.g. prism-core / Claude Code) — add to your mcp_servers.json / .mcp.json:

{
  "mcpServers": {
    "music-perception": {
      "command": "python",
      "args": ["A:\\Prismcode\\music-perception-mcp\\server\\music_perception_server.py"]
    }
  }
}

Dependencies & licensing

All dependencies are permissive (BSD/MIT/ISC) and pure-pip — no external binary, no ffmpeg. They are confined to this server; the prism-core kernel and the other MCP servers stay zero-dependency. Notably this avoids madmom (non-commercial model weights) and Essentia (AGPL), so the stack stays commercial-friendly.

Roadmap

• separate_stems(path) — Demucs source separation (heavy; CPU-slow). Lets you measure each instrument's loudness/masking.
• (done) listen_subjective — the subjective/mood layer, see above.