README • ⚖️ License

🍿
popcorn

🇨🇳 中文 • 🇯🇵 日本語 • 🇰🇷 한국어 • 🇪🇸 Español • 🇩🇪 Deutsch • 🇫🇷 Français • 🇧🇷 Português • 🇷🇺 Русский • 🇸🇦 العربية • 🇮🇹 Italiano • 🇳🇱 Nederlands • 🇹🇷 Türkçe • 🇻🇳 Tiếng Việt • 🇮🇳 हिन्दी

An agent skill that gives any coding agent the ability to watch and understand video. Works with Claude Code, Codex, and any MCP-compatible agent.

Quick Start • How It Works • MCP Tools • Transcription • Configuration • Troubleshooting • License

Popcorn enables AI agents to watch and understand long-form videos by extracting transcripts, detecting scene changes, and returning key frames. Everything runs locally—no external APIs, no fees, complete privacy.

---

Quick Start

# Install FFmpeg (required)
brew install ffmpeg                    # macOS
sudo apt install ffmpeg                # Ubuntu/Debian

# Install Popcorn
git clone https://github.com/anthropics/popcorn.git
cd popcorn && npm install && npm run build

# Optional: Install a transcription backend
pip install mlx-whisper                # Apple Silicon (fastest)
pip install openai-whisper             # Any platform

Add to Claude Desktop config (~/Library/Application Support/Claude/claude_desktop_config.json):

{
  "mcpServers": {
    "popcorn": {
      "command": "node",
      "args": ["/path/to/popcorn/dist/index.js"]
    }
  }
}

Key Features:

• Scene Detection — Captures frames at visual transitions, not fixed intervals
• Local Transcription — 4 backend options (mlx-whisper, faster-whisper, whisper-cpp, whisper)
• Inline Images — Returns key frames directly in MCP responses
• Smart Presets — Auto-configures for screencasts, presentations, movies, interviews
• Zero Config — Just pass a video path and it works
• Privacy First — Everything runs locally, no data leaves your machine

---

Documentation

Getting Started

• Quick Start — Installation & setup
• Tutorial — Step-by-step usage guide
• MCP Tools — Available tools reference

Guides

• Transcription Backends — Choose the best backend for your system
• Video Types & Objectives — Presets for different content
• Configuration — Advanced parameters

Reference

• Troubleshooting — Common issues & solutions
• Agent Skill — Instructions for AI agents
• API Reference — Tool schemas & responses

---

How It Works

Core Components:

1. FFprobe — Extracts video metadata (duration, resolution, codecs)
2. FFmpeg Scene Detection — Finds visual transitions using select='gt(scene,N)' filter
3. Parallel Frame Extraction — Captures JPEGs at scene change timestamps
4. Multi-Backend Transcription — Whisper variants convert audio to timestamped text
5. Analysis Bundle — Results saved to .popcorn/ directory
6. MCP Response — Returns metadata + inline base64 images

Video File ──▶ FFprobe ──▶ FFmpeg ──▶ Whisper ──▶ Analysis Bundle
                 │           │          │              │
                 ▼           ▼          ▼              ▼
              metadata    frames    transcript    MCP Response

---

MCP Tools

Tool	Description
popcorn_analyze	Main analysis — extracts frames, transcribes audio, returns results
popcorn_suggest	Probe video metadata and get recommended settings
popcorn_presets	List available video types and objectives
popcorn_backends	Detect your system and show transcription options
popcorn_read	Read transcript slices with time filtering

Basic Usage

{
  "tool": "popcorn_analyze",
  "arguments": {
    "path": "/path/to/video.mp4"
  }
}

With Presets

{
  "tool": "popcorn_analyze",
  "arguments": {
    "path": "/path/to/video.mp4",
    "videoType": "screencast",
    "objective": "detailed"
  }
}

---

Video Types

Type	Best For	Scene Detection
screencast	Tutorials, coding sessions, UI demos	Low threshold
presentation	Slides, lectures, keynotes	Slide transitions
movie	Films, TV shows	Balanced
interview	Podcasts, talking heads	Transcription priority
surveillance	Security footage, dashcam	High threshold
sports	Live events, fast action	High frame rate

Objectives

Objective	Use When
summary	Quick overview needed
detailed	Don't miss anything
find_moment	Searching for specific content
transcribe	Audio/speech is most important
visual_only	Only care about visuals
quick_scan	Fast preview needed

---

Transcription Backends

Popcorn auto-detects your system and recommends the best backend.

Backend Comparison

Backend	Speed	Best For	Install
mlx-whisper	Fastest	Apple Silicon (M1/M2/M3/M4)	pip install mlx-whisper
faster-whisper	Fast	NVIDIA GPUs	pip install faster-whisper
whisper-cpp	Moderate	Cross-platform	brew install whisper-cpp
whisper	Slow	Most compatible	pip install openai-whisper

Processing Times (60-min video)

Backend	Time
mlx-whisper	3-8 min
faster-whisper	5-10 min
whisper-cpp	10-20 min
whisper	30-60 min

Force a Backend

{
  "tool": "popcorn_analyze",
  "arguments": {
    "path": "/path/to/video.mp4",
    "backend": "mlx-whisper"
  }
}

---

Configuration

All Parameters

Parameter	Type	Description
path	string	Required. Absolute path to video file
videoType	string	Video type preset
objective	string	Analysis objective preset
transcribe	boolean	Enable/disable transcription
backend	string	Transcription backend
model	string	Whisper model (tiny, base, small, medium, large)
language	string	Language code (e.g., en, es, fr)
frameMode	string	scene or interval
sceneThreshold	number	Scene sensitivity (0-1)
maxFrames	number	Maximum frames to extract
inlineFrames	number	Frames to return as base64

Output Structure

.popcorn/<video>_<timestamp>/
├── analysis.json          # Full metadata
├── transcript.txt         # Plain text
├── transcript.json        # Timestamped segments
├── transcript.chunks.json # LLM-friendly chunks
└── assets/
    ├── audio.wav
    └── frames/
        ├── scene_000001.jpg
        └── ...

---

Troubleshooting

FFmpeg not found

brew install ffmpeg          # macOS
sudo apt install ffmpeg      # Ubuntu/Debian

No transcription backend

pip install mlx-whisper      # Apple Silicon
pip install openai-whisper   # Any platform

Too few frames detected

{ "sceneThreshold": 0.15, "minSceneInterval": 2 }

Too many frames detected

{ "sceneThreshold": 0.5, "minSceneInterval": 10 }

See Troubleshooting Guide for more solutions.

---

Development

npm install          # Install dependencies
npm run build        # Build
npm run dev          # Development mode
npm start            # Run server

Project Structure

popcorn/
├── src/
│   ├── index.ts        # MCP server
│   ├── analyze.ts      # Analysis pipeline
│   ├── ffmpeg.ts       # Video processing
│   ├── transcribe.ts   # Multi-backend transcription
│   ├── presets.ts      # Video type presets
│   └── commands.ts     # Shell execution
├── docs/               # Documentation
└── skills/             # Agent skills

---

Contributing

1. Fork the repository
2. Create a feature branch (git checkout -b feature/amazing)
3. Commit changes (git commit -m 'Add amazing feature')
4. Push to branch (git push origin feature/amazing)
5. Open a Pull Request

---

License

MIT License — see LICENSE for details.

---

Acknowledgments

• OpenAI Whisper — Speech recognition
• whisper.cpp — C++ port
• MLX Whisper — Apple Silicon
• faster-whisper — CTranslate2
• FFmpeg — Video processing
• Model Context Protocol — MCP spec

---

Made with 🍿 for AI agents everywhere