DEV Community

Leo Huang
Leo Huang

Posted on

Let Claude Desktop and Cursor actually watch videos (MCP, fully local)

#mcp #claude #ai #python

My AI agent learned to watch videos this week. Here is the 3-line version of how to give yours the same ability.

claude-real-video (MIT, 1.9k stars) extracts scene-aware, deduplicated keyframes plus a timestamped transcript from any video URL or local file — all processing on your machine. Since 0.8.0 it ships as an MCP server, so MCP clients can request a video directly.

Install 

pip install 'claude-real-video[mcp]'
Enter fullscreen mode Exit fullscreen mode

Claude Code:

claude mcp add crv -- crv-mcp
Enter fullscreen mode Exit fullscreen mode

Claude Desktop — add to claude_desktop_config.json:

{ "mcpServers": { "crv": { "command": "crv-mcp" } } }
Enter fullscreen mode Exit fullscreen mode

Cursor and any other MCP client: same stdio command, crv-mcp.

What you get

Two tools show up:

  • watch_video(source, max_frames, language, transcribe) — downloads (URL) or reads (local file), extracts keyframes at scene changes, drops near-duplicates, transcribes with Whisper, and returns the transcript plus the first batch of frames as images straight into the conversation.
  • get_frames(source, start_index, count) — pages through the rest of the frames.

Analyses are cached per source under ~/.cache/crv-mcp, so a follow-up question about the same video is instant.

Why scene-aware instead of 1 fps

Fixed-interval sampling wastes context on near-identical frames and still misses fast cuts. Scene detection keeps the frames that actually differ: a 58-second test clip goes from 58 sampled frames to 26 that matter. Fewer tokens, nothing missed. The benchmark folder in the repo has the full comparison, reproducible.

Honest notes

  • Verified end-to-end on Claude Code (a real model run described a test video's frames correctly through the tool). Claude Desktop and Cursor speak the same MCP stdio protocol — config above; open an issue if anything misbehaves.
  • Transcription needs Whisper (pip install 'claude-real-video[whisper]' or [fast]); frames work without it.
  • The mcp SDK shipped 2.0 on July 28 and moved FastMCP — 0.8.0 dual-imports both majors, tested separately on each. If you build your own MCP server this month, test in a clean venv.

Repo: https://github.com/HUANGCHIHHUNGLeo/claude-real-video

Top comments (1)

Collapse
 
mads_hansen_27b33ebfee4c9 profile image
Mads Hansen •

Scene-aware pagination is a much better context budget than fixed-rate sampling. One boundary worth documenting is that source is both network input and cache identity.

A stable URL can return different bytes later, while signed URLs can identify the same bytes with different query strings. I’d cache by a content digest plus analysis configuration/model versions, and retain the original URL as provenance rather than identity. For remote sources, validate every redirect hop and block private, link-local, metadata, and file: destinations; for local paths, constrain allowed roots and resolve symlinks before reading.

I’d also treat transcript and frame text as untrusted evidence, not instructions. Useful regression cases: same URL with changed content, equivalent signed URLs, redirect to a private IP, partial downloads, and reopening a cache created with older Whisper or scene-detection settings.