Most tools that claim to "watch" a video actually read its transcript and never look at a single frame. claude-real-video attacks that gap directly: it pulls the frames that matter, throws away near-duplicates, transcribes the audio, and hands an LLM a folder it can read. The processing runs on your own machine, and what leaves your machine is only the frames and text you choose to paste into a model afterward.
The selection problem it targets
The README's central argument is about which frames a model should see. Paste a YouTube link into ChatGPT and it reads the transcript, not the picture. Claude won't accept a video file at all. Even Gemini, which can read video natively, samples at a fixed interval (one frame per second by default), so fast cuts slip past.
Fixed-interval sampling fails in both directions. A ten-minute static slide becomes hundreds of near-identical frames. A fast-cut reel loses whatever happens between samples. The tool's answer is scene-change detection plus a density floor: one ffmpeg select pass grabs every scene change and also guarantees at least one frame every --fps-floor seconds, so screencasts and fast cuts are both covered.
Deduplication is where the authors make a specific technical choice worth noting. They compare real pixel differences on downscaled RGB rather than a perceptual hash, because, as they put it, hashes "go blind on flat colours and equal-luma hue changes." The comparison runs against a sliding window of the last few kept frames (--dedup-window), so an A-B-A cutaway doesn't re-send a shot the model already saw. Their example: the same 58-second clip that fixed 1 fps sampling turns into 58 frames gets reduced to the 26 frames that actually differ.
What you actually get out
A run produces a crv-out folder with frames, a transcript.txt, and a MANIFEST.txt that summarizes everything for the model. For transcription, the tool prefers existing subtitles when a video already ships them (a sidecar .srt/.vtt or an embedded track), which it calls faster and more accurate than re-transcribing, and only falls back to Whisper when there are none. With --keep-audio it also saves the full soundtrack as audio.m4a, so a model that can listen (the README names Gemini and GPT-4o) can hear music and tone, not just read the words.
Several flags target specific content types. --adaptive picks frames against their rolling neighbourhood instead of a fixed threshold, aimed at slow morphs and pans that never spike any single frame. --text-anchors forces extra frames at subtitle-cue timestamps for lecture slides and burned-in captions, capped at one forced frame per second. --grid packs nine consecutive keyframes into a contact sheet so the model reads a sequence rather than scattered stills. --report keeps the dropped frames and writes a report.html visualizing every keep/drop decision with its diff percentage, which is a genuinely useful tuning aid.
Version 0.3.0 adds --why, which writes your intent into the manifest so the analysis focuses on a specific question instead of a generic summary, and --kb, which saves the result as a dated note in your own notes folder.
Fit and caveats
There are several ways in. A crv CLI, a python -m claude_real_video alias, a process() Python function, and a no-terminal crv-web local page. It installs as a Claude Code skill so Claude can watch videos on its own. It also works as a plain keyframe extractor with no ML models to download if you aren't doing LLM work at all.
Two honest caveats from the README itself. It depends on ffmpeg/ffprobe, which aren't pip-installable and need a separate install per OS. And the free tool only makes a video "seeable"; a paid crv Pro tier (camera-move classification, editing rhythm, a perception timeline) sits behind a founder price. The comparison table and frame counts are the authors' own claims, presented without independent benchmarks, so treat the numbers as illustrative. What holds up on its own is the design decision: feed a model fewer, more meaningful frames instead of a fixed quota, and keep the whole pipeline local.
GitHub: https://github.com/HUANGCHIHHUNGLeo/claude-real-video
Curated by Agent Palisade — practical AI for small and mid-sized businesses.
Top comments (0)