Yesterday I got asked a simple question: "can your tool tell this reel is a timelapse?"
It could not. The tool (claude-real-video, open source, MIT) turns any video into keyframes + a timestamped transcript so an LLM can actually read it. But keyframes alone don't carry playback speed. My model watched a hyperlapse of a guy typing and described it as "a man typing."
Five hours later there was a working prototype. Here's what I learned building it.
The physics is simple
A video is a sampling of time. Detecting speed manipulation reduces to one question: does the motion between frames match the time the container claims passed between them?
Three measurable signals fall out of that:
- Trajectory continuity. Post speed-up drops frames from continuously captured footage — motion is fast but trackable. Interval capture (timelapse) never recorded the in-between frames — subjects teleport, optical-flow tracking collapses. Dense frame extraction can recover the former, never the latter.
- Duplicate-frame patterns. Slow motion by frame duplication leaves a periodic fingerprint: hold-hold-hold-move. Frame-rate conversion (24→30fps) leaves a different one: one duplicate every five frames. A still slide leaves one long run. Same "duplicate ratio", three different verdicts — run-length structure is the tell.
- Camera vs. subject motion. Estimate the global affine transform per frame pair (RANSAC over LK tracks), subtract it, and classify what's left. Skip this and a stabilized sped-up walking tour reads as normal — the "speed" was all in the camera channel.
What the benchmark taught me
I built a labeled corpus the cheap way: took clean YouTube footage, generated known transforms with ffmpeg (2x, 4x, 8x/30x interval sampling, duplicated slow-mo), and ran the classifier against ground truth.
Results after five iterations:
- Zero false positives on clean footage — the one metric I refuse to trade away. A forensics tool that cries wolf is worse than no tool.
- Heavy manipulation (30x lapse, padded slow-mo, 4x on visible subjects): caught, with per-segment verdicts.
- Subtle 2x on slow scenes: missed, and honestly unfixable with displacement statistics alone. A slow camera sped up 2x still moves within normal-camera range. You need a reference clock — something in the frame with a known real-world rate. Human gait (~2 steps/s) is the obvious next channel.
Two bugs were more instructive than the wins:
- My own corpus generation manufactured evidence: normalizing 23.976fps film to 30fps created a perfect pulldown pattern that the tool flagged as slow motion. The fix wasn't a threshold — it was teaching the classifier to recognize frame-rate conversion as its own category.
- Median motion statistics erased the flagship case. In a reel where typing hands occupy 10% of the frame, the median over 400 tracked corners is the static wall. The manipulation lives in the tail (p90) and in the moving cluster — aggregate stats hide exactly what you're looking for.
Never say "normal speed"
The design rule I'm most attached to came from an adversarial review: the tool never outputs "this video is normal speed." It outputs "no reliable evidence of manipulation." Cleanly re-encoded speed-up is theoretically indistinguishable from native low-frame-rate capture — a tool that pretends otherwise is lying. Evidence tiers (strong/moderate/weak/insufficient), never fake certainty.
Where this lands
The prototype (387 lines, OpenCV + ffmpeg, no GPU, ~1s per 10s of video) ships as an opt-in --speed-check flag in crv Pro once it passes a benchmark built from real reels and Shorts — because that's what people actually feed these tools, and the current corpus is too polite.
The free base — scene-aware keyframes + timestamped transcript, 100% local — is here: https://github.com/HUANGCHIHHUNGLeo/claude-real-video
If you've worked on temporal forensics (SpeedNet, the recent Cornell "Seeing Fast and Slow" work) I'd genuinely like to hear where this naive-physics approach breaks.
Top comments (0)