DEV Community: larry

I benchmarked 3 ways to detect slide changes in video — fixed-interval screenshots, pixel diff, and block diff

larry — Thu, 16 Jul 2026 06:19:30 +0000

Disclosure up front: I maintain video-slide-extractor (MIT, zero dependencies) and Video2Any, the product it came out of. The benchmark below is reproducible from two scripts, and it includes a case where my own defaults lose.

If you have ever tried to turn a recorded lecture, webinar, or Zoom call back into its slides, you have probably done one of these:

Screenshot every N seconds (what most tutorials do)
Compare pixels between frames and keep the frame when "enough" changed
Something smarter

I wanted actual numbers instead of vibes, so I built a benchmark where the ground truth is exact by construction — and the results changed how I document my own library's defaults.

The problem with benchmarking slide detection

The honest way to evaluate a detector is against labeled transitions: precision, recall, F1. But hand-labeling a 45-minute lecture is hours of work, the labels have their own errors, and nobody can reproduce your benchmark without your labels.

So I flipped it: render the videos from known slide sequences. If you choose when each slide appears, every transition timestamp is ground truth by construction. Two scripts, fully reproducible:

git clone https://github.com/larry-xue/video-slide-extractor
node bench/generate-fixtures.mjs   # renders the videos (ffmpeg)
node bench/run.mjs                 # scores the methods, writes RESULTS.md

The fixtures:

Two slide sources. A 30-slide synthetic deck drawn by ffmpeg, and the 46 real slides that Video2Any extracted from MIT OCW 6.0001 Lecture 1 (CC BY-NC-SA 4.0, fetched at run time, not committed).
Three variants each, same timeline: clean (crf 23, hard cuts), noisy (crf 45 + down/upscale), and overlay (an animated "webcam" box in the corner — the classic meeting-recording trap).
Slide durations 4–12 s from a seeded PRNG, so every machine renders identical timelines.

One caveat that belongs in the article and not in a footnote: this measures detection on re-renderings with hard cuts, not on an original classroom recording. Crossfades, camera wobble, and in-slide builds are separate failure classes (protocol here).

The three contestants

All three see the exact same input: one 160×90 RGBA frame every 2 seconds, sampled by ffmpeg.

Fixed interval — keep a frame every 10 seconds. No content awareness at all.

Naive pixel diff — a pixel "changed" if its mean |ΔRGB| > 25; keep the frame when > 2% of pixels changed since the last kept frame.

Block diff (video-slide-extractor) — split the frame into 8×8 blocks, compute each block's mean absolute difference, count a block as changed when its mean exceeds a threshold, keep the frame when > changedRatio of blocks changed. The averaging is the point: single-pixel codec noise dilutes inside a block, while a real slide change lights up whole blocks at once. Run twice: library defaults (changedRatio: 0.02) and tuned (0.10).

Results (MIT deck, 46 slides)

Matching: each detection pairs with at most one ground-truth transition within ±2.5 s. Coverage = share of slides that got at least one capture.

Method	clean F1	noisy F1	overlay F1	overlay precision
Fixed interval (10 s)	0.68	0.68	0.68	0.76
Naive pixel diff (2%)	0.94	0.94	0.91	0.85
Block diff (defaults, 0.02)	0.97	0.97	0.54	0.37
Block diff (ratio 0.10)	0.89	0.89	0.93	1.0

Full tables (both decks, all metrics): bench/RESULTS.md.

What I learned (including where I lose)

1. Fixed-interval capture has a hard ceiling nobody mentions. Its coverage capped at ~0.78–0.80 across every variant: any slide shorter than the capture interval is simply gone, no matter how clean the video is. Tutorials that recommend "grab a frame every 10 seconds" are silently accepting a ~20% slide loss on decks with normal pacing.

2. Naive pixel diff is better than its reputation — until something animates. At a sane per-pixel threshold (25), crf-45 compression noise stayed under the trigger. Where it degrades is the webcam overlay: a small pixel population sits permanently above threshold and precision drops to 0.85.

3. My own defaults flood on webcam overlays. This is the number I care most about publishing: with the default changedRatio: 0.02, the animated overlay occupies more than 2% of blocks, so every sampled frame re-triggers — 125 captures for 46 slides, precision 0.37. Raising the ratio to 0.10 restores precision to 1.0. Thresholds are inputs to report, not constants to trust. (The full product deals with this properly — per-video auto-calibration and masking the animated region — but the library's README now says this out loud instead of hoping.)

4. Neither detector hits recall 1.0 on clean input. A few consecutive MIT slides differ by roughly one bullet line; at 160×90 that change hovers at the trigger floor. Incremental builds are the honest hard case for any frame-differencing approach.

The takeaway

If you are building a video-to-slides pipeline in JavaScript:

Don't use fixed-interval capture; the miss rate is structural.
Frame differencing needs noise absorption (blocks, or a robust threshold) and a story for persistently-animated regions — masking, calibration, or a higher trigger ratio.
Whatever you build, benchmark it against constructed ground truth. It costs two scripts and removes all labeling excuses: methodology.

The detector is ~80 lines, MIT-licensed, zero dependencies, runs in the browser and Node: github.com/larry-xue/video-slide-extractor. The finished product built on top of it (local processing, no upload) is video2any.com.

How to evaluate a slide-change detector without fooling yourself

larry — Wed, 15 Jul 2026 04:49:31 +0000

I maintain video-slide-extractor,
a small JavaScript frame-differencing package used in a browser video-to-slides
pipeline. The detector is intentionally simple. Evaluating it honestly is not.

The easiest demo is to show a lecture clip, export a plausible deck, and say it
worked. That proves almost nothing.

A detector can produce a good-looking output while missing brief slides,
capturing half-faded transitions, or keeping the same slide five times. It can
also look inaccurate because the evaluator sampled too slowly to ever see the
missing slide.

Here is the protocol I now use.

First, define the task

Three jobs are often mixed together:

Slide-change detection: find timestamps where the visible frame changed.
Original-slide recovery: choose a clean representative image for every slide shown in the recording.
Deck generation: use a transcript or model to create a new presentation.

A frame-difference detector only addresses the first job. It should not get
credit for OCR or export quality, and it should not be blamed for a PowerPoint
writer stretching an image.

Build ground truth before tuning

Label the source video manually:

slide_id,start_seconds,end_seconds,notes
1,0.0,42.4,title slide
2,42.4,88.1,incremental bullet builds
3,88.1,123.7,crossfade transition

Do this before changing thresholds. Otherwise it is too easy to redefine
"correct" around the output you already have.

Record the source, license, duration, resolution, and access date. If the video
cannot be redistributed, publish the labels and source link instead.

Freeze the sampling configuration

The sampling interval is part of the detector.

If a slide is visible for 1.4 seconds and you sample every 2 seconds, the
detector cannot recover it. That is a sampling false negative, not a threshold
false negative.

For every run, record:

sample interval;
comparison resolution;
crop or mask;
detector options;
runtime and package version.

For video-slide-extractor, that looks like:

const detect = createSlideDetector(160, 90, {
  blockSize: 8,
  blockDelta: 14,
  changedRatio: 0.02,
});

The defaults are starting points, not an accuracy guarantee.

Match detections once

Give each detected timestamp a tolerance window, such as two seconds, and match
it to at most one labeled transition.

A matched detection is a true positive.
An unmatched detection is a false positive.
An unmatched label is a false negative.

Then calculate:

precision = TP / (TP + FP)
recall    = TP / (TP + FN)
F1        = 2 * precision * recall / (precision + recall)

This catches a common bad evaluation: counting two detections around the same
transition as two successes.

Score visual usability separately

A timestamp can be correct while the captured image is unusable.

Track at least:

transition frames;
consecutive duplicates;
a slide revisited later;
low-resolution or blurred captures;
frames dominated by a presenter or webcam overlay.

I report duplicate rate and transition-frame rate separately from detection F1.
Those failures often belong to temporal cleanup and export-frame selection, not
the first-pass change detector.

Tag the failure class

The aggregate number tells you whether a version improved. The failure class
tells you what to build next.

Useful tags include:

incremental build;
crossfade or wipe;
cursor/webcam motion;
camera footage around a screen;
compression noise;
repeated earlier slide;
sample interval too wide.

For example, raising changedRatio can reduce cursor-related false positives
while making small text-only changes easier to miss. A single "accuracy" number
hides that tradeoff.

Compare equal pipelines

When comparing two libraries, use the same decoded frames and sampling interval.
When comparing two finished products, report the entire pipeline instead:

what input each accepted;
whether processing was local or remote;
how much manual cleanup was needed;
whether the final PPTX contained images, OCR text, or native objects.

Do not rank a transcript-to-new-deck generator against an original-slide
recovery tool as if they produced the same result.

Publish the failures

The minimum reproducible evidence is:

ground-truth labels;
detector configuration;
detected timestamps;
metric calculation;
package/runtime versions;
known blockers and excluded cases.

I added a reusable version of this protocol to the repository:

Evaluate a Slide-Change Detector

The detector itself is zero-dependency JavaScript and accepts ordered RGBA
frames in the browser or Node:

npm install video-slide-extractor

The larger lesson is simple: show the labels, the settings, and the ugly frames.
If a comparison publishes only the winners, it is marketing, not a benchmark.

How I Built a Browser-Based Video to PowerPoint Converter

larry — Sat, 04 Jul 2026 14:16:50 +0000

Long lecture recordings are a strange format.

The useful part is often not the video itself. It is the slide deck trapped inside the video: a lecture, webinar, product walkthrough, Zoom recording, or screen share where the presenter spends most of the time on mostly-static slides.

So I built Video2Any, a browser-based tool that turns videos and screen recordings into PowerPoint decks, PDFs, image frames, and subtitles. The unusual part is that the video does not need to be uploaded. The browser does the decoding, slide detection, and export locally.

On my tests, an hour-long lecture-style video can usually be sampled in under a minute on a normal laptop. The exact speed depends on the video, browser, codec, resolution, and device, but the important design choice is this: the tool does not need to understand every pixel of every frame at full resolution.

It only needs to answer one question quickly:

Did the visible slide change?

The first version was not AI

The tempting answer today is: use AI.

But for this problem, AI is not the first tool I reach for.

A recorded slide deck has a useful property: it is mostly still. The same slide remains on screen for seconds or minutes, then a large part of the frame changes when the presenter advances to the next slide.

That means a simple visual detector can work surprisingly well:

Sample the video every few seconds.
Downscale each sampled frame to a small size, such as 160x90.
Compare the current frame to the last kept slide frame.
Keep the frame if enough visual blocks changed.
Export the kept frames into a deck.

This is not semantic understanding. It does not know what the title says. It does not know whether a chart is important. It only detects meaningful visual changes.

For lecture videos, webinars, and screen recordings, that is often enough to get a useful first deck.

Why downscale first?

A 1080p frame has more than two million pixels. Comparing full-resolution frames repeatedly is wasteful if the goal is only to detect slide transitions.

Instead, Video2Any compares a small version of the frame. At 160x90, the detector is looking at 14,400 pixels. That is small enough to be fast, but still large enough to tell whether a slide changed.

The export step can still capture a higher-quality image later. Detection and export do not need to use the same resolution.

This split matters:

Low-resolution frames for fast detection.
Higher-resolution frames for final PowerPoint/PDF/image export.

Block-based frame differencing

A naive frame diff compares individual pixels. That is too sensitive for compressed video.

Video compression creates small changes even when the slide did not actually change. A cursor might move. A webcam bubble might flicker. A progress bar might animate.

So the detector uses blocks.

At a high level:

function changedEnough(previousFrame, currentFrame) {
  const blockSize = 8;
  const blockDelta = 14;
  const changedRatio = 0.02;

  let changedBlocks = 0;
  let totalBlocks = 0;

  for (const block of blocks(currentFrame, blockSize)) {
    const diff = meanAbsoluteDifference(previousFrame, currentFrame, block);
    if (diff > blockDelta) changedBlocks++;
    totalBlocks++;
  }

  return changedBlocks / totalBlocks > changedRatio;
}

The exact implementation has more edge cases, but the idea is simple: a block counts as changed only if its average visual difference is high enough. Then a frame becomes a new slide only if enough blocks changed.

This makes the detector less jumpy than pixel-by-pixel comparison.

Why the browser is enough

Modern browsers can do a lot of media work locally:

Decode video from a File or Blob.
Seek through timestamps.
Draw frames to a canvas.
Read image data for diffing.
Generate PowerPoint, PDF, or ZIP exports in JavaScript.

The privacy benefit is obvious: if the browser does the work, the file does not need to touch a server.

The cost benefit is also important. If users are converting large private lecture recordings, uploading and processing everything server-side becomes expensive and slow. Local processing makes the free tier much more practical.

The open-core detector

I extracted the basic detector into a tiny open-source package:

npm install video-slide-extractor

Usage looks like this:

import { createSlideDetector } from 'video-slide-extractor';

const width = 160;
const height = 90;
const detect = createSlideDetector(width, height);

for (const frame of sampledFrames) {
  const result = detect(frame.rgba);
  if (result.keep) {
    console.log('new slide near', frame.time);
  }
}

The package is here:

It is intentionally small. It does not export PowerPoint files or run OCR. It just detects slide and scene changes from frames.

What is harder than it sounds?

A few cases make this problem more interesting.

Fade transitions

If a deck uses fade animations, the detector might catch a half-transition frame instead of the final clean slide. A better pipeline should refine the timestamp after a change and capture the settled frame.

Webcam overlays

If a webcam bubble moves constantly, it can create visual noise. Block-based diffing helps, but large overlays can still confuse the detector. One improvement is activity masking: identify small regions that move all the time and reduce their weight.

Returning to an earlier slide

Presenters often jump back. Local frame comparison only knows about the last kept frame. A stronger pipeline also compares against earlier kept slides and removes duplicates.

Text-only changes

The hardest useful case is a mostly identical slide where only a line of text changes. The threshold needs to be sensitive enough to catch that, without firing on video noise.

This is where calibration helps. Instead of using one fixed threshold for every video, you can sample the video first and estimate the gap between same-slide noise and real slide changes.

What AI is still useful for

I do not think AI is unnecessary here. I just think it belongs later in the pipeline.

Frame differencing is good for finding slide frames. AI and OCR are useful after that:

OCR to rebuild editable text.
Speech-to-text for subtitles or speaker notes.
Layout analysis to turn screenshots into structured slides.
Summarization to produce study notes.

That split keeps the first pass fast, private, and cheap.

Where this works best

Video2Any works best for:

Lecture recordings.
Webinars.
Conference talks.
Zoom, Meet, and Teams recordings.
Screen recordings and product walkthroughs.
Mostly-static slide decks captured as video.

It is less ideal for:

Handheld camera footage.
Highly animated videos.
Videos where slides are tiny or blurred.
Recordings where the presenter covers the slide content.

Try it

The browser tool is here:

https://video2any.com

The open-source detector is here:

https://github.com/larry-xue/video-slide-extractor

I also started a small curated list of related tools and libraries:

https://github.com/larry-xue/awesome-video-to-slides

I would be especially interested in feedback from people who deal with long lecture recordings, internal training videos, or webinar archives. The core question is simple: is the extracted deck good enough to replace scrubbing through the video?