DEV Community

Plexescor (Abhijot Singh)
Plexescor (Abhijot Singh)

Posted on

C++ Tool That Extracts Text From Scrolling Screen Recordings

#ai #cpp #performance #showdev

Palimpsest: I Built a C++ Tool That Extracts Text From Scrolling Screen Recordings (94% Accuracy, 323 FPS on a Laptop)

Most of the code was AI-generated from a detailed architecture prompt. I designed the pipeline, the AI wrote the C++. Here's what we built.

I had a problem. My dad takes 1.5 hour long banking regulation classes recorded as screen recordings of someone scrolling through RBI circulars and banking law documents. No download link. No PDF. Just a video of text scrolling past.

I didn't want to type it all out manually.

So instead of spending 3 hours typing, I spent an afternoon designing a solution and prompting an AI to write the C++. The result is Palimpsest — a high-performance CLI tool that extracts text from screen recordings of scrolling documents, deduplicates overlapping frames, and outputs a clean .txt or .docx file.

It runs at 323 FPS on a laptop iGPU and gets 94.56% accuracy on dense banking regulatory text.

GitHub: https://github.com/plexescor/Palimpsest

The Problem Nobody Has a Good Tool For

If you've ever needed to extract text from a screen recording, you know the pain:

  • Someone shared a video of a PDF instead of the PDF itself
  • A lecture recording of someone scrolling through study notes
  • An old recording where the source file is lost forever
  • A tutorial video showing code or documentation you need to copy
  • Someone screen-recorded a locked document walkthrough

Existing tools fail here. Generic OCR tools handle single images. Cloud video OCR services (ScreenApp, etc.) charge monthly subscriptions, cap video length, and you're uploading private content to someone's server. Manual transcription of a 1.5 hour recording is genuinely painful.

There was no good free, local, offline tool that specifically handled the scrolling document problem — deduplicating overlapping frame content and stitching it into a clean linear document.

So I built one.

The Name

Palimpsest — a historical term for a manuscript page that has been scraped clean and rewritten over. Exactly what this tool does to a scroll recording.

Architecture — 8 Stage Lock-Free Pipeline

This is where it gets interesting. The tool isn't just "loop through frames and OCR each one." That would be slow and produce terrible results due to duplicate content across scroll frames.

The architecture is a strictly staged producer-consumer pipeline with lock-free SPSC (Single Producer Single Consumer) ring buffers between every stage:

[Decoder Thread]
     ↓ SPSC RingBuffer<RawFrame, 128>
[Frame Filter & Dwell Gatekeeper]
     ↓ SPSC RingBuffer<FilteredFrame, 64>
[Preprocessor Thread]
     ↓ WorkQueue<PreprocessedFrame>
[OCR Worker Pool — N Threads]
     ↓ ResultQueue<OcrResult> (unordered)
[Reorder Buffer]
     ↓ ordered stream
[Dedup & Stitch Thread]
     ↓
[Structure Recovery]
     ↓
[Output Writer — TXT or DOCX]
Enter fullscreen mode Exit fullscreen mode

Every queue is bounded. Backpressure propagates upstream automatically — if OCR workers slow down, the work queue fills, the preprocessor blocks, the frame filter blocks, the decoder blocks. RAM stays bounded regardless of video length. A 3 hour video uses the same memory as a 5 minute one.

The Key Insight — Dwell Gating

The most important design decision in the whole tool is something I call dwell gating, and it's what separates Palimpsest from naive frame-by-frame OCR approaches.

Most people's first instinct for this problem is: detect when the frame is changing (scrolling happening) and capture those frames. This is completely backwards.

Here's why. The frames captured during active scrolling are:

  • The most motion-blurred frames in the video
  • The most heavily compressed by H.264/H.265 (encoder allocates fewer bits to moving content)
  • Transitional frames showing half old content, half new content simultaneously
  • The absolute worst possible input for Tesseract LSTM

What you actually want are the frames where the educator stopped scrolling and is talking about the content — because those frames are:

  • Perfectly sharp and still
  • Fully H.264 quality (encoder has all bits available for static content)
  • Complete content sections, not mid-scroll transitions
  • Exactly what the presenter intended to be read

Dwell gating works by tracking frame stability over time. A frame is only forwarded to OCR when it has been continuously stable for --stable-dwell-ms milliseconds (default 5000ms — five full seconds). During scrolling, the stability counter resets. During pauses, it accumulates. Only settled, stable, clean frames ever reach Tesseract.

The result: from 21,000 frames in a 5 minute 50 second video, only 5 frames were forwarded to OCR in normal operation. Five perfect frames instead of thousands of garbage ones. 94.56% accuracy on the output.

The Deduplication Problem

Even with dwell gating, consecutive captured frames overlap significantly. Someone scrolls down, pauses — you see the bottom half of section A and the top half of section B. Next pause — full section B. Next — overlap of B and C.

Palimpsest handles this with fuzzy suffix-prefix stitching via rapidfuzz:

  1. Take the tail of the current document (last --overlap-window characters, default 300)
  2. Take the head of the new frame's text (first --overlap-window characters)
  3. Run rapidfuzz::fuzz::partial_ratio to find the overlap boundary
  4. If overlap score > --overlap-threshold (default 75): find exact boundary, append only new content
  5. If score < threshold (fast scroll, content gap): insert [GAP IN CONTENT] marker, append full frame

A rolling seen-lines history handles cross-GAP deduplication — if someone scrolls back up and the same content reappears later, it's detected and skipped via fuzzy matching.

Hardware Backends

Two preprocessing backends, selected via --backend:

CPU (default) — standard cv::Mat pipeline, Gaussian blur, Otsu threshold, Hough deskew. Fully portable, zero dependencies beyond OpenCV.

OpenCL — same pipeline but using cv::UMat throughout, dispatched transparently to the iGPU via OpenCV's OpenCL backend. On startup, enumerates devices and prints which one was selected. Falls back to CPU silently if no OpenCL device found.

OCR always runs on CPU regardless of backend — Tesseract's OpenCL path is unreliable and slower on integrated GPUs due to driver overhead.

Benchmarks

Real hardware: AMD Ryzen AI 7 350 (8 cores, 16 threads), Radeon 860M iGPU (8 CU, 8GB shared), 16GB DDR5 @ 5600 MT/s, Arch Linux.

Video: 1920×1200, H.264 @ 6 Mbps, 60 FPS, 21,000 frames (5 minutes 50 seconds).

Normal operation (dwell mode):

Backend Forwarded to OCR Time Avg FPS Real-Time Factor
CPU 5 1m 06s 318 FPS ~5.3× faster than real-time
OpenCL 5 1m 05s 323 FPS ~5.4× faster than real-time

In dwell mode the bottleneck is purely the video decoder. CPU and OpenCL are identical because almost no frames reach preprocessing.

Preprocessing + OCR stress test (20,000 frames forced through full pipeline):

Backend Time Avg FPS Per-Frame
CPU 2m 47s ~120 FPS ~8.3ms/frame
OpenCL 2m 41s ~107 FPS ~9.4ms/frame

CPU beats OpenCL in the stress test because the bottleneck is Tesseract (CPU-only) and OpenCL dispatch overhead costs more than the preprocessing savings at this scale.

Accuracy benchmark on banking regulatory text (RBI circulars, committee names, legal language):

94.56% accuracy on a real-world banking regulation document. The errors were: one line missed due to fast scroll (tunable with --min-interval-ms), one phantom character from compression artifact (discipline → disciplinec), minor list prefix heuristic misfires. All fixable in a 5 minute review pass.

CLI Usage 

# Simplest run
./build/bin/palimpsest recording.mkv

# OpenCL backend + DOCX output + crop region
./build/bin/palimpsest recording.mkv -b opencl -f docx --crop 20,60,1880,1100 -o output.docx

# Config file (recommended for repeated use)
./build/bin/palimpsest recording.mkv -c config.csv

# Filter watermarks and recurring headers
./build/bin/palimpsest recording.mkv --ignore ignore_phrases.txt

# Print hardware info and exit
./build/bin/palimpsest recording.mkv -b opencl --noshit
Enter fullscreen mode Exit fullscreen mode

Key flags:

  • --stable-dwell-ms — how long a frame must stay stable before OCR (default 5000ms)
  • --crop x,y,w,h — exclude UI chrome, taskbars, window ribbons
  • --ignore — strip watermarks and recurring boilerplate
  • --dump-images — dump the exact frames that got OCR'd, for debugging
  • --bench-preprocess N — stress test mode, force N frames through full pipeline

Full flag reference and a documented config.csv are in the repo.

Dependencies — All Via CMake FetchContent

No vcpkg. No system package manager. One command builds everything from source:

git clone https://github.com/plexescor/Palimpsest.git
cd Palimpsest
./compile.sh
Enter fullscreen mode Exit fullscreen mode

CMake FetchContent pulls and builds:

  • OpenCV 4.11.0 (core, imgproc, imgcodecs, videoio only)
  • Tesseract 5.3.4 + Leptonica 1.84.1
  • rapidfuzz-cpp v3.0.4 (header only)
  • miniz 3.0.2 (for DOCX zip writing)
  • {fmt} 10.2.1
  • CLI11 v2.4.1

Runtime requirement: GStreamer or FFmpeg for video decoding (standard on any Linux with video playback), tessdata in ./tessdata/ or via TESSDATA_PREFIX.

How Much Code Did AI Write?

Almost all of it.

I designed the full pipeline architecture in conversation with Claude — every stage, the ring buffer design, the dwell gating concept, the dedup stitching algorithm, the backend abstraction interface, the DOCX XML structure. Then I wrote a detailed prompt describing the entire system and Claude generated the C++ implementation in one shot.

The AI wrote:

  • The lock-free SPSC ring buffer template
  • All 8 pipeline stage implementations
  • The CMakeLists.txt with FetchContent for all dependencies
  • The HOCR parser extracting bounding boxes from Tesseract output
  • The rapidfuzz overlap stitching logic
  • The Office Open XML writer producing valid .docx files
  • The OpenCL backend using cv::UMat
  • The progress reporting and CLI interface

What I contributed was the architecture, the dwell gating insight, the understanding of why motion-based extraction is wrong for this use case, the benchmark methodology, and the real-world test on actual banking lecture content.

The result was 94.56% accuracy on first real-world test. That's what happens when you understand the problem well enough to describe the solution precisely.

Who Is This For?

If you've ever had any of these problems:

  • Students with lecture recordings of someone scrolling through study material — banking exams, CA preparation, law, medicine, engineering
  • Researchers with video archives containing on-screen text
  • Professionals with compliance recordings of document walkthroughs
  • Developers with tutorial videos showing code they need to extract
  • Anyone whose source document is lost and only the recording survives

Palimpsest is free, runs locally, never uploads your content anywhere, processes any video length, and takes about a minute for a typical lecture recording.

Current Limitations

  • Linux actively tested (Arch / Ubuntu 22.04+). Windows should work with MSVC but is currently untested.
  • Best on recordings where the presenter pauses on content. Continuous non-stop scrolling with no dwell produces sparse output — lower --stable-dwell-ms or set it to 0 for motion-threshold mode.
  • ~5% of content may need manual correction — minor OCR artifacts, occasional missed lines from fast scrolls.
  • Compilation from source required. No prebuilt binaries yet.

What's Next

  • Prebuilt binaries for Linux and Windows
  • Windows testing and validation
  • Potentially a simple GUI wrapper for non-technical users

Try It 

git clone https://github.com/plexescor/Palimpsest.git
cd Palimpsest
./compile.sh
./build/bin/palimpsest your_recording.mkv
Enter fullscreen mode Exit fullscreen mode

Drop the output quality issues you hit in the comments — real feedback on diverse content types helps tune the defaults.

Built with C++17, OpenCV, Tesseract, rapidfuzz, miniz, {fmt}, CLI11. Named after ancient manuscript pages scraped clean and rewritten over.

GitHub: https://github.com/plexescor/Palimpsest
Enter fullscreen mode Exit fullscreen mode

Top comments (0)