How I extract burned-in subtitles from TikTok videos with Python (and why AWS/Google are overkill)

#ai #opensource #python #webdev

If you've ever tried scraping content or transcripts from TikTok, Instagram Reels, or YouTube Shorts, you've probably noticed a common issue: most social media videos don't have separate .srt caption tracks.

Creators hardcode animated captions, promo codes, and visual hooks directly into the video pixels.

When I started building automated video pipelines, I looked at Google Cloud Video Intelligence and AWS Rekognition. But charging $0.10 to $0.15 per video minute gets expensive fast (processing a few hundred hours can cost thousands of dollars).

Here is the lightweight architecture I built in Python to extract hardcoded subtitles into clean .srt files on a standard GPU (or CPU) without breaking the bank.

1. The 3 Bottlenecks with Traditional Video OCR

If you just run ffmpeg + Tesseract on every single video frame, three things happen:

CPU Choke: Running OCR on 30 frames per second takes 10+ minutes for a 60-second clip.
Duplicate Spam: Text staying on screen for 3 seconds produces 90 duplicate lines without proper timestamps.
Flickering Typos: Character recognition varies slightly from frame to frame, breaking continuity.

2. The Solution: Frame-Diffing + Temporal Fusion

To solve this, the pipeline uses two simple optimizations:

Step A: Frame Difference Skipping (SSIM)

Instead of feeding every frame to the OCR model, we sample at 1-2 fps and compare consecutive frames with structural similarity. If the pixels didn't change significantly, we skip neural OCR inference. This cuts GPU compute time by ~80%.

Step B: Temporal Text Deduplication

Adjacent detections that share high Levenshtein text similarity (>85%) are merged into a single continuous timecode (00:00:01,000 --> 00:00:04,500).

3. Python Implementation

Here is how you can use the open-source package:

pip install tiktok-subtitle-ripper

In your Python script:

from tiktok_subtitle_ripper import rip_video

# Pass any TikTok, Reels, or direct MP4 link
rip_video("https://www.tiktok.com/@creator/video/1234567890", output_srt="subtitles.srt")

Or directly from the terminal:

tiktok-rip "https://www.tiktok.com/@creator/video/1234567890" -o my_subtitles.srt

4. Free Web Demo & API

I also put together a free web demo on Hugging Face Spaces where you can paste any link and test it without installing anything:
👉 TikTok & Reels Subtitle Ripper on Hugging Face Spaces

For high-volume scraping pipelines, I also published the API backend on RapidAPI (starting at $2.99/mo).

Would love any feedback or edge cases where font outlines get tricky!

DEV Community