How I Optimized FFmpeg Filters with Slice Threading (My First Contribution)

#c #opensource #ffmpeg #performance

Contributing to FFmpeg has always been a "final boss" level goal for me. As a Mathematics and Computing Engineering student aiming for Google Summer of Code (GSoC), I knew I needed to do more than just fix a typo. I wanted to touch the core performance of the software.

My mission? Slice Threading.

FFmpeg has hundreds of video filters, but many of them still run on a single CPU core. My task was to take vf_alphamerge (which merges an alpha channel into a video) and vf_blackframe (which detects black frames), and modernize them to run in parallel across all available cores.

Here is the story of how I modified the code, broke the Windows build, and learned to love C90 strictness.

The Challenge: Why was it slow?

Video processing is computationally expensive. In the original code for vf_alphamerge, the pixel processing happened in a simple, nested for loop. This meant that even though my laptop has 16 logical cores, the filter was only using one of them. The other 15 sat idle.

Here is a simplified look at the "Before" code:

// The old, single-threaded way
for (int y = 0; y < height; y++) {
    for (int x = 0; x < width; x++) {
        // ... heavy pixel copying logic ...
        // This runs sequentially, row by row.
    }
}

The Solution: Slice Threading

To fix this, I needed to implement Slice Threading. This technique splits the video frame into horizontal "slices" (strips) and hands each slice to a different thread to process simultaneously.

I replaced the standard loop with FFmpeg's avfilter_execute callback system. I moved the processing logic into a new function alphamerge_slice and created a ThreadData struct to pass pointers.

It felt great. The code compiled on my Linux machine, the logic seemed sound, and I confidently sent my patch to the ffmpeg-devel mailing list using git send-email.

The "Red 1" Error: A Lesson in C90

Then, I saw it. The dreaded Red "Fail" on the Patchwork dashboard.

My patch passed on Linux and macOS, but the Windows (MSVC) build bot failed with a compilation error. I was confused. It compiles fine for me! What is wrong?

The issue wasn't my logic; it was my standard. FFmpeg adheres to the strict C90 (ISO C) standard because it needs to run on everything from supercomputers to ancient embedded devices. The Microsoft Visual C++ compiler (MSVC) is notoriously strict about this.

The Mistake: I had declared variables inside loops and if blocks—standard practice in modern C++ or Python, but illegal in C90.

// ❌ C99 Style (What I wrote initially)
if (s->is_packed_rgb) {
    int x = 0; // Error! Declaration after statement
    // ...
}

I had to refactor the entire function, forcing every single variable declaration to the absolute top of the function scope.

// ✅ C90 Style (What FFmpeg demands)
int x;
if (s->is_packed_rgb) {
    x = 0;
    // ...
}

It felt tedious, but it taught me a valuable lesson about writing portable, low-level code.

Verification: Debugging with GDB

Fixing the compiler error was one thing, but how could I prove it was actually multithreaded?

I fired up GDB (GNU Debugger). This led to my favorite moment of the project. I set a breakpoint inside my new slice function and inspected the nb_jobs (number of jobs) variable.

(gdb) break alphamerge_slice
(gdb) run
...
[New Thread 0x7fffb2ff56c0 (LWP 25133)]
Thread 42 hit Breakpoint 1, alphamerge_slice (jobnr=2, nb_jobs=16)

nb_jobs = 16.

Seeing that number confirmed that FFmpeg was spinning up 16 separate threads to crunch the video data. I also learned a cool GDB trick: set scheduler-locking on. Without this, the threads raced each other so fast that my debugger kept jumping between them!

Conclusion

This contribution taught me that Open Source isn't just about writing algorithms. It's about:

Respecting Standards: Legacy support matters (C90 vs C99).
Tooling: Mastering git send-email and GDB is a superpower.
Persistence: A "Fail" on the dashboard isn't a rejection; it's just a to-do list item.

My patches for vf_alphamerge and vf_blackframe are now under review. If you are a student scared to contribute to open source, just dive in. The errors are scary, but solvable.