DEV Community

hao jia
hao jia

Posted on

Alpha is just a plane in the pixel format, and FFmpeg drops it in three quiet places

"Transparent video" sounds like a property of a file, but inside FFmpeg it is a property of a frame in memory: either the frame has a fourth plane, or it does not. FFmpeg names that difference yuva420p versus yuv420p. Every step on the path keeps the plane, drops it, or makes up a new one. In the cases I ran, none of the steps printed an error when it dropped the plane.

The clip is one 8-second, 540×960 WebM with VP9 alpha, exported from ImgIng (https://imging.ai/), a browser tool for video background removal. It is Beta, it is built for desktop Chrome and Edge, it processes the video locally, and it does not run on phones. Everything below is that one clip, FFmpeg 7.1 on one machine, one run each. I did not try other FFmpeg builds or any editing software.

The decoder can hand over a frame with no alpha plane

The first drop is at the decoder. With the default VP9 decoder, ffmpeg -i prints the stream as yuv420p. Name libvpx-vp9 as the decoder and the same stream is printed as yuva420p. The extra a is the whole difference, and it is the only hint you get.

On frame 60, the default decoder gives an alpha minimum, average and maximum of 255, so every pixel is opaque. With libvpx-vp9 the average is 46.7618 and the minimum is 0. I read a raw frame from the correct path as well: 1,296,000 bytes, which is 518,400 for luma, two chroma planes of 129,600 each, and 518,400 for alpha at full luma size. So the alpha plane is a real, full-size plane, and its absence on the other path is not a rounding problem. Why the default decoder skips it, I did not look into.

A filter chain can throw the plane away for good

The second drop happens even when the decoder is already right. Frame 60 again, decoded with libvpx-vp9, then two filter chains that differ only in a detour.

$ ffmpeg -hide_banner -c:v libvpx-vp9 -i input.webm -vf "select=eq(n\,60),format=yuva420p,alphaextract,signalstats,metadata=print" -an -f null -
lavfi.signalstats.YMIN=0
lavfi.signalstats.YAVG=46.7618
lavfi.signalstats.YMAX=255

$ ffmpeg -hide_banner -c:v libvpx-vp9 -i input.webm -vf "select=eq(n\,60),format=yuv420p,format=yuva420p,alphaextract,signalstats,metadata=print" -an -f null -
lavfi.signalstats.YMIN=255
lavfi.signalstats.YAVG=255
lavfi.signalstats.YMAX=255
Enter fullscreen mode Exit fullscreen mode

The second chain leaves the plane and comes back, and what comes back is a plane filled with 255. Stats trimmed to the three lines; the rest of the log is omitted. I tried two more detours, through rgb24 and through gray, and both gave 255 as well. Staying inside formats that have alpha was fine: yuva420p, rgba and yuva444p all kept the 46.7618 average.

This matters because a pipeline rarely has one filter. A chain can pick up format=yuv420p early to keep a scaler happy and format=yuva420p later "to be safe". The second one looks like it restores the transparency. It does not. It only makes the stream claim to have alpha. The same thing happens on the default-decoder path: forcing format=yuva420p after it still reads 255 on frame 60.

The encoder's pixel format gets negotiated for you

The third drop is in the encoder. When the output codec does not take the incoming pixel format, FFmpeg inserts a conversion on its own. Running with -v verbose, decoding with libvpx-vp9 and encoding with libx264 without any -pix_fmt, the log has a line saying it is auto-inserting auto_scale_0, and another showing the conversion fmt:yuva420p -> fmt:yuv420p. The command succeeded and the output stream is yuv420p. H.264 has no alpha, so this is correct behavior. I only ran this with -v verbose, so what the default log level prints for the same conversion is untested.

The same conversion helps explain the worst case in these runs. Default decoder into ProRes 4444 with -pix_fmt yuva444p10le: the log shows auto_scale_0 converting fmt:yuv420p -> fmt:yuva444p10le, and the output stream line says yuva444p10le. The file looks like an alpha file by its metadata. Reading the alpha back gives 255 everywhere. That the converter supplies an opaque plane because the input had none is my inference from the readback, not something I inspected in the filter.

Read the alpha back after the step you suspect

The stream line is a first gate, not the last one. If the line right after the decoder does not say yuva, nothing downstream can fix it, and a later format=yuva... only paints a plane of 255. For anything that has to cross a team boundary, such as handing a clip to an editor, run the finished file back through the alphaextract,signalstats pair above and read the minimum. A minimum of 255 means the file is opaque, whatever its stream line says.

For this clip the number to compare against is 0 / 46.7618 / 255 on frame 60.

Top comments (0)