The state of media processing in Rust (2026): what each crate actually covers

Every few months someone asks how to decode video or pull audio samples in Rust, and the thread converges on the same advice: just run the ffmpeg binary. This post is a map of why — six paths through the ecosystem as of July 2026, what each one actually covers, how it is maintained, and where each one honestly wins. All numbers are monthly downloads or GitHub stars as verified in July 2026; where I could not verify a number, the table shows a dash instead of a guess.

A note on method: all six paths get the same treatment — where it wins, the honest limitation, a one-line verdict — and the limitation column does not soften for whichever path is newest. One disambiguation up front: the Rust crate ez-ffmpeg discussed below is unrelated to the JavaScript "Ez FFmpeg" that once made the HN front page; same name, different thing.
One more ground rule: there are no benchmarks in this piece — runnable benchmarks later beat adjectives now.

The landscape at a glance

Path	Execution model	Coverage highlights (verified)	Key limitations (verified)	Data point (July 2026)
std::process::Command + ffmpeg CLI	subprocess	everything the CLI can do; every recipe on the internet applies as-is	ship the binary; data re-enters your process via stdout/temp files	FFmpeg 8.0 is out; its Whisper support hit the HN front page (1033 points)
ffmpeg-sidecar	subprocess	iterate any video "as if it were an array of raw RGB frames"; no libav linking	still a child process; an ffmpeg executable must be present at runtime	~131K downloads/month
ffmpeg-next	in-process (low-level bindings)	the full libav control plane; the ecosystem's foundation	"maintenance-only mode"; hand-written decode loops (~40 lines before the first frame); third fork in a lineage	1.28M downloads/month
rsmpeg	in-process (safe-wrapper style)	tracks FFmpeg 6/7/8 (0.18.0 added 8.0)	slow cadence: roughly 11 months without major movement	877 stars
video-rs	in-process (high-level)	RGB ndarrays in ten lines; README encodes rainbow.mp4 from ndarray frames; seek_to_frame	no audio API; self-described "still a work-in-progress… some parts not flushed out", "Use with caution"; successor rave "not ready for use yet"	~32K downloads/month
symphonia + rubato	in-process (pure Rust)	demux + decode audio, resample to 16 kHz; symphonium does it in one call; mutter feeds whisper-rs in five lines	HE-AAC and Opus listed unfinished; no AC-3; video out of scope	symphonia ~1.07M downloads/month, 0.6.0 (May 2026); whisper-rs ~170K/month (maintained on Codeberg)
ac-ffmpeg	in-process	touches all three domains (video, audio, encoding)	stays at the packet-loop level; no release in 14 months	—
ez-ffmpeg	in-process (runtime)	API and internal scheduling modeled 1:1 on the FFmpeg CLI — command-line knowledge transfers as-is; 0.15 runs pasted ffmpeg commands in-process (cli facade, verified subset); FrameExtractor / SampleExtractor / VideoWriter as first-class APIs; color-tag-correct YUV→RGB by default	still links libav — the build pain stays; new APIs marked experimental; early-stage	~340 stars

A dash means I could not verify it, and nothing in this article depends on it.

One boundary note: gstreamer-rs is a real option too — ~564K downloads a month and actively maintained — but it is neither FFmpeg-based nor a few-lines-of-code style of API, so it sits outside this FFmpeg-centered map.

Path 1: std::process::Command + the ffmpeg binary

Where it wins. Thirty years of FFmpeg knowledge exists in command-line form, and all of it applies unchanged. The CLI cookbook culture is alive and huge — "FFmpeg by Example" alone scored 920 points on HN. No FFI, no linking, no version matrix. On rustcc, a Chinese Rust forum, a thread on these bindings landed on shelling out as "the most reliable option, with far fewer pitfalls" (translation mine) — not a joke, but the rational endpoint of a lot of scar tissue.

The honest limitation. You ship an ffmpeg binary with your app, and the moment data has to come back into your process — frames, PCM samples — you are parsing stdout or juggling temp files. jaredly's issue from 2015 describes the pattern exactly: "rendering to images then using ffmpeg on the cli". A decade later, plenty of people still work that way. Not because it is good — because the alternatives were never paved.

Verdict: when the output is a file and the recipe exists, still the right answer. When data must land in memory, the tax meter starts.

Path 2: ffmpeg-sidecar — the subprocess, made livable

Where it wins. ~131K downloads a month, and a self-description that gets straight to the point: treat any video "as if it were an array of raw RGB frames". It turns shelling out into a structured API — you write an iterator, not a stdout parser. And because it talks to ffmpeg over stdin/stdout, it never links libav: the entire build/link/platform circus does not apply to you. If you want frames without touching FFI, this is the most comfortable path in the list.

The honest limitation. It is still a child process. An ffmpeg executable has to exist wherever you deploy, and every frame crosses a process boundary. In-process filter graphs and fine-grained control are outside this model. Whether the boundary matters for your workload is a measurement, not an opinion — this survey quotes no numbers for it.

Verdict: if a child process is acceptable, the most comfortable way to get frames.

Path 3: the raw bindings — ffmpeg-next and rsmpeg

Where it wins. When you need the full libav control plane — custom IO, obscure containers, packet-level surgery — nothing else covers that surface. ffmpeg-next sits at 1.28M downloads a month; ez-ffmpeg itself links libav through it. That number is the measure of how much of the ecosystem stands on this layer.

The honest limitation. Start with maintenance: ffmpeg-next is frozen by policy, not abandoned — the maintainer's own words are "maintenance-only mode for the most part… Any PR to improve existing API is unlikely to be merged" — and the crate sits in a fork lineage already three generations long: ffmpeg → ffmpeg-next → ffmpeg-the-third. Then the daily cost: decoding a frame means hand-writing the send_packet/receive_frame ceremony, roughly forty lines before you see pixels, and even "transcode audio and video together" can send you searching — one developer's verdict after going through the examples: "Examples… do not include transcoding of audio+video". When kornel wrote "avoid ffmpeg… all incomplete and poorly maintained", this category's overall state is what he was describing.

One counterintuitive fact about this layer: even a wrapper sold on safety has shipped public APIs annotated "might trigger undefined behavior" (zmwangx/rust-ffmpeg#225). A safe skin over an FFI skeleton is an engineering promise, not a proof.

On rsmpeg specifically: 877 stars, hosted under the larksuite organization, supporting FFmpeg 6, 7 and 8 (0.18.0 added 8.0). The cadence is slow — roughly 11 months without major movement — but calling it dormant overstates it. If you need a low-level wrapper that tracks current FFmpeg majors, it is the fresher one.

Verdict: as a foundation, indispensable; as a daily API, you pay full price.

Path 4: video-rs — high-level video, self-described work in progress

Where it wins. For years this has been the one serious high-level option (~32K downloads/month). PyAV-style decode/encode has been there since 2023: RGB ndarrays within ten lines, a README whose example encodes rainbow.mp4 from ndarray frames, and a seek_to_frame (frame accuracy undeclared). If you want a PyAV-like video experience in Rust today, this is the most convenient existing answer.

The honest limitation. The limitations are written in its own README: "still a work-in-progress… some parts not flushed out", "Use with caution". There is no audio API. The official successor, rave, aims to move off FFmpeg entirely — and is "not ready for use yet".

Verdict: strong on video, and video only; no audio plus self-declared WIP keeps it a specialist.

Path 5: symphonia + rubato — pure Rust until the codec wall

Where it wins. Pure Rust can pull the audio track out of an MP4/MKV and resample it to 16 kHz (symphonia + rubato; symphonium does both in one call; mutter feeds whisper-rs in five lines). The decode-and-resample chain is Rust top to bottom — of the six paths here, the only one that involves no C artifact at any point: no libav to link, no ffmpeg binary to ship. If a clean dependency tree is the goal, this path owns it. One correction to a rumor while I am here: whisper-rs is archived on GitHub but actively maintained on Codeberg, at ~170K downloads a month — do not file it under dead.

The honest limitation. "Any video, any codec, one line" hits a codec wall: symphonia still lists HE-AAC and Opus as unfinished and has no AC-3 at all — which is exactly where an FFmpeg backend remains irreplaceable. And video is out of scope by design.

Verdict: inside its codec table, the cleanest dependency story on this list; outside it, you are back to FFmpeg.

Path 6: ez-ffmpeg — the in-process runtime

The project's stated aim: "the actively-developed high-level in-process FFmpeg runtime for Rust."

The core is not any single feature — it is the ergonomics of moving an FFmpeg command into Rust unchanged. One builder chain reads like one command:

FfmpegContext::builder()
    .input("input.mkv")
    .output(
        Output::from("output.mp4")
            .set_video_codec("libx264")            // -c:v libx264
            .set_video_codec_opt("crf", "23")      // -crf 23
            .set_video_codec_opt("preset", "fast") // -preset fast
            .set_audio_codec("aac"),               // -c:a aac
    )
    .build()?
    .start()?
    .wait()?;

Migration is mechanical, because nothing has to be relearned: seconds become microseconds (-ss 10 → set_start_time_us(10_000_000)); option names work verbatim minus the leading dash (crf, preset, movflags go through set_*_opt straight into FFmpeg's AVOption system — whatever name the CLI accepts, this accepts); the string after -vf drops into filter_desc character for character; stream selection uses the CLI's own specifier syntax (0:a:0). Years of command-line muscle memory stay valid, line for line.

As of 0.15 even the translation step is optional — the command can be pasted in as-is. With the cli feature enabled, from_cli takes a whole ffmpeg command string, parses it, builds the pipeline, and runs it inside your own process — no subprocess:

use ez_ffmpeg::cli::from_cli;

fn main() -> Result<(), Box<dyn std::error::Error>> {
    from_cli("ffmpeg -i input.mp4 -c:v libx264 -crf 28 -preset veryfast -c:a aac -y output.mp4")?
        .start()?
        .wait()?;
    Ok(())
}

The attitude behind it is the part worth noting. Commands sort into three classes: verified shapes may run (six today, each reconciled against the real ffmpeg CLI by a semantic golden suite); unverified ones are translated but never executed; unrecognized ones are rejected on the spot, anchored to the offending token. A sibling function, emit_rust_code, turns the same command into a compile-ready builder program you can extend and maintain. It would rather refuse you clearly than "run, but come out subtly different from ffmpeg."

The isomorphism does not stop at the API surface — it runs down to the source. The internal scheduler is modeled on the ffmpeg CLI's own architecture: fftools' ffmpeg_demux.c / ffmpeg_dec.c / ffmpeg_filter.c / ffmpeg_enc.c / ffmpeg_mux.c / ffmpeg_sched.c / sync_queue.c map file for file to demux_task / dec_task / filter_task / enc_task / mux_task / ffmpeg_scheduler / sync_queue. Two practical consequences: when behavior has to match the CLI, there is a line-by-line reference to diff against — a divergence can be traced to its fftools counterpart; and if you have ever wanted to understand FFmpeg's internals, this codebase reads as a typed, ownership-annotated companion to fftools — a far gentler slope than the C.

On the feature side: history first, then the new parts. Even in 0.13, ez-ffmpeg let you intercept and rewrite every frame via FrameFilter — the repository ships custom tile/volume filter examples. 0.14 adds the dedicated FrameExtractor / SampleExtractor / VideoWriter — turning "possible" into "one line", with color-tag-correct YUV→RGB by default. Concretely: FrameExtractor pulls decoded frames with sampling policies (EveryNth / UniformN / Keyframes); SampleExtractor hands you 16 kHz mono f32 ready for whisper-rs; VideoWriter pushes frames you rendered yourself into an encoding pipeline (write / write_owned), with filter-graph validation; its output surface covers files and RTMP live targets (the live-target claim is API-surface and module-doc backed, not exercised against a live server here). The color detail deserves its own sentence: HD video converts to RGB per BT.709 by default, whereas to_ndarray('rgb24') in older PyAV releases applied BT.601 across the board, which skews saturated colors.

Who should not use ez-ffmpeg

• If linking FFmpeg is your blocker, ez-ffmpeg does not remove it. It links libav through ffmpeg-next — the very layer criticized above, whose frozen maintenance and fork-lineage risk it inherits wholesale. The developer who wrote "I've spent the last 12 hours… willing to spend another week" fighting the build would fight the same build here. What it removes is the hundred-line pipeline after linking succeeds — not the linking.
• The three new APIs are marked experimental in the README. That word is there on purpose. The cli facade has boundaries of its own: it is an optional feature you must enable, execution currently requires an FFmpeg 7.1 runtime (anything else fails before any I/O), and the subset is deliberately narrow — only those six verified shapes run; two-pass encoding, complex filtergraphs and hardware acceleration are outside it.
• It is early-stage: ~340 stars, and effectively one maintainer — bus factor is a real risk. A safe Rust API surface is not the same thing as freedom from C CVEs — libav is still underneath, and the wrapper carries its own unsafe glue: the "safe skin over an FFI skeleton" line above applies to this crate too.
• No parity claims with decord's GPU batch decoding or torchaudio's multi-backend DSP.
• If a pure-Rust supply chain is the goal, symphonia is your answer — not ez-ffmpeg.

Verdict: if you can already write the ffmpeg command, this is the lowest-migration-cost way to run it inside your own process; when frames and samples must land in memory with FFmpeg-grade format coverage, it is also the shortest path. When linking is the problem, nothing on this layer saves you.

The set-difference that 0.14 targets

Let me state the full picture first, so nothing here reads as a strawman: Rust has never been unable to hand you decoded frames — video-rs iterates RGB ndarrays in ten lines, and ffmpeg-sidecar turns any video into an RGB frame iterator (via subprocess). What was missing is a one-liner with sampling policies and correct color handling inside a single in-process, FFmpeg-grade runtime.

So the set-difference looks like this: in Python that's three libraries — PyAV, decord, torchaudio. In Rust, video-rs skips audio, symphonia skips video, sidecar forks a process. ez-ffmpeg 0.14 is the first to put all three jobs inside one in-process FFmpeg runtime.

And the scope of that claim, stated precisely: as far as I can verify (July 2026), ez-ffmpeg is the only actively-maintained in-process FFmpeg runtime in Rust where decoded-RGB export, 16 kHz f32 audio export, and frame-push encoding are all first-class APIs: video-rs has no audio API at all and self-declares WIP (its successor rave is "not ready for use yet"), while ac-ffmpeg covers all three domains but at packet-loop level, with no release in 14 months. gstreamer-rs's appsink/appsrc can genuinely do all three in-process too — but through a different pipeline model, not an FFmpeg runtime, which is exactly why the boundary note above places it on a different map.

Two honest footnotes. First, PyAV is itself a set of libav bindings — ez-ffmpeg runs on the same engine. So the claim is not "Rust rewrote PyAV"; it is the same engine wearing a native high-level Rust face, without raw FFI and without a child process. Second, why this gap is worth closing now: candle, burn, ort, tract and whisper-rs are all growing, and what they need is frames and audio as data — while step one of feeding them, today, is often a line of shell. For a sense of how absurd the gap can get: someone implemented a decord-style video reader in Rust — video_reader-rs — and ships it only on PyPI. You cannot cargo add it.

When shelling out is the right answer

A selection guide that funnels every case toward one crate is an ad. For these, the plain CLI is the right answer:

• The output is a file and the recipe exists. Transcodes, clips, one-off batch jobs — the ML audio world's step one is still ffmpeg -i in.mp4 -ar 16000 -ac 1 -c:a pcm_s16le out.wav. If that line is your whole requirement, do not add a crate.
• Your environment ships the ffmpeg binary anyway. The deployment cost is already paid; no need to also pay the linking cost.
• You want process isolation. A crashed transcode job that cannot take down your service is a feature of the subprocess model, not a defect (an engineering judgment, not a quote).
• Your team knows the CLI, not the libav API, and the task is not on a hot path. The migration cost exceeds the benefit.

The flip side: the moment frames or samples must land in your process memory — feeding a model, real-time processing, logic interleaved with decoding — the CLI starts charging: stdout parsing, temp WAV files, mismatched parameters. jaredly's pattern, running from 2015 to today, is that tax's ten-year invoice.

Picking by task

• Output is a file, recipe exists → CLI
• Frames without FFI or linking → ffmpeg-sidecar
• Full libav control plane → ffmpeg-next (tracking newer FFmpeg majors → rsmpeg)
• Video frames only, WIP acceptable → video-rs
• Audio only, pure Rust, codecs within the support table → symphonia + rubato
• You already write ffmpeg commands and want them in-process at minimal migration cost → ez-ffmpeg (option names, filter strings and stream specifiers are all CLI syntax)
• Frames, samples and frame-push all in-process, FFmpeg-grade coverage → ez-ffmpeg (remember both caveats: the linking pain stays, and the new APIs are experimental)

Every path on this map wins somewhere; none of them replaces the others. Pick by task, not by tribe.

The Rust crate ez-ffmpeg lives at github.com/YeautyYE/ez-ffmpeg.