DEV Community

Mason K
Mason K

Posted on

Benchmark NVENC vs CPU transcoding (and find your real break-even) with FFmpeg

#video #ffmpeg #tutorial #devops

📦 Code: github.com/USER/nvenc-vs-cpu-bench - replace before publishing

TL;DR

A GPU encodes faster than a CPU, but "faster" and "cheaper" are different claims. We'll build a small FFmpeg + VMAF harness that times software (libx264/SVT-AV1) against hardware (h264_nvenc/av1_nvenc), then plug the results into a dollars-per-encoded-minute formula so you find your break-even instead of trusting a benchmark blog.

We're using FFmpeg 7.1.x (current stable line) and an NVIDIA GPU with NVENC. Same approach works for Intel QSV (*_qsv) and AMD AMF (*_amf) if you swap the encoder names.

Why this isn't obvious

NVENC is a fixed-function hardware block, not "the GPU doing x264 in parallel." It's extremely fast and barely touches the CPU, but it exposes fewer rate-control knobs and gives up a little compression efficiency versus a slow software preset. The gap has narrowed a lot, but it's still there at the quality-obsessed end.

So the decision is per-job, and it comes down to one number: dollars per encoded minute = (instance $/hr) ÷ (minutes encoded/hr). GPU instances cost more per hour but encode many streams in parallel, so the answer depends on whether you can keep the encoder saturated.

Let's measure instead of argue.

1. Set up the encoders

Three contenders. One representative source file (use real footage, not a synthetic clip).

# software H.264, quality-leaning preset
ffmpeg -y -i source.mp4 -c:v libx264 -preset slow -crf 21 -an out_cpu.mp4

# NVENC H.264, quality-tuned
ffmpeg -y -hwaccel cuda -i source.mp4 -c:v h264_nvenc -preset p6 -tune hq \
  -rc vbr -cq 23 -an out_gpu.mp4

# AV1: software (SVT-AV1) vs hardware (needs Ada / RTX 40+)
ffmpeg -y -i source.mp4 -c:v libsvtav1 -preset 6 -crf 30 -an out_svtav1.mp4
ffmpeg -y -hwaccel cuda -i source.mp4 -c:v av1_nvenc -preset p5 -cq 30 -an out_av1nvenc.mp4
Enter fullscreen mode Exit fullscreen mode

💡 Tip: -preset p1 (fastest) through -preset p7 (slowest/highest quality) for NVENC. p6/p7 is where it competes on quality; p1-p3 is where it competes on raw throughput.

If av1_nvenc errors with Cannot load nvEncodeAPI or "encoder not found," your GPU is pre-Ada and doesn't have the AV1 block. That's fine, drop it from the comparison.

2. Time each encode 

# bench.sh - time an encode and print seconds
#!/usr/bin/env bash
set -euo pipefail
label="$1"; shift
start=$(date +%s.%N)
"$@" >/dev/null 2>&1
end=$(date +%s.%N)
printf "%s\t%.2fs\n" "$label" "$(echo "$end - $start" | bc)"
Enter fullscreen mode Exit fullscreen mode 
./bench.sh cpu      ffmpeg -y -i source.mp4 -c:v libx264 -preset slow -crf 21 -an out_cpu.mp4
./bench.sh nvenc    ffmpeg -y -hwaccel cuda -i source.mp4 -c:v h264_nvenc -preset p6 -tune hq -rc vbr -cq 23 -an out_gpu.mp4
Enter fullscreen mode Exit fullscreen mode

Realistic output:

cpu     63.40s
nvenc   18.10s
Enter fullscreen mode Exit fullscreen mode

Faster, clearly. But don't migrate yet, we haven't checked quality or cost.

3. Measure quality with VMAF (not bitrate)

Bitrate alone tells you nothing about whether the picture held up. Use libvmaf:

ffmpeg -i out_gpu.mp4 -i source.mp4 \
  -lavfi "[0:v]setpts=PTS-STARTPTS[dist];[1:v]setpts=PTS-STARTPTS[ref];[dist][ref]libvmaf=log_fmt=json:log_path=vmaf_gpu.json" \
  -f null -

python3 -c "import json;print('VMAF', json.load(open('vmaf_gpu.json'))['pooled_metrics']['vmaf']['mean'])"
Enter fullscreen mode Exit fullscreen mode

Run it for each output. You'll typically see software at a slow preset edge out NVENC by a few VMAF points at a similar file size, and the two converge as bitrate climbs. A few points of VMAF may not matter for streaming; it absolutely matters for an archival master.

4. The break-even math

Now put it together. Say:

  • CPU instance: $0.20/hr, encodes 1 stream → ~57 min/hr of this content.
  • GPU instance: $0.55/hr, encodes ~8 of these streams in parallel → ~456 min/hr. 
# breakeven.py
cpu = {"cost_per_hr": 0.20, "minutes_per_hr": 57}
gpu = {"cost_per_hr": 0.55, "minutes_per_hr": 456}

for name, m in {"cpu": cpu, "gpu": gpu}.items():
    print(name, "$/encoded-min =", round(m["cost_per_hr"] / m["minutes_per_hr"], 5))
Enter fullscreen mode Exit fullscreen mode 
cpu $/encoded-min = 0.00351
gpu $/encoded-min = 0.00121
Enter fullscreen mode Exit fullscreen mode

The GPU wins here, but only because it's saturated (8 parallel streams). Drop minutes_per_hr to 57 (one stream, idle GPU) and the GPU becomes more expensive than the CPU. That's the whole lesson: the GPU is cheaper per minute only when you actually feed it.

⚠️ Note: these instance numbers are placeholders. Plug in your cloud's real prices and your measured parallel-stream count. The break-even is yours, not mine.

5. Saturate the GPU (or the math lies)

The break-even above assumed 8 parallel streams. That number is not free; you have to actually run encodes in parallel to hit it, because one ffmpeg process will not max out a modern encoder block on its own. Measure your real parallel ceiling:

# parallel-throughput.sh - how many concurrent NVENC encodes before it degrades
#!/usr/bin/env bash
set -euo pipefail
N="${1:-8}"
start=$(date +%s.%N)
for i in $(seq 1 "$N"); do
  ffmpeg -y -hwaccel cuda -i source.mp4 -c:v h264_nvenc -preset p5 \
    -rc vbr -cq 23 -an "out_$i.mp4" >/dev/null 2>&1 &
done
wait
end=$(date +%s.%N)
echo "encoded ${N} streams in $(echo "$end - $start" | bc)s"
Enter fullscreen mode Exit fullscreen mode 
./parallel-throughput.sh 8
# encoded 8 streams in 22.40s
Enter fullscreen mode Exit fullscreen mode

Watch nvidia-smi while it runs; the Enc utilization column tells you whether you are actually feeding the block or just queuing. If utilization plateaus below 100% at, say, 6 streams, then 6, not 8, is your real divisor in the cost formula. Consumer cards also cap concurrent NVENC sessions in the driver, so a datacenter card may be required to reach high parallelism at all.

6. Rate control matters more than preset

For a transcoding pipeline you usually want consistent quality at a predictable size, which means the rate-control mode is a bigger lever than the preset:

Mode NVENC flag Use when
Constant quality -rc vbr -cq N VOD, you want stable quality, variable size
Capped CRF-like -rc vbr -cq N -maxrate M -bufsize 2M ABR ladder rungs with a bitrate ceiling
Constant bitrate -rc cbr -b:v M Live, fixed-bandwidth delivery 
# ABR rung: quality target with a hard ceiling so the rung stays in its lane
ffmpeg -y -hwaccel cuda -i source.mp4 -c:v h264_nvenc -preset p6 -tune hq \
  -rc vbr -cq 23 -maxrate 4M -bufsize 8M -an rung_1080.mp4
Enter fullscreen mode Exit fullscreen mode

⚠️ Note: NVENC's -cq is not the same scale as x264's -crf. Do not copy a CRF value across; pick -cq by measuring VMAF on your own footage.

Two more flags claw back most of the remaining quality gap with software, at almost no throughput cost: enable B-frames and a lookahead window so the encoder can plan bit allocation across upcoming frames.

ffmpeg -y -hwaccel cuda -i source.mp4 -c:v h264_nvenc -preset p6 -tune hq \
  -rc vbr -cq 23 -bf 3 -rc-lookahead 20 -spatial-aq 1 -an out_tuned.mp4
Enter fullscreen mode Exit fullscreen mode

Re-run your VMAF check after adding these; on most content they lift the score a point or two for free, which can be the difference that makes NVENC "good enough" for your quality bar.

When to pick which

Workload Pick Why
High-volume / near-real-time / live NVENC Throughput dominates; keeps the block saturated
Large catalog, steady upload firehose NVENC Cost-per-minute wins when utilized
Archival / mastering ladders Software (x264/x265) Quality-per-bit compounds forever
AV1 for storage savings SVT-AV1 (software) Currently beats av1_nvenc on efficiency
Spiky, low-volume jobs Software Idle GPU is wasted money

What's next

  • Wire the timing + VMAF steps into CI so a preset change can't silently tank quality or throughput.
  • Try QSV (h264_qsv, av1_qsv on Arc) if you're on Intel, the same harness works.
  • Measure with footage that matches your real catalog; grain, motion, and resolution all move the numbers.

Top comments (0)