Every time someone benchmarks an audio model on a laptop, the same thing happens: the spec sheet says real time, the stopwatch disagrees, and the gap gets blamed on "optimization".
Most of the time the model is fine. The bottleneck is memory traffic, and you can estimate it before you run anything.
The number that actually matters
A diffusion-style audio model does not touch its weights once per track. It touches them once per denoising step.
Take a 1.5B parameter model at fp16. That is roughly 3 GB of weights. If the sampler runs 1,500 steps, the model streams about 4.5 TB through the memory bus to produce a single clip.
Now divide by real bandwidth. A laptop with 150 GB/s of unified memory gets:
4.5 TB / 150 GB/s = 30 seconds
That is RTF ~ 1.0 — exactly real time, with no optimization and no discrete GPU. The spec sheet and the stopwatch agree once you account for the right quantity.
Why quantization moves the needle so much
The step count is fixed by the sampler. The bytes per step are not.
At 4-bit weights the same model is roughly 0.75 GB. Same 1,500 steps, same bus:
0.75 TB / 150 GB/s = 5 seconds
RTF drops to about 0.4. Nothing about the architecture changed. You moved four times fewer bytes, so you finished four times sooner. "We quantized it and it got 3x faster" is not a trick, it is arithmetic.
What actually surprises people
Two things get misattributed constantly.
Long clips do not scale linearly in quality. Most models are trained on short windows. Past roughly 30 seconds, structure starts to drift: the same chord loop returns, a vocal stops matching the lyric. That is a modeling limit, not a speed problem, and quantization will not touch it.
First-run latency is not throughput. Loading weights from disk is a one-time cost. If your first generation takes 40 seconds and the second takes 12, you were measuring cold start.
Measure these three
- RTF — generation seconds divided by audio seconds
- Peak RSS — whether the model fits, or spills into swap
- Bytes per step — the quantity you can predict before running anything
If RTF is worse than bytes divided by bandwidth predicts, the problem is somewhere else: swap, a CPU fallback, or a sampler doing more work than you assumed.
The useful habit is to do this division before you install anything. It tells you in thirty seconds whether the model can run at real time on the machine you already own, and it tells you which knob will help.
Disclosure: I work on Creatune AI Music Generator. This arithmetic is the same one we use when deciding what to run locally.
Top comments (0)