For thirteen releases, my Ogg/MP3 decoder's README said this:
it decodes the same bytes to the same samples on every platform
I believed it. The decoding happens in C, the C is compiled from the same source everywhere, the tests pass on macOS, Linux and Windows. Where would a difference even come from?
I was getting the package ready to freeze at 1.0.0, which is the point where a documented promise stops being a sentence and becomes a contract. So instead of reading the claim again, I tested it. It is false, and it has been false since the first release.
Testing a claim about two architectures needs two architectures
That sounds obvious written down. It is exactly what my CI did not do.
On an Apple laptop you can do the whole thing locally, because clang will cross-compile and Rosetta will run the result. The driver ships in the repo at doc/blog/cross-arch/detdrv.c; from a fresh clone, run these at the repo root:
cp doc/blog/cross-arch/detdrv.c .
clang -arch arm64 -O3 -Isrc/third_party -o det_arm64 detdrv.c src/*.c
clang -arch x86_64 -O3 -Isrc/third_party -o det_x86_64 detdrv.c src/*.c
detdrv.c is about fifty lines: read a file, call the package's own decode function, print the channel count, the sample rate, the frame count, and an FNV-1a hash of the decoded samples. Then run both binaries over the six test fixtures and compare the hashes. src/*.c here is the package's two C files, and -O3 is what the build actually uses, so this is the production configuration rather than a special one. The x86-64 binary runs under Rosetta, but Rosetta only executes the instructions it is handed; anything the two binaries disagree on was decided when clang compiled them, not at run time.
Five of the six came back different.
fixture arm64 x86_64 same?
sine_44100_mono_1s.mp3 26f26e4d00bd4680 db3ed41489553717 NO
sine_44100_mono_1s.ogg 741546cc3f0e3857 9b9fa075856e40b1 NO
sine_44100_stereo_1s.mp3 62655177b196b90f db39b9b9973061b7 NO
sine_44100_stereo_1s.ogg b712d80468229e7f ab1522a686a648df NO
sine_48000_mono_halfsec.mp3 03a0cfb92eff3109 03a0cfb92eff3109 yes
sine_48000_mono_halfsec.ogg 0e38543dbad39c7e db914b5a5a724d93 NO
Same source, same compiler, same optimisation level, same input bytes. Different samples. The one that matched, a short MP3, matched by luck rather than by rule: the divergence only shows up when a sample lands close enough to a rounding boundary to fall the other way, and that clip happened to have none.
How big, exactly
A hash mismatch tells you nothing about severity, so the next step is to dump both PCM buffers and diff them sample by sample:
sine_44100_mono_1s.ogg 12 of 44100 samples differ (0.027%), max delta 1 LSB
sine_44100_mono_1s.mp3 3 of 47232 samples differ (0.006%), max delta 1 LSB
sine_44100_stereo_1s.ogg 16 of 88200 samples differ (0.018%), max delta 1 LSB
sine_44100_stereo_1s.mp3 10 of 94464 samples differ (0.011%), max delta 1 LSB
sine_48000_mono_halfsec.ogg 5 of 24000 samples differ (0.021%), max delta 1 LSB
That is all five of the fixtures that diverged. Every difference is one least-significant bit, on a fraction of a percent of samples. In 16-bit audio one LSB is one part in 32768 of peak amplitude. Nobody is going to hear it. The frame count, channel count and sample rate matched on both architectures for all six fixtures.
So the audio is fine. The sentence is not.
Where the bit goes missing
Both decoders do their arithmetic in floating point, and the C standard lets a compiler contract a multiply followed by an add into a single fused multiply-add. An fma computes a * b + c with one rounding instead of two, so it is more accurate, and it produces a different number than the unfused version.
arm64 has an fmadd instruction and clang uses it freely. Baseline x86-64 does not guarantee FMA, so on that target the same expression compiles to a separate multiply and add, with a rounding in between. One rounding versus two, in a decoder that then scales to 16-bit integers, and occasionally a sample lands on the other side of .5.
This is not a bug in the two decoders this package vendors — stb_vorbis for Ogg and minimp3 for MP3 — nor in clang or my shim. It is the default behaviour of C floating point, and -ffp-contract=off turns it off. I checked what the build actually passes: only an optimisation level, no floating-point flags, so contraction is on, and the divergence I had documented as impossible is exactly what the default asks for.
For stb_vorbis that flag is the whole story. Rebuilt with contraction off, all three Ogg fixtures decode identically on both architectures. But the two MP3 fixtures still differ. I disassembled the contraction-off arm64 build: it contains zero fused multiply-add instructions, so whatever is left cannot be a fused-versus-unfused split.
It is minimp3 hand-writing its final float-to-int16 conversion once per instruction set, and the two versions round differently. The SSE path converts with _mm_cvtps_epi32, which rounds to nearest and breaks ties to even. The NEON path adds 0.5f and truncates. The float values feeding that step are bit-identical on the two architectures once contraction is off — I decoded to float on both and hashed the result to check — so the entire residual difference is those two rounding rules disagreeing, on a handful of samples, by one. Force the plain scalar path with -DMINIMP3_NO_SIMD, which also adds 0.5f and truncates, so x86-64 now rounds the way the NEON path does, and with contraction off as well all six fixtures finally match.
The part worth generalising
My CI matrix is three jobs: macOS (arm64), Ubuntu (x64), Windows (x64). It looks like it covers this. It cannot.
Each job builds the package and runs the test suite, and the test suite checks that decoding produces the right frame count, the right sample rate, and a waveform whose dominant frequency is the tone it should be. Every one of those assertions is a comparison against itself, inside one job. Nothing ever compares the macOS output against the Ubuntu output, because nothing in the matrix has both.
The trap is not specific to audio:
No number of independent self-comparisons tests a cross-environment claim. The only test is to carry an output from one environment into the other and compare — which a matrix that never moves an output between jobs does not do.
The fix in CI, if you need the guarantee, is to have jobs upload a hash of the output and have a final job assert they all match. The fix in the docs, if you do not need the guarantee, is to stop claiming it.
What the docs say now
I took the second option, because bit-identical cross-architecture output is not worth turning off contraction and minimp3's SIMD paths for a decoder whose output goes to a waveform or an FFT:
Decoding is deterministic for a given build: the same bytes decode to the same samples every time, and the geometry (channel count, sample rate, frame count) is the same everywhere. Sample values are not bit-identical across CPU architectures, though. [...] Measured across this package's own fixtures, that is around 0.03% of samples differing by one least-significant bit: inaudible, but enough that a checksum of decoded PCM will not match across a mixed-architecture fleet.
That last clause is the only case where any of this bites. If you decode audio on a build machine and compare a hash against audio decoded on a user's machine, or you cache by content hash across a mixed fleet, you will get misses that look like corruption and are not. Everyone else can ignore the whole thing, which is why the honest version of the sentence is longer than the false one.
If you maintain a native package that computes in floating point, the check costs one afternoon: build for two architectures, hash the output, compare. I would not assume the answer.
Top comments (0)