Hi folks!
We are HardenedLinux, an old-school free software community. SonicBoom is our open-source (GPLv3) inference runtime: a compiler-first alternative to the usual Python stack. No Python at build or run time — one shared library, a C ABI, and Guile Scheme as the first-class language.
What v0.0.3 does on a desktop RTX 3050 8 GB (same session, same GGUF, vs llama.cpp -ngl 99):
On real multi-turn agent workloads, time-to-first-token dropped several-fold since v0.0.2 (prefix reuse + plan caching). Output tokens are bit-identical to our golden runs, and prefix reuse is bitwise identical to a full prefill.
How: all graph optimization and kernel generation happen on Boom IR, an MLIR dialect; hot quantized matmuls run as attributed ggml-derived kernels. Every optimization has a switch and an ablation number, so you can see what each one buys.
Where we are honest about gaps: Qwen decode at depth 512 is 0.98×, and we have only one GPU. That is exactly why we need you.
The invitation: every release gets one benchmark issue. If you have an NVIDIA GPU (any generation — Ada, Hopper, Blackwell especially welcome), one command produces a report you can paste. Everyone whose results we publish is credited in the README.
→ v0.0.3 benchmark issue: https://github.com/hardenedlinux/SonicBoom/issues/2
Please post results and problems in the issue rather than here, so everything stays in one place.
Top comments (0)