At one point in this project, two documents in my own notes argued opposite positions about the same compile flag.
Document one, the backend research: "The current build has AVX2 disabled. Priority 1: recompile with AVX2. Expected improvement, 15 to 30% on prompt processing."
Document two, the ecosystem plan: "AVX2 is off by design, to preserve CPU headroom for the rest of the stack while the GPU serves inference."
One says the flag is off and that's a problem. The other says it's off and that's a feature. They can't both be right.
Neither was. The flag was on the whole time.
Where both documents went wrong
The evidence behind "AVX2 is disabled" was the cmake build cache, which showed the SIMD options set to OFF. Build-directory archaeology: inspect the configuration, infer the artifact.
But llama.cpp's build enables native CPU optimizations through its own path regardless of those cached toggles, and the shipped binary is perfectly willing to tell you what it contains. Its startup system info prints the actual capability set. Mine printed:
AVX = 1 | AVX2 = 1 | F16C = 1 | FMA = 1 | BMI2 = 1
The running binary had AVX2 enabled all along. The "disabled" reading described cmake defaults that the real build had overridden. And the second document's clever theory about why disabled-AVX2 was good design was a rationalization of a condition that didn't exist.
The regression incident from post 2 has a footnote here too: that four-variables-at-once rebuild included "enable AVX2" as one of its four changes. In its own words, in the post-mortem: redundant. One of the four simultaneous changes was a no-op, which muddied attribution further.
The hierarchy this settled
My notes rank evidence quality in six rungs, and this incident fixed the ordering of two of them permanently:
Runtime artifact inspection beats build-system archaeology. Always. The cmake cache tells you what the build system was asked. The binary's own system info tells you what you are actually running. When they disagree, the binary wins, because the binary is what serves your traffic.
The general form: verify the premise before optimizing it. "Recompile to enable AVX2" was a well-reasoned recommendation, correctly derived from its evidence, aimed at a switch that was already in the right position. Everything about the plan was sound except the fact it stood on.
Cheap checks, expensive assumptions
What makes this sting is how cheap the correct check was. The system info line prints at every server start. It was in every log I'd ever launched. Nobody had read it, because everyone was reading the build directory instead, where the interesting-looking configuration lives.
Since then, the rule in my fleet is that claims about a binary come from the binary: its startup banner, its version string, its measured behavior. Configuration files describe intent. Artifacts describe reality. Optimization work starts from reality.
Two documents argued about a switch that was already in the right position. The moral isn't that documents are bad. It's that both documents cited the same wrong source, and one five-second look at the right source would have ended the argument before it started.
Top comments (1)
The two-document contradiction is the scariest version of this because both documents were internally coherent. A recommendation derived correctly from wrong evidence is indistinguishable from a good one until something executes it.
Same trap, different layer for me: services on my box that start through wrapper scripts. The script assumes a binary version and capability set; the binary in $PATH got replaced by an update days ago. Nothing disagrees out loud because nothing ever asks the binary. What I've settled on is close to your runtime-inspection rule — capture the binary's own version/capability report at service startup and write it into the service log, so every post-mortem starts with "what was actually running" instead of "what we think was running".
One question, since your notes clearly get read by more than just you: when agents — not you — write and revise those documents, what enforces the evidence hierarchy? A human can be taught "runtime beats build cache", but if an agent drafts the next optimization plan from whichever note is nearest, the stale one wins by default. Does the ranking live in a prompt, a template, or a review step?