Every start script in my fleet carried the same line, under the same banner comment:
# CRITICAL: Forcing MMQ kernels for Pascal GPUs
export GGML_CUDA_FORCE_MMQ=1
My hardware constraints doc listed it as a requirement. Community guidance for Tesla P40s repeats it everywhere: Pascal has no Tensor Cores, its strength is INT8 matmul through the dp4a instruction, and MMQ is the kernel path that uses it. Forcing MMQ on is the single most repeated piece of P40 advice on the internet.
In May 2026, while writing build documentation for a new stack, I actually read the kernel selection code.
The env var is dead code. On upstream llama.cpp, ggml_cuda_should_use_mmq() selects MMQ unconditionally on compute capability 6.1. Pascal meets the DP4A minimum and has no FP16 tensor core path, so there is no other kernel the code could choose. The runtime environment variable is never read. Only a cmake option by a similar name exists, and it does something different at build time.
The kernels I was "forcing" were always on. They could not have been off.
The interesting part is why nobody noticed
The advice was harmless. That's exactly what made it invisible. Exporting the variable cost nothing, broke nothing, and the resulting performance was good, so the ritual survived every review. If the variable had hurt performance, someone would have caught it years ago.
This is the definition of a cargo-cult flag: a config line that travels from tutorial to tutorial because removing it feels riskier than keeping it, and no one's measurement would change either way.
I want to be precise about what was wrong here, because the distinction matters. The underlying claim was true: INT8 MMQ matmul genuinely is the right kernel path for a P40. Only the mechanism claim was false, the belief that you had to force it. A true conclusion propped up by a false mechanism is still a landmine, because you'll carry the false mechanism into the next decision.
What source verification buys you
My notes rank evidence in six rungs, and this incident is why "read the source" outranks "benchmark it yourself." No benchmark would have caught this. A/B testing the env var produces identical numbers on both sides, which is easy to misread as "the flag is so important it's saturated," rather than "the flag is disconnected."
Only the code answers what a flag actually does:
- Community claims told me to set it.
- My own measurements couldn't distinguish it.
- The source told me it was never read.
The fix in my fleet was deliberately conservative. New stacks drop the export and document why. Old stacks keep it, because it's harmless and their configs are frozen with their measurements. And the constraints doc got the stale line flagged for review rather than silently edited, because silently rewriting your own historical record is how you lose the ability to trust it.
The takeaway
Before you optimize a flag, verify the flag is connected to anything. The cheapest possible check, reading the selection logic, settled in minutes what years of repeated community advice never questioned.
Somewhere in your config, right now, there is a line that does nothing. It's probably the one with "CRITICAL" in the comment.
Top comments (0)