The Mask Is Not the Model: We Audited Eight Released Models for Causal Leakage, and Two Failed
An autoregressive model is only meaningful if position t depends on positions ≤ t and nothing else. Almost nobody checks. We built a check that fits on one page, ran it on eight public checkpoints, and found real defects in two of them.
Paper: arXiv:2608.22876 · 24 pages
The property nobody checks
Model releases report parameter counts, context lengths, training tokens, benchmark scores. They do not report whether the released implementation is causally correct — whether the forward pass actually respects the constraint that makes autoregressive modeling meaningful in the first place.
Every practical use of these models silently assumes it: teacher-forced training, perplexity evaluation, incremental decoding, KV-cache reuse, speculative decoding. All of them.
Historically this omission was tolerable. Decoder-only Transformers have one dominant temporal mixing mechanism — self-attention — and its causal behaviour is governed by an explicit mask. So people inspected the mask and called it an audit.
That assumption no longer holds. Modern stacks mix heterogeneous mixers in a single model: local attention, sparse attention, linear recurrences, structured state-space models, state-space duals, recurrent-attention hybrids, convolutional alternatives. In such systems explicit attention masks govern only a subset of layers, and in some layers there is no mask at all.
The traditional audit stopped covering most of the computation graph, and nobody replaced it.
Why silent leakage is the dangerous failure mode
Causal leakage is not a crash. It does not raise an exception. It does not produce obviously broken output.
It makes your metrics look better.
If future information reaches earlier positions, next-token prediction becomes artificially easier. Training loss drops. Validation perplexity drops. Benchmark scores computed from teacher-forced likelihoods improve. Everything on your dashboard says the change worked.
The asymmetry is the problem: leakage improves the very metrics you use to select models, and only surfaces during free-running generation, deployment, or downstream evaluation. By the time anyone notices, the architecture comparisons that justified the design are already contaminated.
This is the same shape as data leakage in ML research, which has been identified as a major contributor to reproducibility failures across scientific fields. The difference is that this one lives inside the forward pass.
Why we built this: three faults in a row
This started as an engineering tool, not a research project.
While developing an internal hybrid sequence-model stack we hit three separate causal failures:
- A doubled output shift. Two independently correct alignment operations combined to expose one future token.
- A wrong causal condition in a hand-written attention path. A causal neighbourhood was replaced with a symmetric one.
- A differential-attention aggregation fault. A subtraction between attention maps happened after a sequence-wide aggregation had already mixed information across time.
Every one of them took substantial manual debugging. Every one of them violated the same underlying property. None of them showed up in conventional training telemetry.
Three bespoke debugging sessions for three instances of one bug is a signal that you are missing an abstraction.
The entire test, in eight lines
INPUT model f with layers [1..L]; sequence length T; threshold τ
OUTPUT verdict ∈ {CLEAN, LEAK}; first offending layer ℓ*
1 x₁ ← random token sequence of length T
2 x₂ ← copy of x₁ with position T−1 replaced by a different token
3 attach a forward hook to every layer, capturing its output
4 h₁ ← f(x₁) with caching disabled # forward pass 1
5 h₂ ← f(x₂) with caching disabled # forward pass 2
6 for ℓ = 1..L:
Δℓ ← max |h₁ℓ[0:T−1] − h₂ℓ[0:T−1]| # exclude the differing position
7 ℓ* ← min{ℓ : Δℓ > τ}, or NONE
8 return (LEAK, ℓ*) if ℓ* exists else (CLEAN, −)
That is the whole method. Two forward passes. No training, no gradients, no labels, no accelerator — our census ran on CPU.
We print it in full deliberately. The argument of the paper is that this check is cheap enough to be mandatory, and a check that does not fit on a page will not become mandatory.
Four design choices carry the weight:
Per-layer hooks, not output logits. Reading only the final logits tells you whether the model leaks. It cannot tell you where, because the output projection has already mixed every layer's contribution. The deliverable of this test is a line of code for an engineer to look at, so the layer index is the point. In our ablation the logits-only variant detected every injected fault and localized none of them.
Exclude the final position. Position T−1 legitimately differs — that is the perturbation. Comparing [0, T−1) isolates exactly the prefix that must be invariant.
Disable caching during the test. Incremental caching changes which code path executes.
A single-token difference is enough. It is sufficient to expose any violation, and it is the cheapest perturbation to construct.
What we found
192 injected faults: mask inspection 0, our audit 192
Eight patterns of injected leak, applied at three depths each (an early layer, the middle, a late layer), across eight checkpoints.
| Method | Detected | Localized |
|---|---|---|
| Attention-mask inspection | 0 / 192 | 0 |
| Logits-only comparison | 192 / 192 | 0 |
| Per-layer prefix audit | 192 / 192 | 192 |
Mask inspection finding nothing is not a bug in mask inspection. It is the point: the leaks were not in the masks. They were in scans and normalization — code paths a mask never touches.
Two released models leak, exactly at their chunk boundary
Beyond the injected faults, the audit surfaced defects in two publicly released hybrid models.
| Model | Leak begins at | Chunk parameter |
|---|---|---|
| Zamba2-1.2B | sequence length 256 | chunk = 256 |
| Nemotron-H-8B | sequence length 128 | chunk = 128 |
The leakage starts exactly at each model's chunked-scan boundary. Below the chunk size, both look clean. That is not a coincidence you argue about — it is a fingerprint that points at a specific implementation path.
And we did not stumble on these. A static analysis of chunked-scan implementations predicted causal violations in two released hybrid models, and the dynamic audit confirmed both predictions.
Telling a real leak from numerical noise
A float32 forward pass is not bit-exact, so any Δ-based test needs a discriminator. Ours is an ε-sweep: perturb by varying magnitudes and watch how Δ responds.
Zamba2 slope 0.63 ± 0.07 ← a real leak scales with the perturbation
Granite-4.0-h flat ← numerical floor
Jamba-tiny flat ← numerical floor
A real leak climbs. A numerical floor stays flat. Granite-4.0-h and Jamba-tiny are clean under this test, and it matters that we can say so with evidence rather than by not looking.
Two auditing norms this forced on us
1. A clean result is uninterpretable without a positive control on the same loaded checkpoint.
We learned this from an audit that failed. If your harness reports CLEAN, you have two explanations — the model is correct, or your harness is not wired up. Injecting a known fault into that exact loaded checkpoint and confirming the harness catches it is the only thing that separates them. Anything else is trusting your own tooling because it agreed with you.
2. Audit sequence length must exceed the architecture's chunk, kernel, or window parameter.
Our default was T = 48. At T = 48, Zamba2 looks clean. Its chunk size is 256; the relevant execution path is never entered. A test that never reaches the buggy branch reports what you want to hear.
This one generalizes past causality: your test has to exercise the path the bug lives on, and architectural constants tell you where those paths begin.
Where this leaves model adoption
We build this into AX-RAY, our model diagnostics system, because the practical problem it solves is not academic.
When an organization adopts an external model, it inherits that model's implementation. Benchmark scores do not tell you whether the implementation is causally correct — and as shown above, a causal defect can make those benchmark scores higher. The one number people use to decide is the number the defect inflates.
So the useful gate is not "does this model score well." It is:
A causal-leakage finding cannot be overridden by a good benchmark score.
That ordering is the whole product argument. Capability metrics and correctness evidence are different kinds of claim, and one does not substitute for the other.
What we are actually asking for
Parameter counts, context lengths, and benchmark tables ship with every model release. A causal-correctness certificate should ship with them too.
Not because model authors are careless — the two defects we found are in serious models from serious teams, and they are exactly the kind of bug that hides from every standard check. That is the argument. If a bug class is invisible to the tools everyone uses, the answer is a new tool that is cheap enough that everyone runs it.
Two forward passes. No gradients. Runs on a CPU. Fits on a page.
There is no good reason not to.
Paper: arXiv:2608.22876 — The Mask Is Not the Model: Auditing Prefix Invariance in Attention, State-Space, and Hybrid Sequence Models
Authors: Taebong Kim, Youngsik Hong, Minsik Kim, Sunyoung Choi, Jaewon Jang, Minseo Kim — VIDRAFT AI Research
AX-RAY: vidraft.net
Top comments (0)