I ran a causal audit on sparse autoencoder features and found that up to 77% of "recovered" features never actually activate when their concept is present — even at cosine similarity ≈ 1.000.
TL;DR
I spent the last few months building and running a causal audit on Sparse Autoencoders (SAEs) — the tool the mechanistic interpretability field uses to decompose neural network activations into human-interpretable features. The result is now on arXiv:
From Geometric Recovery to Causal Validation: A Reproducible Audit of Sparse Autoencoder Features (cs.LG)
Code: sae-causal-audit · superposition-to-monosemanticity
The headline finding: up to 77% of SAE features that pass the field's standard recovery metric (cosine similarity ≥ 0.90) never actually fire when their feature is present — including matches at cosine ≈ 1.000, in an otherwise well-trained SAE. Correlational recovery and causal behavior are not the same claim, and the metric everyone uses can't tell them apart.
The problem, in one sentence
SAEs are evaluated almost entirely by decoder geometry: does the learned decoder atom point in roughly the same direction as the ground-truth (or probed) feature? That's a correlational claim. Whether the encoder ever actually selects that atom when the feature is genuinely present is a completely different, causal claim — and cosine similarity, by construction, cannot distinguish the two.
An SAE under pressure to reconstruct well can place a decoder atom at the geometrically perfect location for a feature, while the paired encoder weight almost never wins the TopK (or ReLU) selection competition for that feature — because something else usually wins first. The atom is "recovered." It's also silent.
What I actually did
Rather than argue this abstractly, I built a fully controlled toy setting where ground truth is known exactly, and ran the whole pipeline from scratch:
Reproduced the superposition phase diagram from Elhage et al. (2022) — and caught a convergence artifact along the way: at high sparsity, under-training was silently masquerading as superposition. Fixing the step budget resolved it and surfaced a third, under-described geometric regime (diffuse sharing) at extreme overcompleteness.
Reproduced the TopK-vs-L1 comparison from Gao et al. (2024), including a direct activation-refinement test that measures L1 shrinkage rather than just asserting it (91.2% of the L1 SAE's reconstruction gap closes when you refine magnitudes on a frozen support; TopK's magnitude barely moves at all).
Built a causal validation battery — ablation and sign-correct steering, propagated through the model's actual output stage, not a proxy — and ran it against every feature that passed the standard cosine bar.
That last step is where it gets uncomfortable.
The central result
Two TopK SAEs, opposite ends of the Pareto front:
| Correlationally recovered (of 22) | Causally inert (fired_frac = 0) | |
|---|---|---|
| Good SAE (k=4, precision 1.00) | 22/22 | 2 (9%) — both at cosine > 0.9999 |
| Degraded SAE (k=13, precision 0.11) | 18/22 | up to 17 (77%) |
"Causally inert" means: across 500 samples where the ground-truth feature is genuinely present, the matched decoder atom's encoder entry never fires once. Not weak. Not noisy. Exactly zero. Ablating an atom that never activates cannot possibly change anything — the specificity ratio isn't small, it's undefined-and-reported-as-zero, for a structurally different reason than "weak effect."
Why the good SAE still has inert features
This is the part I didn't expect going in. The two inert features in the well-trained SAE aren't noise — they're the exact same geometric event, twice. Feature 8 matches atom 78 at cosine 0.9997, anti-aligned, fired_frac = 0.00. But that same atom 78 is also the best match for feature 1 — positively aligned, firing 100% of the time, ablation specificity 535.
One atom, two "recovered" features. The toy model's weights explain it completely: cos(W1, W8) = -0.9998 — an antipodal pair, the exact Wi ≈ -Wj configuration the superposition literature predicts. The SAE learned one atom per shared axis, which is the correct thing to do for reconstruction. But TopK-then-ReLU only passes positive pre-activations, so the atom fires for one side of the axis and never the other.
Antipodal inertness is not a training failure — it's superposition geometry faithfully compressed. A better SAE pushes the anti-aligned cosine closer to 1.000, which makes the inert match look more convincing by the correlational metric, not less. No amount of SAE quality removes this on its own; only interface-level awareness of match sign does.
Read-inert ≠ write-inert
Here's the part that I think matters most practically. The antipodal features are ablation-inert (you can't remove what never activates) — but they are absolutely not steering-inert. Forcing the same atom on with the correct sign raises the paired feature's output at steering specificity 143–310, some of the highest specificities in the entire audit, attached to atoms whose ablation effect is exactly zero.
So "causally inert" isn't one property — it decomposes into two separable, dissociable claims:
- Read-inert: the encoder never selects the atom → monitoring built on it is blind
- Write-inert: interventions along the atom don't move the feature → steering built on it is impotent
All five antipodal pairs across both SAEs dissociate these completely. A feature can be simultaneously unmonitorable and highly steerable, through the same atom. If you're using an SAE feature as a monitoring signal, that's a distinction with real safety implications — read-validation and write-validation are not interchangeable, and passing one tells you nothing about the other.
The instrument, not just the finding
A result that only exists inside one-off experiment scripts is a claim, not a tool. So I packaged the methodology as sae-causal-audit: a model-agnostic instrument built around two runtime-checkable structural protocols (no inheritance, no adapter needed) —
@runtime_checkable
class SparseAutoencoder(Protocol):
W_dec: torch.Tensor
def encode(self, h: torch.Tensor) -> torch.Tensor: ...
def decode(self, f: torch.Tensor) -> torch.Tensor: ...
sae_lens.SAE satisfies this out of the box. The division of labor restates the finding structurally: matching uses W_dec (correlational, what the field's metric measures); every causal metric routes through encode/decode (what decoder geometry can't certify).
The package includes a hash-verified deterministic reproduction pipeline. Getting that byte-exact promise to actually hold surfaced three layers of cross-environment nondeterminism — a serialization leak (wall-clock timing hashed into results), a multi-threaded Adam optimizer silently reordering floating-point accumulation, and the discovery that "the same torch version" on Windows vs. Linux is not the same compiled program. Byte-exact cross-platform reproducibility isn't hard to achieve — it's unavailable by construction for this class of workload. The repo now makes two separate guarantees with disjoint scopes: byte-exact within one pinned CI environment, semantic (rtol = 1e-4) everywhere else.
Does it show up on a real model?
Toy models are what make exact ground truth and causal certification possible — they're also exactly what makes every number a calibration point for the easy case, not a benchmark. So I ran the harness against a published production SAE (gpt2-small-res-jb, one hook layer) against 83 hand-authored concepts spanning unrelated domains.
Of 83 matched pairs, 7 cleared a (lowered, probe-proxy-adjusted) recovery bar; 1 (14%) was causally inert. Small-scale, but qualitatively consistent with the toy regime. More interesting: a handful of decoder atoms — not one, several — recur as the nearest correlational match for dozens of semantically unrelated concepts (astronomy, cryptography, law, all matching the same atom at cosine 0.30–0.70). This replicated across three independently constructed, growing concept batches, and a controlled negative case ruled out prompt-templating as the artifact. It's the same dictionary-under-splitting signal the toy antipodal-pair mechanism was built to make legible — now showing up at production scale.
Practical guidance, if you're building on SAEs
- Never gate an automated action directly on raw SAE feature activation — in one realistic configuration, 77% of correlationally matched features were acting-on-noise candidates.
- Validate causally for the direction you actually use. Monitoring needs read-validation. Steering needs write-validation. This work measured five cases where one direction passes at specificity 143–310 while the other sits at exactly zero.
- Re-run causal validation whenever the SAE is retrained. A feature's causal relationship to a concept is a property of one trained checkpoint, not a portable abstraction.
- Version-pin SAE checkpoints to the exact model and data distribution they were validated against.
Try it yourself
git clone https://github.com/mohamed-bal/sae-causal-audit
cd sae-causal-audit
make reproduce # regenerates every number in the paper, ~3 min on CPU
make verify # checks results within tolerance, any machine
make verify-hashes # byte-exact check, pinned CI environment
The protocol interface accepts any SAE that can encode, decode, and expose its dictionary — including sae_lens.SAE directly.
This is my first arXiv submission as an independent researcher, and I want to say a genuine thank-you to Adam Karvonen (lead author of SAEBench) for the cs.LG endorsement that made this possible. SAEBench's benchmark-scale diagnosis — that proxy metrics don't reliably predict practical performance — is a big part of what motivated pushing the same question down to the level of individual features with a causal instrument.
Happy to answer questions, get pushback, or hear about places this breaks. That's the point of putting the instrument out there rather than just the number.
Paper: https://arxiv.org/abs/2607.12166
Code: https://github.com/mohamed-bal/sae-causal-audit
Top comments (0)