I Ran a Causal Test on Sparse Autoencoders — 77% of 'Recovered' Features Turned Out to Be Causally Inert

What Your Model Is Hiding in Plain Sight: A Rigorous, Reproducible Tour of Superposition, Dictionary Learning, and the Measured Limits of Mechanistic Interpretability

Show me the code: the complete implementation (toy models, sparse autoencoders, feature-recovery metrics, causal validation, full test suite) is available on GitHub: mohamed-bal/superposition-to-monosemanticity. Every number in this piece is written by code in that repository to a results/*.json file — nothing here is hand-typed from memory, and every figure is regenerated by re-running the corresponding script.

One clarification before anything else

Mechanistic interpretability is not a solved problem, and nothing in this piece claims otherwise. What this piece does is reproduce, from scratch and with proper statistical rigor (multiple random seeds, explicit convergence checks, honest reporting of where the methodology breaks), the foundational results the entire modern interpretability toolkit is built on — sparse autoencoders, feature dictionaries, the class of tools currently used by Anthropic, OpenAI, and Google DeepMind's interpretability teams — and then measures, precisely, where that toolkit succeeds and where it falls short, even in the most favorable setting that can be constructed for it: a small synthetic model where the ground truth is fully known and fully controlled.

The real state of the field as of mid-2026: sparse autoencoders scale to production models. Anthropic extracted millions of features from Claude 3 Sonnet's residual stream (Templeton et al., 2024); OpenAI and Google DeepMind have since published comparable open-weight SAE suites for their own models (Gao et al., 2024; Lieberum et al., 2024, "Gemma Scope"). But recovering a sparse, plausible-looking decomposition is not the same as proving you've found the decomposition the model actually uses — and a 2024 survey of the field flags a sharper, unresolved concern: as models get more capable, there's a live research question about whether they could develop internal structure that actively resists or misleads the very analysis techniques used to probe them. No interpretability method today gives a certified, complete account of a model's internal computation. Anything presented as though it does is either premature or marketing.

What actually exists is a well-defined, falsifiable hypothesis — the superposition hypothesis — a specific training procedure that produces structure consistent with it, and a growing but incomplete body of validation. That is the real subject here: what the hypothesis claims, why it should be true on purely geometric grounds, and what happens when you go and check.

The theoretical problem

Start from the assumption most people make, often without noticing they're making it, about what a hidden layer "means": that each dimension corresponds to roughly one interpretable concept, the way a variable in ordinary code corresponds to one thing. Real trained networks violate this constantly. A single neuron in a real language model frequently activates for several unrelated concepts at once — a phenomenon called polysemanticity — which is precisely why reading meaning directly off individual neurons has never worked reliably as a general interpretability strategy.

Elhage et al. (Anthropic, 2022) proposed a precise, two-part explanation:

1. The linear representation hypothesis — features correspond to directions in activation space, not to individual basis coordinates.
2. The superposition hypothesis — when a model would benefit from representing more features than it has dimensions, and those features are sparse (rarely all active at once), it can represent more features than dimensions by assigning them almost-orthogonal — not exactly orthogonal — directions. Simultaneously active features interfere with each other, but if that's rare enough, the expected benefit of representing more features outweighs the expected cost of occasional interference.

This isn't an ad hoc trick; it has a name and a well-studied justification in high-dimensional geometry. The Johnson–Lindenstrauss phenomenon says that in a $d$-dimensional space you can fit exponentially more than $d$ near-orthogonal directions — with pairwise inner products bounded arbitrarily close to zero — as long as you're willing to accept that "close to zero" instead of demanding "exactly zero." Superposition, under this view, is a neural network discovering and exploiting exactly this geometric fact under ordinary gradient descent — nothing about it needs to be designed in.

The geometry gets specific: antipodal pairs and polytopes

The original paper goes further than "features overlap somehow" — it characterizes how they overlap once a model settles into a solution, and the structure is surprisingly rigid rather than smoothly continuous. Two features forced to share a single dimension under symmetric competition don't split it 60/40 or drift to an arbitrary angle; the stable configuration found by gradient descent is $W_i = -W_j$ — an antipodal pair, exactly opposite unit vectors sharing one dimension, each still perfectly distinguishable from the other because a real input is (with overwhelming probability, under sparsity) never simultaneously "feature $i$ active" and "feature $j$ active." With more features competing for the same handful of dimensions, the geometry generalizes to higher-order symmetric configurations — the paper documents pentagons, tetrahedra, and other regular polytope-like arrangements — which is the same optimization structure that shows up in the century-old Thomson problem (how do you arrange $n$ mutually-repelling point charges on a sphere to minimize total potential energy?). A network resolving superposition is, geometrically, doing the same kind of job: packing feature directions to minimize mutual interference subject to a fixed number of dimensions.

This piece doesn't reproduce the polytope geometry in full (that requires studying 3-8 feature toy models in isolation, a genuinely separate and equally interesting reproduction exercise). What it does reproduce, with the rigor of multi-seed statistics, is the two-feature antipodal case exactly, and the resulting large-scale statistical structure — the phase diagram — across dozens of features and a full sparsity/overcompleteness sweep.

The toy model, precisely

Every experiment in this piece builds on one small, fully from-scratch model — no pretrained weights, no external data, runs on a CPU in seconds — implemented in src/toy_model.py:

hidden = W x                (n_hidden-dimensional bottleneck; linear, no nonlinearity here)
x_hat  = ReLU(Wᵀ hidden + b)

W has shape (n_hidden, n_features), and the interesting regime is n_hidden < n_features — a bottleneck that forces a choice about what to represent. Inputs are synthetic and sparse: each of the n_features entries is independently zero with probability sparsity, and drawn Uniform(0,1) otherwise. Training minimizes an importance-weighted reconstruction loss:

loss = sum_i I_i * (x_i - x_hat_i)^2   # I_i = per-feature importance weight

Two different importance schedules matter across this piece, and it's worth being explicit that they answer different questions:

• Decayed importance ($I_i = 0.9^i$), which breaks symmetry between features so the model has a principled reason to prioritize which features it represents when it can't represent all of them equally well. This is the schedule to use if you want to study prioritization — and it produces a real, sharp finding on its own: training a 32-feature model into an 8-dimensional bottleneck at sparsity=0.95, the optimizer doesn't spread capacity thinly across all 32 features — it fully drops the 10 lowest-importance ones (their columns in W collapse to near-zero) and represents the other 22 well. This emergent triage falls directly out of gradient descent on the importance-weighted loss; nothing hand-codes which features get dropped.
• Uniform importance ($I_i = 1$ for all $i$), which removes prioritization as a confound entirely and is the correct schedule for studying the geometric phase structure of superposition itself — how much interference emerges, and what shape it takes — independent of any priority ordering. This is the schedule used for the phase diagram below.

@dataclass(frozen=True)
class ToyModelConfig:
    n_features: int
    n_hidden: int
    sparsity: float
    importance_decay: float = 0.9   # set to 1.0 for uniform importance
    seed: int = 0

Measuring superposition precisely: feature dimensionality

A single scalar per feature, defined in Elhage et al. (2022) §3, captures exactly how much of a "dedicated dimension" each feature effectively owns:

$$D_i = \frac{\lVert W_i \rVert^2}{\sum_j (W_i \cdot W_j)^2}$$

def feature_dimensionality(W: torch.Tensor) -> torch.Tensor:
    norms_sq = (W**2).sum(dim=0)          # ||W_i||^2, shape (n_features,)
    gram = W.T @ W                         # W_i . W_j, shape (n_features, n_features)
    denom = (gram**2).sum(dim=1)           # sum_j (W_i . W_j)^2
    return norms_sq / denom.clamp_min(1e-12)

Three exact cases, each checked by a unit test in the repo rather than merely asserted:

• Orthonormal features ($W$ = identity): $D_i = 1$ for every feature — confirmed exactly (test_feature_dimensionality_is_one_for_exactly_orthonormal_features).
• An antipodal pair ($W = [\,1, -1\,]$, one shared dimension): $D_i = 0.5$ for both features, matching the geometric picture above exactly (test_feature_dimensionality_is_half_for_antipodal_pair).
• A subtlety that cost an afternoon to catch: $D_i$ is a ratio of squared norms, so it is not, by itself, a magnitude detector. Construct $W = \begin{bmatrix}1 & 0 & 10^{-6}\ 0 & 1 & 10^{-6}\end{bmatrix}$ — the third feature's column norm is $\sqrt{2}\times10^{-6}$, utterly negligible in absolute terms. Its $D_i$ is nonetheless 1.0, not near zero (test_feature_dimensionality_alone_cannot_detect_a_dropped_feature), because both the numerator and the dominant denominator term scale the same way with the column's overall magnitude. A genuinely dropped feature and a genuinely-represented-but-tiny-relative-to-its-competitors feature can report the identical $D_i$. Telling them apart requires checking $\lVert W_i \rVert^2$ directly, not just $D_i$ — a distinction that turns out to matter for a real finding below, not just as a footnote.

The phase diagram: does superposition structure follow the predicted pattern?

Methodology

• $n_{hidden} = 20$, fixed throughout.
• Overcompleteness ratio $n_{features}/n_{hidden} \in {1, 2, 4, 8}$ (so $n_{features} \in {20, 40, 80, 160}$).
• Sparsity $\in {0.0, 0.3, 0.6, 0.8, 0.9, 0.95, 0.99}$.
• Uniform importance throughout, isolating pure geometric structure from prioritization effects.
• 3 independent seeds per (ratio, sparsity) cell — every number below is a mean $\pm$ standard deviation across those seeds, not a single lucky (or unlucky) run.
• Each feature is classified by its final $D_i$: dedicated ($D_i > 0.9$), superposed ($0.1 \le D_i \le 0.9$), or a candidate "not really there" case ($D_i < 0.1$) — which, per the subtlety above, is further split by checking $\lVert W_i \rVert^2$ directly into truly dropped ($\lVert W_i \rVert^2 < 0.1$, i.e. actually near-zero weight) versus diffusely shared ($\lVert W_i \rVert^2 \ge 0.1$, i.e. substantially represented, just spread thin across many simultaneous overlaps rather than concentrated in one dedicated or antipodal direction).

A convergence bug, caught and fixed — reported here because it's more useful published than hidden

The first full run of this sweep produced a strange result at ratio=1 (where $n_{features} = n_{hidden} = 20$, so the model has exactly enough capacity to represent every feature orthogonally — the correct converged answer is 100% dedicated, at every sparsity level, with no exceptions): at sparsity 0.95 and 0.99, roughly 8% and 82% of features respectively showed up as "superposed" instead. That's not a plausible finding — there is no capacity pressure whatsoever forcing superposition in this configuration — so it was investigated rather than reported. Re-running the suspicious cells at 8,000 training steps instead of the sweep's default 1,500 fully resolved the discrepancy: dedicated fraction returns to exactly $1.00 \pm 0.00$ with loss $\approx 0$ at every sparsity level. The mechanism is straightforward once seen: at high sparsity, any given feature is active in only a small fraction of training batches, so gradient signal for that feature arrives more rarely — convergence is slower, not harder. A flat step budget tuned for the easy (low-sparsity) cells silently under-trained the hard-to-reach-but-not-hard-to-solve ones. The fix (steps_for_cell() in src/part1_phase_diagram.py) allocates 8,000 steps specifically to the cheap ($n_{features}=20$) ratio=1 cells at sparsity $\ge 0.9$, leaving the rest of the sweep at 1,500 steps. This is the kind of error multi-seed, sanity-checked reporting is supposed to catch — and it's reported here in full rather than quietly patched, because "we found this looked wrong, checked why, and fixed it" is exactly the epistemic standard the rest of this piece is asking readers to hold every other number to.

Results

Ratio = 1 ($n_{features} = n_{hidden} = 20$) — the capacity-sufficient control case:

Sparsity	Dedicated	Superposed	Final loss
0.00 – 0.99 (all 7 levels)	1.00 ± 0.00	0.00 ± 0.00	$\approx 0$

Exactly as it should be: with no capacity pressure, the model never chooses superposition, at any sparsity. This row is as much a sanity check on the whole pipeline as it is a result.

Ratio = 2, 4, 8 ($n_{features} = 40, 80, 160$) — genuine overcompleteness:

Sparsity	Ratio=2 dedicated	Ratio=4 dedicated	Ratio=8 dedicated
0.00	1.00 ± 0.00	1.00 ± 0.00	1.00 ± 0.00
0.30	0.26 ± 0.01	1.00 ± 0.00	1.00 ± 0.00
0.60	0.02 ± 0.02	0.51 ± 0.02	0.47 ± 0.07
0.80	0.00 ± 0.00	0.00 ± 0.00	0.00 ± 0.00
0.90	0.00 ± 0.00	0.00 ± 0.00	0.00 ± 0.00
0.95	0.00 ± 0.00	0.00 ± 0.00	0.00 ± 0.00
0.99	0.00 ± 0.00	0.00 ± 0.00	0.00 ± 0.00*

(*at ratio=8, sparsity=0.99, the 0% "dedicated" comes with 0% "superposed" too — see the diffuse-sharing finding below; the missing 100% is a different regime entirely, not an omission.)

The qualitative pattern matches the theory directly: more overcompleteness needs less sparsity before superposition becomes the dominant regime. At 2x overcompleteness, the transition is already well underway by sparsity=0.3 (74% superposed). At 4x and 8x, the transition point is later (around sparsity=0.6) but converges to a similar critical sparsity for both ratios — consistent with the idea that once superposition is "worth it" at all, further overcompleteness mostly changes how much gets packed rather than whether packing starts.

An unplanned third regime: diffuse sharing

The most interesting cell in the entire sweep is the most extreme one: ratio=8, sparsity=0.99 (160 features into 20 dimensions, 99% of inputs zeroed per sample). Every feature there reports $D_i \approx 0.08$ — comfortably below the 0.1 "candidate not-there" threshold — which the raw classification would call "dropped." But checking $\lVert W_i \rVert^2$ directly (exactly the subtlety flagged above) tells a completely different story: mean $\lVert W_i \rVert^2 \approx 1.40$, larger than the norm of a single cleanly-represented orthogonal feature ($\lVert W_i \rVert^2 = 1$ at the ratio=1 optimum). Every one of the 160 features is substantially represented — not remotely dropped — but in a highly symmetric, diffuse configuration where each feature overlaps a little with very many others rather than owning a dimension outright (as in the "dedicated" regime) or sharing exactly one dimension with exactly one partner (as in the antipodal "superposed" regime). Reconstruction loss at this cell is low (0.136 ± 0.009) — the model is doing a good job, just via a third geometric strategy the paper's clean dedicated/antipodal/dropped trichotomy doesn't name. Whether this "many-way diffuse packing" regime is a known, previously-documented phase or a genuinely under-described corner of the phase diagram is left as an open question here — but it would not have been visible at all without checking $\lVert W_i \rVert^2$ alongside $D_i$, which is the entire reason that check was built into the pipeline in the first place rather than added after the fact.

Every claim above is about the encoding side: given sparse features and a bottleneck, gradient descent finds a specific, geometrically characterizable, empirically reproducible way to pack more information than the bottleneck should naively allow. That's already enough to explain polysemanticity as a real, useful-to-the-model phenomenon rather than a training artifact — but it raises the obvious next question: if a hidden layer really is a superposed mixture of more features than it has dimensions, can anything recover the individual features back out of it? That's dictionary learning, and that's where L1 sparse autoencoders and their modern TopK successors enter — including the honest, measured answer to how well recovery actually works, not just whether it works at all.

From encoding to recovery

Everything so far establishes that a network will pack more features than it has dimensions when those features are sparse — and characterizes the geometry of that packing precisely enough to catch a real convergence bug in our own pipeline. None of it says anything yet about whether the packing can be undone. If a hidden layer really is a superposed mixture, can anything recover the individual, monosemantic features back out of it? That is the dictionary-learning question, and it has a history longer than the current interpretability boom: sparse coding — representing a signal as a sparse combination of an overcomplete set of basis vectors ("atoms") — goes back to Olshausen & Field's 1996 work explaining simple-cell receptive fields in visual cortex as an emergent property of sparse coding applied to natural images. The mechanistic-interpretability version of the idea (Bricken et al., Anthropic, 2023; scaled to a production model by Templeton et al., 2024) is the same mathematical object trained on a transformer's activations instead of image patches.

Why L1, and what it costs you

The natural formulation is combinatorial: find the sparsest code $f$ (fewest nonzero entries — its $L_0$ "norm") that reconstructs $h$ well. $L_0$ minimization is NP-hard in general, so the standard move — in compressed sensing, in Lasso regression, and in every SAE published before 2024 — is the convex relaxation: replace the $L_0$ penalty with an $L_1$ penalty on the code, which is the tightest convex relaxation of sparsity and is tractable by ordinary gradient-based optimization:

loss = ((h - h_hat) ** 2).sum(dim=-1).mean() + l1_coeff * f.abs().sum(dim=-1).mean()

The relaxation is not free. Tibshirani (1996) — the original Lasso paper — already documents the mechanism: an $L_1$ penalty systematically shrinks every coefficient toward zero, correct ones included, because the model is rewarded for making any active entry smaller purely to reduce the penalty term, independent of whether that entry's true optimal value is large. Applied to SAEs, this means an $L_1$-trained dictionary's reported activation strengths are a biased under-estimate of a feature's true presence — a real, quantifiable problem, not a theoretical nicety, and the reason the field moved to an alternative.

TopK: sparsity without a penalty to game

Gao et al. (OpenAI, 2024, "Scaling and Evaluating Sparse Autoencoders") propose removing the penalty term entirely and enforcing sparsity structurally instead: the encoder computes pre-activations for every dictionary atom as usual, then a TopK operation keeps only the $k$ largest values and zeros everything else — $k$ directly is the $L_0$, chosen once, not tuned indirectly through a coefficient:

def encode(self, h):
    pre_act = (h - self.b_dec) @ self.W_enc + self.b_enc
    topk_vals, topk_idx = torch.topk(pre_act, k=self.config.k, dim=-1)
    topk_vals = torch.relu(topk_vals)
    f = torch.zeros_like(pre_act)
    f.scatter_(-1, topk_idx, topk_vals)
    return f

loss = ((h - h_hat) ** 2).sum(dim=-1).mean()   # no L1 term at all

With no penalty on activation magnitude, there is nothing for the optimizer to game by shrinking values — the only pressure on the $k$ surviving activations is to reconstruct $h$ as well as possible. TopK does introduce its own new failure mode instead of the old one: forcing exactly $k$ active features per sample means an unlucky feature can go a long time without ever being selected, and once a dictionary atom stops firing it stops receiving gradient — it "dies." Following Gao et al., this repository's TopK implementation includes a simplified auxiliary loss: dictionary atoms that haven't fired within a rolling window get a secondary reconstruction objective (fitting the current residual using only their own top activations), which gives dead atoms an occasional gradient signal instead of leaving them permanently at zero. This is a simplified version of the real auxk mechanism (production implementations track a much larger EMA-based window at far greater scale) — adequate to demonstrate the mechanism at this toy scale, not a claim of production-readiness.

Head-to-head: L1 vs. TopK on the same superposed activations

Setup

Both SAE variants are trained on activations from the same toy model used for the feature-recovery study earlier in this piece — 32 features compressed into an 8-dimensional bottleneck, sparsity=0.95, decayed importance (22 of 32 features well-represented, 10 fully dropped) — with a 16x-overcomplete dictionary (128 atoms) in both cases. Every configuration below is trained across 3 seeds, reported as mean $\pm$ standard deviation, so the L1-vs-TopK comparison isn't resting on single lucky runs for either method.

Results: the Pareto front

Method	Hyperparameter	$L_0$ (mean$\pm$std)	Recon. loss	Recovery precision (well-represented features, cos-sim $\ge$ 0.90)
L1	$\lambda=0.003$	$14.68 \pm 0.58$	$0.00309 \pm 0.00039$	$0.15 \pm 0.06$
L1	$\lambda=0.01$	$10.69 \pm 0.58$	$0.00334 \pm 0.00021$	$0.23 \pm 0.10$
L1	$\lambda=0.03$	$7.90 \pm 0.73$	$0.00515 \pm 0.00029$	$0.27 \pm 0.07$
L1	$\lambda=0.1$	$4.42 \pm 0.32$	$0.01427 \pm 0.00074$	$0.77 \pm 0.11$
TopK	$k=13$	$12.92 \pm 0.07$	$0.00031 \pm 0.00010$	$0.11 \pm 0.06$
TopK	$k=9$	$8.99 \pm 0.00$	$0.00060 \pm 0.00009$	$0.23 \pm 0.07$
TopK	$k=6$	$6.00 \pm 0.00$	$0.00166 \pm 0.00104$	$0.53 \pm 0.13$
TopK	$k=4$	$4.00 \pm 0.00$	$0.01064 \pm 0.00210$	$\mathbf{1.00 \pm 0.00}$

Two comparisons at matched (or near-matched) $L_0$ tell the whole story:

• At $L_0 \approx 4$: TopK reaches perfect recovery precision on the well-represented features (1.00 $\pm$ 0.00 across all 3 seeds — every single one of the 22 ground-truth features the toy model actually represents is cleanly recovered, every seed, no variance at all), against L1's $0.77 \pm 0.11$ at essentially the same sparsity level. This is not a marginal win.
• At $L_0 \approx 9$–$13$: TopK's reconstruction loss is roughly an order of magnitude lower than L1's at comparable sparsity ($0.0003$–$0.0006$ vs. L1's $0.003$–$0.005$), while recovery precision degrades on a similar curve for both methods as $L_0$ grows (more simultaneously active features naturally means more opportunities for cross-feature interference to corrupt any single feature's match).

TopK dominates L1 on this Pareto front — both fidelity and interpretability-relevant recovery precision, at every $L_0$ level tested — which is exactly the qualitative claim in Gao et al. (2024), reproduced here from scratch on an unrelated small synthetic model rather than taken on faith from the original production-scale paper.

Direct evidence of shrinkage

The comparison above shows TopK wins, but doesn't yet show why — it's consistent with TopK just being a generally better sparsity mechanism for unrelated reasons. Gao et al. propose a specific, falsifiable mechanism (shrinkage) and a specific empirical test for it, reproduced exactly here: freeze which dictionary atoms an SAE selected as active for a batch of inputs (its "support"), then re-optimize only the magnitudes of those already-selected atoms — via projected gradient descent with a non-negativity constraint, decoder frozen — to directly minimize reconstruction error. If the SAE's own reported activations were already optimal given its own chosen support, this refinement could not improve anything. If it does improve substantially, the original magnitudes were biased.

At matched sparsity ($L_0 \approx 4$, using the $k=4$ TopK model and the $\lambda=0.1$ L1 model from the table above, tested on a held-out batch of 5,000 activations):

Method	$L_0$	MSE improvement from refinement	Mean active-code magnitude change
L1	4.23	91.2%	+22.5%
TopK	4.00	64.8%	$-0.05\%$ (essentially zero)

This is the cleanest possible confirmation of the proposed mechanism. Refining the L1 SAE's activations closes 91% of its reconstruction gap and does so by pushing magnitudes up by nearly a quarter on average — direct, quantitative evidence that the $L_1$ penalty was suppressing the true feature strengths, exactly as Tibshirani's shrinkage argument predicts. Refining the TopK SAE's activations still closes some of the gap (65% — refinement is a strictly-better-or-equal optimization by construction, so some residual improvement is expected even for an unbiased method, since the original activations were found by a single gradient-descent trajectory rather than solved exactly), but the mean magnitude of its active entries doesn't move at all (a change indistinguishable from zero, $-0.05\%$) — there is no systematic direction to correct, because there was no penalty pushing values in a systematic direction in the first place.

What this changes about how to read a "found feature"

The practical upshot for anyone applying either recipe to a real model: an $L_1$ SAE's reported activation strength for a feature is not just noisy, it is systematically biased low, in a direction and magnitude this experiment now quantifies rather than merely asserts. If a downstream system uses SAE activation magnitude for anything more than presence/absence — ranking which features matter most for a given output, thresholding an anomaly-detection signal, comparing feature strength across inputs — an $L_1$-trained dictionary's numbers need that bias kept in mind; a TopK-trained dictionary's don't carry the same known bias, though it introduces the separate, different cost of a rigid per-sample sparsity budget and the dead-latent problem that comes with it.

None of this closes the loop yet. Recovery precision even for the best configuration tested here (TopK, $L_0=4$) is reported as 1.00 on the well-represented subset of features — but that's still a correlational claim (does a learned direction point at the right ground-truth feature?), not a causal one (does intervening on that direction actually change model behavior the way the label implies?). That gap is what the rest of this piece confronts directly.

From correlation to causation

A decoder atom can be geometrically aligned with a feature's true direction in activation space without the encoder ever actually activating that atom when the feature is genuinely present — two different failure modes hiding behind one cosine-similarity number. Closing that gap requires an intervention, not another measurement of the same representation.

The experiment: ablation and steering

For every well-represented ground-truth feature $i$, matched by cosine similarity to a dictionary atom $j$ (exactly as in the dictionary-learning comparison, with the match's sign also recovered — a real bug caught during development, below), two interventions are run, both propagated through the actual toy-model output stage ($\hat{x} = \text{ReLU}(\hat{h}W + b)$), not an arbitrary proxy:

• Ablation: present an input where only feature $i$ is active, encode it through the SAE, zero out atom $j$'s code, decode, and measure how much the reconstructed output for feature $i$ specifically drops — versus how much every other feature's reconstructed output moves. The ratio of these two (targeted drop over mean off-target effect) is the specificity ratio: a causally faithful match should hit feature $i$ hard and everything else barely at all.
• Steering: the reverse direction — present an input where feature $i$ is off, force atom $j$ to a fixed positive value, and measure how much the reconstructed output for feature $i$ rises.

f_ablated = f.clone()
f_ablated[:, atom_idx] = 0.0
h_hat_ablated = sae.decode(f_ablated)
x_hat_ablated = torch.relu(h_hat_ablated @ toy_model.W + toy_model.b)

targeted_drop = (x_hat_baseline[:, feature_idx] - x_hat_ablated[:, feature_idx]).mean()
off_target_effect = (x_hat_baseline - x_hat_ablated).abs()[:, other_features].mean()
specificity_ratio = targeted_drop / off_target_effect

A sign bug, caught by the test suite, worth reporting because of what it demonstrates: cosine-similarity matching (best_match_cosine_similarity, used throughout this piece) takes the absolute value of cosine similarity, because a feature and its exact negation are equally good correlational matches — sensible for measuring recovery, but wrong for an intervention, which is direction-sensitive. The first version of the steering test always pushed the matched atom toward $+1$, silently assuming a positive match. test_steering_effect_runs_and_returns_expected_keys caught this immediately: on a deliberately simple sanity-check model, the induced effect came back as exactly 0.0 instead of positive, because the actual match happened to be anti-aligned (the dominant weight was $-0.82$, not $+0.82$) — steering "on" in the wrong direction pushed the reconstruction toward an invalid negative input region, which the ReLU output clips straight to zero. The fix (best_match_indices_and_signs) recovers the signed match explicitly rather than assuming positive, and steering_effect now requires that sign as an argument so the bug can't quietly reappear through the same code path. This is the second real bug this piece has caught through its own test suite and multi-seed validation discipline (the first was the ratio=1 convergence artifact in the phase diagram) — reported in full both times because a piece about measuring interpretability honestly should hold its own pipeline to the same standard it's applying to the SAEs.

Results: does a good correlational match predict a good causal match?

Two SAEs from the earlier Pareto front, at opposite ends of it: TopK, $k=4$ (recovery precision 1.00, the best configuration found) and TopK, $k=13$ (recovery precision 0.11, deliberately poor). For every one of the 22 well-represented features, both an ablation and a steering intervention are run on the atom each SAE matched to it.

SAE	Median ablation specificity	Min	Max	Median steering specificity
TopK, $k=4$ (good, precision 1.00)	168.8	0.0	2608.6	149.9
TopK, $k=13$ (bad, precision 0.11)	0.0	$-902.5$	163.0	21.4

The headline number: the good SAE's median specificity ratio (168.8 — ablating the matched atom drops the targeted feature's output roughly 169 times more than it moves every other feature, on average) against the bad SAE's median of exactly zero. Correlation between cosine similarity and log-specificity across both SAEs combined: $r = 0.657$ — a real, meaningfully positive relationship (a better correlational match does predict a better causal match, on average), but nowhere near $r=1$, and the exceptions are the informative part.

Why the bad SAE's median is exactly zero, not just small

Seventeen of the bad SAE's 22 matched pairs show ablation_specificity_ratio == 0.0 exactly — not approximately, not "small but nonzero." Investigating why (atom_fired_frac, tracked in ablation_effect alongside the specificity ratio) gives an unambiguous answer: in every one of those seventeen cases, the matched atom never fires at all when the corresponding ground-truth feature is presented in isolation ($\text{atom_fired_frac} = 0.0$ across 500 test samples). Ablating a code entry that was already exactly zero cannot change anything — the specificity ratio isn't small, it's undefined-and-reported-as-zero, for a completely different reason than "the effect is weak." One of these seventeen (feature 20) even has a cosine similarity of $0.92$ — comfortably above the $\ge 0.90$ threshold used throughout to call a feature "recovered." By the correlational metric used everywhere else in this piece, feature 20 counts as a recovery success. By the causal test, the atom credited with recovering it never once activates for that feature. This is not a rare edge case dredged up after the fact — it's 17 of 22 pairs for the low-precision SAE, and it is the single most important finding here for anyone tempted to treat a cosine-similarity match as sufficient evidence on its own.

The good SAE shows the same failure mode, just far more rarely: 2 of its 22 matches (features 5 and 6, both with cosine similarity $> 0.9999$ — as close to a perfect geometric match as this experiment produces) also have atom_fired_frac = 0.0. Even the best configuration tested here is not immune to "geometrically perfect, causally inert" matches — it just has the problem at a 9% rate instead of a 77% rate.

What "decoder direction" and "encoder activation" are actually measuring

The mechanism is straightforward in hindsight, and worth naming precisely because it generalizes past this toy setup: cosine-similarity matching, as used throughout this piece (and in the real interpretability literature it reproduces), compares ground-truth directions against decoder dictionary atoms ($W_{dec}$) — the vectors used to reconstruct activations from a code. Whether an atom's corresponding encoder entry actually fires for a given input is a separate question, governed by the encoder weights and the TopK/ReLU selection mechanism, not by the decoder geometry at all. A high-$L_0$ SAE under real pressure to reconstruct well can apparently learn a decoder atom that sits at the geometrically "correct" location for reconstruction purposes — useful for the reconstruction objective the SAE is actually trained on — while its paired encoder entry essentially never wins the TopK competition for that specific feature, because something else usually wins first. Decoder-geometry recovery and encoder-activation recovery are two different empirical claims that this piece's earlier correlational metric could not, by construction, tell apart — only an intervention can.

Production framework: what this changes about deploying any of this

Decision framework

Validation level	What it actually establishes	Cost	What it still can't tell you
Cosine similarity to a known ground truth (only possible in a toy setting)	Decoder geometry alignment	Lowest	Whether the encoder ever activates that atom for the matching input at all
Published open-weight SAE + qualitative "does this feature look interpretable" review	A human-plausible story for what a feature means	Moderate	Whether the story is causally load-bearing or a post-hoc rationalization of a spurious correlation
Activation patching / ablation on your own model (the methodology demonstrated in this piece, applied to a real model)	Causal specificity — does intervening on this feature change the targeted behavior more than everything else?	Highest (requires a working intervention harness on your actual model, not just an evaluation script)	Whether the effect you're seeing is the mechanism you think it is, versus a correlate of it — causal specificity within your test distribution doesn't guarantee it holds out of distribution

Where SAE-based signals belong in a real system

Given everything measured in this piece, the responsible design for any production use is narrow: an SAE-derived signal (anomalous feature co-activation, a feature flagged as correlating with a risky behavior) is a routing signal for closer review, layered on top of — never substituting for — existing output-level safety filtering. Concretely:

• Never gate an automated action directly on raw SAE feature activation. Given this piece's own measurement that 17 of 22 correlationally-matched features in a realistic (imperfect) SAE configuration turned out causally inert, an automated pipeline trusting activation-magnitude-as-ground-truth would be acting on noise disguised as signal a majority of the time.
• Any feature used for monitoring or steering needs its own causal validation pass (ablation and/or steering, exactly as in this piece) before being trusted — a cosine-similarity match to a hand-labeled or LLM-labeled concept is not sufficient evidence on its own, precisely because this experiment shows concretely how often it isn't.
• Re-run causal validation whenever the SAE is retrained, not just once at deployment — a decoder atom's causal relationship to a labeled concept is a property of one specific trained checkpoint, not a property of the "feature" as an abstract, portable concept.
• Version-pin an SAE checkpoint to the exact model checkpoint and data distribution it was validated against, same discipline as any other derived-artifact-from-a-model-checkpoint.

Security and epistemic notes

• A geometric match is not a causal claim, and this piece now has a concrete, quantified demonstration of exactly how far apart the two can be — not a hypothetical caveat, a measured 77% divergence rate in one realistic configuration.
• The interpretability-adversarial-robustness question raised at the start of this piece remains open, and this finding sharpens why it matters: if decoder-geometry alignment can diverge this far from encoder-activation reality without any adversarial pressure at all — purely as an ordinary training-dynamics artifact — a model under actual incentive to obscure a mechanism from interpretability tooling has an even larger space of "geometrically plausible, causally misleading" solutions available to it than this experiment explores.
• Treat every SAE-derived interpretability claim, including every one in this piece, as tied to the specific checkpoint, sparsity setting, and validation method that produced it. A number that held for TopK at $k=4$ on this toy model is not a license to assume the same causal reliability for a different $k$, a different model, or a different training run — the whole point of measuring the $k=4$ vs. $k=13$ gap directly was to make that non-transferability concrete rather than asserted.

Limitations, stated directly

• Every quantitative claim in this piece comes from small, fully synthetic toy models — real transformer residual streams involve orders of magnitude more features, genuine cross-layer and cross-token structure, and, critically, no accessible ground truth to validate against the way this piece repeatedly did. The toy setting is what makes rigorous validation possible; it is also exactly what makes every absolute number here a calibration point for the easy case, not a benchmark for a production model.
• The causal validation methodology above uses feature isolation (only one ground-truth feature active at a time) to get a clean read on specificity — real inputs to a real model activate many features simultaneously, and interaction effects between simultaneously-active features are not tested here at all.
• Two real bugs were caught and fixed during this piece's own development (the ratio=1 phase-diagram convergence artifact, and the steering sign-convention bug) — both are reported here in full because they were caught, not because the pipeline is now guaranteed bug-free. A third, uncaught issue of the same character is not a hypothetical risk; it's the expected steady-state for any codebase of comparable complexity, and the honest response is the testing discipline demonstrated throughout, not a claim of completeness.
• Sample sizes for the causal experiments (22 well-represented features, 2 SAE configurations) are small enough that the exact specificity numbers should be read as "this is the shape of the effect, measured once, on this model" rather than as a tight statistical estimate — the qualitative finding (correlational match quality is informative but leaves a large, characterizable gap) is the reproducible claim, not the precise value 168.8.

Closing synthesis

Three claims, each checked by an experiment rather than asserted, and each holding up with a real, quantified limitation attached: sparse features get packed non-orthogonally into a smaller space when doing so is cheap under sparsity, in a geometric pattern precise enough that a bug in reproducing it was catchable by comparing against the theoretically-required answer; an overcomplete sparse autoencoder can partially undo that packing, with TopK's structural sparsity mechanism dominating the older L1 penalty's Pareto front for a specific, mechanistically-identified reason (shrinkage) rather than by unexplained magic; and a geometric recovery, even a very good one, is not yet a causal one, by a margin this piece measured directly rather than gestured at. None of the three claims required taking the literature's word for it, and none of them came back "trivially true" once measured — the phase diagram had a real convergence bug hiding in it, the shrinkage measurement needed the right $L_0$ regime to show a clean signal, and the causal validation surfaced a 77%-divergence finding nobody was looking for going in. That — measure it, expect the measurement to complicate the clean story somewhat, report the complication anyway — is the actual discipline mechanistic interpretability is asking practitioners to adopt, at whatever scale they're working at.

---

Sources

• Olshausen, B. A. & Field, D. J. (1996), "Emergence of simple-cell receptive field properties by learning a sparse code for natural images," Nature.
• Tibshirani, R. (1996), "Regression Shrinkage and Selection via the Lasso," Journal of the Royal Statistical Society.
• Elhage, N. et al. (2022), "Toy Models of Superposition," Anthropic.
• Bricken, T. et al. (2023), "Towards Monosemanticity: Decomposing Language Models With Dictionary Learning," Anthropic.
• Templeton, A. et al. (2024), "Scaling Monosemanticity: Extracting Interpretable Features from Claude 3 Sonnet," Anthropic.
• Gao, L. et al. (2024), "Scaling and Evaluating Sparse Autoencoders," arXiv:2406.04093.
• Rajamanoharan, S. et al. (2024), "Improving Dictionary Learning with Gated Sparse Autoencoders," arXiv:2404.16014.
• Bussmann, B. et al. (2024), "BatchTopK Sparse Autoencoders," arXiv:2412.06410.
• Lieberum, T. et al. (2024), "Gemma Scope: Open Sparse Autoencoders Everywhere All at Once on Gemma 2," arXiv:2408.05147.
• "Mechanistic Interpretability for AI Safety — A Review," arXiv:2404.14082.