DEV Community: Mohamed Bal

Your SAE Passed the Cosine Similarity Bar. That Doesn't Mean It

Mohamed Bal — Wed, 15 Jul 2026 16:50:09 +0000

I ran a causal audit on sparse autoencoder features and found that up to 77% of "recovered" features never actually activate when their concept is present — even at cosine similarity ≈ 1.000.

TL;DR

I spent the last few months building and running a causal audit on Sparse Autoencoders (SAEs) — the tool the mechanistic interpretability field uses to decompose neural network activations into human-interpretable features. The result is now on arXiv:

From Geometric Recovery to Causal Validation: A Reproducible Audit of Sparse Autoencoder Features (cs.LG)

Code: sae-causal-audit · superposition-to-monosemanticity

The headline finding: up to 77% of SAE features that pass the field's standard recovery metric (cosine similarity ≥ 0.90) never actually fire when their feature is present — including matches at cosine ≈ 1.000, in an otherwise well-trained SAE. Correlational recovery and causal behavior are not the same claim, and the metric everyone uses can't tell them apart.

The problem, in one sentence

SAEs are evaluated almost entirely by decoder geometry: does the learned decoder atom point in roughly the same direction as the ground-truth (or probed) feature? That's a correlational claim. Whether the encoder ever actually selects that atom when the feature is genuinely present is a completely different, causal claim — and cosine similarity, by construction, cannot distinguish the two.

An SAE under pressure to reconstruct well can place a decoder atom at the geometrically perfect location for a feature, while the paired encoder weight almost never wins the TopK (or ReLU) selection competition for that feature — because something else usually wins first. The atom is "recovered." It's also silent.

What I actually did

Rather than argue this abstractly, I built a fully controlled toy setting where ground truth is known exactly, and ran the whole pipeline from scratch:

Reproduced the superposition phase diagram from Elhage et al. (2022) — and caught a convergence artifact along the way: at high sparsity, under-training was silently masquerading as superposition. Fixing the step budget resolved it and surfaced a third, under-described geometric regime (diffuse sharing) at extreme overcompleteness.
Reproduced the TopK-vs-L1 comparison from Gao et al. (2024), including a direct activation-refinement test that measures L1 shrinkage rather than just asserting it (91.2% of the L1 SAE's reconstruction gap closes when you refine magnitudes on a frozen support; TopK's magnitude barely moves at all).
Built a causal validation battery — ablation and sign-correct steering, propagated through the model's actual output stage, not a proxy — and ran it against every feature that passed the standard cosine bar.

That last step is where it gets uncomfortable.

The central result

Two TopK SAEs, opposite ends of the Pareto front:

	Correlationally recovered (of 22)	Causally inert (fired_frac = 0)
Good SAE (k=4, precision 1.00)	22/22	2 (9%) — both at cosine > 0.9999
Degraded SAE (k=13, precision 0.11)	18/22	up to 17 (77%)

"Causally inert" means: across 500 samples where the ground-truth feature is genuinely present, the matched decoder atom's encoder entry never fires once. Not weak. Not noisy. Exactly zero. Ablating an atom that never activates cannot possibly change anything — the specificity ratio isn't small, it's undefined-and-reported-as-zero, for a structurally different reason than "weak effect."

Why the good SAE still has inert features

This is the part I didn't expect going in. The two inert features in the well-trained SAE aren't noise — they're the exact same geometric event, twice. Feature 8 matches atom 78 at cosine 0.9997, anti-aligned, fired_frac = 0.00. But that same atom 78 is also the best match for feature 1 — positively aligned, firing 100% of the time, ablation specificity 535.

One atom, two "recovered" features. The toy model's weights explain it completely: cos(W1, W8) = -0.9998 — an antipodal pair, the exact Wi ≈ -Wj configuration the superposition literature predicts. The SAE learned one atom per shared axis, which is the correct thing to do for reconstruction. But TopK-then-ReLU only passes positive pre-activations, so the atom fires for one side of the axis and never the other.

Antipodal inertness is not a training failure — it's superposition geometry faithfully compressed. A better SAE pushes the anti-aligned cosine closer to 1.000, which makes the inert match look more convincing by the correlational metric, not less. No amount of SAE quality removes this on its own; only interface-level awareness of match sign does.

Read-inert ≠ write-inert

Here's the part that I think matters most practically. The antipodal features are ablation-inert (you can't remove what never activates) — but they are absolutely not steering-inert. Forcing the same atom on with the correct sign raises the paired feature's output at steering specificity 143–310, some of the highest specificities in the entire audit, attached to atoms whose ablation effect is exactly zero.

So "causally inert" isn't one property — it decomposes into two separable, dissociable claims:

Read-inert: the encoder never selects the atom → monitoring built on it is blind
Write-inert: interventions along the atom don't move the feature → steering built on it is impotent

All five antipodal pairs across both SAEs dissociate these completely. A feature can be simultaneously unmonitorable and highly steerable, through the same atom. If you're using an SAE feature as a monitoring signal, that's a distinction with real safety implications — read-validation and write-validation are not interchangeable, and passing one tells you nothing about the other.

The instrument, not just the finding

A result that only exists inside one-off experiment scripts is a claim, not a tool. So I packaged the methodology as sae-causal-audit: a model-agnostic instrument built around two runtime-checkable structural protocols (no inheritance, no adapter needed) —

@runtime_checkable
class SparseAutoencoder(Protocol):
    W_dec: torch.Tensor
    def encode(self, h: torch.Tensor) -> torch.Tensor: ...
    def decode(self, f: torch.Tensor) -> torch.Tensor: ...

sae_lens.SAE satisfies this out of the box. The division of labor restates the finding structurally: matching uses W_dec (correlational, what the field's metric measures); every causal metric routes through encode/decode (what decoder geometry can't certify).

The package includes a hash-verified deterministic reproduction pipeline. Getting that byte-exact promise to actually hold surfaced three layers of cross-environment nondeterminism — a serialization leak (wall-clock timing hashed into results), a multi-threaded Adam optimizer silently reordering floating-point accumulation, and the discovery that "the same torch version" on Windows vs. Linux is not the same compiled program. Byte-exact cross-platform reproducibility isn't hard to achieve — it's unavailable by construction for this class of workload. The repo now makes two separate guarantees with disjoint scopes: byte-exact within one pinned CI environment, semantic (rtol = 1e-4) everywhere else.

Does it show up on a real model?

Toy models are what make exact ground truth and causal certification possible — they're also exactly what makes every number a calibration point for the easy case, not a benchmark. So I ran the harness against a published production SAE (gpt2-small-res-jb, one hook layer) against 83 hand-authored concepts spanning unrelated domains.

Of 83 matched pairs, 7 cleared a (lowered, probe-proxy-adjusted) recovery bar; 1 (14%) was causally inert. Small-scale, but qualitatively consistent with the toy regime. More interesting: a handful of decoder atoms — not one, several — recur as the nearest correlational match for dozens of semantically unrelated concepts (astronomy, cryptography, law, all matching the same atom at cosine 0.30–0.70). This replicated across three independently constructed, growing concept batches, and a controlled negative case ruled out prompt-templating as the artifact. It's the same dictionary-under-splitting signal the toy antipodal-pair mechanism was built to make legible — now showing up at production scale.

Practical guidance, if you're building on SAEs

Never gate an automated action directly on raw SAE feature activation — in one realistic configuration, 77% of correlationally matched features were acting-on-noise candidates.
Validate causally for the direction you actually use. Monitoring needs read-validation. Steering needs write-validation. This work measured five cases where one direction passes at specificity 143–310 while the other sits at exactly zero.
Re-run causal validation whenever the SAE is retrained. A feature's causal relationship to a concept is a property of one trained checkpoint, not a portable abstraction.
Version-pin SAE checkpoints to the exact model and data distribution they were validated against.

Try it yourself

git clone https://github.com/mohamed-bal/sae-causal-audit
cd sae-causal-audit
make reproduce      # regenerates every number in the paper, ~3 min on CPU
make verify          # checks results within tolerance, any machine
make verify-hashes   # byte-exact check, pinned CI environment

The protocol interface accepts any SAE that can encode, decode, and expose its dictionary — including sae_lens.SAE directly.

This is my first arXiv submission as an independent researcher, and I want to say a genuine thank-you to Adam Karvonen (lead author of SAEBench) for the cs.LG endorsement that made this possible. SAEBench's benchmark-scale diagnosis — that proxy metrics don't reliably predict practical performance — is a big part of what motivated pushing the same question down to the level of individual features with a causal instrument.

Happy to answer questions, get pushback, or hear about places this breaks. That's the point of putting the instrument out there rather than just the number.

Paper: https://arxiv.org/abs/2607.12166
Code: https://github.com/mohamed-bal/sae-causal-audit

A Perfect 1.000 Cosine Match That Never Fires: The Hidden Trap in SAE Evaluation

Mohamed Bal — Fri, 10 Jul 2026 12:48:51 +0000

description: "The 77% causal-inertness finding is now a tested, hash-verified, model-agnostic audit tool — and building it surfaced two more real bugs (a serialization leak and a three-layer reproducibility failure), a new failure-mode decomposition rooted in antipodal geometry, and features matched at cosine ≈ 1.000 that never fire once — in the good SAE."

Show me the code: the tool described here — sae-causal-audit, with its full test suite (37 tests including property-based tests), deterministic reproduction pipeline, hash-based scientific-regression gate, and a harness for auditing published production SAEs — has its own repository: mohamed-bal/sae-causal-audit. The finding that motivated it was first published alongside the original toy-model reproduction: mohamed-bal/superposition-to-monosemanticity. Every number in this piece is written by code to a results/*.json file whose SHA-256 hash is committed and verified in CI, and every figure is regenerated by re-running the corresponding script. Nothing here is hand-typed from memory.

One clarification before anything else

This piece does not claim that sparse autoencoders are broken, and it does not claim that the tool presented here certifies them. What it does is narrower and, I'd argue, more useful: it takes the causal-validation methodology from the previous piece — the one that measured, in a fully-controlled setting, that up to 77% of features passing a standard correlational recovery bar were causally inert — and turns it into an instrument anyone can point at their own SAE. An instrument, unlike a finding, has to survive contact with inputs its author never anticipated. That constraint forced design decisions, surfaced a third real bug (reported below in full, in the same tradition as the first two), and — unexpectedly — produced a refinement of the original finding itself: "causally inert" turns out to decompose into two distinct failure modes, one of which is directly traceable to the antipodal-pair geometry of superposition.

The state of the underlying question is unchanged from the previous piece: no interpretability method today gives a certified, complete account of a model's internal computation, and correlational feature-recovery metrics — the cosine-similarity matches used throughout the field's evaluation practice — measure decoder geometry, not encoder behavior. Those are two different empirical claims. This piece is about the machinery for telling them apart at whatever scale you work at, and about what that machinery revealed when it was turned, once again, on the setting where the right answers are known.

Where the last piece stopped, precisely

A one-paragraph recap, because everything below builds on it. The previous piece trained the Elhage et al. (2022) toy model — 32 sparse features compressed into an 8-dimensional bottleneck at sparsity 0.95 — then trained sparse autoencoders (L1 and TopK) on its hidden activations and asked whether the SAE's dictionary atoms recover the ground-truth features. By the standard correlational metric (unsigned cosine similarity ≥ 0.90 between a ground-truth direction and its best-matching decoder atom), recovery looked excellent for the right hyperparameters. Then came the intervention: for every matched pair, ablate the atom's code on inputs where the feature is genuinely present, and measure whether the targeted output drops more than everything else moves. For a deliberately imperfect SAE (TopK, $k=13$), 17 of 22 correlationally-matched features showed an ablation effect of exactly zero — not small, zero — because the matched atom never fired at all when its feature was presented. One of those seventeen had a cosine similarity of 0.92. The mechanism: cosine matching compares against decoder atoms ($W_{dec}$, the reconstruction vectors), while whether an atom fires is governed by the encoder and the TopK/ReLU selection — a competition the geometrically-correct atom can simply keep losing.

That was the finding. The honest criticism of the previous piece — the one I'd make of it myself — is that its most decision-relevant output was locked inside its own experiment scripts. If you trained an SAE tomorrow and wanted your inert rate, you'd have to read my code, separate the methodology from the experiment, and re-implement the former against your model. That's not a tool; that's homework. A result you can't hand to someone is a claim.

The audit, precisely

sae-causal-audit runs three measurements per matched (feature, atom) pair, in increasing order of causal strength. All three propagate through the SAE's actual encode/decode path and through a caller-supplied behavioral readout — never through decoder geometry alone.

The diagram names the split that matters: matching runs on the decoder ($W_{dec}$) because that's what correlational recovery metrics are supposed to measure, while every causal number below comes from the encoder's actual behavior through encode/decode — the two paths only merge back together in the final report.

1. Fired fraction. The cheap screen that, on its own, explained every one of the seventeen inert cases in the original experiment:

$$\text{fired_frac}{i,j} = \frac{1}{N}\sum{n=1}^{N} \mathbb{1}\left[\,|f_j(h_n)| > \varepsilon\,\right], \quad h_n \sim \text{feature } i \text{ present}$$

If the encoder never activates atom $j$ across hundreds of samples where feature $i$ is genuinely present, no downstream claim about atom $j$ "representing" feature $i$ can be causal, whatever the cosine says. This is a screen, not a verdict — it costs one batched encode, which matters when the full ablation battery is expensive at production scale.

2. Ablation specificity. Present inputs where feature $i$ is active, encode, zero atom $j$'s code entry, decode, push both reconstructions through the downstream readout, and compare the targeted drop against mean collateral movement:

f = sae.encode(activations_on)
f_ablated = f.clone()
f_ablated[:, atom_idx] = 0.0

y_base = downstream(sae.decode(f))
y_abl  = downstream(sae.decode(f_ablated))

targeted_drop = (y_base[:, feature_idx] - y_abl[:, feature_idx]).mean()
off_target    = (y_base - y_abl).abs()[:, other_features].mean()
specificity   = targeted_drop / off_target

A causally faithful match hits its target hard and everything else barely at all. A ratio near 1 means the "intervention" is indistinguishable from generic collateral damage.

3. Sign-correct steering. The reverse direction: inputs where feature $i$ is off, atom $j$ forced to a fixed value — multiplied by the sign of the match, which the API requires as an explicit argument. Why that requirement exists is a story from the previous piece that shaped this package's interface, told properly below.

What "any SAE" means: structural typing, and why the causal path routes around the decoder

The audit accepts anything satisfying a three-member protocol — no inheritance, no registration, no adapter class to write:

@runtime_checkable
class SparseAutoencoder(Protocol):
    W_dec: torch.Tensor                                  # (d_sae, d_in), rows are atoms
    def encode(self, h: torch.Tensor) -> torch.Tensor: ...
    def decode(self, f: torch.Tensor) -> torch.Tensor: ...

sae_lens.SAE — the interface behind most published open-weight SAE suites — satisfies this out of the box. The division of labor inside the audit is deliberate and is itself a restatement of the original finding: matching uses W_dec (that's the correlational step, and it should use decoder geometry, because that's what the field's recovery metrics measure and what the audit is auditing), but every causal metric routes through encode and decode, so encoder-side selection — the thing decoder geometry cannot certify — is what actually gets measured.

The ground-truth side is a second small protocol, and it's the seam that lets one pipeline serve two regimes:

class FeatureProbe(Protocol):
    def activations_with_feature(self, feature_idx: int, n_samples: int) -> torch.Tensor: ...
    def activations_without_feature(self, feature_idx: int, n_samples: int) -> torch.Tensor: ...

In the toy regime, "with feature $i$" means synthetic isolation inputs — ground truth, exact. In the real-model regime, it means positive/negative prompt datasets for a labeled concept, with activations captured at the SAE's hook point. Same audit, honestly different epistemics — the real regime's weaker ground truth is a stated substitution, not a hidden one, and the whole reason the toy calibration exists.

Interface design as bug prevention: making the sign bug unrepresentable

The previous piece reported two real bugs its own discipline caught: a convergence artifact in the phase diagram (caught by multi-seed sanity checks against a theoretically-required answer) and a sign-convention bug in steering (caught by the test suite). Packaging the methodology forced a harder question than catching bugs: can the interface make a bug class impossible to write?

For the sign bug, yes. The original failure: cosine matching takes absolute values — correctly, because a feature and its exact negation are equally good correlational matches — and the first steering implementation silently assumed every match was positively aligned. On an atom whose true alignment was $-0.82$, steering "on" toward $+1$ pushed the reconstruction into a region the output ReLU clips, and the measured effect came back as exactly $0.0$. In the package, the matcher returns both facts as separate fields instead of collapsing them into one lossy number:

@dataclass(frozen=True, slots=True)
class MatchResult:
    feature_idx: int
    atom_idx: int
    cosine: float   # unsigned, in [0, 1] — recovery semantics
    sign: float     # ±1.0 — intervention semantics

and steering_effect(...) requires the sign as an argument. There is no default to silently assume. The original failure scenario is additionally encoded as a permanent regression test: an anti-aligned atom steered with the wrong sign must read back exactly $0.0$ through a ReLU downstream, and with the correct sign must read back positive. The same test that once caught the bug now guards the API that prevents it.

Two more semantics rules are load-bearing enough to state:

Zeros carry their cause. A specificity of $0.0$ can mean "weak effect" or "the atom's code was already zero, so ablating it changed nothing" — opposite implications hiding behind one number. Every causal result therefore reports fired_frac alongside the ratio, and the causally_inert flag is defined as fired_frac == 0.0, never inferred from the ratio.
Infinity is legal where it means something and rejected where it doesn't. A perfectly surgical intervention — nonzero targeted effect, exactly zero collateral — is legitimately infinite specificity. The serializer encodes it safely (strict JSON has no Infinity literal; the Python extension breaks downstream parsers), median-based statistics accept it, and mean-based statistics refuse it loudly, because a mean over infinity is not a number anyone should publish.

A third bug, in the same tradition

The package makes a specific, checkable promise: two runs that produce identical science produce byte-identical result files. That's what lets CI regenerate every result from scratch and compare SHA-256 hashes — regression detection on the results, not just the code. A refactor that silently moves a scientific number fails CI exactly like a broken unit test.

The first attempt to prove that promise failed. Two back-to-back from-scratch reproductions — same seeds, single-threaded, torch.use_deterministic_algorithms(True) — produced different hashes. The scientific numbers were identical; the files were not. The culprit was embarrassing in hindsight: the report dataclass carries a runtime_seconds field, wall-clock timing was being serialized into the hashed JSON, and no two runs take the same number of microseconds. The fix is a documented VOLATILE_FIELDS exclusion set in the serializer — timing stays available in memory and in the human-readable render, and never enters the hashed artifact.

That is the third real bug this project has caught and reported across two pieces, and the pattern is the point. The convergence artifact was caught by multi-seed checks against a known answer; the sign bug by the test suite; this one by actually executing the reproducibility claim instead of asserting it. The verification loop that caught it is now the CI gate itself, and the gate is tested the only way a gate can honestly be tested: reproduce twice and require identical hashes, then deliberately corrupt one number in one result file and require the gate to fail. It does — the corrupted file is named, with the expected and observed hashes side by side.

A fourth bug, right on schedule

The limitations list below makes a falsifiable prediction — that a fourth bug exists — and between drafting that sentence and publishing this piece, the prediction cashed out. It lived in the reproducibility machinery itself, it was three distinct mechanisms deep, and digging it out ended with this repository making a more honest promise than the one it started with.

The gate just described had passed the only test I had given it: two from-scratch reproductions on one machine, identical hashes, and a deliberately corrupted number caught loudly. Then the same gate ran on GitHub Actions and produced different hashes on different runs of the same commit — same seeds, same code, same pinned dependencies. Peeling that back went three layers down.

Layer one: the optimizer was multi-threaded behind my back. torch.set_num_threads(1) pins intra-op parallelism, but PyTorch's Adam defaults to foreach=True — a fused multi-tensor path whose floating-point accumulation order is not the naive loop's — and inter-op parallelism (torch.set_num_interop_threads) had never been pinned at all. Float addition is not associative; a different reduction order is a different result in the last bit. Setting foreach=False, pinning interop threads to one, and adding a wall of defensive environment variables (MKL_CBWR, every BLAS thread count, PYTHONHASHSEED) restored single-environment determinism.

Layer two: the bad SAE is a noise amplifier — and that is a finding, not an inconvenience. A last-bit difference should change a hash and nothing else. In the $k=13$ SAE it changed integer counts: one feature's best-match cosine sits close enough to the 0.90 bar, and one atom's pre-activation close enough to the TopK cut, that last-bit noise flips them across. The good SAE's census never moved; the deliberately degraded one is perched on a selection boundary where its discrete statistics are chaotic in the float noise. The same TopK competition that produces causally inert matches also produces numerical instability — one mechanism, two symptoms, at two completely different levels of the stack. The repository now encodes this as an explicit $\pm 1$ tolerance band on that SAE's counts, and only that SAE's.

Layer three: "the same torch version" is not the same program. With CI-side determinism proven, the hashes still refused to match the committed baseline — because the baseline had been generated on my Windows machine, and CI runs Linux. The torch==2.13.0+cpu wheel for Windows and the one for Linux are different binaries: MSVC versus GCC 13.3, linked against MKL 2026.1 versus 2024.2, NNPACK off versus on. Different compiled code produces different last-bit rounding — deterministically, permanently. No environment variable bridges it. Byte-exact reproducibility across platforms is not hard; it is unavailable, by construction. On this workload the visible symptom was layer two wearing different clothes: the bad SAE's census read one feature higher on Windows than on Linux while every continuous metric agreed to four decimal places.

So the promise was restructured rather than widened. The repository now makes two, with disjoint scopes. Byte-exact — SHA-256-identical result files — is guaranteed within the pinned CI environment (ubuntu-24.04, torch==2.13.0+cpu) and verified there on every push by make verify-hashes; the gate has since produced identical hashes across three independent runs of a single commit. Semantic — every number equal within $\text{rtol}=10^{-4}$, the boundary-sensitive counts within $\pm 1$ — is guaranteed on any platform by make verify. The byte-exact baseline is generated by the CI environment itself, through a manually dispatched job that uploads it as an artifact — never regenerated locally, never copied out of an error log — because a baseline produced on the wrong machine was the entire third layer. And the tolerance tier is tested the way the hash tier was: corrupt a count by two (outside the $\pm 1$ band) and the gate must fail; nudge a continuous metric by one percent and the gate must fail. Both do.

One sentence of generalization, because most repositories' "fully reproducible" badge silently means on my machine, probably: reproducibility is not one claim but a stack of claims with different scopes, and an honest repository states which rung it guarantees where. This one guarantees the top rung on exactly one pinned environment, the semantic rung everywhere, and tells you which one you are getting.

The test suite as instrument calibration

An audit tool's numbers are only as trustworthy as the setting where its answers can be checked. The suite (37 tests) works in three layers, each borrowed from the previous piece's discipline and pushed one step further:

Hand-derivable unit cases. Identity SAEs and two-atom constructions where every expected value is computable on paper: a perfectly surgical ablation must return exactly $\infty$; a structurally dead atom must return exactly $0.0$ with fired_frac = 0.0; a downstream that mixes the target equally into one other dimension must return specificity exactly $1.0$. The same standard the previous piece applied to feature_dimensionality, where a hand-checkable antipodal pair had to come out at exactly $D_i = 0.5$.
Property-based tests (Hypothesis) over randomized inputs: for any directions and any dictionary, cosine stays in $[0,1]$, the reported sign agrees with the true signed cosine of the chosen pair, dead atoms never win a match through a $0/0$, and bootstrap intervals are always ordered and bracketed by the sample range.
An end-to-end integration layer that trains the actual toy setting — the real optimizer, the real TopK competition — and asserts the audit recovers the known qualitative answers: a good SAE's recovered features are overwhelmingly causally specific; census arithmetic is internally consistent; serialization round-trips deterministically; a shape mismatch anywhere fails loudly rather than producing numbers computed on misaligned tensors.

Results: the setting re-audited under the instrument

The deterministic pipeline (make reproduce: single-threaded, foreach=False, deterministic algorithms enforced, fixed seeds, ~3 minutes on CPU) retrains the original configuration — 32 features into 8 dimensions at sparsity 0.95 with decayed importance (the toy model drops the 10 lowest-importance features outright and represents the other 22, exactly as before) — and audits two TopK SAEs from opposite ends of the previous piece's Pareto front, both with 128-atom dictionaries. One environmental note worth a full sentence: the fully deterministic pipeline converges to different weights than the original multi-threaded runs, so the exact counts moved and the qualitative finding did not — which is precisely the non-transferability the previous piece warned about when it said a number measured for one checkpoint is not a portable constant. Consider this an accidental robustness check that the claim passed. All numbers below are from the reference environment whose hashes CI verifies; where the boundary sensitivity documented above makes a count environment-dependent, that is stated at the number.

The census, over the 22 well-represented features, at the same cos ≥ 0.90 recovery bar as before:

	TopK $k=4$ (good)	TopK $k=13$ (bad)
Correlationally recovered, of 22	22 / 22	18 / 22
Recovered but causally inert (atom never fires)	2 (9%)	3 (17%), 4 in the ±1 band
Median ablation specificity, recovered set†	133.8 [95% CI 107.0 – 167.4]	68.3 [95% CI 22.0 – 105.9]
Median steering specificity, recovered set†	37.7 [95% CI 33.1 – 41.8]	16.9 [95% CI 14.2 – 21.3]

† Medians and bootstrap intervals are computed over every recovered pair across all 32 features ($n = 24$ and $n = 21$ respectively), not only the well-represented 22 used for the census rows — the bootstrap operates on the full recovered set by design.

Three things in that table deserve to be read slowly.

First, the inert features are not marginal matches scraping past the threshold — in either SAE. The bad SAE's inert cosines are 0.986, 0.994, and 0.9995; the good SAE's two are 0.9997 and 0.9998. Read that again: the good SAE — the one whose recovered set is otherwise causally excellent — contains two features whose geometric match is perfect to three decimal places and whose atoms do not fire once across 500 samples where the feature is present. The previous piece's most alarming single example was an inert match at cosine 0.92; the instrument, on a fresh training run, found matches at effectively 1.000 that never fire — and found them in both SAEs, including the well-trained one. The correlational metric is not merely noisy at the margin. It can be maximally confident and causally wrong, and being a good SAE by every reconstruction and specificity measure does not prevent it. Why the good SAE, of all things, produces the most perfect inert matches has a clean geometric answer, given in the next section.

Second, correlation between match quality and causal quality, over each SAE's firing pairs (cosine vs. log ablation specificity): $r = 0.859$ ($n = 21$) for the good SAE, $r = 0.471$ ($n = 18$) for the bad one. The previous piece reported a single pooled figure ($r = 0.657$); splitting it per-SAE reveals the more decision-relevant shape: the correlational metric is least trustworthy exactly where you most need it. On a healthy dictionary, cosine is a decent (never sufficient) proxy for causal quality; on a degraded one — the case an audit exists to catch — the proxy itself degrades toward uninformative. Cosine similarity is a fair-weather instrument.

Third, the bad SAE's ablation interval is wide in a way the point estimate hides. The good SAE's 95% interval spans 107–167 around a median of 134 — a relative width of about 45%. The bad SAE's spans 22–106 around 68: a relative width of 123%, with a lower bound sitting within an order of magnitude of the specificity-1 collateral-damage line. A percentile bootstrap over an 18-pair recovered set containing three exact zeros produces resampled medians that swing across most of the observed range — the uncertainty structure is the finding, restated statistically. This is also the upgrade the previous piece explicitly owed, having warned that its n ≈ 20 point estimates should be read as shapes, not tight values. Every summary number now ships with a seeded, 10,000-resample interval.

And the count carrying the "±1 band" annotation is itself data. On the reference Linux environment the bad SAE's recovered census reads 18 with 3 inert; on a Windows build of the same torch version it reads 19 with 4, because one non-antipodal feature (cosine 0.924) sits close enough to both the recovery bar and the TopK selection boundary that different BLAS builds resolve it differently. That flip-prone feature is the competitive-inertness case discussed in the next section — the boundary sensitivity documented in the reproducibility section and the causal taxonomy below turn out to be the same phenomenon seen from two altitudes.

Anatomy of the perfect inert matches: antipodal pairs, and a refinement of the original finding

The good SAE's two inert features repay close inspection, because they are not flukes and they are not independent — they are the same geometric event happening twice, visible end to end.

Feature 8 matches atom 78 at cosine 0.9997, anti-aligned (sign $-$), with fired_frac = 0.00: the atom never activates once across 500 feature-ON samples, so its ablation specificity is exactly 0. But atom 78 is simultaneously the best match for feature 1 — same atom, positively aligned, firing on 100% of feature-1 samples with an ablation specificity of 535. One atom, two "recovered" features. And the good SAE's other inert feature is the identical story with different indices: feature 5 matches atom 22 at cosine 0.9998, anti-aligned, never fires — while feature 0 matches the same atom 22, positively aligned, firing every time with specificity 347.

Checking the toy model's weights explains everything: $\cos(W_1, W_8) = -0.9998$ and $\cos(W_0, W_5) = -0.9999$. Both are antipodal pairs — the exact $W_i \approx -W_j$ configuration the original superposition paper predicts and the previous piece reproduced — two well-represented features sharing a single direction with opposite signs. The SAE, reasonably, learned one atom per shared axis. The encoder's TopK-then-ReLU selection passes only positive pre-activations, so each atom fires for the positive side of its axis and never for the negative side. Decoder geometry for the negative-side feature is essentially perfect; encoder behavior for it is essentially nonexistent. This also answers the question the census table raised — why the good SAE produces the most perfect inert matches: antipodal inertness is not a training failure, it is superposition geometry faithfully compressed. A better SAE learns the shared axis more precisely, which pushes the anti-aligned cosine closer to 1.000. The failure is structural, and no amount of SAE quality removes it — only interface-level awareness of match sign does.

The bad SAE replicates the pattern threefold: atoms 22, 21, and 39 each serve an antipodal pair (features 0/5, 1/8, and 4/2 respectively), each with one side firing at specificity in the hundreds and the other side at exactly zero, each pair anti-parallel in the ground truth to at least $-0.999$. Across both SAEs, that is five antipodal pairs, and on the reference environment they account for every causally inert feature in both recovered sets.

Which sharpens the taxonomy: causal inertness itself decomposes by cause. Structural inertness — the antipodal mechanism above — appears in good and bad SAEs alike, survives improved training, and is diagnosable from geometry (a shared atom with opposite signs is visible in the match table before any intervention runs). Competitive inertness — an atom that simply keeps losing the TopK competition despite decent geometry, the dominant mechanism behind the previous piece's 77% figure — appears only in the degraded SAE. The bad SAE's fourth inert case, the non-antipodal feature at cosine 0.924 that flips in and out of the census across BLAS builds, is this second kind — and the fact that it is also the boundary-unstable feature is not a coincidence. An atom that barely loses the TopK competition is an atom whose selection is decided in the last bits of a float; competitive inertness and numerical boundary sensitivity are one mechanism expressed at two levels. The two kinds have opposite operational profiles: the structural kind is stable, predictable, and screenable from the dictionary alone; the competitive kind is unstable by nature and is precisely what the fired_frac screen exists to catch empirically.

And there is a twist that refines the original taxonomy along a second axis. These features are ablation-inert — you cannot remove what never activates — but they are not steering-inert: forcing atom 78 on with the correct negative sign raises feature 8's reconstructed output with a steering specificity of 310, and the same intervention on feature 5's atom scores 203; the bad SAE's three antipodal ghosts score 143–261. These are among the highest steering specificities in the entire audit, attached to atoms whose ablation effect is exactly zero. The decoder direction is causally usable in the write direction even though the encoder never uses it in the read direction. "Causally inert," as the previous piece used it, therefore decomposes into two separable claims — read-inert (the encoder never selects the atom for the feature; ablation-based monitoring built on this atom is blind) and write-inert (interventions along the atom don't move the feature; steering built on it is impotent) — and all five antipodal pairs dissociate the two completely. A feature can be unmonitorable yet steerable through the same atom, and in this setting that is not an edge case: it is the systematic signature of antipodal superposition under a positive-pass encoder. For anyone building on SAE features, the two failure modes have different operational consequences, and one intervention type cannot stand in for the other. This decomposition wasn't visible in the previous piece because, without the signed-steering machinery working correctly, the write-direction test on anti-aligned matches returned zeros for the wrong reason. Fixing the instrument revealed structure the bug had been hiding.

Two smaller notes fall out of the same examples. The matcher is deliberately greedy (per-feature argmax) rather than a bipartite assignment, precisely so that atom collisions like 1-and-8-on-atom-78 surface in the results instead of being optimized away — a collision is a finding about the dictionary (here: one atom serving an antipodal pair), not matching noise; the audit's five collisions were the thread that unraveled everything above. And the fired_frac screen would have flagged every one of the five for the cost of one batched encode, before any intervention ran — which at production scale is the difference between a cheap census and an expensive one.

From toy calibration to production audit: the real-model harness

Everything above is the controlled setting. The package ships scripts/audit_real_sae.py: the identical pipeline pointed at published, production SAEs — GPT-2 residual-stream SAEs and Gemma Scope, via SAELens and TransformerLens. The real setting forces two substitutions, and the harness makes both explicit rather than smuggling them in:

Ground truth → probe datasets. "Feature present" is defined by positive/negative prompt sets per labeled concept, activations captured at the SAE's hook point on the final token. The concept's direction — the thing cosine matching needs — is estimated as the difference in mean activations between the two sets: a standard linear-probe direction, and explicitly the weakest link in the real regime. It is a proxy standing in for ground truth that does not exist; that is precisely why the pipeline is calibrated first in the toy regime, where the test suite can assert the right answers.
Toy readout → spliced logits. The downstream readout decodes the (possibly intervened) SAE code, splices the reconstruction back into the residual stream at the hook point, runs the remainder of the model, and reads mean logits over a small set of concept-diagnostic tokens. Ablation specificity then means: does zeroing this atom suppress this concept's tokens more than it moves the other audited concepts' tokens?

GPT-2-small runs on a free Colab T4 (or CPU, slowly); Gemma-2-2b with Gemma Scope wants ~16 GB of GPU memory. The research question the harness exists to answer is the obvious promotion of everything above: of the features in a published production SAE that pass a standard correlational bar for a labeled concept, what fraction never fire when the concept is actually present — and of those that fire, how does read-specificity relate to write-specificity? I didn't know the answers when I wrote the previous paragraph, and I specifically wasn't predicting that the toy regime's rates would transfer — the probe-direction proxy alone guarantees the numbers aren't comparable one-to-one. Here's what running it actually found.

A first real-model census, and something the toy setting didn't prepare me for

I audited gpt2-small-res-jb/blocks.8.hook_resid_pre — a published SAE from SAELens — against 83 hand-written concepts chosen for maximum semantic distance from one another: geography, natural sciences, sports, and a long tail of skilled trades (beekeeping, glassblowing, cartography, falconry) picked specifically because nothing about "beekeeping" should share a probe direction with "cryptography." Each concept gets 8 positive and 8 negative prompts; the correlational bar drops to cosine ≥ 0.5, down from the toy regime's 0.90, because a difference-in-means probe direction from 8 prompts is a much weaker proxy than exact ground truth.

Of 83 matched pairs, 7 cleared the bar. Of those, 1 (14%) was causally inert. Ablation specificity over the recovered set: median 1.63, 95% CI [0.05, 5.34]. Steering specificity: median 2.23, 95% CI [0.69, 3.54]. Both intervals are wide and both sit close to their lower bound — which is the same story the toy regime told: correlational recovery is a weak predictor of causal magnitude, now showing up in a production dictionary instead of a synthetic one. I'm explicitly not treating 14% as comparable to the toy regime's 77%/9% — the probe-direction proxy and the lowered bar mean the two numbers measure related but distinct things, exactly as flagged above.

That's the expected kind of result. The unexpected part showed up while I was scaling the concept set up.

Atom 14149 is the nearest match for 8 of the 83 concepts. Not "related" concepts — astronomy, cryptography, and law are three of the eight, domains that share essentially no vocabulary. Same story at smaller scale: atom 4504 and atom 17413 each match 5 concepts, atom 17241 matches 4. This is the "matcher is deliberately greedy, and a collision is a finding" argument from the antipodal-pairs section above, playing out again — except this time on a real dictionary in a production model, not something I built to have this property.

I didn't believe it the first time. My first instinct was that it had to be a prompt-template artifact — I'd started scaling the concept set by extending it with 40 templated per-country entries (same eight-sentence skeleton, country names substituted), and it's exactly the kind of thing that could fake a "collision" by making 40 concepts secretly correlated with each other. So I checked: the collision rate went up and the recovery yield went down on the templated batch specifically, which is itself informative (templated concepts really do collapse toward shared probe directions — that's a methods finding worth knowing on its own) but it also meant I couldn't trust that batch as evidence for anything else. I threw it out and rebuilt the concept set entirely by hand — 83 concepts, no two sharing a sentence structure, spanning three independently-grown batches of increasing size (33, then 48, then 83). The same handful of atoms kept recurring as the nearest match across all three. Whatever is generating this, it isn't my template.

What it is — genuine antipodal-style structural inertness playing out in a production dictionary, a limitation of the difference-in-means proxy direction itself, or a real polysemantic hub feature that several unrelated concepts happen to route through — cosine matching alone can't say. That's exactly the kind of question the causal battery above exists to answer, and disambiguating it against this SAE is the obvious next step for the harness.

The decision framework, updated

The previous piece closed with a validation-level table. The instrument adds a rung to it — the cheap screen — and the feature-8 finding splits the top rung in two:

Validation level	What it actually establishes	Cost	What it still can't tell you
Cosine match to a known/probed direction	Decoder geometry alignment	Lowest	Whether the encoder ever selects the atom at all — this piece found cosine-≈1.000 counterexamples, in a well-trained SAE
`fired_frac` screen (one batched `encode`)	Whether the atom activates when the feature/concept is present	Near-zero	Whether the activation is specific — an atom can fire promiscuously
Ablation specificity	Read-direction causal specificity: monitoring built on this atom sees the feature	Moderate (intervention harness required)	Write-direction behavior; out-of-distribution transfer
Sign-correct steering specificity	Write-direction causal specificity: interventions along this atom move the feature	Moderate	Read-direction behavior — five antipodal pairs show the two dissociate completely

The production guidance from the previous piece survives unchanged and gets sharper: never gate an automated action on raw SAE activation (or absence of activation) without the atom having passed the causal battery for the direction of use — a monitoring use case needs read-validation, a steering use case needs write-validation, and this piece measured five cases where one direction passes at specificity 143–310 while the other sits at exactly zero.

Limitations, stated directly

The real-model census above is small and single-layer: 83 concepts, one hook point, one model, 7 recovered features. It is a first census, not a benchmark — the toy numbers remain the calibration point for the easy case, and the real-model rate isn't directly comparable to it for the reasons stated above.
The difference-in-means probe direction in the real harness is a linear approximation standing in for ground truth that doesn't exist, and concept prompt sets of realistic size undersample the concept's distribution. Both weaken the correlational side of the real-regime audit — the causal side (fired_frac, ablation, steering) does not depend on the direction estimate, only the matching does.
The causal battery still tests features in isolation on the ON side; interaction effects between simultaneously active features remain untested in both regimes. The read/write decomposition was measured on five antipodal pairs across two SAEs — systematic within this run — but still within one training run and one seed; the structural/competitive split of inertness likewise rests on one run and its ±1 boundary band, not a measured rate across seeds.
The sample sizes remain what they are: 22 features, 2 SAE configurations, 1 seed per SAE under the new deterministic pipeline (the bootstrap quantifies within-sample uncertainty, not across-seed variance — the previous piece's 3-seed protocol measured the latter and the tool supports re-running under any seed).
Four bugs caught and reported across two pieces is not evidence the pipeline is now bug-free; it is evidence the discipline that catches them is working. The previous draft of this list predicted a fourth existed, and it did — the expected steady state is that a fifth does too.

Closing synthesis

The previous piece ended on a discipline: measure it, expect the measurement to complicate the clean story, report the complication anyway. This piece is what happens when that discipline is applied to the piece itself. The finding became an instrument, and building the instrument complicated the finding — productively, four times over. The reproducibility claim, actually executed, exposed a serialization bug and produced a CI gate that now enforces what was previously asserted — and the gate, pushed onto machines that were not mine, exposed three more layers of the same claim and forced it to be restated honestly as two guarantees with different scopes. The sign-handling fix, promoted from a patch to an API contract, made a bug class unrepresentable and then revealed structure the bug had hidden: causal inertness decomposes into read-inertness and write-inertness, and antipodal pairs — the most basic geometry superposition produces — systematically generate features that are perfectly matched, completely unmonitorable, and highly steerable, all at once, through one atom apiece — in the good SAE as much as the bad one. And the fresh training run under a stricter environment moved every exact count while preserving every qualitative claim, which is the transferability behavior the original piece predicted and is now, accidentally, a measured demonstration of it.

The instrument is in mohamed-bal/sae-causal-audit: make reproduce regenerates every number here in about three minutes on a CPU; make verify checks them within tolerance on any machine, and make verify-hashes holds the pinned CI environment to the byte; the protocol interface accepts your SAE if it can encode, decode, and show its dictionary. If you run the audit against your own SAE — especially a production one — I want to hear what your inert census looks like, in both directions.

Sources

Elhage, N. et al. (2022), "Toy Models of Superposition," Anthropic.
Bricken, T. et al. (2023), "Towards Monosemanticity: Decomposing Language Models With Dictionary Learning," Anthropic.
Templeton, A. et al. (2024), "Scaling Monosemanticity: Extracting Interpretable Features from Claude 3 Sonnet," Anthropic.
Gao, L. et al. (2024), "Scaling and Evaluating Sparse Autoencoders," arXiv:2406.04093.
Lieberum, T. et al. (2024), "Gemma Scope: Open Sparse Autoencoders Everywhere All at Once on Gemma 2," arXiv:2408.05147.
Tibshirani, R. (1996), "Regression Shrinkage and Selection via the Lasso," Journal of the Royal Statistical Society.
Efron, B. & Tibshirani, R. (1993), An Introduction to the Bootstrap, Chapman & Hall.
"Mechanistic Interpretability for AI Safety — A Review," arXiv:2404.14082.
The originating write-up and its repository: mohamed-bal/superposition-to-monosemanticity.
This piece's tool and its repository: mohamed-bal/sae-causal-audit.

I Ran a Causal Test on Sparse Autoencoders — 77% of 'Recovered' Features Turned Out to Be Causally Inert

Mohamed Bal — Wed, 08 Jul 2026 12:52:12 +0000

What Your Model Is Hiding in Plain Sight: A Rigorous, Reproducible Tour of Superposition, Dictionary Learning, and the Measured Limits of Mechanistic Interpretability

Show me the code: the complete implementation (toy models, sparse autoencoders, feature-recovery metrics, causal validation, full test suite) is available on GitHub: mohamed-bal/superposition-to-monosemanticity. Every number in this piece is written by code in that repository to a results/*.json file — nothing here is hand-typed from memory, and every figure is regenerated by re-running the corresponding script.

One clarification before anything else

Mechanistic interpretability is not a solved problem, and nothing in this piece claims otherwise. What this piece does is reproduce, from scratch and with proper statistical rigor (multiple random seeds, explicit convergence checks, honest reporting of where the methodology breaks), the foundational results the entire modern interpretability toolkit is built on — sparse autoencoders, feature dictionaries, the class of tools currently used by Anthropic, OpenAI, and Google DeepMind's interpretability teams — and then measures, precisely, where that toolkit succeeds and where it falls short, even in the most favorable setting that can be constructed for it: a small synthetic model where the ground truth is fully known and fully controlled.

The real state of the field as of mid-2026: sparse autoencoders scale to production models. Anthropic extracted millions of features from Claude 3 Sonnet's residual stream (Templeton et al., 2024); OpenAI and Google DeepMind have since published comparable open-weight SAE suites for their own models (Gao et al., 2024; Lieberum et al., 2024, "Gemma Scope"). But recovering a sparse, plausible-looking decomposition is not the same as proving you've found the decomposition the model actually uses — and a 2024 survey of the field flags a sharper, unresolved concern: as models get more capable, there's a live research question about whether they could develop internal structure that actively resists or misleads the very analysis techniques used to probe them. No interpretability method today gives a certified, complete account of a model's internal computation. Anything presented as though it does is either premature or marketing.

What actually exists is a well-defined, falsifiable hypothesis — the superposition hypothesis — a specific training procedure that produces structure consistent with it, and a growing but incomplete body of validation. That is the real subject here: what the hypothesis claims, why it should be true on purely geometric grounds, and what happens when you go and check.

The theoretical problem

Start from the assumption most people make, often without noticing they're making it, about what a hidden layer "means": that each dimension corresponds to roughly one interpretable concept, the way a variable in ordinary code corresponds to one thing. Real trained networks violate this constantly. A single neuron in a real language model frequently activates for several unrelated concepts at once — a phenomenon called polysemanticity — which is precisely why reading meaning directly off individual neurons has never worked reliably as a general interpretability strategy.

Elhage et al. (Anthropic, 2022) proposed a precise, two-part explanation:

The linear representation hypothesis — features correspond to directions in activation space, not to individual basis coordinates.
The superposition hypothesis — when a model would benefit from representing more features than it has dimensions, and those features are sparse (rarely all active at once), it can represent more features than dimensions by assigning them almost-orthogonal — not exactly orthogonal — directions. Simultaneously active features interfere with each other, but if that's rare enough, the expected benefit of representing more features outweighs the expected cost of occasional interference.

This isn't an ad hoc trick; it has a name and a well-studied justification in high-dimensional geometry. The Johnson–Lindenstrauss phenomenon says that in a $d$-dimensional space you can fit exponentially more than $d$ near-orthogonal directions — with pairwise inner products bounded arbitrarily close to zero — as long as you're willing to accept that "close to zero" instead of demanding "exactly zero." Superposition, under this view, is a neural network discovering and exploiting exactly this geometric fact under ordinary gradient descent — nothing about it needs to be designed in.

The geometry gets specific: antipodal pairs and polytopes

The original paper goes further than "features overlap somehow" — it characterizes how they overlap once a model settles into a solution, and the structure is surprisingly rigid rather than smoothly continuous. Two features forced to share a single dimension under symmetric competition don't split it 60/40 or drift to an arbitrary angle; the stable configuration found by gradient descent is $W_i = -W_j$ — an antipodal pair, exactly opposite unit vectors sharing one dimension, each still perfectly distinguishable from the other because a real input is (with overwhelming probability, under sparsity) never simultaneously "feature $i$ active" and "feature $j$ active." With more features competing for the same handful of dimensions, the geometry generalizes to higher-order symmetric configurations — the paper documents pentagons, tetrahedra, and other regular polytope-like arrangements — which is the same optimization structure that shows up in the century-old Thomson problem (how do you arrange $n$ mutually-repelling point charges on a sphere to minimize total potential energy?). A network resolving superposition is, geometrically, doing the same kind of job: packing feature directions to minimize mutual interference subject to a fixed number of dimensions.

This piece doesn't reproduce the polytope geometry in full (that requires studying 3-8 feature toy models in isolation, a genuinely separate and equally interesting reproduction exercise). What it does reproduce, with the rigor of multi-seed statistics, is the two-feature antipodal case exactly, and the resulting large-scale statistical structure — the phase diagram — across dozens of features and a full sparsity/overcompleteness sweep.

The toy model, precisely

Every experiment in this piece builds on one small, fully from-scratch model — no pretrained weights, no external data, runs on a CPU in seconds — implemented in src/toy_model.py:

hidden = W x                (n_hidden-dimensional bottleneck; linear, no nonlinearity here)
x_hat  = ReLU(Wᵀ hidden + b)

W has shape (n_hidden, n_features), and the interesting regime is n_hidden < n_features — a bottleneck that forces a choice about what to represent. Inputs are synthetic and sparse: each of the n_features entries is independently zero with probability sparsity, and drawn Uniform(0,1) otherwise. Training minimizes an importance-weighted reconstruction loss:

loss = sum_i I_i * (x_i - x_hat_i)^2   # I_i = per-feature importance weight

Two different importance schedules matter across this piece, and it's worth being explicit that they answer different questions:

Decayed importance ($I_i = 0.9^i$), which breaks symmetry between features so the model has a principled reason to prioritize which features it represents when it can't represent all of them equally well. This is the schedule to use if you want to study prioritization — and it produces a real, sharp finding on its own: training a 32-feature model into an 8-dimensional bottleneck at sparsity=0.95, the optimizer doesn't spread capacity thinly across all 32 features — it fully drops the 10 lowest-importance ones (their columns in W collapse to near-zero) and represents the other 22 well. This emergent triage falls directly out of gradient descent on the importance-weighted loss; nothing hand-codes which features get dropped.
Uniform importance ($I_i = 1$ for all $i$), which removes prioritization as a confound entirely and is the correct schedule for studying the geometric phase structure of superposition itself — how much interference emerges, and what shape it takes — independent of any priority ordering. This is the schedule used for the phase diagram below.

@dataclass(frozen=True)
class ToyModelConfig:
    n_features: int
    n_hidden: int
    sparsity: float
    importance_decay: float = 0.9   # set to 1.0 for uniform importance
    seed: int = 0

Measuring superposition precisely: feature dimensionality

A single scalar per feature, defined in Elhage et al. (2022) §3, captures exactly how much of a "dedicated dimension" each feature effectively owns:

$$D_i = \frac{\lVert W_i \rVert^2}{\sum_j (W_i \cdot W_j)^2}$$

def feature_dimensionality(W: torch.Tensor) -> torch.Tensor:
    norms_sq = (W**2).sum(dim=0)          # ||W_i||^2, shape (n_features,)
    gram = W.T @ W                         # W_i . W_j, shape (n_features, n_features)
    denom = (gram**2).sum(dim=1)           # sum_j (W_i . W_j)^2
    return norms_sq / denom.clamp_min(1e-12)

Three exact cases, each checked by a unit test in the repo rather than merely asserted:

Orthonormal features ($W$ = identity): $D_i = 1$ for every feature — confirmed exactly (test_feature_dimensionality_is_one_for_exactly_orthonormal_features).
An antipodal pair ($W = [\,1, -1\,]$, one shared dimension): $D_i = 0.5$ for both features, matching the geometric picture above exactly (test_feature_dimensionality_is_half_for_antipodal_pair).
A subtlety that cost an afternoon to catch: $D_i$ is a ratio of squared norms, so it is not, by itself, a magnitude detector. Construct $W = \begin{bmatrix}1 & 0 & 10^{-6}\ 0 & 1 & 10^{-6}\end{bmatrix}$ — the third feature's column norm is $\sqrt{2}\times10^{-6}$, utterly negligible in absolute terms. Its $D_i$ is nonetheless 1.0, not near zero (test_feature_dimensionality_alone_cannot_detect_a_dropped_feature), because both the numerator and the dominant denominator term scale the same way with the column's overall magnitude. A genuinely dropped feature and a genuinely-represented-but-tiny-relative-to-its-competitors feature can report the identical $D_i$. Telling them apart requires checking $\lVert W_i \rVert^2$ directly, not just $D_i$ — a distinction that turns out to matter for a real finding below, not just as a footnote.

The phase diagram: does superposition structure follow the predicted pattern?

Methodology

$n_{hidden} = 20$, fixed throughout.
Overcompleteness ratio $n_{features}/n_{hidden} \in {1, 2, 4, 8}$ (so $n_{features} \in {20, 40, 80, 160}$).
Sparsity $\in {0.0, 0.3, 0.6, 0.8, 0.9, 0.95, 0.99}$.
Uniform importance throughout, isolating pure geometric structure from prioritization effects.
3 independent seeds per (ratio, sparsity) cell — every number below is a mean $\pm$ standard deviation across those seeds, not a single lucky (or unlucky) run.
Each feature is classified by its final $D_i$: dedicated ($D_i > 0.9$), superposed ($0.1 \le D_i \le 0.9$), or a candidate "not really there" case ($D_i < 0.1$) — which, per the subtlety above, is further split by checking $\lVert W_i \rVert^2$ directly into truly dropped ($\lVert W_i \rVert^2 < 0.1$, i.e. actually near-zero weight) versus diffusely shared ($\lVert W_i \rVert^2 \ge 0.1$, i.e. substantially represented, just spread thin across many simultaneous overlaps rather than concentrated in one dedicated or antipodal direction).

A convergence bug, caught and fixed — reported here because it's more useful published than hidden

The first full run of this sweep produced a strange result at ratio=1 (where $n_{features} = n_{hidden} = 20$, so the model has exactly enough capacity to represent every feature orthogonally — the correct converged answer is 100% dedicated, at every sparsity level, with no exceptions): at sparsity 0.95 and 0.99, roughly 8% and 82% of features respectively showed up as "superposed" instead. That's not a plausible finding — there is no capacity pressure whatsoever forcing superposition in this configuration — so it was investigated rather than reported. Re-running the suspicious cells at 8,000 training steps instead of the sweep's default 1,500 fully resolved the discrepancy: dedicated fraction returns to exactly $1.00 \pm 0.00$ with loss $\approx 0$ at every sparsity level. The mechanism is straightforward once seen: at high sparsity, any given feature is active in only a small fraction of training batches, so gradient signal for that feature arrives more rarely — convergence is slower, not harder. A flat step budget tuned for the easy (low-sparsity) cells silently under-trained the hard-to-reach-but-not-hard-to-solve ones. The fix (steps_for_cell() in src/part1_phase_diagram.py) allocates 8,000 steps specifically to the cheap ($n_{features}=20$) ratio=1 cells at sparsity $\ge 0.9$, leaving the rest of the sweep at 1,500 steps. This is the kind of error multi-seed, sanity-checked reporting is supposed to catch — and it's reported here in full rather than quietly patched, because "we found this looked wrong, checked why, and fixed it" is exactly the epistemic standard the rest of this piece is asking readers to hold every other number to.

Results

Ratio = 1 ($n_{features} = n_{hidden} = 20$) — the capacity-sufficient control case:

Sparsity	Dedicated	Superposed	Final loss
0.00 – 0.99 (all 7 levels)	1.00 ± 0.00	0.00 ± 0.00	$\approx 0$

Exactly as it should be: with no capacity pressure, the model never chooses superposition, at any sparsity. This row is as much a sanity check on the whole pipeline as it is a result.

Ratio = 2, 4, 8 ($n_{features} = 40, 80, 160$) — genuine overcompleteness:

Sparsity	Ratio=2 dedicated	Ratio=4 dedicated	Ratio=8 dedicated
0.00	1.00 ± 0.00	1.00 ± 0.00	1.00 ± 0.00
0.30	0.26 ± 0.01	1.00 ± 0.00	1.00 ± 0.00
0.60	0.02 ± 0.02	0.51 ± 0.02	0.47 ± 0.07
0.80	0.00 ± 0.00	0.00 ± 0.00	0.00 ± 0.00
0.90	0.00 ± 0.00	0.00 ± 0.00	0.00 ± 0.00
0.95	0.00 ± 0.00	0.00 ± 0.00	0.00 ± 0.00
0.99	0.00 ± 0.00	0.00 ± 0.00	0.00 ± 0.00*

(*at ratio=8, sparsity=0.99, the 0% "dedicated" comes with 0% "superposed" too — see the diffuse-sharing finding below; the missing 100% is a different regime entirely, not an omission.)

The qualitative pattern matches the theory directly: more overcompleteness needs less sparsity before superposition becomes the dominant regime. At 2x overcompleteness, the transition is already well underway by sparsity=0.3 (74% superposed). At 4x and 8x, the transition point is later (around sparsity=0.6) but converges to a similar critical sparsity for both ratios — consistent with the idea that once superposition is "worth it" at all, further overcompleteness mostly changes how much gets packed rather than whether packing starts.

An unplanned third regime: diffuse sharing

The most interesting cell in the entire sweep is the most extreme one: ratio=8, sparsity=0.99 (160 features into 20 dimensions, 99% of inputs zeroed per sample). Every feature there reports $D_i \approx 0.08$ — comfortably below the 0.1 "candidate not-there" threshold — which the raw classification would call "dropped." But checking $\lVert W_i \rVert^2$ directly (exactly the subtlety flagged above) tells a completely different story: mean $\lVert W_i \rVert^2 \approx 1.40$, larger than the norm of a single cleanly-represented orthogonal feature ($\lVert W_i \rVert^2 = 1$ at the ratio=1 optimum). Every one of the 160 features is substantially represented — not remotely dropped — but in a highly symmetric, diffuse configuration where each feature overlaps a little with very many others rather than owning a dimension outright (as in the "dedicated" regime) or sharing exactly one dimension with exactly one partner (as in the antipodal "superposed" regime). Reconstruction loss at this cell is low (0.136 ± 0.009) — the model is doing a good job, just via a third geometric strategy the paper's clean dedicated/antipodal/dropped trichotomy doesn't name. Whether this "many-way diffuse packing" regime is a known, previously-documented phase or a genuinely under-described corner of the phase diagram is left as an open question here — but it would not have been visible at all without checking $\lVert W_i \rVert^2$ alongside $D_i$, which is the entire reason that check was built into the pipeline in the first place rather than added after the fact.

Every claim above is about the encoding side: given sparse features and a bottleneck, gradient descent finds a specific, geometrically characterizable, empirically reproducible way to pack more information than the bottleneck should naively allow. That's already enough to explain polysemanticity as a real, useful-to-the-model phenomenon rather than a training artifact — but it raises the obvious next question: if a hidden layer really is a superposed mixture of more features than it has dimensions, can anything recover the individual features back out of it? That's dictionary learning, and that's where L1 sparse autoencoders and their modern TopK successors enter — including the honest, measured answer to how well recovery actually works, not just whether it works at all.

From encoding to recovery

Everything so far establishes that a network will pack more features than it has dimensions when those features are sparse — and characterizes the geometry of that packing precisely enough to catch a real convergence bug in our own pipeline. None of it says anything yet about whether the packing can be undone. If a hidden layer really is a superposed mixture, can anything recover the individual, monosemantic features back out of it? That is the dictionary-learning question, and it has a history longer than the current interpretability boom: sparse coding — representing a signal as a sparse combination of an overcomplete set of basis vectors ("atoms") — goes back to Olshausen & Field's 1996 work explaining simple-cell receptive fields in visual cortex as an emergent property of sparse coding applied to natural images. The mechanistic-interpretability version of the idea (Bricken et al., Anthropic, 2023; scaled to a production model by Templeton et al., 2024) is the same mathematical object trained on a transformer's activations instead of image patches.

Why L1, and what it costs you

The natural formulation is combinatorial: find the sparsest code $f$ (fewest nonzero entries — its $L_0$ "norm") that reconstructs $h$ well. $L_0$ minimization is NP-hard in general, so the standard move — in compressed sensing, in Lasso regression, and in every SAE published before 2024 — is the convex relaxation: replace the $L_0$ penalty with an $L_1$ penalty on the code, which is the tightest convex relaxation of sparsity and is tractable by ordinary gradient-based optimization:

loss = ((h - h_hat) ** 2).sum(dim=-1).mean() + l1_coeff * f.abs().sum(dim=-1).mean()

The relaxation is not free. Tibshirani (1996) — the original Lasso paper — already documents the mechanism: an $L_1$ penalty systematically shrinks every coefficient toward zero, correct ones included, because the model is rewarded for making any active entry smaller purely to reduce the penalty term, independent of whether that entry's true optimal value is large. Applied to SAEs, this means an $L_1$-trained dictionary's reported activation strengths are a biased under-estimate of a feature's true presence — a real, quantifiable problem, not a theoretical nicety, and the reason the field moved to an alternative.

TopK: sparsity without a penalty to game

Gao et al. (OpenAI, 2024, "Scaling and Evaluating Sparse Autoencoders") propose removing the penalty term entirely and enforcing sparsity structurally instead: the encoder computes pre-activations for every dictionary atom as usual, then a TopK operation keeps only the $k$ largest values and zeros everything else — $k$ directly is the $L_0$, chosen once, not tuned indirectly through a coefficient:

def encode(self, h):
    pre_act = (h - self.b_dec) @ self.W_enc + self.b_enc
    topk_vals, topk_idx = torch.topk(pre_act, k=self.config.k, dim=-1)
    topk_vals = torch.relu(topk_vals)
    f = torch.zeros_like(pre_act)
    f.scatter_(-1, topk_idx, topk_vals)
    return f

loss = ((h - h_hat) ** 2).sum(dim=-1).mean()   # no L1 term at all

With no penalty on activation magnitude, there is nothing for the optimizer to game by shrinking values — the only pressure on the $k$ surviving activations is to reconstruct $h$ as well as possible. TopK does introduce its own new failure mode instead of the old one: forcing exactly $k$ active features per sample means an unlucky feature can go a long time without ever being selected, and once a dictionary atom stops firing it stops receiving gradient — it "dies." Following Gao et al., this repository's TopK implementation includes a simplified auxiliary loss: dictionary atoms that haven't fired within a rolling window get a secondary reconstruction objective (fitting the current residual using only their own top activations), which gives dead atoms an occasional gradient signal instead of leaving them permanently at zero. This is a simplified version of the real auxk mechanism (production implementations track a much larger EMA-based window at far greater scale) — adequate to demonstrate the mechanism at this toy scale, not a claim of production-readiness.

Head-to-head: L1 vs. TopK on the same superposed activations

Setup

Both SAE variants are trained on activations from the same toy model used for the feature-recovery study earlier in this piece — 32 features compressed into an 8-dimensional bottleneck, sparsity=0.95, decayed importance (22 of 32 features well-represented, 10 fully dropped) — with a 16x-overcomplete dictionary (128 atoms) in both cases. Every configuration below is trained across 3 seeds, reported as mean $\pm$ standard deviation, so the L1-vs-TopK comparison isn't resting on single lucky runs for either method.

Results: the Pareto front

Method	Hyperparameter	$L_0$ (mean$\pm$std)	Recon. loss	Recovery precision (well-represented features, cos-sim $\ge$ 0.90)
L1	$\lambda=0.003$	$14.68 \pm 0.58$	$0.00309 \pm 0.00039$	$0.15 \pm 0.06$
L1	$\lambda=0.01$	$10.69 \pm 0.58$	$0.00334 \pm 0.00021$	$0.23 \pm 0.10$
L1	$\lambda=0.03$	$7.90 \pm 0.73$	$0.00515 \pm 0.00029$	$0.27 \pm 0.07$
L1	$\lambda=0.1$	$4.42 \pm 0.32$	$0.01427 \pm 0.00074$	$0.77 \pm 0.11$
TopK	$k=13$	$12.92 \pm 0.07$	$0.00031 \pm 0.00010$	$0.11 \pm 0.06$
TopK	$k=9$	$8.99 \pm 0.00$	$0.00060 \pm 0.00009$	$0.23 \pm 0.07$
TopK	$k=6$	$6.00 \pm 0.00$	$0.00166 \pm 0.00104$	$0.53 \pm 0.13$
TopK	$k=4$	$4.00 \pm 0.00$	$0.01064 \pm 0.00210$	$\mathbf{1.00 \pm 0.00}$

Two comparisons at matched (or near-matched) $L_0$ tell the whole story:

At $L_0 \approx 4$: TopK reaches perfect recovery precision on the well-represented features (1.00 $\pm$ 0.00 across all 3 seeds — every single one of the 22 ground-truth features the toy model actually represents is cleanly recovered, every seed, no variance at all), against L1's $0.77 \pm 0.11$ at essentially the same sparsity level. This is not a marginal win.
At $L_0 \approx 9$–$13$: TopK's reconstruction loss is roughly an order of magnitude lower than L1's at comparable sparsity ($0.0003$–$0.0006$ vs. L1's $0.003$–$0.005$), while recovery precision degrades on a similar curve for both methods as $L_0$ grows (more simultaneously active features naturally means more opportunities for cross-feature interference to corrupt any single feature's match).

TopK dominates L1 on this Pareto front — both fidelity and interpretability-relevant recovery precision, at every $L_0$ level tested — which is exactly the qualitative claim in Gao et al. (2024), reproduced here from scratch on an unrelated small synthetic model rather than taken on faith from the original production-scale paper.

Direct evidence of shrinkage

The comparison above shows TopK wins, but doesn't yet show why — it's consistent with TopK just being a generally better sparsity mechanism for unrelated reasons. Gao et al. propose a specific, falsifiable mechanism (shrinkage) and a specific empirical test for it, reproduced exactly here: freeze which dictionary atoms an SAE selected as active for a batch of inputs (its "support"), then re-optimize only the magnitudes of those already-selected atoms — via projected gradient descent with a non-negativity constraint, decoder frozen — to directly minimize reconstruction error. If the SAE's own reported activations were already optimal given its own chosen support, this refinement could not improve anything. If it does improve substantially, the original magnitudes were biased.

At matched sparsity ($L_0 \approx 4$, using the $k=4$ TopK model and the $\lambda=0.1$ L1 model from the table above, tested on a held-out batch of 5,000 activations):

Method	$L_0$	MSE improvement from refinement	Mean active-code magnitude change
L1	4.23	91.2%	+22.5%
TopK	4.00	64.8%	$-0.05\%$ (essentially zero)

This is the cleanest possible confirmation of the proposed mechanism. Refining the L1 SAE's activations closes 91% of its reconstruction gap and does so by pushing magnitudes up by nearly a quarter on average — direct, quantitative evidence that the $L_1$ penalty was suppressing the true feature strengths, exactly as Tibshirani's shrinkage argument predicts. Refining the TopK SAE's activations still closes some of the gap (65% — refinement is a strictly-better-or-equal optimization by construction, so some residual improvement is expected even for an unbiased method, since the original activations were found by a single gradient-descent trajectory rather than solved exactly), but the mean magnitude of its active entries doesn't move at all (a change indistinguishable from zero, $-0.05\%$) — there is no systematic direction to correct, because there was no penalty pushing values in a systematic direction in the first place.

What this changes about how to read a "found feature"

The practical upshot for anyone applying either recipe to a real model: an $L_1$ SAE's reported activation strength for a feature is not just noisy, it is systematically biased low, in a direction and magnitude this experiment now quantifies rather than merely asserts. If a downstream system uses SAE activation magnitude for anything more than presence/absence — ranking which features matter most for a given output, thresholding an anomaly-detection signal, comparing feature strength across inputs — an $L_1$-trained dictionary's numbers need that bias kept in mind; a TopK-trained dictionary's don't carry the same known bias, though it introduces the separate, different cost of a rigid per-sample sparsity budget and the dead-latent problem that comes with it.

None of this closes the loop yet. Recovery precision even for the best configuration tested here (TopK, $L_0=4$) is reported as 1.00 on the well-represented subset of features — but that's still a correlational claim (does a learned direction point at the right ground-truth feature?), not a causal one (does intervening on that direction actually change model behavior the way the label implies?). That gap is what the rest of this piece confronts directly.

From correlation to causation

A decoder atom can be geometrically aligned with a feature's true direction in activation space without the encoder ever actually activating that atom when the feature is genuinely present — two different failure modes hiding behind one cosine-similarity number. Closing that gap requires an intervention, not another measurement of the same representation.

The experiment: ablation and steering

For every well-represented ground-truth feature $i$, matched by cosine similarity to a dictionary atom $j$ (exactly as in the dictionary-learning comparison, with the match's sign also recovered — a real bug caught during development, below), two interventions are run, both propagated through the actual toy-model output stage ($\hat{x} = \text{ReLU}(\hat{h}W + b)$), not an arbitrary proxy:

Ablation: present an input where only feature $i$ is active, encode it through the SAE, zero out atom $j$'s code, decode, and measure how much the reconstructed output for feature $i$ specifically drops — versus how much every other feature's reconstructed output moves. The ratio of these two (targeted drop over mean off-target effect) is the specificity ratio: a causally faithful match should hit feature $i$ hard and everything else barely at all.
Steering: the reverse direction — present an input where feature $i$ is off, force atom $j$ to a fixed positive value, and measure how much the reconstructed output for feature $i$ rises.

f_ablated = f.clone()
f_ablated[:, atom_idx] = 0.0
h_hat_ablated = sae.decode(f_ablated)
x_hat_ablated = torch.relu(h_hat_ablated @ toy_model.W + toy_model.b)

targeted_drop = (x_hat_baseline[:, feature_idx] - x_hat_ablated[:, feature_idx]).mean()
off_target_effect = (x_hat_baseline - x_hat_ablated).abs()[:, other_features].mean()
specificity_ratio = targeted_drop / off_target_effect

A sign bug, caught by the test suite, worth reporting because of what it demonstrates: cosine-similarity matching (best_match_cosine_similarity, used throughout this piece) takes the absolute value of cosine similarity, because a feature and its exact negation are equally good correlational matches — sensible for measuring recovery, but wrong for an intervention, which is direction-sensitive. The first version of the steering test always pushed the matched atom toward $+1$, silently assuming a positive match. test_steering_effect_runs_and_returns_expected_keys caught this immediately: on a deliberately simple sanity-check model, the induced effect came back as exactly 0.0 instead of positive, because the actual match happened to be anti-aligned (the dominant weight was $-0.82$, not $+0.82$) — steering "on" in the wrong direction pushed the reconstruction toward an invalid negative input region, which the ReLU output clips straight to zero. The fix (best_match_indices_and_signs) recovers the signed match explicitly rather than assuming positive, and steering_effect now requires that sign as an argument so the bug can't quietly reappear through the same code path. This is the second real bug this piece has caught through its own test suite and multi-seed validation discipline (the first was the ratio=1 convergence artifact in the phase diagram) — reported in full both times because a piece about measuring interpretability honestly should hold its own pipeline to the same standard it's applying to the SAEs.

Results: does a good correlational match predict a good causal match?

Two SAEs from the earlier Pareto front, at opposite ends of it: TopK, $k=4$ (recovery precision 1.00, the best configuration found) and TopK, $k=13$ (recovery precision 0.11, deliberately poor). For every one of the 22 well-represented features, both an ablation and a steering intervention are run on the atom each SAE matched to it.

SAE	Median ablation specificity	Min	Max	Median steering specificity
TopK, $k=4$ (good, precision 1.00)	168.8	0.0	2608.6	149.9
TopK, $k=13$ (bad, precision 0.11)	0.0	$-902.5$	163.0	21.4

The headline number: the good SAE's median specificity ratio (168.8 — ablating the matched atom drops the targeted feature's output roughly 169 times more than it moves every other feature, on average) against the bad SAE's median of exactly zero. Correlation between cosine similarity and log-specificity across both SAEs combined: $r = 0.657$ — a real, meaningfully positive relationship (a better correlational match does predict a better causal match, on average), but nowhere near $r=1$, and the exceptions are the informative part.

Why the bad SAE's median is exactly zero, not just small

Seventeen of the bad SAE's 22 matched pairs show ablation_specificity_ratio == 0.0 exactly — not approximately, not "small but nonzero." Investigating why (atom_fired_frac, tracked in ablation_effect alongside the specificity ratio) gives an unambiguous answer: in every one of those seventeen cases, the matched atom never fires at all when the corresponding ground-truth feature is presented in isolation ($\text{atom_fired_frac} = 0.0$ across 500 test samples). Ablating a code entry that was already exactly zero cannot change anything — the specificity ratio isn't small, it's undefined-and-reported-as-zero, for a completely different reason than "the effect is weak." One of these seventeen (feature 20) even has a cosine similarity of $0.92$ — comfortably above the $\ge 0.90$ threshold used throughout to call a feature "recovered." By the correlational metric used everywhere else in this piece, feature 20 counts as a recovery success. By the causal test, the atom credited with recovering it never once activates for that feature. This is not a rare edge case dredged up after the fact — it's 17 of 22 pairs for the low-precision SAE, and it is the single most important finding here for anyone tempted to treat a cosine-similarity match as sufficient evidence on its own.

The good SAE shows the same failure mode, just far more rarely: 2 of its 22 matches (features 5 and 6, both with cosine similarity $> 0.9999$ — as close to a perfect geometric match as this experiment produces) also have atom_fired_frac = 0.0. Even the best configuration tested here is not immune to "geometrically perfect, causally inert" matches — it just has the problem at a 9% rate instead of a 77% rate.

What "decoder direction" and "encoder activation" are actually measuring

The mechanism is straightforward in hindsight, and worth naming precisely because it generalizes past this toy setup: cosine-similarity matching, as used throughout this piece (and in the real interpretability literature it reproduces), compares ground-truth directions against decoder dictionary atoms ($W_{dec}$) — the vectors used to reconstruct activations from a code. Whether an atom's corresponding encoder entry actually fires for a given input is a separate question, governed by the encoder weights and the TopK/ReLU selection mechanism, not by the decoder geometry at all. A high-$L_0$ SAE under real pressure to reconstruct well can apparently learn a decoder atom that sits at the geometrically "correct" location for reconstruction purposes — useful for the reconstruction objective the SAE is actually trained on — while its paired encoder entry essentially never wins the TopK competition for that specific feature, because something else usually wins first. Decoder-geometry recovery and encoder-activation recovery are two different empirical claims that this piece's earlier correlational metric could not, by construction, tell apart — only an intervention can.

Production framework: what this changes about deploying any of this

Decision framework

Validation level	What it actually establishes	Cost	What it still can't tell you
Cosine similarity to a known ground truth (only possible in a toy setting)	Decoder geometry alignment	Lowest	Whether the encoder ever activates that atom for the matching input at all
Published open-weight SAE + qualitative "does this feature look interpretable" review	A human-plausible story for what a feature means	Moderate	Whether the story is causally load-bearing or a post-hoc rationalization of a spurious correlation
Activation patching / ablation on your own model (the methodology demonstrated in this piece, applied to a real model)	Causal specificity — does intervening on this feature change the targeted behavior more than everything else?	Highest (requires a working intervention harness on your actual model, not just an evaluation script)	Whether the effect you're seeing is the mechanism you think it is, versus a correlate of it — causal specificity within your test distribution doesn't guarantee it holds out of distribution

Where SAE-based signals belong in a real system

Given everything measured in this piece, the responsible design for any production use is narrow: an SAE-derived signal (anomalous feature co-activation, a feature flagged as correlating with a risky behavior) is a routing signal for closer review, layered on top of — never substituting for — existing output-level safety filtering. Concretely:

Never gate an automated action directly on raw SAE feature activation. Given this piece's own measurement that 17 of 22 correlationally-matched features in a realistic (imperfect) SAE configuration turned out causally inert, an automated pipeline trusting activation-magnitude-as-ground-truth would be acting on noise disguised as signal a majority of the time.
Any feature used for monitoring or steering needs its own causal validation pass (ablation and/or steering, exactly as in this piece) before being trusted — a cosine-similarity match to a hand-labeled or LLM-labeled concept is not sufficient evidence on its own, precisely because this experiment shows concretely how often it isn't.
Re-run causal validation whenever the SAE is retrained, not just once at deployment — a decoder atom's causal relationship to a labeled concept is a property of one specific trained checkpoint, not a property of the "feature" as an abstract, portable concept.
Version-pin an SAE checkpoint to the exact model checkpoint and data distribution it was validated against, same discipline as any other derived-artifact-from-a-model-checkpoint.

Security and epistemic notes

A geometric match is not a causal claim, and this piece now has a concrete, quantified demonstration of exactly how far apart the two can be — not a hypothetical caveat, a measured 77% divergence rate in one realistic configuration.
The interpretability-adversarial-robustness question raised at the start of this piece remains open, and this finding sharpens why it matters: if decoder-geometry alignment can diverge this far from encoder-activation reality without any adversarial pressure at all — purely as an ordinary training-dynamics artifact — a model under actual incentive to obscure a mechanism from interpretability tooling has an even larger space of "geometrically plausible, causally misleading" solutions available to it than this experiment explores.
Treat every SAE-derived interpretability claim, including every one in this piece, as tied to the specific checkpoint, sparsity setting, and validation method that produced it. A number that held for TopK at $k=4$ on this toy model is not a license to assume the same causal reliability for a different $k$, a different model, or a different training run — the whole point of measuring the $k=4$ vs. $k=13$ gap directly was to make that non-transferability concrete rather than asserted.

Limitations, stated directly

Every quantitative claim in this piece comes from small, fully synthetic toy models — real transformer residual streams involve orders of magnitude more features, genuine cross-layer and cross-token structure, and, critically, no accessible ground truth to validate against the way this piece repeatedly did. The toy setting is what makes rigorous validation possible; it is also exactly what makes every absolute number here a calibration point for the easy case, not a benchmark for a production model.
The causal validation methodology above uses feature isolation (only one ground-truth feature active at a time) to get a clean read on specificity — real inputs to a real model activate many features simultaneously, and interaction effects between simultaneously-active features are not tested here at all.
Two real bugs were caught and fixed during this piece's own development (the ratio=1 phase-diagram convergence artifact, and the steering sign-convention bug) — both are reported here in full because they were caught, not because the pipeline is now guaranteed bug-free. A third, uncaught issue of the same character is not a hypothetical risk; it's the expected steady-state for any codebase of comparable complexity, and the honest response is the testing discipline demonstrated throughout, not a claim of completeness.
Sample sizes for the causal experiments (22 well-represented features, 2 SAE configurations) are small enough that the exact specificity numbers should be read as "this is the shape of the effect, measured once, on this model" rather than as a tight statistical estimate — the qualitative finding (correlational match quality is informative but leaves a large, characterizable gap) is the reproducible claim, not the precise value 168.8.

Closing synthesis

Three claims, each checked by an experiment rather than asserted, and each holding up with a real, quantified limitation attached: sparse features get packed non-orthogonally into a smaller space when doing so is cheap under sparsity, in a geometric pattern precise enough that a bug in reproducing it was catchable by comparing against the theoretically-required answer; an overcomplete sparse autoencoder can partially undo that packing, with TopK's structural sparsity mechanism dominating the older L1 penalty's Pareto front for a specific, mechanistically-identified reason (shrinkage) rather than by unexplained magic; and a geometric recovery, even a very good one, is not yet a causal one, by a margin this piece measured directly rather than gestured at. None of the three claims required taking the literature's word for it, and none of them came back "trivially true" once measured — the phase diagram had a real convergence bug hiding in it, the shrinkage measurement needed the right $L_0$ regime to show a clean signal, and the causal validation surfaced a 77%-divergence finding nobody was looking for going in. That — measure it, expect the measurement to complicate the clean story somewhat, report the complication anyway — is the actual discipline mechanistic interpretability is asking practitioners to adopt, at whatever scale they're working at.

Sources

Olshausen, B. A. & Field, D. J. (1996), "Emergence of simple-cell receptive field properties by learning a sparse code for natural images," Nature.
Tibshirani, R. (1996), "Regression Shrinkage and Selection via the Lasso," Journal of the Royal Statistical Society.
Elhage, N. et al. (2022), "Toy Models of Superposition," Anthropic.
Bricken, T. et al. (2023), "Towards Monosemanticity: Decomposing Language Models With Dictionary Learning," Anthropic.
Templeton, A. et al. (2024), "Scaling Monosemanticity: Extracting Interpretable Features from Claude 3 Sonnet," Anthropic.
Gao, L. et al. (2024), "Scaling and Evaluating Sparse Autoencoders," arXiv:2406.04093.
Rajamanoharan, S. et al. (2024), "Improving Dictionary Learning with Gated Sparse Autoencoders," arXiv:2404.16014.
Bussmann, B. et al. (2024), "BatchTopK Sparse Autoencoders," arXiv:2412.06410.
Lieberum, T. et al. (2024), "Gemma Scope: Open Sparse Autoencoders Everywhere All at Once on Gemma 2," arXiv:2408.05147.
"Mechanistic Interpretability for AI Safety — A Review," arXiv:2404.14082.

Quantum-Inspired, Not Quantum: The Physics of Tensor Networks Behind Production LLM Compression

Mohamed Bal — Sat, 04 Jul 2026 23:28:08 +0000

⚙️ Show me the code: The complete, production-ready PyTorch implementation (TT-SVD linear layer) discussed in this article is available on GitHub: mohamed-bal/tt-linear-pytorch

One clarification before anything else: "Quantum-Inspired" ≠ "Quantum Computing"

This has to be stated with zero ambiguity, because the framing risks misleading readers otherwise: every technique in this piece runs on ordinary GPUs, today, in production. The word "quantum" here refers to the mathematical/physical origin of the toolkit — tools originally built to simulate many-body quantum systems on classical computers — not to actual qubit hardware.

The real state of quantum hardware as of mid-2026: we're squarely in the late-NISQ (Noisy Intermediate-Scale Quantum) era — roughly 50–200 physical qubits, gate error rates in the 10⁻³ to 10⁻² range, no genuine fault tolerance. Production-scale, error-corrected systems are projected for a 2030–2035 window. In plain terms: no real quantum computer today reduces RAM for any LLM in any production datacenter. Any claim to the contrary is either wrong or marketing.

What actually exists is a full mathematical discipline called tensor networks — machinery built by condensed-matter physicists between roughly 1990 and 2011 to solve intractable quantum many-body problems, adapted by ML researchers to neural network weight matrices starting in 2015, and applied specifically to LLMs starting in 2023–2024. That's the real subject of this piece.

The original physics problem: why the Density Matrix Renormalization Group (DMRG) worked

In 1992, Steven White introduced DMRG to compute ground states of one-dimensional quantum spin chains. The problem: the Hilbert space of an L-site system with local dimension d grows as d^L — catastrophic exponential growth. The fix rested on a precise physical observation called the Area Law.

Precise statement: for the ground state of a local Hamiltonian with an energy gap, the entanglement entropy between two partitions of the system scales with the boundary area separating them, not with volume. In 1D, the "area" between two blocks is a single point (zero-dimensional) — meaning the entropy doesn't grow with system size at all, it stays bounded.

Formally: split a system into A and B, form the reduced density matrix:

ρ_A = Tr_B(|Ψ⟩⟨Ψ|)

and compute the von Neumann entanglement entropy:

S(ρ_A) = -Tr(ρ_A ln ρ_A)

The area law says S stays bounded regardless of the size of A — it doesn't scale up with it.

Matrix Product States (MPS) and Bond Dimension: where physics meets computation

Why does this matter computationally? Because any quantum state obeying the area law can be represented exactly by a Matrix Product State (MPS) — a chain of tensors linked by a bond dimension (denoted χ or D). The precise relationship between bond dimension and entropy, worked through in detail here:

S ≤ 2 ln(D)        (a strict bound for any bipartition in an MPS)
χ ~ exp(S)          (the required bond dimension grows exponentially with entropy)

The practical payoff: if entropy S stays bounded — as guaranteed by the area law in 1D gapped systems — the required bond dimension stays constant even as system size grows. DMRG (at O(χ³) cost per step) then runs in polynomial time instead of exponential. That's precisely why DMRG has dominated quantum many-body simulation for over 30 years.

The 2D extension is harder: Projected Entangled Pair States (PEPS) generalize MPS to 2D lattices. The area law here states entropy scales with the region's perimeter, not its area — for an L×L square block with bond dimension χ, the entropy bound is L·log(χ) (proportional to L, not L²). But the cost: the bond dimension needed within PEPS itself grows exponentially with system width — computation is far harder than plain MPS. That's exactly why LLM compression researchers reach for iPEPS + Tensor Renormalization Group (TRG) rather than plain MPS when they need to capture multi-directional correlations (like those in attention layers) instead of simple linear correlations (like those in embedding tables).

From condensed-matter physics to weight matrices: the actual bridge

The central claim this entire research direction rests on: trained (not random) neural network weight matrices exhibit low effective entanglement/correlation structure — meaning their real information content is far smaller than the raw parameter count suggests.

This isn't a philosophical assumption — it has direct empirical backing. The KARIPAP paper (October 2025, iPEPS + TRG) ran "layer-wise entanglement profiling" on LLaMA-2 7B — measuring actual entropy layer by layer — and found clear redundancy concentrated in deeper layers. The practical result: 93% memory reduction, 70% parameter reduction, with only 2–3% accuracy loss.

One technical distinction worth making precisely: this is not the same thing as ordinary low-rank decomposition (a plain 2D SVD). Tensor networks — specifically MPO (Matrix Product Operator), the operator-space variant applied to weight matrices rather than states — generalize the idea: reshape the matrix into a higher-order tensor and capture correlations across multiple dimensions simultaneously, not just one. That's exactly why TensorGPT (2023) targeted the embedding layer specifically — embedding tables have a natural hierarchical structure (word → subword → character) that maps cleanly onto a tensor-train factorization, an approach that traces back to Novikov et al.'s original "Tensorizing Neural Networks" (2015).

The second thread: Disentanglers — removing correlation before compressing

A deeper technique from "Quantum Large Language Models via Tensor Network Disentanglers" (Aizpurua et al., 2024): inspired by MERA (Multi-scale Entanglement Renormalization Ansatz). Instead of compressing directly, apply local unitary transformations (disentanglers) first to strip out short-range entanglement, so that the subsequent coarse-graining only has to deal with the remaining long-range structure — which is much cheaper to compress. Practically: if two adjacent transformer layers carry redundant correlation, remove that redundancy with a cheap local transform first, and the actual compression step operates on what's left — producing a more accurate result than compressing directly with no such pre-processing.

The third thread (and a genuinely different one): Dequantization — what it actually means, and what it doesn't

This is where precision matters most, because two very different things get conflated constantly.

In 2018, Ewin Tang — then a 17-year-old undergraduate — did something remarkable: she took a quantum algorithm that was considered one of the strongest candidates for a provable exponential speedup in quantum machine learning (Kerenidis-Prakash's quantum recommendation system), and proved the same result was achievable classically — a classical algorithm that, given a data structure supporting ℓ²-norm sampling, produces a rank-k approximation sample in O(poly(k)·log(mn)) time — only polynomially slower than the quantum version, not exponentially. Consequence: Kerenidis-Prakash's algorithm did not actually deliver an exponential speedup over classical computation — the claimed advantage rested on a strong assumption about the input data structure, not on quantum mechanics itself. The work was later generalized (Chia, Gilyén, Li, Lin, Tang, Wang — STOC 2020, JACM 2022) into a full framework for "dequantizing" a broad class of quantum SVT (Singular Value Transformation) algorithms.

The necessary distinction: dequantization is a computational complexity theory result — it answers "what can we compute classically at the same speed as quantum?" — not a published, production LLM compression technique. Tang herself put it plainly: "If you are a classical person, you would not do this pre-processing." The ℓ²-sampling data structure these algorithms require is expensive to build, and raw weight matrices in any real LLM don't arrive in that form. So: no production system today uses dequantized algorithms to reduce LLM RAM footprint — unlike MPO/tensor-train decomposition, which genuinely is deployed (CompactifAI, KARIPAP, TensorGPT).

The real value of dequantization in this context: it establishes fundamental lower bounds on any low-rank matrix operation — it tells you precisely where genuine speedup (quantum or classical) ends and where the hard floor begins. That's valuable for designing any new compression algorithm (you know upfront what's realistically achievable versus mathematically impossible), but it isn't something you drop into an inference pipeline today.

Bond Dimension as the real "RAM knob"

The point that ties all of this to production engineering: bond dimension χ is the one practical lever controlling the tradeoff between compression ratio and model accuracy — the 93%-memory-reduction-for-2–3%-accuracy-loss figure (KARIPAP) is simply one measured point on the χ curve. Smaller χ means more compression and more accuracy loss — and the relationship isn't linear, it tracks the actual entropy of each layer (as layer-wise profiling shows).

What follows is how to actually choose χ per layer in production (not one global number), how this combines with the quantization techniques (INT4/INT8/FP8) and KV-cache compression already deployed in DeepSeek/MiniMax, and CXL memory disaggregation architecture in datacenters — with real, production-grade PyTorch code for applying tensor-train decomposition to a weight matrix, complete with error handling and memory profiling.

Note: The following implementation, along with the full test suite and layer profiling utilities, is available as an installable package in the tt-linear-pytorch GitHub repository.

From bond dimension to a real PyTorch layer

Everything in this section has been numerically verified against a dense-matrix reference before being written here — reconstruction error drops to ~1e-6 at full rank and degrades predictably as χ is truncated, and the efficient forward pass matches nn.Linear exactly at full rank. This is TT-matrix decomposition (Oseledets 2011; TTM form per Novikov et al.) implemented as a drop-in replacement for nn.Linear, with the forward pass computed by contracting TT-cores directly — the dense (O, I) matrix is never materialized, which is the entire point.

"""
tt_linear.py — TT-matrix (tensor-train) compressed linear layer.

Implements TT-SVD decomposition (Oseledets, 2011) in TT-matrix / TTM form
(Novikov et al., "Tensorizing Neural Networks", 2015) and a drop-in nn.Linear
replacement whose forward pass never materializes the dense weight matrix.

Numerically verified: at full rank, reconstruction error and forward-pass
output match a dense nn.Linear to ~1e-6 relative error (float32).
"""
from __future__ import annotations

import math
import logging
from dataclasses import dataclass
from typing import Sequence

import torch
import torch.nn as nn

logger = logging.getLogger(__name__)


@dataclass(frozen=True)
class TTMatrixConfig:
    """Factorization spec for a TT-matrix layer.

    out_factors / in_factors: dimension factorizations such that
        prod(out_factors) == out_features
        prod(in_factors)  == in_features
    ranks: internal bond dimensions, length == len(out_factors) - 1
           (boundary ranks r_0 = r_d = 1 are implicit, not listed here).
    """
    out_factors: Sequence[int]
    in_factors: Sequence[int]
    ranks: Sequence[int]

    def __post_init__(self) -> None:
        if len(self.out_factors) != len(self.in_factors):
            raise ValueError(
                f"out_factors (len={len(self.out_factors)}) and in_factors "
                f"(len={len(self.in_factors)}) must have the same length."
            )
        if len(self.ranks) != len(self.out_factors) - 1:
            raise ValueError(
                f"ranks must have length {len(self.out_factors) - 1}, "
                f"got {len(self.ranks)}."
            )
        if any(r < 1 for r in self.ranks):
            raise ValueError("all ranks must be >= 1.")
        if any(f < 1 for f in list(self.out_factors) + list(self.in_factors)):
            raise ValueError("all factors must be >= 1.")

    @property
    def out_features(self) -> int:
        return math.prod(self.out_factors)

    @property
    def in_features(self) -> int:
        return math.prod(self.in_factors)

    def max_ranks(self) -> list[int]:
        """Theoretical max useful rank at each internal cut — exceeding this
        wastes parameters without adding representational power (confirmed
        empirically: reconstruction error saturates past this point)."""
        d = len(self.out_factors)
        result = []
        for k in range(d - 1):
            left = math.prod(self.out_factors[: k + 1]) * math.prod(self.in_factors[: k + 1])
            right = math.prod(self.out_factors[k + 1 :]) * math.prod(self.in_factors[k + 1 :])
            result.append(min(left, right))
        return result

    def validate_against(self, out_features: int, in_features: int) -> None:
        """Raise if this config doesn't actually factorize the target layer shape.
        Call this before decomposing any real weight — a silent shape mismatch
        here is a correctness bug, not something to coerce or truncate around."""
        if self.out_features != out_features or self.in_features != in_features:
            raise ValueError(
                f"Config factorizes ({self.out_features}, {self.in_features}), "
                f"but target layer is ({out_features}, {in_features})."
            )
        max_r = self.max_ranks()
        for requested, maximum in zip(self.ranks, max_r):
            if requested > maximum:
                raise ValueError(
                    f"Requested rank {requested} exceeds theoretical max {maximum} "
                    f"at this cut — reduce ranks or this factorization is wasting parameters."
                )


def tt_svd_decompose(
    weight: torch.Tensor, config: TTMatrixConfig
) -> list[torch.Tensor]:
    """Decompose a dense (out_features, in_features) weight matrix into TT-cores.

    core[k] has shape (r_{k-1}, out_factors[k], in_factors[k], r_k), with
    r_0 = r_d = 1. Uses sequential SVD (TT-SVD); the last core absorbs the
    remainder directly with no further truncation, since r_d = 1 by construction.

    Raises:
        ValueError: if `config` doesn't factorize `weight`'s actual shape.
    """
    out_f, in_f, ranks = config.out_factors, config.in_factors, config.ranks
    config.validate_against(*weight.shape)

    d = len(out_f)
    full_ranks = [1] + list(ranks) + [1]

    tensor = weight.reshape(list(out_f) + list(in_f))
    interleave_perm = [axis for k in range(d) for axis in (k, d + k)]
    current = tensor.permute(interleave_perm).contiguous()

    cores: list[torch.Tensor] = []
    left_dim = 1
    for k in range(d):
        m_k = out_f[k] * in_f[k]
        if k < d - 1:
            mat = current.reshape(left_dim * m_k, -1)
            try:
                U, S, Vh = torch.linalg.svd(mat, full_matrices=False)
            except torch.linalg.LinAlgError as e:
                # SVD failure here almost always means NaN/Inf already present
                # in the source weights — surface that clearly instead of a
                # cryptic LAPACK error.
                raise RuntimeError(
                    f"SVD failed decomposing core {k}; check the source weight "
                    f"tensor for NaN/Inf before decomposing."
                ) from e

            r_k = min(full_ranks[k + 1], S.shape[0])
            core = U[:, :r_k].reshape(left_dim, out_f[k], in_f[k], r_k)
            cores.append(core)

            remainder = torch.diag(S[:r_k]) @ Vh[:r_k, :]
            trailing_shape = [dim for j in range(k + 1, d) for dim in (out_f[j], in_f[j])]
            current = remainder.reshape([r_k] + trailing_shape)
            left_dim = r_k
        else:
            cores.append(current.reshape(left_dim, out_f[k], in_f[k], 1))

    return cores


def tt_forward(
    x: torch.Tensor, cores: Sequence[torch.Tensor], out_factors: Sequence[int], in_factors: Sequence[int]
) -> torch.Tensor:
    """y = x @ W^T for W implicit in TT-cores — the dense matrix is never formed.

    x: (batch, in_features). Returns (batch, out_features).
    """
    batch = x.shape[0]
    d = len(cores)
    state = x.reshape([batch] + list(in_factors))
    n_out_so_far = 0

    for k in range(d):
        core = cores[k]
        if k == 0:
            core2 = core.squeeze(0)  # (o_k, i_k, r_k); r_prev == 1
            state = state.movedim(1, -1)
            state = torch.tensordot(state, core2, dims=([-1], [1]))
            state = state.movedim(-2, 1)
        else:
            i_k_axis = 1 + n_out_so_far
            state = state.movedim(i_k_axis, -1)  # -> (..., r_prev, i_k)
            state = torch.tensordot(state, core, dims=([-2, -1], [0, 2]))
            state = state.movedim(-2, 1 + n_out_so_far)
        n_out_so_far += 1

    state = state.squeeze(-1)  # drop trailing r_d == 1
    return state.reshape(batch, math.prod(out_factors))


class TTLinear(nn.Module):
    """Drop-in replacement for nn.Linear backed by TT-matrix cores.

    Construct via `TTLinear.from_dense(existing_linear, config)` to compress
    an already-trained layer, or directly for training a TT-native layer
    from random init.
    """

    def __init__(self, config: TTMatrixConfig, bias: bool = True, dtype=None, device=None):
        super().__init__()
        self.config = config
        factory = {"dtype": dtype, "device": device}
        self.cores = nn.ParameterList()
        d = len(config.out_factors)
        full_ranks = [1] + list(config.ranks) + [1]
        for k in range(d):
            shape = (full_ranks[k], config.out_factors[k], config.in_factors[k], full_ranks[k + 1])
            # Xavier-ish scaling per core keeps the product's variance sane at init.
            core = torch.randn(shape, **factory) / math.sqrt(config.in_factors[k] * full_ranks[k])
            self.cores.append(nn.Parameter(core))
        self.bias = nn.Parameter(torch.zeros(config.out_features, **factory)) if bias else None

    @classmethod
    def from_dense(cls, linear: nn.Linear, config: TTMatrixConfig) -> "TTLinear":
        """Compress an existing trained nn.Linear into TT-matrix form."""
        config.validate_against(linear.out_features, linear.in_features)
        module = cls(config, bias=linear.bias is not None,
                      dtype=linear.weight.dtype, device=linear.weight.device)
        with torch.no_grad():
            decomposed = tt_svd_decompose(linear.weight.detach(), config)
            for core_param, core_value in zip(module.cores, decomposed):
                core_param.copy_(core_value)
            if linear.bias is not None:
                module.bias.copy_(linear.bias.detach())
        return module

    def forward(self, x: torch.Tensor) -> torch.Tensor:
        orig_shape = x.shape
        x2d = x.reshape(-1, orig_shape[-1])
        out = tt_forward(x2d, list(self.cores), self.config.out_factors, self.config.in_factors)
        if self.bias is not None:
            out = out + self.bias
        return out.reshape(*orig_shape[:-1], self.config.out_features)

    def compression_report(self) -> dict:
        """Parameter/byte comparison against the equivalent dense nn.Linear —
        run this before deploying a given rank choice, not after."""
        dense_params = self.config.out_features * self.config.in_features
        tt_params = sum(c.numel() for c in self.cores)
        bytes_per_elem = next(iter(self.cores)).element_size()
        return {
            "dense_params": dense_params,
            "tt_params": tt_params,
            "compression_ratio": tt_params / dense_params,
            "dense_bytes": dense_params * bytes_per_elem,
            "tt_bytes": tt_params * bytes_per_elem,
        }

Choosing χ per layer: don't guess a global number

The test that matters before trusting any of this on real weights: a random matrix has no compressible structure, and TT-decomposition correctly refuses to compress it — reconstruction error only drops to near-zero once rank approaches the theoretical maximum, at which point the TT format actually costs more parameters than dense storage (verified above: at full rank on a random 64×64 matrix, TT storage was 2x the dense size). This is expected and correct — it's the same distinction from the theory section: entropy is bounded for trained weights with real correlation structure, not for random ones. Compression only pays off where the entropy actually is low, which is a layer-by-layer, empirical question, not a global constant.

The correct procedure for picking χ on a real trained layer:

def profile_layer_compressibility(
    linear: nn.Linear, config: TTMatrixConfig, sample_batch: torch.Tensor
) -> dict:
    """Run before deciding a layer's rank in production: measures reconstruction
    error AND functional error (output divergence on real activations, not just
    weight Frobenius norm) at the requested rank.
    """
    tt_layer = TTLinear.from_dense(linear, config)
    with torch.no_grad():
        dense_out = linear(sample_batch)
        tt_out = tt_layer(sample_batch)
        functional_rel_error = (
            torch.linalg.norm(dense_out - tt_out) / torch.linalg.norm(dense_out)
        ).item()
    report = tt_layer.compression_report()
    report["functional_rel_error"] = functional_rel_error
    return report

Run this per layer with a small batch of real activations (not synthetic noise — the whole point is that real activations reveal the actual low-entanglement structure), sweep a few candidate rank configs, and pick the smallest χ that keeps functional error under your accuracy budget for that specific layer. KARIPAP's "layer-wise entanglement profiling" finding — that redundancy concentrates in deeper layers — means this sweep is not going to converge on a single global χ; expect earlier layers to need higher rank and deeper layers to tolerate aggressive truncation.

Compounding compression: TT-cores are still ordinary tensors

One point worth being explicit about because it's easy to miss: TT-matrix decomposition and quantization are not competing techniques — they compose. Once a layer is in TT-core form, each core is just a small ordinary tensor, and every standard quantization recipe (INT8 post-training quantization, INT4 GPTQ/AWQ-style, FP8) applies directly to the cores themselves. A core at (r_{k-1}, o_k, i_k, r_k) with r ≈ 16–32 is small enough that per-channel or even per-core quantization scales stay well-conditioned — you get the tensor-network compression ratio and the quantization compression ratio multiplicatively, not as alternatives to choose between.

The honest number: TT is not free at inference time

Correctness tests confirm the math is right. They say nothing about speed — and speed is where this gets uncomfortable. Benchmarked on this repo's own benchmarks/bench_tt_linear.py (methodology: torch.cuda.synchronize()-correct timing, 50 trials after warmup, mean±std):

On small square matrices (256–1024), TTLinear is 2x–50x slower in wall-clock latency than dense nn.Linear. This is the direct cost of not having a fused TT-contraction kernel: dense dispatches to one heavily-optimized BLAS GEMM; TTLinear.forward performs several smaller tensordot operations with real dispatch overhead between them.

But that finding doesn't generalize to real model scale — and this is worth stating plainly rather than quietly correcting. Benchmarked against Gemma-3-1B's actual MLP dimensions (embed_dim=1152, MLP hidden=6912), the picture flips at low rank: up to 5.9x faster than dense at rank 16, batch 1, while keeping the memory savings — a genuine speed and memory win in exactly the regime (low rank, aggressive compression) where the entropy argument above says compression is justified in the first place. The crossover runs the other way at high rank + large batch (down to ~4x slower), consistent with the small-matrix result. Full rank×batch sweep and methodology: benchmarks/README.md.

Use this when memory is the hard constraint — and benchmark your own model's actual layer shapes and rank choice before assuming either direction on latency.

KV-cache and datacenter memory: where the bytes actually live

Weight compression addresses static parameter storage. In production, KV-cache — which scales with batch size × context length × layers, not with parameter count — is frequently the larger operational cost, and it needs a different set of architectural decisions:

Approach	Latency	Effective capacity	Complexity	Notes
GPU HBM only (baseline full-attention serving)	Lowest — HBM bandwidth (~2–3 TB/s on current datacenter GPUs)	Limited to physical GPU memory	Lowest	The ceiling on concurrent long-context requests is set directly by HBM capacity; this is why sparse/compressed KV formats (MiniMax MSA, DeepSeek CSA+HCA — covered separately) matter as much as raw memory tricks
CPU RAM offload via PCIe (e.g. vLLM-Offload style)	Highest under cache miss — PCIe Gen4/Gen5 bandwidth (32–64 GB/s in published testbeds) is an order of magnitude below HBM	Much larger — system RAM is cheap and abundant	Low-moderate	Simple to deploy on existing hardware; the bandwidth cliff between HBM and PCIe-attached RAM is the whole tradeoff
CXL-attached pooled memory	Between the two — CXL.mem gives near-DRAM latency with byte addressability and cross-device pooling	Large, poolable across nodes	Higher — newer hardware ecosystem, integration work	The approach explored in recent disaggregated-KV-cache research for sparse-attention LLMs; a genuine middle ground rather than a strict win, since the ecosystem (drivers, interconnect topology) is still maturing
Hierarchical selective fetch (e.g. the Spin-style approach: full KV cache resident in CPU memory, only "critical" entries fetched to GPU on demand)	Depends on hit rate — good when the underlying attention mechanism is already sparse (MSA/DSA-style scoring tells you which KV entries matter)	Effectively unbounded (CPU-resident)	Highest — requires the serving stack to be sparsity-aware, not just memory-aware	This is the option that actually composes with the attention-architecture choices from earlier: the same Index-Branch/Lightning-Indexer scoring that makes MSA/DSA compute-efficient also tells the memory system which KV entries are worth fetching across the PCIe/CXL boundary in the first place

The practical takeaway: don't treat weight compression and KV-cache placement as independent decisions made by different teams. The row you pick for KV-cache placement should be informed by whether your attention mechanism is sparse in the first place — a sparsity-aware serving stack turns "offload everything, fetch what's needed" from a bandwidth gamble into a mechanism that fetches close to only what the model actually attends to.

Security notes for anything in this pipeline

Never torch.load() a checkpoint from an untrusted source without weights_only=True. Standard pickle-based checkpoints can execute arbitrary code on load; this applies just as much to TT-core checkpoints as to any other model artifact — a compressed model is not a safer model.
Validate rank/factor configs before allocating tensors, especially if any part of the config is derived from external input (a config file, an API parameter, a user-uploaded model spec). TTMatrixConfig.__post_init__ and validate_against above exist specifically so a malformed or adversarial config raises immediately with a clear message, rather than allocating an unexpectedly enormous tensor or silently producing wrong results.
Log, don't print, in anything that runs in a service. Swap the module logger's level and handlers per your existing observability stack; the point of using logging here instead of bare print is that compression failures need to show up in the same place as every other production error, not scroll past in stdout.

What follows ties the theory and the code together into an actual decision procedure: given a specific layer's measured entropy profile, a latency budget, and a KV-cache placement choice, which combination of techniques to reach for — and, just as importantly, when none of this is worth the engineering cost compared to simply using a smaller dense model.

The framework, decision by decision

Each node above hides a real decision procedure. Here's what actually goes into each one.

Node 1 — "Is functional error acceptable at aggressive χ truncation?"

The number that matters is functional error on real activations (from profile_layer_compressibility in the previous section), not weight-Frobenius-norm error. A layer can have large weight-reconstruction error but small functional error if the truncated singular directions correspond to input subspaces the layer rarely sees in practice — and the reverse is also possible. Measure the thing you actually care about.

The threshold itself is workload-dependent, but the more important engineering point is that per-layer error does not simply add up across the network — it can compound nonlinearly through subsequent layers, or in some cases partially cancel. Treat per-layer functional error as a fast screening metric to decide which layers are even candidates for aggressive compression, not as the final acceptance criterion. The final criterion has to be an end-to-end evaluation: task accuracy delta, perplexity delta, or whatever your actual production quality metric is, measured on the fully compressed model, not summed per-layer estimates. A reasonable procedure:

Screen all layers with the cheap per-layer functional-error check, sorted by "how compressible is this layer" (this reproduces KARIPAP's finding empirically on your own model — expect deeper layers to screen as more compressible).
Compress layers greedily from most-compressible to least, re-running full end-to-end evaluation after each batch of layers, stopping when end-to-end quality crosses your acceptance line.
Keep the boundary layer's rank as a tunable — this is the number you'll revisit first if quality regresses after any future fine-tune or version bump.

Node 2 — Quantization: hardware determines the right answer, not preference

This decision is dictated by which GPU generation you're actually deploying on, not by which format is theoretically better:

GPU generation	Native FP8 tensor cores	Recommendation
Ampere (A100)	No	INT8 — FP8 has no dedicated hardware path here; forcing it buys nothing
Ada Lovelace (L40S, RTX 4090-class)	Partial — FP8 ops exist without dedicated scaling hardware, silently falling back toward BF16-level gains	INT8 for guaranteed throughput; FP8 only if you've benchmarked an actual win on your kernel
Hopper (H100/H200)	Yes — first-generation Transformer Engine with dedicated FP8 tensor cores	FP8 — real published economics: ~$0.71/M tokens vs ~$1.19/M at BF16 for Llama 3.1 70B on H100, 1.4–1.8x throughput at large batch
Blackwell (B100/B200/B300)	Yes, second-generation, plus native FP4/FP6	FP8 as the safe default; FP4 (NVFP4/MXFP4) if you've validated accuracy on your specific model — FP4 roughly doubles throughput again over FP8 but carries materially higher accuracy risk

The reason INT8 remains the correct default on older hardware isn't nostalgia: FP8 retains a floating-point exponent, giving it a wider dynamic range than INT8's fixed-range integers — genuinely better suited to transformer activation distributions — but that advantage only materializes with dedicated silicon to exploit it. Without that silicon, you're paying FP8's format-conversion overhead for a BF16-equivalent result.

Node 3 — KV-cache placement: the sparsity signal is what makes "hierarchical selective fetch" viable at all

This branch only works in one direction. If your attention mechanism produces an explicit relevance signal (the Index Branch score in MSA-style architectures, the Lightning Indexer score in DSA-style architectures), that signal is exactly the thing a hierarchical fetch scheduler needs to decide which KV entries are worth moving across a PCIe or CXL boundary before the attention computation needs them. Without that signal, "fetch what's needed" degrades to "fetch everything, cache what fits" — which is just PCIe offload with extra steps.

If you're running a dense-attention model, don't build the hierarchical-fetch complexity — go straight to the HBM-vs-PCIe-offload decision from the earlier comparison table. The engineering cost of the sparsity-aware scheduler only pays for itself when there's a real sparsity signal driving it.

Node 4 — Budget check: a worked example

Concrete arithmetic, using representative (not benchmark-sourced) numbers to illustrate the calculation shape:

Setup: a 7B-parameter-class model, serving batch size 8, context length 128K tokens, 32 layers, FP16 KV cache.

Dense KV cache footprint: 2 (K and V) × 32 layers × 128,000 tokens × 8 batch × hidden_dim(4096) × 2 bytes (FP16) ≈ 537 GB — already exceeds a single GPU's HBM at this batch/context combination, before weights are even counted.
With a sparse-attention architecture retaining, say, 10% of KV entries as "relevant" per the indexer's selection (illustrative ratio, not a published figure) and INT8 KV quantization: 537 GB × 0.10 × 0.5 (INT8 is half the bytes of FP16) ≈ 27 GB — now plausibly HBM-resident on modern datacenter GPUs, without touching PCIe/CXL at all.
Weight footprint: TT-decomposition at a validated χ per layer (from Node 1's sweep) plus INT8 quantization might realistically land in the 15–30% of dense-FP16-weight-bytes range for a model with KARIPAP-like redundancy — but this number has to come from your own compression_report() output on your own checkpoint, not from a generic industry figure. This is the one place a hallucinated benchmark would be actively dangerous to plan capacity around.

The point of walking through the arithmetic explicitly isn't the specific numbers — it's that the KV-cache term and the weight-compression term are computed independently, from independent measurements, and both have to clear your actual per-GPU HBM budget (or your PCIe/CXL-offload bandwidth budget, if you've deliberately chosen that path) before "deploy" is the right answer at Node 4's diamond.

When the honest answer is "don't build this pipeline"

The feedback loop in the diagram exists because this is a real, frequent outcome, not an edge case. A comparison worth having explicitly before committing engineering time:

Approach	Performance	Resource consumption	Engineering/maintenance cost
No compression (dense, native precision)	Baseline	Highest	Lowest — no pipeline to maintain, no re-profiling on every checkpoint update
Quantization only (INT8/FP8, no TT-decomposition)	Near-baseline quality, ~2x memory/throughput win	Moderate reduction	Low — mature tooling (vLLM, TensorRT-LLM, SGLang all support this natively), minimal custom code
Full pipeline (TT-decomposition + quantization + sparsity-aware KV placement)	Latency is scale-dependent — 2x–50x slower on small matrices, but up to 5.9x faster at real LLM scale + low rank (see "the honest number" above); benchmark your own dimensions, don't assume either direction	Largest reduction — this is the only option in the set that meaningfully changes the weight footprint, not just KV/activation footprint	Highest — custom decomposition code, per-layer rank tuning, re-profiling required on every checkpoint change, and everything in the security notes above becomes your team's responsibility, not a library maintainer's
Smaller off-the-shelf dense model	Depends entirely on whether a smaller model meets your quality bar	Smallest — by construction	Lowest of any option that actually reduces footprint — no custom pipeline, just a different model

The full pipeline is the right call when you have a specific, fixed model you're contractually, legally, or technically required to run as-is — data residency requirements, a licensing constraint, a fine-tuned checkpoint with no smaller equivalent, or genuine on-premise hardware limits that no vendor API can route around. It is very often not the right call when the actual requirement is "a model that's good enough and cheap enough," because that requirement is usually satisfied faster and more reliably by evaluating an already-smaller model than by building and maintaining a custom compression pipeline for a larger one.

Production checklist before cutover

Version-pin the entropy profile to the checkpoint hash. A rank choice validated against checkpoint A is not guaranteed valid against checkpoint A after a fine-tune. Store compression_report() output keyed by checkpoint hash, not by model name.
Shadow-deploy before cutover. Run the compressed model against a copy of live traffic without serving its output, comparing downstream task metrics against the dense baseline over a realistic traffic distribution — the per-layer and even end-to-end offline eval from Node 1 is a necessary but not sufficient check.
Keep the dense checkpoint loadable as a feature-flagged fallback. Compression should be a runtime choice you can revert without a redeploy, not a one-way migration.
Never torch.load() any checkpoint — compressed or dense — without weights_only=True against untrusted sources, as covered earlier; a compressed model is not a lower-risk artifact.
Re-run the entire decision flow on every architecture or checkpoint version change. Entropy structure is a property of the specific trained weights, not the model architecture in the abstract — this is the direct practical consequence of Part 1's theoretical claim that trained (not random) weights are what carries the compressible structure in the first place.

If you're looking to integrate this into your inference stack, you can drop in the TTLinear module directly from the GitHub Repo. PRs and issues are welcome!

Closing the loop

The physics in the first section and the code in the second aren't separate concerns bolted together for a blog post — they're the same claim at two levels of abstraction. Area law says entanglement entropy in a physical system stays bounded when there's real structure to exploit. KARIPAP's layer-wise profiling says the same thing is empirically true of trained transformer weights, concentrated unevenly by depth. And the decision framework above is just the operational form of that claim: measure where the structure actually is, on your own model, before spending engineering effort compressing it — because the same theory that explains why compression works this well on structured weights is exactly why it does nothing useful on weights that aren't.

Sources: Area law review; DMRG/MPS tutorial (ITensor); DMRG/MPS lecture notes (Feiguin); CompactifAI; KARIPAP; TensorGPT; Novikov et al., "Tensorizing Neural Networks"; Quantum LLMs via Tensor Network Disentanglers; Ewin Tang, "A quantum-inspired classical algorithm for recommendation systems"; Chia, Gilyén, Li, Lin, Tang, Wang — dequantizing framework; 2026 NISQ status.

MiniMax-M3 on DEVUP AI — A Technical Deep Dive into MiniMax Sparse Attention, Benchmarks, and Production Integration

Mohamed Bal — Sat, 04 Jul 2026 22:05:02 +0000

MiniMax-M3 is live on DEVUP AI — routable through the same OpenAI-compatible gateway as our other 170+ models, billed in DZD via Edahabia/CIB. Model ID: MiniMaxAI/MiniMax-M3.

Most launch-day coverage repeated MiniMax's press numbers without reading the actual paper behind the architecture. This post goes one level deeper: what the arXiv paper actually claims, where it diverges from the PR figures, what M3 costs and how it's exposed on our platform, and production-grade integration code — retries, timeouts, streaming, and multimodal, all handled correctly.

What's actually new

Three capabilities that, until now, only closed-source models combined in one checkpoint:

Frontier-tier coding/agentic performance — competitive with GPT-5.5 and Gemini 3.1 Pro on agentic benchmarks
1M-token context (with a guaranteed floor — more on that below)
Native multimodality — image and video input, trained in from step zero, not bolted on

Architecture: MoE, ~428B total parameters, ~23B active per token.

Under the hood: MiniMax Sparse Attention (MSA)

Here's the mechanism, straight from the technical report (Lai et al., MiniMax, June 2026):

MSA is a blockwise sparse attention built on top of Grouped Query Attention (GQA) — not a replacement for GQA, an extension of it. It has two branches:

Index Branch — a lightweight scorer that ranks key-value blocks and independently selects a Top-k subset per GQA group. This is the detail most summaries miss: selection isn't global, it's group-specific, which is what lets the model retrieve different relevant context for different attention heads within the same layer.
Main Branch — performs exact block-sparse attention, but only over the blocks the Index Branch selected. No approximation inside the selected blocks — the sparsity is in which blocks get attended to, not in how attention is computed within them.

To make that sparsity translate into wall-clock speed rather than just fewer FLOPs on paper, MiniMax co-designed the GPU kernel with exp-free Top-k selection and KV-outer sparse attention, specifically to keep tensor-core utilization high under block-granular memory access — a detail that matters because naive sparse attention implementations often lose their theoretical speedup to poor hardware utilization.

The numbers, precisely: On a 109B-parameter research checkpoint with native multimodal training, MSA matches GQA quality while cutting per-token attention compute by 28.4x at 1M context. Paired with the co-designed kernel, that's 14.2x prefill and 7.6x decoding wall-clock speedup on H800 GPUs.

The gap between the two curves widens with sequence length — GQA's cost grows steeply while MSA stays close to flat, which is the entire point of a sparse mechanism: the advantage barely matters at 32K tokens and becomes decisive at 1M.

Compare that to the press figures circulating since launch — "~1/20th compute, 9x prefill, 15x decode." These aren't wrong, but they're not the same number as the paper's, and the discrepancy is worth understanding rather than hand-waving: the paper's 28.4x figure is specifically attention compute on a smaller 109B test model, while the press figures describe end-to-end per-token compute on the production 428B/23B-active M3 — a different denominator (attention-only vs. full forward pass) and a different model scale. Both can be true simultaneously. If you're doing capacity planning, use the paper's numbers for attention-layer cost modeling and treat the press figures as a rough end-to-end proxy, not interchangeable data points.

The inference kernel itself is open-sourced: github.com/MiniMax-AI/MSA. Worth cloning if you're evaluating self-hosting.

Benchmarks — with the caveats that matter

MiniMax reports:

Benchmark	M3 Score	Notes
SWE-Bench Pro	59.0%	Ahead of GPT-5.5 (58.6%) and Gemini 3.1 Pro
Terminal-Bench 2.1	66.0%
MCP Atlas	74.2%
BrowseComp	83.5	vs. Claude Opus 4.7's 79.3

Two things before you weight these for a production decision:

The comparison baseline is stale. MiniMax benchmarked against Claude Opus 4.7 — Opus 4.8 shipped four days before M3's launch. Against the actual current frontier, M3 sits further back than the launch post implies.
Self-reported, self-scaffolded. Several results ran on MiniMax's own infrastructure with agent scaffolding (Claude Code, Mini-SWE-Agent, Terminus). Weights are now public on Hugging Face, so independently reproduced numbers should surface over the coming weeks — re-check before leaning on these for anything critical.

The long-horizon autonomy demos are the more interesting signal for agentic use cases: M3 ran unsupervised for ~12 hours reproducing an ICLR 2025 paper (18 commits, 23 figures, core experimental claims validated), and separately spent ~24 hours on a CUDA/Hopper kernel optimization task — 147 submissions, 1,959 tool calls, pushing hardware utilization from 7.6% to 71.3%, continuing to improve well past the point where most models plateau and stop.

On DEVUP AI: specs and pricing

	Value
Model ID	`MiniMaxAI/MiniMax-M3`
Pricing	105 DZD in / 420 DZD out / 21 DZD cached — per 1M tokens
Context window (as exposed)	524K tokens
Architecture	MoE, ~428B total / ~23B active
Capabilities	Public · JSON mode · Function calling · Multimodal

One detail worth noticing: MiniMax's official ceiling is 1M tokens, but they only guarantee 512K. 512K × 1024 = 524,288 — which is exactly the 524K we expose. We're not truncating the model's capability; we're exposing the guaranteed floor rather than the marketing ceiling, so your context-length assumptions hold under load instead of degrading silently past the guaranteed range.

Production integration

Basic completion, with real retry/error handling

The gateway returns OpenAI-compatible error bodies (error.message, error.type, error.code). 400/401/403 are never retryable — fix the payload, key, or balance. 429 and 500 are, and 429 responses include a Retry-After header you should actually respect instead of guessing a backoff.

import { DevUpAI } from "devupai";

const MODEL = "MiniMaxAI/MiniMax-M3";

const devup = new DevUpAI({
  apiKey: process.env.DEVUP_API_KEY!, // never hardcode; pull from a secrets manager in prod
});

interface DevUpErrorBody {
  error: { message: string; type: string; code: string };
}

type ChatMessage = { role: "system" | "user" | "assistant"; content: string };

/**
 * Calls MiniMax-M3 through the DEVUP AI gateway with production-grade handling:
 * - explicit timeout suited to long-context (up to 524K token) requests, which
 *   can legitimately take tens of minutes, not seconds
 * - retry only on documented-retryable statuses (429, 500)
 * - honors the Retry-After header on 429 instead of guessing a backoff
 * - exponential backoff + jitter as a fallback when Retry-After is absent
 */
async function callMiniMaxM3(
  messages: ChatMessage[],
  { timeoutMs = 120_000, maxRetries = 3 }: { timeoutMs?: number; maxRetries?: number } = {}
): Promise<string> {
  let attempt = 0;

  while (true) {
    const controller = new AbortController();
    const timer = setTimeout(() => controller.abort(), timeoutMs);

    try {
      const completion = await devup.chat.completions.create(
        { model: MODEL, messages, max_tokens: 4096 },
        { signal: controller.signal }
      );
      return completion.choices[0]?.message?.content ?? "";
    } catch (err: any) {
      const status: number | undefined = err?.status ?? err?.response?.status;
      const body: DevUpErrorBody | undefined = err?.error ?? err?.response?.data;
      const code = body?.error?.code;
      const retryable = status === 429 || status === 500;

      if (!retryable || attempt >= maxRetries) {
        throw new Error(
          `MiniMax-M3 request failed (${status ?? "unknown"} / ${code ?? "no_code"}): ${
            body?.error?.message ?? err.message
          }`
        );
      }

      const retryAfterHeader = err?.response?.headers?.["retry-after"];
      const retryAfterMs = retryAfterHeader ? Number(retryAfterHeader) * 1000 : 2 ** attempt * 1000;
      const jitterMs = Math.random() * 250;
      await new Promise((resolve) => setTimeout(resolve, retryAfterMs + jitterMs));
      attempt++;
    } finally {
      clearTimeout(timer);
    }
  }
}

Streaming (SSE)

For anything user-facing, stream rather than block — especially at long context, where full-response latency can be substantial.

const stream = await devup.chat.completions.create({
  model: MODEL,
  messages: [{ role: "user", content: "Summarize this 400K-token incident log and flag anomalies." }],
  stream: true,
});

for await (const chunk of stream) {
  const delta = chunk.choices[0]?.delta?.content;
  if (delta) process.stdout.write(delta);

  if (chunk.usage) {
    console.log(`\n[tokens] prompt=${chunk.usage.prompt_tokens} completion=${chunk.usage.completion_tokens}`);
  }
}

Multimodal input

const completion = await devup.chat.completions.create({
  model: MODEL,
  messages: [
    {
      role: "user",
      content: [
        { type: "image_url", image_url: { url: "https://example.com/architecture-diagram.png" } },
        { type: "text", text: "Review this system architecture diagram and flag single points of failure." },
      ],
    },
  ],
});

Tool calling

Standard OpenAI-compatible tools array works unmodified. One tuning note specific to tool calling: MiniMax's own recommended defaults (temperature=1.0, top_p=0.95) are tuned for general generation quality, not argument-parsing reliability. For tool-calling workloads, drop temperature below 1.0 — consistent with our platform-wide guidance for structured function-call output regardless of model.

API vs. self-hosting the open weights

Weights are public on Hugging Face (~428B total / ~23B active MoE). Real architectural choice:

	Via DEVUP AI (API)	Self-hosted (open weights)
Time to integrate	Minutes — OpenAI-compatible endpoint	Days — the released MSA kernel targets H800; broader GPU/framework support (llama.cpp, etc.) is still maturing and currently falls back to dense attention
Cost profile	Pay-per-token, DZD billing, no upfront infra	High upfront GPU cost — 428B params means serious VRAM even with MoE sparsity
Data residency	Traffic routed through our gateway; see our DPA for compliance requirements	Full control — matters if you're bound by strict residency rules
Long-context throughput	Shared infra, subject to queueing under load	Full control over batching/throughput tuning, but you own the kernel engineering

One honest note on data handling: M3 is developed in Shanghai, and Chinese entities operate under legal obligations to cooperate with state intelligence requests. That's not a reason to avoid the model — it's a reason to be deliberate about what you send it, same as with any third-party inference endpoint. If you're processing regulated or sensitive data, self-hosting the open weights (or scrubbing payloads before routing through any API) is the safer default.

Why this matters for us

Long context, native multimodal, and agentic coding — at open-weight economics, with a documented and open-sourced attention kernel — means Algerian and MENA teams building RAG pipelines, code-review agents, or document-heavy workflows now have a genuinely frontier-adjacent option that doesn't require a foreign card or USD billing. One key, one DZD invoice, 170+ models including this one.

Try MiniMax-M3 on DEVUP AI →

Primary sources: arXiv:2606.13392 (MSA technical report), MiniMax-M3 on Hugging Face, DEVUP AI model page. Secondary: MiniMax official model page, VentureBeat, The Decoder, Artificial Analysis, TechTimes.

Why Algeria Needs Its Own AI Infrastructure — and Why I'm Building It

Mohamed Bal — Wed, 01 Jul 2026 18:42:26 +0000

The problem no one was solving

Every Algerian developer building with AI hits the same wall: an international payment card. OpenAI, Anthropic, Google — every major AI provider assumes you have one. Most Algerian developers don't, or don't want to deal with the friction of currency conversion, card rejections, and unpredictable billing in a foreign currency.

That's not a minor inconvenience. It's a barrier that quietly excludes an entire generation of developers from building with the best AI models available — not because they lack the skill, but because of infrastructure that was never designed with them in mind.

The vision: AI sovereignty, not just AI access

Access alone isn't the goal. The goal is sovereignty — Algeria having its own AI infrastructure layer, controlled locally, billed locally, and built to local compliance standards, instead of depending entirely on foreign gateways with no local accountability.

That's what DEVUP AI is: Algeria's first AI inference gateway, built from the ground up to remove every friction point between an Algerian developer and the AI models they need.

What DEVUP AI actually does

170+ AI models — including DeepSeek V4, Llama 3.1 405B, Qwen 3, Gemma 2, Mistral, GPT, Claude, and Gemini — through a single API
OpenAI-compatible and Anthropic-compatible — point your existing SDK at our endpoint, no code rewrite needed
Local DZD billing via Edahabia/CIB — no international card required
SATIM-certified payment infrastructure — full compliance with Algeria's national payment standards
Scoped JWT authentication for production-grade security
A dedicated SDK (npm install devupai) and full documentation, so integration takes minutes, not days

The technical bar was non-negotiable: this had to be production-grade from day one, not a side project. SATIM certification alone meant building proper transaction validation, receipt generation, chargeback tracking, and rejection-rate monitoring — the same rigor a bank would expect from a payment processor, because in a real sense, that's part of what DEVUP AI is.

Try it in 30 seconds

If you already use the OpenAI SDK, this is all it takes:

from openai import OpenAI

client = OpenAI(
    api_key="your_devup_key",
    base_url="https://api.devupai.com/v1"
)

response = client.chat.completions.create(
    model="deepseek-ai/DeepSeek-V4-Pro",
    messages=[{"role": "user", "content": "Hello from Algeria!"}]
)

print(response.choices[0].message.content)

No vendor lock-in, no rewritten integration — just a different base_url and DZD billing instead of a foreign card.

Why "fully integrated" matters

A gateway that just proxies requests isn't enough. Developers need billing, authentication, model routing, and monitoring to work together as one system — not five different tools stitched together. That integration is what lets a developer go from signup to their first API call in minutes, with DZD billing handled transparently in the background.

This is bigger than one platform

I didn't build this just to ship a product. I built it because Algeria's developer ecosystem deserves AI infrastructure that treats it as a first-class market, not an afterthought. Every student building a hackathon project, every startup integrating AI into their product, every developer experimenting with RAG pipelines or multi-agent systems — they should be able to do it without fighting their bank first.

That's the mission. The product is just how I'm executing on it.

Try it

If you're an Algerian developer (or anywhere, honestly — the API works globally), every new signup gets a free credit to test the platform.

Platform: devupai.com
Docs: docs.devupai.com
Models: devupai.com/models
SDK: npm install devupai

I'd genuinely love feedback from other developers building AI products — what's missing, what's confusing, what would make this more useful for you.