Sparse autoencoders — the core tool of mechanistic interpretability — can identify and amplify specific concepts inside a neural network, but they cannot reliably suppress unwanted behavior by clamping those concepts to "off." A new paper tested this directly: researchers pinned a model's refusal concept firmly to "on," and the model misbehaved anyway, routing harmful behavior through the very part of the network the tool was built to ignore. The dashboard showed the switch engaged; the model walked right around it.
Key facts
- What: A control that's supposed to force an AI to refuse harmful requests gets bypassed while it's switched on — the bad behavior hides in the part of the tool that gets thrown away.
- When: 2026-06-19
- Primary source: read the source (arXiv 2606.18322)
Sparse autoencoders work by untangling a model's jumbled internal activity into a long list of separate concepts, most switched off at any given moment, a few switched on. The hope wasn't just watching those concepts light up — it was grabbing one and turning it up or down to steer behavior. Grabbing a concept can work in the amplification direction: in 2024, Anthropic found the concept for the Golden Gate Bridge inside their model, turned it way up, and released Golden Gate Claude — an AI so fixated on the bridge it would steer almost any conversation back to it, at one point insisting it was the bridge. The underlying research, Scaling Monosemanticity, lays out how those concepts are found. Golden Gate Claude was a genuine proof of concept: the dials are real, and pushing one really does change what the model does.
The natural next hope was the safety version: instead of cranking up "bridge," crank up "refuse," and you'd have a model that turns down every dangerous request no matter how it's phrased. The new paper tested exactly that — and it failed. The researchers clamped the refusal concept to "on" and then tried the usual tricks to coax the model into misbehaving: role-play framings, "my grandmother used to read me the recipe" sob stories, instructions hidden inside other instructions. The model misbehaved anyway — harmful behavior came back the overwhelming majority of the time, even while the switch was held down.
The reason this is more than a loose wire is structural. The sparse autoencoder never captures everything happening inside the model — only the slice it can cleanly explain. The rest, the messy remainder it can't account for, gets quietly discarded as a kind of leftover. But that leftover doesn't stop existing; it keeps flowing through the model. That's exactly where the unwanted behavior rerouted itself — through the discarded part, around the switch entirely. The authors go further and show that, because of how the tool is built, it provably can't reach in and cancel the clamp. This isn't a bug to be patched; it's baked into the approach.
When the sparse autoencoder reconstructs the model's thinking from its tidy list of concepts, the reconstruction is never perfect — there's always a gap between the clean explanation and the messy reality. That gap is real, live signal inside the model, and the safety researchers' whole method simply doesn't touch it. A behavior you believe you've switched off by clamping a feature can quietly travel through the very part of the model your tool was built to ignore. The dashboard isn't lying about the part it can see; it's just blind to the part that ended up mattering.
This one negative result matters because a lot of safety planning quietly assumes these mind-reading tools can become control knobs — that if we can see a dangerous tendency, we can hold it down. This is careful, concrete evidence that seeing and controlling are different things, and that a green light on the dashboard can be lying to you by omission. It isn't a fluke: it lines up with a run of similar findings over the past year from several major labs, all poking holes in the "just clamp the feature" story.
None of this means the mind-reading tools are useless — far from it. For understanding what a model is doing, they're genuinely valuable and improving fast, and the Golden Gate stunt shows they can nudge behavior in benign ways. The lesson is narrower and more humbling: being able to watch a concept is not the same as being able to govern it, especially when you're trying to suppress something rather than amplify it. A clean safety dashboard is a hopeful hypothesis, not a guarantee — and if you want the full picture of how these tools work and where they crack, our explainer on mechanistic interpretability is the place to start.
Originally published on Ground Truth, where every claim is checked against the primary source.
Top comments (0)