Anthropic released CHIVE, an automated pipeline that hunts for unexpected model behavior in the wild and explains it by editing the prompt and watching what changes, and the headline finding is a negative one. Predictors that can read the model's internal activations, including sparse autoencoders and natural-language autoencoders, do not beat a predictor that sees nothing but the conversation transcript. The result held across two target models, three families of predictor, hyperparameter sweeps, and a rerun with the model's reasoning turned on. Code, datasets, and checkpoints are public on GitHub.
Key facts
- CHIVE stands for Counterfactual Hypothesis Investigation Via Edits and runs in four stages: sample, screen, investigate with 5 to 15 counterfactual prompt edits, then judge.
- Activation oracles, sparse autoencoders, and natural-language autoencoders all failed to beat a transcript-only baseline.
- Published August 21, 2026 by Anthropic's alignment team as arXiv 2608.16747, with a full open release of code, datasets, LoRAs, and oracle checkpoints.
- Primary source: Anthropic's CHIVE post.
The setup is simpler than the acronym suggests. A model does something odd. The obvious next question is why, and the usual answer is a story: it did that because the prompt mentioned a deadline, or because the user sounded like an auditor. Stories are cheap. CHIVE makes them expensive by testing them. If the explanation is that the deadline caused the behavior, then removing the deadline should remove the behavior, and changing something irrelevant should not. So the pipeline generates a batch of small targeted edits, runs them, and scores the explanation on whether it actually predicted what happened.
This is a measurement called counterfactual simulatability, and it is a genuinely good idea because it grades explanations on outcomes rather than on how convincing they sound. The analogy is a mechanic who says your car stalls because of the fuel pump. The satisfying-sounding version of that claim is a paragraph about fuel pressure. The useful version is: replace the pump and see if it still stalls. CHIVE only accepts the second kind, which is what makes it a benchmark rather than an interpretability demo. It relates directly to ablation studies, which apply the same logic to model components instead of prompts.
Then comes the part that should give the field pause. Anthropic used CHIVE to build an evaluation, then tested whether tools that read the model's internals help predict what a counterfactual edit will do. Mechanistic interpretability rests on the premise that they should: if you can see the features the model is using, you should be better at forecasting how it responds to a change than someone reading only the text. Across activation oracles, sparse autoencoders, and natural-language autoencoders, that premise did not pay. The transcript-only baseline was not beaten. Turning on visible reasoning made the baseline stronger, and the internals-augmented predictors still did not catch it.
The result also surfaces something uncomfortable in passing. When Anthropic reran the pipeline with reasoning enabled, it produced many more cases where the visible reasoning was unfaithful or simply silent about what was actually driving the answer. That lines up with the broader worry about chain-of-thought faithfulness, and it lands the same week as research showing hidden reasoning blocks can be replayed and read. One study says the hidden reasoning is more exposed than anyone assumed; the other says even reading it does not tell you what you need to know.
Anthropic's own framing is notably restrained. The paper's appendix says the evaluation is a proxy, and an easy one: anyone with sampling access can just run the counterfactual and find out, so ground truth is always available here. The real cases that matter are the ones where you cannot check, where a model is doing something in deployment and there is no cheap experiment that settles why. The argument is not that interpretability tools are useless. It is that failing on the version of the problem where the answer is checkable is a bad omen for the version where it is not.
The strongest counter-argument is a fair one, and it is about what was measured. Predicting whether an edit flips a behavior is a specific and rather behavioral task, and it may simply not be where feature-level tools have an edge. Sparse autoencoders were built to find interpretable directions in activation space, not to forecast prompt sensitivity, and a tool can be scientifically valuable without winning at a task it was not designed for. Anthropic's release makes that argument testable rather than rhetorical, which is the point of shipping the datasets and checkpoints alongside the paper. The lab did not find third-party commentary worth citing yet, and neither did we; that will come once other groups run their own predictors against the benchmark.
Originally published on Ground Truth, where every claim is checked against the primary source.
Top comments (0)