Last week, Anthropic published a paper on a discovery they made regarding Claude's internal reasoning they call the J-space. It seems to be the steps that Claude works through before it commits to a final word: the closest thing we've come to seeing the inside of an LLM's "brain."
Although it holds a median of only 6–7% of a concept's representation inside the model and never more than about a tenth of the model's activity at any layer, if you switch it off, Claude's multi-step reasoning collapses to almost nothing. Fluent speech and simple recall remain intact.
I quite liked the video that Anthropic released with the research post. It's a long research post though, so I thought I'd write a summary of the parts I considered the coolest and how I interpreted the mathematics of the J-space.
This is a preview of a post from my blog. Read the full post with the interactive charts →
Top comments (0)