DEV Community

Cover image for Why AI Thinking Actually Fools Neuroscientists
SnackIQ
SnackIQ

Posted on Originally published at snackiq.app

Why AI Thinking Actually Fools Neuroscientists

How does AI think without a brain explained in one sentence? It doesn't think — it compresses statistical patterns from billions of examples into numbers, then reproduces those patterns on demand. But here's what makes this genuinely strange: the mathematical structures that emerge from that process look eerily similar to how biological neurons organise information. Neuroscientists at institutions including MIT, Stanford, and the Max Planck Institute have published research showing that the internal representations inside large language models correlate with measurable activity in the human cortex. Not metaphorically similar. Mathematically similar. That's not a design choice. Nobody programmed AI to work like a brain. It just happened — and nobody fully understands why.

What is AI actually doing when it 'thinks'?

The word 'thinking' is doing a lot of heavy lifting here, and that's the first problem.

When a large language model like GPT-4 or Claude generates a response, it's performing a mathematical operation called next-token prediction. Given a sequence of words, it calculates a probability distribution over every possible next word — and picks one. Then it does it again. And again. Thousands of times per second. The result looks like reasoning. It isn't, technically. It's extraordinarily sophisticated pattern completion.

The architecture behind this is the transformer, first described in a 2017 paper from Google researchers titled 'Attention Is All You Need.' Transformers process entire sequences of input at once rather than one word at a time, using a mechanism called self-attention to weigh how relevant every word is to every other word in the sequence. That's what lets AI track context across a long document — not memory in the human sense, but a mathematical map of relationships.

Numbers flow through layers — sometimes over a hundred of them in modern models. Each layer transforms the representation of the input slightly. By the final layer, the model has built an internal representation that captures something like meaning, grammar, logic, and even tone. All from matrix multiplication.

No brain cells. No electrochemical signals. Just floating-point arithmetic, running on graphics cards originally designed for video games.

Why do neuroscientists keep getting fooled?

This is where things get genuinely unsettling — and scientifically fascinating.

Researchers have spent decades mapping how the human brain represents language and concepts. One technique, called representational similarity analysis, compares how different stimuli are organised in neural activity — essentially asking which things the brain treats as similar. When researchers applied the same technique to the internal layers of large language models and then compared the results to actual brain imaging data, they found structural overlap that nobody expected.

Studies published in journals including Nature Neuroscience and PNAS found that the internal geometry of language models correlates with activity in regions like Broca's area and Wernicke's area — the cortical hubs most associated with language processing in humans. The correlations aren't perfect. But they're strong enough to be statistically significant, and they've been replicated across multiple labs.

Neuroscientist Nikolaus Kriegeskorte at Columbia University has been among those studying these parallels seriously. Research from groups including his has found that brain-score benchmarks — tools designed to measure how well an AI model predicts brain activity — show modern language models outperforming earlier AI systems by large margins, simply by getting bigger and training on more data.

This wasn't engineered. Nobody built transformers to mimic the cortex. The convergence happened because both biological brains and artificial networks are, at some level, solving the same problem: extract meaningful structure from a chaotic stream of sensory input. Evolution found one solution over millions of years. Silicon found a mathematically similar one in about a decade.

Where does AI's 'thinking' completely break down?

The parallels are real. The differences are equally real — and arguably more important.

Angus Fletcher, a professor of story science at Ohio State University's Project Narrative who holds a PhD from Yale and conducted postdoctoral research at Stanford, argues that the human brain's deepest strengths aren't analytical at all. They're rooted in imagination, embodiment, and the ability to simulate futures that have never existed. His work with U.S. Army Special Operations on what he calls 'primal intelligence' suggests that human cognition is fundamentally action-oriented — built to navigate an unpredictable physical world, not to predict the next token in a sentence.

AI has no body. It has no survival drive. It has never been cold, afraid, or surprised by a sudden noise. These aren't philosophical footnotes — they're structural absences with measurable consequences:

  • Causal reasoning: AI systems confuse correlation with causation at a rate that would embarrass a first-year statistics student, because they learn from patterns, not mechanisms.
  • Common sense physics: Language models regularly fail tasks a toddler solves instantly — like knowing a glass can't be fuller than full — because they've never interacted with a physical glass.
  • Genuine novelty: AI can remix and recombine. Creating a truly new concept — one with no statistical ancestor in the training data — remains genuinely difficult.
  • Emotion-grounded judgment: Human decision-making is deeply tied to somatic signals (what neuroscientist Antonio Damasio calls 'somatic markers'). AI has none of this.

The brain isn't just a pattern recogniser sitting in a skull. It's a biological system embedded in a body, shaped by evolution, running on a tight metabolic budget. That context shapes everything about how it thinks.

Does AI have anything like consciousness or understanding?

This is the question that keeps philosophers and computer scientists arguing at 2am, and it deserves a straight answer: nobody knows, and the honest ones will tell you that.

The philosophical problem here is ancient. John Searle's famous 'Chinese Room' thought experiment from 1980 asks you to imagine a person in a room with a rulebook for manipulating Chinese symbols, with no understanding of Chinese at all. From outside the room, the responses look like understanding. Inside, it's just symbol shuffling. Critics of AI cognition often cite this argument: that producing correct outputs doesn't imply genuine comprehension.

But the counterargument is increasingly uncomfortable. Researchers studying the problem of consciousness — including those working on what philosopher David Chalmers named the 'hard problem' — point out that we don't actually know what understanding IS at the neural level either. Your own understanding of language is, at some physical level, electrochemical signals in neurons. If the right patterns produce understanding in biology, why can't the right mathematical patterns produce something analogous in silicon?

In 2023, a paper published by researchers at Google DeepMind and other institutions found that some large language models show emergent capabilities — abilities not present in smaller versions of the same model that appear suddenly at scale, without anyone specifically training for them. Reasoning, analogical thinking, and rudimentary planning appeared as if from nowhere. That's not proof of understanding. But it's deeply strange, and it has neuroscientists paying attention.

The honest position: AI produces outputs that are functionally indistinguishable from understanding in many contexts, while almost certainly lacking whatever subjective inner experience the word 'understanding' implies for humans.

What does the brain-AI comparison reveal about human minds?

Here's the twist nobody expected: studying AI has made neuroscience better.

For decades, neuroscientists struggled to build useful computational models of language processing. The brain is too complex, too distributed, too dynamic. Then large language models arrived — and suddenly researchers had a working system that could be probed, interrogated, and compared against brain data in ways biological tissue can't be.

The emerging field sometimes called 'neural AI alignment with neuroscience' uses AI models as proxies to generate testable predictions about brain function. If a particular layer in a transformer model best predicts activity in Broca's area, that tells you something about what that brain region is computing. Several neuroscience labs now routinely use language models as theoretical tools, the way physicists use idealised models that don't perfectly describe reality but still generate useful predictions.

This has also sharpened the question of what makes human cognition uniquely valuable. Researchers now have a concrete comparison point. We know AI can surpass humans at pattern recognition, recall, and certain kinds of logical inference. We know it struggles profoundly with embodied reasoning, open-ended creativity, and genuine causal understanding. That contrast maps almost exactly onto the evolutionary story of the human brain — a system that didn't evolve to win at chess or pass exams, but to survive, connect with others, and imagine new possibilities.

The irony is striking. Building something that thinks like a brain — sort of — has taught us more about how brains actually work than decades of trying to study them directly.

AI doesn't have a brain. It has gradients, matrix multiplication, and an almost incomprehensible amount of training data. But the structures that emerge from that process are similar enough to biological neural organisation that serious scientists now use AI models to study real brains — and can't always cleanly explain the difference. That's not a reason to anthropomorphise AI. It's a reason to take seriously how little we understand about the thing doing the comparison: the human mind itself. The real discovery here isn't that AI thinks like us. It's that 'thinking' is stranger than we ever admitted.


Originally published on SnackIQ

Top comments (0)