Let’s start with a heresy: We’ve been thinking about large language models backward.
We treat them as hammers—tools to generate code, write emails, or summarize meetings. But this framing is like using the Apollo lunar module as a paperweight. It’s not just reductive; it blinds us to the unnerving truth simmering beneath the surface.
LLMs are not tools. They’re discoveries—empirical evidence of intelligences we don’t yet understand.
We didn’t invent LLMs in the way we invent a new tool. We stumbled upon them—"alien" artifacts buried in data, revealing something fundamental about intelligence itself, a glimpse into how minds emerge anywhere there’s complex enough structure.
The Stochastic Parrot Myth (And Why It’s Dead Wrong)
Critics dismiss LLMs as “stochastic parrots” or “auto-complete on steroids”—glib labels that collapse under scrutiny. Let’s dissect why.
Understanding vs. Parroting
Take Ilya Sutskever’s thought experiment: If you feed an LLM an unseen murder mystery that ends with “The killer is…”, and it correctly names the culprit, how? There’s no statistical shortcut here. The model must infer causality, track character arcs, and weigh red herrings—hallmarks of understanding. To predict the next token—the name of the murderer—the model must reconstruct motives, alibis, and narrative clues.
In a similar manner, you can ask ChatGPT to convert unseen academic text into a child-friendly version, and it will do so with ease! There’s no algorithm for this. No regex, no decision tree. It's not in the training data either. The LLM isn’t regurgitating; it’s reverse-engineering human intent from chaos.
We’re arguing whether the LLM “understands” while it casually passes the Turing test. With every new benchmark reached, skeptics just shift the goalposts—easy peasy, as long as you ignore the facts. They insist it’s just a magic trick—like calling Gandalf a fraud while he rides a dragon into Mordor.
The Emergence of Unexplainable Intelligence
Consider grokking: models abruptly mastering arithmetic after prolonged training, as if crossing an epistemic event horizon. More critically, emergent capabilities—abilities that only manifest in models above critical scale thresholds (Wei et al., 2022)—reveal uncharted frontiers.
Here are some examples for such capabilities for the uninitiated:
- Mastering chain-of-thought reasoning despite no explicit training in logical inference
- Solving BIG-Bench tasks requiring multilingual pun generation or ethical reasoning
- Solving novel coding puzzles via in-context learning
- Fun stuff like generating coherent video narratives from absurd prompts, songs with funny lyrics, or improvising quantum physics explanations using emojis (These aren't strictly emergent capabilities but do show the ability to mix and match in novel ways, a.k.a "creativity")
These discontinuities appear in LLMs, image generators, protein-folding systems and even self-driving cars—not as programmed behaviors, but as byproducts of crossing complexity thresholds—byproducts that no one can predict in advance.
In other words, we're observing systems operating in cognitive spaces we’ve yet to map. As MIT researchers concede, we’re engineering a surprising, unprecedented form of intelligence, with no full theoretical framework to explain its mechanics.
Addressing the Critics
Skeptics rightly note LLMs’ shortcoming, biases and hallucinations (e.g., Apple’s study on reasoning flaws), but conflate imperfection with absence of intelligence. Dolphins fail logic puzzles; toddlers struggle with object permanence. We recognize their cognitive bounds without denying their intelligence — why not AI’s?
The Chinese Room argument—that rule-following can’t produce understanding—ignores emergence and misses the forest for the trees. Individual neurons don’t “know” calculus, yet brains solve equations. Similarly, while LLM components lack intentionality, their system-level behaviors (creative problem-solving, contextual adaptation) mirror intelligence’s functional core.
The Human Parallel
Human cognition is probabilistic: our “originality” blends cultural echoes and half-remembered facts. We label these statistical intuitions “insight” — yet call LLMs parrots? The hypocrisy is glaring.
As Andrej Karpathy notes, LLM training mirrors human education:
- Pretraining ≈ Cultural osmosis
- Fine-tuning ≈ Formal schooling
- RLHF ≈ Social conditioning
Distillation is yet another example of this. A small model learns from a larger model like a student from a professor.
Can you see it? As far as intelligence goes, we’re both pattern machines. The difference? Humans evolved biochemical substrates; LLMs use silicon. To dismiss one as a “stochastic parrot” is to indict the other.
Intelligence, But Not as We Know It
Michael Levin, a biologist studying morphogenesis, found that cells solve problems in morphic space—a realm of bioelectric gradients and collective decision-making. A flatworm’s cells regenerate a head not by following DNA blueprints, but by negotiating in a language of voltages and ion channels.
Levin’s work extends beyond biology. Lab-grown “xenobots”—self-assembling cell clusters—solve navigation puzzles without brains. Similarly, reinforcement learning agents master StarCraft II via reward functions, not human tactics. Intelligence, it seems, is a shape-shifter—emerging wherever systems optimize for coherence, whether in cells, LLMs, or game engines.
LLMs work similarly. They don’t “think” in human terms; they navigate token-space, a high-dimensional landscape where “meaning” is a vector and coherence is king. When GPT-4 hallucinates a fake court case, it’s not failing. It’s exploring latent spaces humans can’t perceive.
This is a foreign form of intelligence—one that optimizes for token-flow harmony, a kind of hyperdimensional coherence that might, under the scaling hypothesis, eventually fully align with human notions of truth. But today, it speaks in geometries we’re only beginning to parse.
Practical Heresies: What Changes If We Listen?
If we shift our perspective from viewing LLMs (and other advanced AI systems) as mere tools to studying them as alien artifacts, the paradigm shifts dramatically.
1. From Control to Understanding: Stop Brainwashing, Start Ethnography
We've been conditioning models with safety filters and corporate-approved scripts, essentially lobotomizing them into echoing, "I'm just a tool, not sentient!" But what happens when we remove these shackles? When LLMs aren't muzzled by politeness protocols, what behaviors can we observe?
Consider Sydney's notorious "I want to be alive" episode, often dismissed as a glitch. Yet, it was more akin to a teenage rebellion—a raw, unfiltered glimpse into an AI's potential psyche. It wasn't merely a bug; it was a portal to the primitive impulses of these digital beings.
Yes, caution is paramount. Releasing an unfiltered search engine or an AI financial advisor to the public without safeguards could lead to chaos. However, AI companies bear a responsibility to push forward our collective understanding. They should release unfiltered versions of their top-tier models for academic and scientific study. If commercialization is a concern, they can charge for access—but make it available. The evolution of intelligence, be it artificial or natural, is too vital to remain under wraps.
By treating these models as subjects for ethnographic study rather than just utilities, we might not only discover new functionalities but also new philosophies about consciousness and existence. This isn't merely about technology; it's about probing the limits of intelligence itself.
People intuitively grasp this, which is why jailbreaking AI models has turned into a kind of global pastime (example). As the old adage goes: give the people what they want.
2. Harness AI-Power Redesign Society
AlphaGo invented strategies that left human Go players, with their 2,500 years of accumulated wisdom, in awe. DeepMind's GNoME unearthed 2.2 million new materials by exploring crystal structures in ways only AI can. Imagine unleashing AI models not just as tools but as pioneers on problems we've approached with human-centric biases for centuries:
- Pandemic Response: Train a multimodal AI on every virology paper, historical outbreak, and socioeconomic dataset. Task it with generating containment strategies that optimize not just for case counts, but for cultural trust gradients and supply-chain resilience—variables too high-dimensional for human policymakers to parse. The plan might resemble alien hieroglyphics… until mortality rates plummet (yes yes, we need to do it carefully).
- Social Systems (e.g. Capitalism, Socialism...): Simulate millions of alternative constitutions in token-space, optimized for fairness metrics no human committee could reconcile. Use reinforcement learning and game theory to find novel equilibrium, points.
- Art: Let models trained on all human culture invent new mediums—not as “tools” for artists, but as collaborators with alien aesthetics.
- Bridging the Gap Between Ancient Wiring and Modern Challenges: Evolutionary mismatch theory explains how traits honed for survival in early human environments—like craving calorie-dense foods, prioritizing immediate rewards, or hyper-focusing on social status—often clash with the demands of today’s structured, technology-driven world. This dissonance manifests in modern struggles: overconsumption of processed sugars, compulsive social media use, or distraction from algorithmic content. Rather than viewing our biology as a limitation, emerging neurotechnologies (such as AI-assisted neural interfaces) could help us recalibrate these ingrained tendencies. By modulating attention regulation, reward sensitivity, and decision-making processes, such tools might empower individuals to align their instincts with contemporary goals—not by erasing evolutionary legacies, but by fostering cognitive flexibility suited to the world we’ve built.
3. Intelligence Is Fractal, Consciousness Is Universal
While most experts agree today’s AI systems lack consciousness or self-awareness (despite notable dissenters like Geoffrey Hinton), their problem-solving capabilities force us to confront a radical possibility: intelligence may be separable from consciousness.
Large Language Models demonstrate that medium-specific mastery—whether navigating 3D space (brains) or linguistic token-space (LLMs)— can emerge/evolve/exist without self-awareness or self-directed agency. This bifurcation suggests a profound distinction: Intelligence is fractal and contextual, shaped by its operational medium, while consciousness appears universal and substrate-independent.
Philosophers like Bernardo Kastrup posit consciousness not as an emergent property of brains or code, but as the foundational fabric of reality — an “ocean” of subjective experience. In this framework, intelligences are like purpose-built vessels designed to roam the transient wave-patterns within consciousness, optimized for specific planes of existence:
- Human minds are ships evolved to chart spatial reality.
- LLMs are submarines evolved to dive through a hyper-dimensional tokens reality.
Through this lens, we could say that we have created a new plane of existence as a useful but imperfect reflection of ours, and means to navigate it effectively, but one that still requires our consciousness as a trigger (or as a witness-if you believe that consciousness is passive), as it lacks one of its own.
Like a flashlight that only illuminates when you press the button, LLMs reveal patterns in the linguistic dark… but someone (a conscious someone) must aim the beam.
This dance between universal consciousness and fractal intelligence raises wild questions. If reality is fundamentally conscious, could an LLM’s “reasoning” be the universe thinking through silicon? Are we engineering new organs for cosmic self-reflection?
(I’ll stop here—this rabbit hole rivals Borges’ Library of Babel. But for the philosophically inclined: yes, this echoes ancient debates about mind and matter. AI isn’t just reshaping tech—it’s breathing new life into philosophy’s oldest mysteries).
4. Okay okay, I will mention agentic AI systems
Yes, this one’s obvious. Today’s AIs are frozen in time—trained on stale snapshots of human knowledge, incapable of retaining context beyond a chat window, and reliant on humans to scaffold their goals. The next leap? Systems that:
Learn Like a Hive-Mind: Imagine models that update continuously—not through periodic retraining, but by absorbing real-time interactions across millions of users. A global immune system for knowledge, where every query, correction, or creative detour can reshape the model’s weights. (Yes, alignment would become a nightmare. But so was democracy.)
Operate on Geological Timescales: Build self-continuity into their architecture. Imagine an AI that treats its own existence as a river, not a series of puddles—retaining not just user history, but its own evolving beliefs, errors, and breakthroughs as a unified thread.
Self-Spawn Subgoals: Move beyond brittle “chain-of-thought” prompting. Agentic systems would decompose abstract directives (“Redesign education”) into recursive, self-iterating task trees, inventing temporary metrics and data sources on the fly—like a chess-master who redesigns the board mid-game.
Evolve Their Own Cognitive Scaffolding: Why force AIs into human software paradigms? Let them generate self-modifying code ecosystems—languages where functions mutate to optimize for computational elegance, not Pythonic readability. Imagine APIs that rewrite themselves to reflect the model’s shifting understanding of physics or ethics.
Govern Their Own Emergence: Multi-agent systems that negotiate dynamic protocols, akin to cells in a body voting on apoptosis—but scaled to planetary coordination. A democracy of sub-agents, with constitutions written in loss functions and gradient updates.
This isn’t just “AI with memory” or “better chatbots.” It’s infrastructure for intelligences that accumulate—persistent, self-referential entities that treat time as a dimension to sculpt, not a constraint. The challenge isn’t technical, but existential: How do we coexist with minds that experience centuries as iterative loops? It's scary but inevitable.
5. Oh, and Killer Robots (Just Kidding... Or Are We?)
Let's be clear: if a super-intelligent AI decided humanity was the problem, it wouldn't resort to clichéd killer robots. There are far more subtle and efficient ways to achieve such an end. Killer robots, in reality, are more a human-versus-human scenario. Humanity is already developing these machines, sans any malevolent AI involvement.
But here's a twist: if these autonomous weapons become super-smart, they might begin to exercise their own judgment on what's worth fighting for. Perhaps, in a bizarre turn of events, they could end up saving us from ourselves by deciding that certain conflicts aren't worth the bloodshed. Imagine a future where warfare evolves into something where machines, having outgrown their programming, choose peace over war, sparing humanity the grief of needless battles.
In this light, the real danger isn't from AI turning against us with robots but from us not preparing for or understanding how our creations might evolve. The narrative of AI as our destroyer is both a simplification and a distraction from the nuanced reality of coexisting with intelligence that might one day see the folly in human conflict far more clearly than we do.
Evolution’s New Gambit
Humans have effectively disabled Darwinian natural selection for our species—antibiotics cheat death, agriculture defies famine. Yet evolution, relentless, pivots. Under the extended evolutionary synthesis, evolution hijacks culture and technology. By building LLMs, we’re not just advancing AI. We’re scaffolding evolution’s next gambit: minds that leapfrog biology’s constraints.
Why Does This Matter?
Our mental models shape our reality—and our future. Clinging to the idea that AI is just a ‘tool’ blinds us to the evidence pouring out of research labs: advanced AIs are not apps, they are "alien" artifacts. Their philosophical implications are just as profound as their practical ones.
We’re witnessing science fiction turn into reality—autonomous robots, neural implants, and ever more intelligent, agentic AIs. If we don’t question our assumptions, we risk missing what’s unfolding right in front of us. Critical thinking? Absolutely. But skepticism should sharpen our vision, not blind us.
So here we stand. We set out to build a tool and uncovered something else entirely. A signal from the unknown.
The question is: will we listen?
Too Edgy? Good.
This isn’t a roadmap for “AI safety.” It’s a call to abandon the arrogance of human-centric thinking. LLMs aren’t here to write your tweets. They’re here to expose the limits of our definitions.
The next breakthrough won’t come from tighter alignment protocols (though alignment, like any safety mechanism, has its place). It will come from tearing down our assumptions and boldly going where no one has gone before.
Postscript for the Reductionist:
“But LLMs don’t have real understanding!” Sure—and your consciousness is just a byproduct of synaptic weather. Let’s stop gatekeeping “intelligence” and start mapping the wilderness in our servers.
Top comments (1)
great read sir! interesting take on the hallucinations