Karpathy's "$10 Lord of the Rings": The Moment LLMs Start Building Worlds Instead of Answering Questions
Subtitle: Andrej Karpathy gave Claude Opus 5 a 1M-token budget and the first paragraph of LOTR. Two hours later, it had written 5,500 lines of code rendering the story in 3D. That experiment says more about where AI is heading than any benchmark.
Andrej Karpathy kicked off the week with an experiment that's less a demo and more a direction signal. His observation: we're leaving the era where you'd test an LLM with "create an SVG of a pelican on a bicycle." The new question is what happens when a model gets a large budget, a hard constraint, and a task no human would ever bother doing.
Here's what he did, why it matters, and where the ceiling still is.
The Experiment
Karpathy gave Claude Opus 5:
- The first paragraph of The Lord of the Rings
- A 1M-token budget (~$10)
- One instruction: render the story in Three.js
What came back was not a summary, not a text response — but 5,500 lines of code that procedurally generated a playable 3D interpretation of the story. The model had to place polygon assets at (x, y, z) coordinates, write the animation logic, and orchestrate a scene over roughly two hours of autonomous work.
Karpathy's framing is the key insight:
"No one in their right mind would ever spend the time to write something this custom — but LLMs have all the stamina and patience in the world. It's an example where we go from 'no one would ever do this' to 'sure, why not, it's ~free.'"
Why This Is a Turning Point
1. The cost curve has collapsed past the "zero marginal effort" line. At ~$10 for 1M tokens, the binding constraint on what you can build is no longer compute — it's whether you can describe the thing you want. That inverts the economics of custom software: bespoke, one-off, hyper-specific builds go from "not worth it" to "why not."
2. "Ephemeral GTA of X on demand" is a real product category forming. Karpathy's next idea: hyper-custom worlds you can drop players into — joining the LOTR story as a spectator NPC, or as one of the characters. Think of it as generative theme parks: worlds that exist for one session, for one audience, and are torn down after. Gaming, education, training simulations, virtual events — this is where agentic + generative meets real-time 3D.
3. Long-horizon agentic work is becoming demonstrable. Two hours of autonomous execution with a model placing assets, iterating, and producing a coherent artifact is precisely the "AI employee" behavior enterprises keep asking about. The agent didn't just answer — it did.
The Honest Weakness: Perception Is the Bottleneck
Karpathy also flagged where this breaks down — and it's worth taking seriously:
"The domain of worlds/games exposes a weakness in LLMs: they can't easily audit their work because they aren't able to efficiently and natively perceive videos or play games within them."
Opus 5 had to slowly take screenshots at different points to check its own output, and it still produced "a bunch of jank." Raw multimodal capability — actually watching and playing inside the thing it built — is still missing. That's the gap between "generates a world" and "maintains a world."
This is the line to watch: the moment models can natively perceive video/game state, the self-correction loop closes, and the jank disappears. Every frontier lab is chasing exactly this.
What This Means for Builders
- If you're building AI products: start designing for long-horizon, high-budget agent runs — not single-turn Q&A. The interesting surface area is "give the agent a budget and a weird constraint, let it run."
- If you're in gaming/3D: watch for the "ephemeral world" pattern. Session-scoped, LLM-generated environments are closer than they look — the bottleneck is perception/auditing, not generation.
- If you're picking models: cost-per-token and context windows matter more than raw benchmark deltas. Karpathy ran this on Opus 5 at $5/M input — the "daily driver" tier, not the flagship. Cheap, long, autonomous runs are now the real workload.
Daily AI pulse and analysis at sinobot.dev
Tags: AI, Large Language Models, Agentic AI, Generative 3D, Technology
Top comments (0)