DEV Community

Cover image for I'm trying to teach a machine to play a game it has never seen
Alan Scott Encinas
Alan Scott Encinas

Posted on • Originally published at alanscottencinas.com

I'm trying to teach a machine to play a game it has never seen

Builder Journal · ARC Prize 2026

Hand a seven-year-old a video game they have never seen. No manual, no tutorial. Within a few minutes they have worked out which button does what, what kills them, and what they are supposed to be chasing. Nobody explained the rules. They learned them by poking at the world and watching what happened.

That casual, almost invisible act is one of the hardest unsolved problems in artificial intelligence. I'm competing to build a machine that can do it.

This is a new thread in my builder's log, and a second competition running alongside the hyperspectral one I have been writing about. Completely different game. Same deal: you get it as it happens, from behind a delay, with the parts that are still my edge kept dark.

What this actually is

The competition is the ARC Prize 2026, built on a benchmark called ARC-AGI. To understand why it matters, you have to know what it was built to embarrass.

ARC stands for the Abstraction and Reasoning Corpus, and it came from François Chollet, the researcher who also built Keras, one of the tools that put deep learning in everyone's hands. In 2019 he wrote a paper called On the Measure of Intelligence with an argument that has aged into a quiet prophecy. Most of what we call AI progress, he said, is not intelligence at all. It is skill. A model that has seen ten million chess games is not smart at chess, it is well-rehearsed. Real intelligence is not how much you know. It is how efficiently you handle something you were never prepared for.

So he designed a test that you cannot rehearse for. ARC is a set of little puzzles where every single one follows a rule you have never seen, and you get only a handful of examples before you have to generalize. You cannot memorize your way through it, because nothing repeats. The only thing that helps is the raw ability to look at a few examples of a brand new pattern and grasp what is going on.

Why it's the field's uncomfortable mirror

Here is the part that keeps ARC in the headlines. Humans, including children, solve these puzzles comfortably. The most powerful AI systems on earth, for years, scored close to nothing.

That gap is the whole point. The same models that pass the bar exam, write working code, and draft a passable essay would sit down in front of ARC and fall apart, because passing the bar exam is a memory-and-pattern feat and ARC is not. It is the benchmark that refuses to be impressed by scale. Every time the models get bigger and quietly start to crack one version, the benchmark gets sharpened into a harder one, and the gap reopens. ARC is the mirror the field keeps walking past, the standing evidence that "knows almost everything" and "can actually think" are not the same sentence.

The ARC Prize exists to do something about that. It is an open competition, with real money behind it, deliberately built to drag the field's attention away from "make the model bigger" and toward "make the model adapt." It rewards efficiency and generalization, the things scale alone does not buy.

The interactive leap

Earlier versions of ARC were static. Look at a few examples, produce one answer, done. ARC-AGI-3, the one I'm competing in, makes it interactive, and that changes everything.

Now the AI is not staring at a snapshot. It is dropped inside a small game it has never encountered and has to win. It is not told the goal, the controls, or the rules. It has to take an action, watch what the world does in response, form a guess, test the guess, and slowly build a working theory of a place nobody described to it. It is the difference between reasoning about a photograph and figuring out a machine by pressing its buttons. That is far closer to how a real mind learns, by doing, by consequence, and it is far harder for a computer.

And you still cannot study for it. The games it gets graded on are hidden. There is no dataset to scrape, no test to pretrain on. The only thing that helps is learning on the fly, in a world you have never seen, from a standing start.

Why anyone should care

Strip away the contest and here is the stakes. Today's AI is dazzling inside the conditions it was trained on and brittle the moment reality steps outside them. It is a brilliant student who has memorized every past exam and panics at a question phrased a new way.

The skill ARC measures, getting your footing in a situation nobody prepared you for, is the exact thing standing between the narrow tools we have now and machines that can handle the truly unfamiliar. A robot in a kitchen it has never seen. A system meeting a problem that was in no textbook. A scientific assistant facing a result nobody predicted. All of that lives or dies on the ability to learn an unfamiliar world quickly, which is precisely what a frontier model still cannot reliably do and a child does before breakfast. That is why a benchmark about cartoon grid-games is, underneath, one of the most serious questions in the field.

The bet I'm making

Here is where I am stubborn, and where a lot of the field would disagree. The fashionable move is to point an enormous language model at the problem and let it sort everything out. I am doing close to the opposite. I am hand-building the mind, the perception, the exploration, the decision-making, all of it as my own code that I can open up and explain, and I am letting a language model do exactly one job: the theory-building part, the place Chollet's own breakdown says most of the difficulty actually lives.

The reason is not nostalgia. I want to be able to explain and defend every single thing my agent does, and "the big model decided" is not an explanation. Learning first, score second. If a shortcut would buy me a better rank at the cost of understanding why it works, it is not a shortcut I am willing to take.

Where I'm standing right now

My best agent wins about a quarter of the hidden games. The number on the board is 0.25.

That is further than it sounds and nowhere near where I want to be. Most of what I have learned here came not from the runs that worked but from the ones that embarrassed me, and this competition has handed me some of the most instructive failures I have had on any project. Those are the entries coming next: the times I was certain I was improving the agent, and the scoreboard told me, flatly, that I had made it worse.

There is a particular kind of humbling that only happens when a number you cannot argue with disagrees with a belief you were sure of. I have a few of those to tell.


More in this series

This is part of an ongoing builder's log written from inside live competitions. You're reading where I was, not where I am.

Top comments (0)