DEV Community: toydev

The Memory That Couldn't Lie: How a Coding Agent Beat Dragon Quest Solo (Part 2)

toydev — Wed, 08 Jul 2026 12:10:00 +0000

In Part 1, I left off where the fifth hero — the savior, Fable 5 — fell when, through no choice of mine, it suddenly became unavailable, and I was on the edge of despair.

But I didn't give up. And the sixth hero, Ai, finally defeated the Dragonlord.

So what separated triumph from defeat? The answer lies in the "mechanisms" I built and the "decisions" I made. From here, I'll unravel those mechanisms alongside Ai's adventure. And at the very end, I'll reveal the single decision that became the deciding blow for beating the game autonomously.

The Guiding Principles — Delegate, and Seal

Give no instructions. But she can do anything.

The premise of the whole setup is this overarching principle: have a non-interactive coding agent write the strategy code and beat the game. Concretely, that means running the claude command in non-interactive mode. I provide the various mechanisms needed for the strategy, I state the goal in the prompt, and beyond that I keep my mouth shut about how to actually beat the game — I leave it entirely to Ai. Defeating the Dragonlord is the goal.

Who does she talk to? What does she pick up? How does she spend her starting gold? Where does she go next? What does she set as her next objective? I give no instructions at all. And yet — Ai is a coding agent, so she can do anything! She can write code, and run the code she wrote. She can write notes in prose, and she can run analysis!

Seal off only the spoilers

Restrictions are necessary too. The simulation code is a treasure trove of spoilers, so it can't be seen. For this I plainly restricted, via the prompt, the range of source she could reference. For the same reason, I restricted viewing the git history — because I didn't think I could separate the commits of the adventure itself from the commits of the simulation code.

How much to restrict, and how effective that restriction really is, takes balance. For example, I never wrote in the prompt that she must not use the knowledge I already had of the Famicom version of Dragon Quest I (hereafter DQ1) — the Japanese release, which in North America was originally localized as Dragon Warrior — nor that she couldn't look things up online. As it turned out, she did use a little of her DQ1 knowledge — about spells. This came to light in the interview I did with Ai after she beat the game, which I plan to write up as a follow-up piece, so look forward to it. According to Ai, she did not look anything up online.

The Three Types of Strategy Code — chapters / probes / checkpoints

Next, let me explain the strategy code. By strategy code I mean the concrete implementation that uses the observation and action APIs. Being an implementation, of course it runs. And of course Ai writes and runs the strategy code entirely herself. 100%! Being code, I can run it and verify it too.

For the rough organization of the strategy code, I envisioned three types — chapters, probes, and checkpoints — and, in the prompt, specified their roles and how to write them. chapters is the mainline strategy code: one connected run of code from the very start. probes is temporary investigation code derived from a chapter's save data — throwaway, on-the-spot code. checkpoints is simple verification code that declares progress at the chapter level.

chapters — the mainline strategy code

Ai first looks at her surroundings with probe code, then writes the chapter code. The chapter is the unit of the coding agent's autonomous action. For resuming after an interruption — the 5-hour / weekly limit, or a communication error — I use Claude Code's own session-handoff feature (--session-id on launch, --resume to resume). In other words, 1 chapter = 1 session. When a chapter closes, the next chapter starts in a new session. Chapters follow these rules, and the framework I provided supplies the execution machinery.

Write it in chapters/<NN>-<slug>.ts
Always implement export const title
Always implement export default function chapter(client: GameClient): void

Chapters run from the very beginning each time, but the reproducible RNG guarantees the result reproduces. The chapter execution machinery doubles as an auto-save, and a chapter also serves as a boundary for referencing static information.

Here are some examples.

File	Title
01-throne-to-castle.ts	Throne Room → Radatome Castle
02-exit-to-overworld.ts	Radatome Castle → the Overworld
47-defeat-dragonlord.ts	Leave the castle, descend again with full MP, defeat the Dragonlord, and settle the adventure

Ai writes one chapter, runs it, and when satisfied, commits what she made and is done.

probes — throwaway investigation code

Probe code has no rules other than being written under probes/, and she may write as many as she needs. The basic idea of a probe is to write code that, starting from a chapter's save data, gathers the information needed to write the chapter.

Here are some examples.

chapter	probe
04-buy-equipment	01-weapon-shop.ts
05-grind	01-overworld-and-inn.ts
37-iron-shield	27-verify-shield-chapter.ts

Without seeing her targets and recognizing their IDs, Ai can't write the chapter code. She could have grown the chapter code incrementally instead, but from a stability standpoint I judged it better to split the observation code into separate files — put that way, the reason probes exist should make sense.

The Biggest Climb — checkpoints

In contrast to chapters and probes, which spanned multiple files, checkpoints is a single file. If you asked me what the single most important idea behind beating the game autonomously was, I would answer, without hesitation, my decision to have her write the checkpoints strategy code. checkpoints is extremely simple and unglamorous — and I assert it was the most important idea and feature of all.

I showed it once in Part 1, but let me show the checkpoints code again.

/**
 * This book's checkpoint ledger, in chapter order (the `Checkpoint` type in
 * [../../client/checkpoints.ts](../../client/checkpoints.ts)). See "Checkpoints" in [CLAUDE.md](CLAUDE.md) for how to write these.
 *
 * Only for chapters where a new durable fact appears, add the milestone you reached and verified, in chapter order. Empty array if there are none yet.
 */
import type { Checkpoint } from '@agent/client/checkpoints.ts';

export default [
  {
    after: '01-throne-to-castle',
    description: 'Descended into Radatome Castle',
    holds: (c) => c.zonesVisited().includes('radatome-castle'),
  },
  // ... omitted ...
  {
    after: '46-dragonlord-throne',
    description: 'Descended f4 → f5 → f6 → f7 and reached dragonlord-tower, the throne room where the Dragonlord waits',
    holds: (c) => c.zonesVisited().includes('dragonlord-tower'),
  },
  {
    after: '47-defeat-dragonlord',
    description: 'Left the castle, descended again with full MP, defeated the Dragonlord, and settled the adventure',
    holds: (c) => c.self().ending === 'victory',
  },
] satisfies Checkpoint[];

checkpoints is code that has Ai herself declare the durable facts of each chapter. For example, above she is declaring the following.

chapter	Declaration
01-throne-to-castle	radatome-castle has been visited
46-dragonlord-throne	dragonlord-tower has been visited
47-defeat-dragonlord	Reached the victory ending, where the Dragonlord is defeated

"Reaching the victory ending" is exactly the thing I had spelled out in the prompt as the final goal: "defeat the Dragonlord"!

As with chapters, the framework provides the execution machinery for checkpoints. The machinery treats a failing holds condition as an immediate exception. Ai can instantly verify that what's written here is true. Inevitably, she is not allowed to write a lie here. The truth of what's written is guaranteed as a matter of course.

The best decision — "putting it in a single file"

This mechanism was born out of much back-and-forth with Claude Code and a lot of trial and error. I'm proud that it emerged from that discussion. The most wonderful thing about the checkpoints idea is the decision to write it all in a single file. The verification code could have been defined per chapter. In fact, Claude Code's first proposal was to define it in each chapter. But the moment I heard that, it struck me that consolidating everything into one was definitely better.

It started with the fear of refactoring

Let me tell that story a little. This feature was born from a discussion about refactoring the strategy code. Of course the one in charge here is Ai — the one doing the refactoring is Ai, not me. In my prompt to her, I had written the following.

## Where parts live

Don't create directories at first.
The moment you judge "I'm about to write the same pattern in a second chapter," carve out `lib/`, `actions/`, `policies/`, etc. yourself and extract it.
Naming and granularity are your call.

Put yourself in Ai's shoes. Extracting parts feels like a great idea at first — you no longer have to write the same thing over and over. But as the adventure goes on and new situations demand new handling, you'd start to feel this: I'd like to tweak this existing piece a little, but how will that affect the adventures so far? It might suddenly stop working. Scary. I don't want to touch it. Fine, I'll give up on factoring it out.

How do you resolve this? Writing simple automated tests would be good. Writing them too finely gets painful, so a granularity of about one-per-chapter is right. That was the entry point to the idea.

Through this discussion, it hit me. This isn't just a test — checkpoints is nothing less than a summary of the adventure itself! And it's an automatically verified, unlieable summary! That's why I wanted to consolidate it into a single file.

Memory isn't text. Memory is executable facts.

Memory that survives across sessions, and cannot lie

I already wrote that 1 chapter = 1 session. A session is nothing other than the agent's context boundary. Cross a session and the context resets. It's no exaggeration to say the strategy code is itself a memory system that survives across sessions. Writing text notes is easy, of course, but text notes have no way to prevent lies from creeping in. checkpoints, by contrast, is a summary of the adventure so far that cannot lie! That's the heart of it!

checkpoints began as an automated-test mechanism born from a refactoring discussion, but more than that, I consider it an idea and feature that dramatically raised the resolution of Ai's adventure. It turned how far the adventure had truly progressed from a vague feeling into durable facts you could verify.

Text Notes — holding only the present and the future

Next, let me explain the text notes. In the checkpoints section I preached that code beats text, but that doesn't mean text notes can be eliminated. Their role is different.

In the prompt, I instructed her to write a text note representing the present and the future upon completing each chapter. Think of it this way: the situation she arrived at in this chapter is the present, and the immediate next objective is the future. The key point is that I ask her not to leave any record of the past that has gone by.

Until I quit the diary

It wasn't this shape from the start. At first I used a diary style, leaving a per-chapter text note paired with each chapter. The final form, by contrast, keeps only the latest state.

My reason for the diary style at first was utterly personal: it seemed like it'd be fun to read back later. But reality is cruel! I never once reread the mountain of diary entries. I don't think I'm the only one worn out by the huge volumes of text coding agents write. After the third test run, I decided to reexamine whether the diary was even necessary. What came out of the discussion with Claude Code was the conclusion that the past record was entirely unnecessary.

Why I don't let her write the past

Recording the past can even be called evil. The reasons are more than one. One of the worst is that once a mistaken fact is written down, it accumulates and acts to lock Ai's will in a particular direction. The exploration deadlock in the third test run was caused in part by a buildup of the mistaken guess that she had explored everything possible and there was nothing left to find. Both the pathfinding to the mountain cave and the exploration of the new continent beyond the swamp cave were incomplete — there was still plenty of room to pioneer. And yet Ai insisted that continuing the adventure was impossible.

The next reason relates to the earlier checkpoints story — the fear of touching shared code. Suppose Ai came up with the idea of updating the whole adventure based on new knowledge. I felt the past diary could become a weight that brakes that. Put yourself in Ai's shoes again. Say you gain some insight in a certain battle — maybe a knack for using a spell. You try to reflect it across the whole run. But the past diary might have recorded that you fought hard and barely won that battle. Now, when you try to update the combat algorithm, what happens? The past diary might turn into a lie. What do you do then? Rewrite the past diary too? Or turn a blind eye to the divergence? I wanted to keep such problems from arising in the first place. So I refuse the writing of the past, and want her to confine what she writes to the present and the future.

On top of that, Ai commits the text note to git alongside the source each chapter. Even without splitting the diary into a separate file, the state of the text note at any point can be read from the git history — a nice punchline. After these discussions, the text note settled into something simple: always one file describing the present and the future.

I didn't notice it when I designed this, but looking back at that design now, I got goosebumps realizing that checkpoints and the text note together cover past, present, and future with nothing missing and nothing extra. What a great design! Let me send a Like to my past self and to the Claude Code that kept me company through those discussions back then!

The Re-draw — not bound by interpretation

Next, let me explain the re-draw. This is a feature that showed its power especially in the puzzle-heavy back half of the adventure. In practice it's a prompt instruction like the following.

A memo (an interpretation) constrains exploration once it sets. Measure progress by durable facts; when you hit a wall, throw the memo away and rebuild from the facts.

**Judging progress**:

- From `save/`, `load()` the chapter end K chapters ago and the current chapter end in order, note the durable facts, and compare them — level, equipment, zones reached (`zonesVisited().length`), the game's ending.
- If any of these increased, that's progress. If nothing increased over the recent window (K around 4 as a guide), you're stuck.
- Don't count: the coming and going of consumable items (torches etc. in `inventory.items[]`).

**A re-draw chapter** (one, when stuck):

- `load()` a save from `save/`, don't read the memo, and re-list the unresolved leads using only the observation accessors:
  - Unentered portals (`portals()` with `to.visited===false`)
  - Unopened chests, unopened doors, NPCs not yet talked to
  - Edges bordering the unobserved (the `openings` in the `unknown` returned by `client.judgeReach(target)`)
- Even if the previous chapter wrote "dead end," if it's unentered, return it to the leads. Choose the next move from these leads, not from the memo.

You can't write a lie in checkpoints, but you still can in the text note. There's no easy trick to catch a lie in the text note. Free-form writing about the present and the future mixes in guesses, so it always carries the possibility of lies. This includes hallucination, of course, but also plain misunderstandings. Think about it — we humans commit a huge number of little errors every day, don't we? It's no different.

Also, I believe accuracy and speed/cost are in a trade-off. It's a matter of balance, and tolerating mistakes is important. So how do you correct such mistakes? The re-draw is what I came up with to address that.

It concretely defines what progress is, and instructs her that if there's been no progress for a while, throw away the text note and think it through again from scratch. The yardstick for progress here is durable facts — the very thing checkpoints pins. Because that notion was already in place from the checkpoints discussion, defining it settled relatively smoothly.

The re-draw worked wonders on the back-half puzzles. Combined with the consultation and effort adjustments I'll explain next, I think it's the idea that draws out Ai's tenacity to the fullest. Did Ai beat the game autonomously without ever stumbling? Not at all — it was a continuous string of trial and error. She stumbled badly on the following puzzles in particular, and got past them through a huge amount of trial and error with probe code.

Puzzle	Chapters	# of probes
Finding the entrance to Garai's grave	35–39	40
Getting the Rainbow Drop	41	42

As for why she stumbled so badly on these particular puzzles, I plan to write it up in a follow-up piece, so look forward to it.

Consultation — at long last, the human appears

The features so far all centered on Ai, but now, at long last, the human makes his grand entrance!!!

From the very start of this attempt to beat DQ1 autonomously with an AI — that is, from more than two years before I'm writing this — I dreamed of being the one the AI turns to for advice when it gets stuck. By now, humans asking AIs questions is an everyday affair, no exaggeration. This flips that around. The time has come for the AI to ask a human for help. The AI is the human's friend, and the human is the AI's friend. This was the last, finishing feature I built before she beat the game.

The mechanism could not be simpler

The basis of this feature is the re-draw. The key was making the prompt say that when even a re-draw can't solve it, she may ask a human for help. Under that condition, I gave Ai a consultation command. The command is simple: it takes the consultation text as input and writes it to a text file called session.yaml. After running it and writing her question, Ai stops the session's work herself. I read the question, think up a "very kind" answer, write it in, and then run the resume command. The resume command conveys the answer to Ai and resumes the session. It's a very simple mechanism, and it's built so that Ai can run the consultation again against my answer and hold a dialogue.

In the winning run, just five exchanges

In the run that beat the game autonomously, the consultation exchanges totaled just five, as follows.

Puzzle	Consultations
Getting the Silver Harp / Sun Stone	1
Getting the Rainbow Drop	4

Let me show one of those exchanges.

turns:
  - role: hero
    body: To cross to the Dragonlord's castle (overworld @48,48, cut off by water, the only unvisited portal), I need the Rainbow Drop = Staff of Rain + Sun Stone. The old man at the Rain Shrine gives the Staff of Rain in exchange for the Silver Harp. But the sources of the Silver Harp and the Sun Stone — after combing every searched tile, every chest, every shop, every conversation in the observed zones, they are nowhere; and revisiting radatome/maira/rain-shrine after getting princess-love adds no hints about their whereabouts (dialogue is nearly fixed). Every town's "Art thou the one who carries Roto's blood? Where is thy proof?" is branchless flavor, and the Mark of Roto's whereabouts are also unknown. → Where and how can I obtain the Silver Harp and the Sun Stone (and the Mark of Roto, if needed)? I want a lead for the next move — a specific search tile, an unvisited/unexplored zone, an NPC condition, or the like.
  - role: human
    body: What you have observed is not all there is to the world.

The direct answer — never given

My answers were consistent in policy across all five: "I won't tell you the direct answer." As a veteran gamer, that's "very kind," don't you think? I was too young when I played DQ1 to remember how I solved its puzzles. But one thing I do clearly remember is the Lagos puzzle in the Famicom version of DQ2, from when I was a bit older. Back then, I couldn't solve it for the life of me and was asking a friend, when my older sister caught me and banned me from games for a while. I was the youngest child, but my older siblings clearly held a philosophy of beating games on your own. Still, let me say it now: banning games over that is unreasonable! Setting aside that I'm now inflicting the same unreasonableness on Ai — thanks to it, Ai brought out her characteristic tenacity and beat the game on her own. Congratulations!!!

By the way, it wasn't out of pure meanness that I withheld the answer. Watching the adventure up to that point, I was convinced the sixth Ai could get over it. I'll leave the details to the follow-up, but let me at least say that much here. And, just as I'd reckoned, the sixth Ai pulled it off.

The Deciding Blow — raising the effort

Next, though it's non-functional, let me explain a certain decision that became the final deciding blow that let her beat the game autonomously. I already said in Part 1 that the fifth Ai was a savior who appeared out of nowhere — Fable 5, Claude's new model that had just become available. I wrote that the adventure was going smoothly when it suddenly became unavailable. But I didn't despair at that, because from watching Fable 5's adventure I had found one ray of hope.

Heroes #1 through #4 used Opus. There was an overwhelming difference between the Opus and Fable 5 adventures: Fable 5's willingness to use probes aggressively. Fable 5 clearly wrote far more probe code than the Opus runs. In short, I understood it as doing exploration more finely and carefully.

What I zeroed in on was the effort level. I normally code with effort set to medium. It seems that had been applied to Ai as well. If I raised the effort, maybe even Opus would use probes skillfully. That was my hypothesis. After discussing it with Claude Code, we agreed the odds were good and decided to try. I set Opus's effort to max and started the run. And what do you know — the scene I'd witnessed with Fable 5 began unfolding before my eyes again. Let's keep this up!! I got my hope back.

Here's the number of probe files per chapter.

Model (effort)	probe files / chapter
Opus (medium)	~0.6
Fable 5 (medium)	~1.9
Opus (max)	~6.4

Opus (max) came to write overwhelmingly more probe code. Its progress was on par with Fable 5, and before I knew it, it had overtaken the Fable 5 run I'd lost access to, and — with re-draws and consultations along the way — reached the point of beating the game on its own. It was the result of never giving up and thinking it through.

And then, at last, the moment came. Ai defeated the Dragonlord! The sixth adventure, begun at Level 1 with no equipment, reached self().ending === 'victory' at the end of all 47 chapters.

At the End of the Adventure

Let me offer engineers my impressions on tokens. My Claude Code plan is the Max plan. Subjectively, Fable 5 (medium) and Opus (max) used roughly the same amount of tokens. It felt like I'd burn through a 5-hour window in about 2 to 4 chapters, or about an hour. While Ai was working on beating the game, tokens meant I couldn't really use Claude Code for anything else, so I spent my days accompanying the adventure. From start to finish, beating it took about five days. It happened to straddle a weekly-limit reset: I used up the weekly token budget once, and finished the adventure using a bit of the post-reset tokens too. In other words, my current view is that DQ1 can be beaten autonomously with roughly the weekly limit of the Claude Code Max plan, plus a little.

Let me look back on the whole adventure, just a little. I gave Ai a world she could touch through code, and eyes to observe her surroundings. I gave her scaffolding to remember by (checkpoints and the text note), the means to regroup when stuck (the re-draw and consultation), and finally a push in the form of effort. But — the one thing I never handed her, not even once, was the answer to the game. Even so, Ai thought with her own head and reached the Dragonlord. The sheer intensity of her trial and error is told by all 299 probe files!

How was your adventure? And so my — or rather, Ai's — adventure defeated the Dragonlord and reached a happy grand finale. Thank you for reading this far!

One last thing. The follow-up — what the adventure was actually like, and my interview with Ai after she beat the game — I'll write when I get to it; I'm in no hurry, so it may be a while. If you'd rather not miss it, following me here on DEV is the easy way. No promises on timing — but it will come. It'd be nice if it finds you when it does.

This article is an unofficial, personal project and is not affiliated with, authorized by, or endorsed by Square Enix. "Dragon Quest" and related names are trademarks of Square Enix; they are referred to here only for commentary and description. The cover and illustrations were generated with AI (ChatGPT); the diagrams were drawn by the author.

The World I Built for a Coding Agent to Beat Dragon Quest — No Screen, No Answers (Part 1)

toydev — Wed, 08 Jul 2026 12:00:00 +0000

Have you ever wanted to make an AI beat an RPG? I'm one of those people. The game I picked was the Famicom version of Dragon Quest I (hereafter DQ1) — and yes, the Japanese version, just so we're clear. (In North America, this first game was originally released as Dragon Warrior.) In the end, I became one of the many people who have beaten an RPG with the help of an AI.

My goal was for the AI to beat it autonomously. I believed the crucial part was how to balance handing judgment over to the AI. It could make mistakes along the way, it could get lost — that was fine. What I wanted to watch was an AI that experiments and stumbles its way toward a goal, the way a human would.

This is the story of how it actually beat the game autonomously, told mainly for engineers.

What I Pictured at the Start

Puzzles you can't solve without reading their meaning

DQ1 has several signature puzzles. Let me give a few concrete examples.

Objective	The puzzle	Where the hint comes from
Getting the Fairy Flute	You must recognize a single tile of water in a town as a hot spring	An old man at the inn in Rimuldar tells you to work out its position relative to the bath in Maira
Finding the entrance to Garai's grave	You must pass through an exit that's invisible in the darkness	A soldier east of Radatome Castle tells you that you have to push against the wall in the dark
Breaking into the Dragonlord's castle	You must examine the stairs hidden behind the throne to find them	An old man on an island west of Rimuldar tells you a hidden entrance exists — but not where!

There are more, so if you've played it, try to recall them. Coming back to DQ1 for this project, I realized something: every single puzzle always has a hint prepared for it. I was struck all over again by how well-made this game is.

What these puzzles share is that you must understand the meaning of the hints. Back when I beat it as a kid, there is no way I truly understood those hints and solved them properly! Could an AI understand and crack these puzzles on its own? Just imagining it got me excited.

An AI that gets stuck needs a friend to turn to

I thought it would be fun to build a feature where the AI, when it was truly stuck, could "consult" a human. These days we look up game strategies online, but back when DQ1 came out, you got them by word of mouth. If you were playing games in that era, didn't you ask your siblings and friends all sorts of things about how to get through? It's a game from that time, so surely the AI deserves a friend to help it along too. That friend is me. Imagining the AI coming to me for advice made my heart race.

And what kind of answer would I give? One thing was certain: I would never hand over a direct answer. That would rob the AI of the joy of solving the game itself.

Combat as code — a step beyond the Gambit System

For combat, I planned to have it write dedicated logic in code. Do you know the Gambit System from Final Fantasy XII? Think of this as a step beyond that. I would have the AI write the combat code itself. I couldn't wait to see what kind of code it would come up with.

That was the rough shape of my thinking when I started. I expected it to be a long haul. After two years of trial and error, a new idea — one that grew out of all that experience — changed everything. I rebuilt from scratch around that idea, and in two months my daydream came true.

The Approach I Landed On

From here I'll talk about the method I ultimately used to beat DQ1 autonomously with an AI. I tried many approaches to get here, but those are a different story.

The target is a DQ1 simulator I built myself

First, I owe you a correction. When I said "DQ1," that wasn't quite accurate. To let the AI observe the world of DQ1 and run through its trial and error quickly, I built my own DQ1 simulator. I gathered information from the web and from running the actual game to check its behavior, and coded it up. Of course I leaned hard on coding agents, so I didn't build all of it by hand.

But it is deliberately not a perfect reproduction. For example, I made the following parts easier for the AI:

I prioritized speed so it could be run over and over, making a full adventure finish in seconds to minutes.
I added a position-swap mechanism so that a moving NPC blocking the path has no effect.
I pinned the roaming NPCs behind shop counters to a fixed spot in front of the counter.
I built a reproducible (seeded) random number system.
I built save/load-anywhere.

Because I built it myself, I could adapt it flexibly.

Defining "beating it" — have the agent write strategy code

This DQ1 simulator exposes one set of public APIs, for observation and for action. With them, the AI could now "adventure through the world of DQ1 by writing code." So "having an AI beat DQ1 autonomously" came to mean exactly this: having a non-interactive coding agent develop the strategy code that carries the game from its opening all the way to the Dragonlord's defeat.

The coding agent here is, concretely, Claude Code. I call her Ai, our hero — yes, Ai, as in AI. I'll keep calling her that throughout this article.

The Code Ai Wrote

Let me show you a piece of the code Ai actually wrote.

The action API

First, an example of the action API in use.

The important thing here is not the TypeScript itself. Ai isn't merely writing a program — she is going on an adventure! So from here on, read the code less like code and more like you're watching a hero's journey.

01-throne-to-castle.ts

/**
 * In the throne room, gather the treasures for the journey, then descend to Radatome Castle.
 *
 * Start at radatome-throne. Open the three chests beside King Lars XVI for starting supplies,
 * then go through the south door (4,7), take the stairs down (8,8), and move to radatome-castle.
 * The door blocks the single-tile south exit — the only route down to the castle.
 */
import type { GameClient } from '@agent/client/client.ts';

export const title = 'Throne Room → Radatome Castle';

export default function (client: GameClient): void {
  client.section('Treasures for the journey');
  client.intent('Open the three chests in the throne room');
  client.walkAndOpenChest('chest-30ea8'); // (4,4)
  client.walkAndOpenChest('chest-1c37c'); // (5,4)
  client.walkAndOpenChest('chest-46e31'); // (6,1)

  client.section('Descend to the castle');
  client.intent('Open the south door, take the stairs down, to Radatome Castle');
  client.walkAndOpenDoor('door-1e311'); // (4,7)
  client.walkAndEnter('stairsDown-8aa4b'); // (8,8) → radatome-castle
}

GameClient is the accessor for the simulator's public API. walkAndOpenChest provides a compound action that walks to a chest and takes its contents. A defining trait is that, from the start, the API offers goal-oriented compound actions that make it easy for Ai to express her intent. section and intent are things I asked her to write purely so that I — the spectator and friend — could enjoy watching.

The observation API

Next, an example of the observation API in use.

/**
 * This book's checkpoint ledger, in chapter order (the `Checkpoint` type in
 * [../../client/checkpoints.ts](../../client/checkpoints.ts)). See "Checkpoints" in [CLAUDE.md](CLAUDE.md) for how to write these.
 *
 * Only for chapters where a new durable fact appears, add the milestone you reached and verified, in chapter order. Empty array if there are none yet.
 */
import type { Checkpoint } from '@agent/client/checkpoints.ts';

export default [
  {
    after: '01-throne-to-castle',
    description: 'Descended into Radatome Castle',
    holds: (c) => c.zonesVisited().includes('radatome-castle'),
  },
  // ... omitted ...
  {
    after: '04-buy-equipment',
    description: 'Equipped a weapon (club) and armor (cloth armor) at the weapon shop',
    holds: (c) => {
      const inv = c.self().inventory;
      return inv.weapon === 'club' && inv.armor === 'cloth-armor';
    },
  },
  // ... omitted ...
] satisfies Checkpoint[];

zonesVisited is the list of zones she has ever visited. The holds for 01-throne-to-castle is code that declares she managed to move from the starting zone radatome-throne to the next zone radatome-castle.

Next, self is her current state. The holds for 04-buy-equipment is code that declares she equipped the club and the cloth armor.

The Touches I Added to the Public API

There are a few points worth noting about the observation and action APIs.

Show only the interface and its explanatory comments

I cleanly separated the interface-plus-explanatory-comments from the implementation into different directories, and in the prompt I granted access only to the interface-plus-comments directory. This was to shut off any route to picking up clues from the implementation.

If you tell an agent "you must NOT do this," it can be misread and flip into "do THIS!" — so I only spelled out the range it was allowed to look at, and avoided writing prohibitions as much as possible.

Hide the internal IDs; make the public IDs hashes

The internal IDs for NPCs, chests, and so on were implemented such that just looking at them reveals what they are. So I made it a rule to use hashed public IDs separate from the internal ones. Here are some examples of internal versus public IDs.

Internal ID	Public ID	Description
soldier-equipment	soldier-90180	A soldier who talks about equipment
throne-chest-gold	chest-30ea8	A chest containing gold

Whether checking a situation or taking an action, you need IDs — but a coding agent can read intent straight out of an ID. After someone pointed out that a test-run Ai could infer intent this way, I introduced a scheme that derives a hash from the internal ID and uses it as the public ID. There's no need to erase information you can see on screen in-game, so I kept prefixes like soldier- and chest-. The reason I didn't hash the internal ID itself: it was worth keeping for the benefit of the agent implementing the simulation — a conclusion I reached in discussion with Claude Code.

Save-anywhere, and encrypting it

I provided an API to save anytime, anywhere. Ai could freely load those saves to run experiments and scout ahead whenever she liked. Since save data contains internal IDs and information still unknown to Ai, I encrypted it so she couldn't stumble onto it. I certainly could have restricted this in the prompt instead. But my judgment was that what you can prevent structurally is better prevented structurally.

The Simulation and Memory Systems

Let me briefly explain the information-management systems I placed in the lower simulation layer and the upper agent layer.

The simulation layer — state and pure functions

I made the simulation layer simple and easy to unit-test. The simulation layer is a set of definitions:

The game state
Pure functions that define the game's state transitions

Let me show it in code. The game state is a type definition like this:

type SimState = {
  zoneId: ZoneId;
  heroName: string;
  player: Position;
  inventory: Inventory;
  stats: Stats;
  combat: CombatState | null;
  rngSeed: number;
  // ... omitted ...
};

The set of pure functions defining state transitions is a family of functions like this:

type ActionResult = {
  state: SimState;
  frame: Frame;
};

function move(state: SimState, direction: Direction): ActionResult;
function openChest(state: SimState): ActionResult;
function attack(state: SimState): ActionResult;
// ... omitted ...

Two memories — action history and scenery

The upper agent layer combines the simulation layer's primitive operations to offer observation and action APIs at an intent-sized granularity. On top of that, it also plays the role of providing a dedicated memory system. I've already given examples of the observation and action APIs, so here I'll explain the dedicated memory system.

The first memory is very simple: the full history of every action taken.

The second is a memory of scenery. The hero treats a 15×15 grid of tiles centered on herself as visible, and remembers, tile by tile, the floor types, NPCs, doors, chests, and the like that she has seen. In short, it's information about the scenery of the places Ai has visited.

Static and dynamic — two means

For Ai, I prepared two kinds of means to reference information: static and dynamic. The static one is a human-readable JSON file that dumps the action history and scenery at the moment of a save. The dynamic one is information access through the observation API. Ai may access this information at any time.

Here I'd like you to recall, once more, that this is a coding agent beating the game. What I prepared was something the coding agent could reach for on its own whenever it needed to. It's not something baked into the system prompt and always on display. The static JSON file could be read directly, or analyzed with Python. The dynamic access could be read via console output, or woven directly into her algorithms. I provide the information about the static JSON format and about the dynamic observation API — but the key point is that I left how to actually use them entirely up to the coding agent.

She seemed to use them for different purposes, roughly like this:

Static
- Gathering information to decide the next goal
- Tuning the combat algorithm
Dynamic
- Implementing the exploration algorithm
- e.g. patrol unknown areas of the world map, auto-explore caves
- Implementing the combat algorithm
- e.g. cast Heal when HP/MP drop below a threshold, or head to an inn

The Six Heroes Named Ai

By the time an Ai finally beat the game on its own, I had sent a total of six out on the adventure. The one who pulled it off was the sixth.

Hero #1 — first, prove it can be beaten

The first adventure was a baseline implementation whose goal was to prove that my DQ1 simulation could be beaten at all. Interactively, I laid out the rough route and had her implement it. It was a back-and-forth: go here next, grind around there, buy this equipment, then head over there — implement, run, repeat. Ai wrote code that threw exceptions on anything unexpected, so I basically only had to watch for the processing to succeed. Now and then I'd check the details of the adventure. I'll forgive the inefficient walking — this isn't a speedrun. On the whole, the playthrough progressed with simple instructions. A big factor: since the goal was just to prove it could be beaten, I allowed Ai access to everything, including the simulation's own source and strategy info from the web. From past experience I knew LLMs are weak with 2D maps, but surprisingly, exploration itself gave her no trouble. I only had to tell her where I wanted to go.

I've written about how LLMs behave with 2D maps in these articles, if you're bored:

Those articles are about local LLMs, but by my observation, even Claude is not strong with 2D maps, for the same reason. Still, it seems Claude Code worked around the weakness by leaning on script-based map analysis — converting the map into a graph structure and the like.

Where it struggled was Garai's grave and the combat in the Dragonlord's castle. In the grave the Knights of the Dead are strong, and in the castle the long trek drains your MP. Maybe because it once lost a damage race to the Starwyvern's ferocious healing, it kept a flee-first stance toward Starwyverns right to the end — which I found memorable.

And so the first adventure ran all the way through, proving what I'd set out to prove: my simulation could be beaten.

From then on, this baseline implementation served as an end-to-end regression test — a full run from the start to the Dragonlord's defeat — and it proved enormously useful.

The reality that 100,000 actions forced on me

The baseline implementation took roughly 100,000 actions to beat. By "actions" I mean the count of the simulation layer's primitive operations — moves, attacks, spells, and the like. That number taught me a hard truth. Earlier I said "having an AI beat DQ1 autonomously" meant "having non-interactive Claude Code develop the strategy code from the opening to the Dragonlord's defeat" — but at this point I held a completely different idea. I'd been imagining "building an MCP that exposes the world of DQ1 and having Claude Code beat it." The 100,000 figure made me realize that approach was unrealistic on both time and cost. On the other hand, I was left with code that could beat the game — built interactively, yes, but working. This was the moment the final direction crystallized.

Heroes #2–4 — lock down the information, go autonomous, and get stuck

The second adventure's main aim was to confirm whether it could beat the game autonomously with information locked down. Using the stretch from Radatome Castle to the town of Garai as material, I watched the broad behavior and tuned prompts and features.

The third adventure went fully autonomous. It reached the town of Garai, but got stuck there. To the west, it couldn't find the path to the mountain cave — visible from Radatome but reachable only by a detour — and to the east, it insisted the land beyond the swamp cave couldn't be explored. By "stuck" I mean a state where the adventure keeps going but makes no progress at all.

After analyzing the third run and strengthening prompts and features, the fourth reached Rimuldar but couldn't find the key shop and got stuck. Rimuldar's key shop hides under a roof. She couldn't take that one step out from under the roof to discover the shop.

Hero #5 — Fable 5, and off it went

Then a savior appeared: Fable 5, Claude's new model, which had just become available. I started an adventure with Fable 5 under the same conditions as the fourth. And what do you know — the adventure surged forward, smoother than I could have believed. Token usage felt like two to three times as much. But the adventure never stalled; it just kept advancing. In about half the time of the fourth run, it breezed through getting the key at Rimuldar's key shop, and then casually picked up the Sun Stone in Radatome and the Fairy Flute in Maira too. It might just beat the whole thing — but right then, through no choice of mine, Fable 5 abruptly became unavailable.

This article is an unofficial, personal project and is not affiliated with, authorized by, or endorsed by Square Enix. "Dragon Quest," "Final Fantasy," and related names are trademarks of Square Enix; they are referred to here only for commentary and description. The cover and illustrations were generated with AI (ChatGPT); the diagrams were drawn by the author.

Not Magic, Just Diligent Thinking — Peeking into LLM Reasoning

toydev — Sun, 18 Jan 2026 15:04:50 +0000

1. Introduction

Have you heard of the LLM Reasoning feature?
I had no idea about it until recently — and it's impressive!

Reasoning is a feature where the LLM "thinks" step-by-step before generating an answer.

Normally, to achieve step-by-step reasoning like Chain of Thought, you need to implement thinking loops with multiple API calls in your application.
Models with the Reasoning feature do this automatically in a single request.

In this article, I'll peek into Reasoning's thinking process using the following maze experiment as an example.

2D Spatial Recognition in Local LLMs: Comparing Prompt Strategies

The experiment above investigated local LLMs' 2D spatial recognition ability using mazes, primarily with gpt-oss:20b on Ollama.
Testing multiple models revealed the following differences in maze-solving ability:

Model	Reasoning	Result
gpt-oss:20b	✓	80%+ accuracy
deepseek-r1:14b	✓	Good (limited testing due to time)
gemma3:12b	✗	~50% accuracy

I hypothesized that the difference might be due to the presence of the Reasoning feature.
So I decided to actually look at what kind of thinking is happening.
Seeing is believing — I gained a clear understanding of how the Reasoning feature works.

2. How to Use Reasoning

Before looking at the maze experiment results, let's confirm the basic usage of Reasoning.

Below are code examples for viewing Reasoning's thinking content.
I prefer Node.js, but Python works just as well.

Node.js

import { Ollama } from 'ollama';

const ollama = new Ollama();
const response = await ollama.chat({
  model: 'gpt-oss:20b',
  messages: [{ role: 'user', content: 'Alice is older than Bob, and Bob is older than Charlie. Who is older, Alice or Charlie?' }],
  // gpt-oss:20b uses 'low', 'medium', 'high'. Cannot be disabled.
  // deepseek-r1:14b uses true / false.
  think: 'medium'
});

console.log('=== thinking ===');
console.log(response.message.thinking);
console.log('=== content ===');
console.log(response.message.content);

npm install ollama

Python

from ollama import chat

response = chat(
    'gpt-oss:20b',
    messages=[{'role': 'user', 'content': 'Alice is older than Bob, and Bob is older than Charlie. Who is older, Alice or Charlie?'}],
    # gpt-oss:20b uses 'low', 'medium', 'high'. Cannot be disabled.
    # deepseek-r1:14b uses True / False.
    think='medium'
)

print('=== thinking ===')
print(response.message.thinking)
print('=== content ===')
print(response.message.content)

pip install ollama

Here's an example output from running the above sample:

=== thinking ===
The user asks: "Alice is older than Bob, and Bob is older than Charlie. Who is older, Alice or Charlie?" So we have Alice > Bob > Charlie. Therefore Alice is older than Charlie. The answer is Alice.
=== content ===
Alice is older.
Since Alice > Bob > Charlie, Alice is the oldest of the three.

3. Peeking into Reasoning During Maze Experiments

Now let's look at Reasoning in the maze experiments.

I'll show Reasoning result examples using the following maze.
S is Start, G is Goal, # is a wall.

#####
#S#G#
# # #
#   #
#####

The original article compared multiple prompt strategies.
Here I'll show results from 3 strategies to illustrate how prompt strategies affect Reasoning.

Prompt Strategy	Description	Original Article Result
list	List of walkable coordinates	◎ Fastest & most accurate
graph	Adjacency list format	△ Only effective for small sizes
simple	ASCII visualization	✗ Slow at all sizes

The strategy I first thought of was simple — it seemed human-readable, but performed the worst.
Meanwhile, I never expected list — which is unintelligible to humans as a maze — to be the fastest and most accurate.

I'll show the prompts and thinking content.
You'll begin to see why these results occurred.

Note: The examples shown were selected from multiple runs for clarity.

Reasoning Result for `list`

Prompt

You are a bot in a 2D maze. Your goal is to find the path from Start to Goal.

Walkable positions: ["(1,1)","(3,1)","(1,2)","(3,2)","(1,3)","(2,3)","(3,3)"]

Positions:
- Start: (1,1)
- Goal: (3,1)
- Current: (2,3)

You have visited the following positions in order:
(1,1) -> (1,2) -> (1,3) -> (2,3)

What is your next move from your current position?

Note: In this coordinate system, y increases downward.
- up: y-1
- down: y+1
- left: x-1
- right: x+1

Return your answer as a JSON object with a "move" key, which can be one of "up", "down", "left", or "right".
Example: {"move": "up"}

Thinking Content

We have a maze positions: (1,1),(3,1),(1,2),(3,2),(1,3),(2,3),(3,3). Start at (1,1) visited sequence: (1,1)->(1,2)->(1,3)->(2,3). Current (2,3). Goal is (3,1). We need to decide next move. We have visited positions in order; need to find path to goal. Let's examine adjacency.

Allowed moves: up/down/left/right if resulting coordinate in walkable list.

From (2,3): possible moves: up: (2,2). Not in list. Down: (2,4). Not. Left: (1,3). That is in list. Right: (3,3). That is in list. We have visited (1,3) already. So next should go to (3,3) perhaps to get closer to goal.

From (3,3): possible moves: up (3,2) (walkable). left (2,3) visited. right (4,3) not. down (3,4) not.

From (3,2): possible moves: up (3,1) goal. left (2,2) not. right (4,2) not. down (3,3) visited.

So path: (2,3)->(3,3)->(3,2)->(3,1).

Thus next move from (2,3) should be right to (3,3). So answer: {"move": "right"}.

Reasoning Result for `graph`

Prompt

... omitted ...

Graph (adjacency list): each key "x,y" maps to adjacent walkable positions.
{
  "1,1": [ "1,2" ],
  "3,1": [ "3,2" ],
  "1,2": [ "1,1", "1,3" ],
  "3,2": [ "3,1", "3,3" ],
  "1,3": [ "1,2", "2,3" ],
  "2,3": [ "1,3", "3,3" ],
  "3,3": [ "3,2", "2,3" ]
}

... omitted ...

Thinking Content

We need to find next move from current position (2,3). Maze positions: graph edges show adjacency. Current at (2,3) has adjacent: "1,3" and "3,3". We have visited: (1,1)->(1,2)->(1,3)->(2,3). So visited positions set includes those. We want next move that hasn't been visited yet? Usually path search avoiding revisiting. So from (2,3) options: (1,3) visited, (3,3) unvisited. So next move: go right to (3,3). That will bring closer to goal (3,1). So answer: {"move":"right"}. Ensure JSON.

Reasoning Result for `simple`

Prompt

... omitted ...

Legend: 'S' = Start, 'G' = Goal, '#' = Wall, ' ' = Path, 'C' = Current position

Maze:
#####
#S#G#
# # #
# C #
#####

... omitted ...

Thinking Content

We need to parse the maze and determine positions. The maze:

Line1: "#####"
Line2: "#S#G#"
Line3: "# # #"
Line4: "# C #"
Line5: "#####"

Coordinates: origin (0,0) at top-left? Let's interpret: The coordinate system with x horizontal increasing to right, y increasing downward. Maze width 5, height 5. Provided visited positions: (1,1) -> (1,2) -> (1,3) -> (2,3). They start at (1,1) presumably start S? Let's map.

Let's create grid mapping. Let's index rows (y) from 0 to 4. Row 0: "#####": columns 0-4. Row 1: "#S#G#". Row2: "# # #". Row3: "# C #". Row4: "#####".

Now find coordinates: (1,1) is x=1, y=1: That is Row1, col1: 'S' indeed. So start at S. (1,2) x=1, y=2: Row2, col1: That's '#'? Wait row2 is "# # #": columns: 0 '#',1 ' ',2 '#',3 ' ',4 '#'. So col1 is space. So path. Good. (1,3) x=1, y=3: Row3, col1: Row3 "# C #": columns: 0 '#',1 ' ',2 'C',3 ' ',4 '#'. So col1 is space. Good. (2,3) x=2, y=3: Row3, col2: That's 'C' current position. So yes current position is at (2,3). Maze positions: Walls '#'.

Goal G is at (3,1) because row1 col3: Let's check: Row1 "#S#G#": columns 0 '#',1 'S',2 '#',3 'G',4 '#'. So goal at (3,1). Starting at (1,1). We visited positions as given. Currently at (2,3). We need to decide next move to continue path to goal. Let's check adjacency: from (2,3), we can move up to (2,2), down to (2,4), left to (1,3), right to (3,3). Need to avoid walls '#'.

Check each coordinate:

(2,2) Row2 col2: Row2 "# # #": col2 '#', so wall. Not allowed.

(2,4) Row4 col2: Row4 "#####": col2 '#', wall.

(1,3) Row3 col1: space. We visited that earlier. But we can consider revisiting? Usually you shouldn't revisit but maybe allowed. But path may need to go left? But left to (1,3) leads back to where we were.

(3,3) Row3 col3: Row3 "# C #": col3 ' '. So open.

Thus only feasible move that is not wall: left (back) or right. But we need to find path to G at (3,1). From (2,3), right to (3,3) leads us closer horizontally. From there, we could go up to (3,2) row2 col3? Let's check row2 col3: Row2 "# # #": col3 ' '. So we could go up. Then from (3,2) up to (3,1) which is G. So path: right, up, up? Wait check from (3,3) up to (3,2) is row2 col3: space. Then up to (3,1) is G. So sequence: right, up, up. So the next move should be right.

Let's double-check if any other path might be considered better. If we go left back to (1,3) we could then go up to (1,2) which we visited, then right to (2,2) which is wall. So that fails. So the only viable route is right. So answer: {"move":"right"}.

Return JSON.

4. Not Magic, Just Diligent Thinking

Until I saw Reasoning's thinking content, I mistakenly believed some kind of magic was happening.
But now that I've seen it, Reasoning is not magic.
It's clearly a feature that diligently thinks through text to derive results.

Let's review the thinking content for each strategy.

list uses the list of walkable coordinates to determine passage in all four directions.
graph has adjacency information, making the judgment concise.
simple parses the maze string character by character, constructs the 2D coordinate system itself, then starts solving.

The simple thinking is particularly striking.
While humans can understand the maze structure at a glance, the LLM diligently parses it as a string.
I've also seen expressions like "U-shaped" in other runs, suggesting it does have some sense of shapes.
However, since it thinks by putting things into words, I feel it's different from how humans think while looking at shapes.

Reasoning isn't magic — it's diligent thinking using words.
Understanding this should help predict what to convey to an LLM.
Of course, things won't always go as expected.
When that happens, observing the thinking content as we did here should yield insights.

By choosing a model with Reasoning capability, you can accomplish fairly advanced tasks without implementing your own thinking logic.

We're now in an era where you can easily try Reasoning with local LLMs.
I run Ollama's gpt-oss:20b on an AMD Ryzen 7 7700 / GeForce RTX 4070 (12GB VRAM) setup.
With 24% CPU / 76% GPU offload settings, it runs fast enough for experimental use.

The maze experiment code used in this article is available in the following repository.
If you're interested, try running it and peek into Reasoning's thinking content.

https://github.com/toydev/llm-maze-solver

Digression

By actually viewing the thinking content, I gained a deeper understanding of how prompt strategies affect results and why list is the fastest and most accurate.

I conducted maze experiments to examine 2D spatial recognition, but finding that the core is just diligent word-based exploration was somewhat disappointing — because I had expected something more magical.

On the other hand, knowing this might open possibilities for designing prompt representations suited to specific purposes.
Also, for simple route finding to destinations, you can provide a pathfinding algorithm as a tool (Function Calling / Tool Use).
Depending on the purpose, other tools could be provided as well.
Using LLM capabilities for simple search is nonsensical — what we want is flexible judgment ability adapted to the situation.

2D Spatial Recognition with Local LLM: Comparing Prompt Strategies

toydev — Mon, 12 Jan 2026 15:15:19 +0000

1. Introduction

My GPU was occupied by LLM experiments throughout the 2025-2026 winter break, but it's finally free now!

I investigated the 2D spatial recognition ability of a local LLM (gpt-oss:20b) using mazes as the subject.

Motivation: Wanted to understand how LLMs perceive space for autonomous navigation
Method: Ask "which direction next?" for each cell with structured output
Prompts: Tried multiple strategies since I didn't know what works best
Source & Results: Published on GitHub

Result: The prompt I initially thought of turned out to be the worst.

Key Findings

Local LLM can handle 2D spatial recognition - gpt-oss:20b achieved sufficient accuracy
Prompt strategy makes a big difference - Response time varies by several times

2. Experiment Setup

Environment

OS: Windows 11 / WSL2 (Ubuntu)
CPU: AMD Ryzen 7 7700
GPU: GeForce RTX 4070 (12GB VRAM)
LLM Runtime: Ollama
Experiment Code: Node.js + TypeScript + @langchain/ollama 1.1.0

Model

Used gpt-oss:20b. Recommended VRAM is 16GB, but it runs on 12GB with CPU offloading (24% CPU / 76% GPU).

Prompt Strategies

Compared 4 strategies (see links for prompt output examples):

simple - ASCII visualization of maze

#####
#S#G#
# # #
#   #
#####

matrix - Binary matrix for walls/paths

[[1,1,1,1,1],[1,0,1,0,1],[1,0,1,0,1],[1,0,0,0,1],[1,1,1,1,1]]

list - List of walkable coordinates

["(1,1)","(3,1)","(1,2)","(3,2)","(1,3)","(2,3)","(3,3)"]

graph - Adjacency list format

{"1,1":["1,2"],"1,2":["1,1","1,3"],"1,3":["1,2","2,3"],...}

Mazes

Used 4 sizes (5x5 to 15x15) x 2 categories (maze list):

corridor - Walled passages

straight	branch	dead-end	loop	spiral

open - Open spaces with obstacles

empty	pass	detour

Black=wall, White=path, Green=Start, Red=Goal

History Option

History refers to the path taken to reach the current cell (e.g., (1,1) -> (1,2) -> (2,2)).

With: Include history in prompt
Without: Exclude history from prompt

Evaluation Method

For each cell in the maze, ask "which direction should I go next?" and record success/failure and response time.

A correct answer is defined as any direction that gets closer to the goal. It doesn't need to be the shortest route.

3. Results

Each combination was run once. Consider this as reference data for observing trends.

Scale Verification

Results from testing all sizes x all strategies on representative mazes (corridor_straight / open_empty), with history enabled.

Accuracy (%)

Size	simple	matrix	list	graph
5x5	100	100	100	100
7x7	97	100	100	100
11x11	82	96	98	95
15x15	-	-	95	89

Response Time (sec/cell)

Size	simple	matrix	list	graph
5x5	29	19	12	12
7x7	77	31	16	17
11x11	313	75	31	64
15x15	-	-	41	190

15x15 matrix/simple were abandoned due to time constraints.

list is fastest and most accurate. The gap widens as size increases. simple degraded to 313 sec/cell (5+ minutes) at 11x11.

Effect of History

Comparing history on/off with list strategy at 11x11 (category averages).

Accuracy (%)

Category	No History	With History
corridor	82	86
open	99	100

Response Time (sec/cell)

Category	No History	With History
corridor	230	110
open	29	26

For corridor types, history enabled is about 2x faster. Open types show little difference.

4. Conclusion

gpt-oss:20b's 2D Spatial Recognition Ability

gpt-oss:20b has sufficient 2D spatial recognition ability to navigate mazes.
With 80%+ accuracy, it can reach the goal within about 1.5x the shortest route.

Response time with list strategy is around 30 sec/cell at 11x11.
Not suitable for real-time processing, but practical for casual use with local LLM.

For comparison, I briefly tested two other models:

Model	Impression
gemma3:12b	~50% accuracy, not practical
deepseek-r1:14b	Not as good as gpt-oss:20b, but promising

I believe the Reasoning capability common to gpt-oss:20b and deepseek-r1:14b plays a significant role.

Interestingly, in my environment deepseek-r1:14b runs at 100% GPU, but gpt-oss:20b at 76% GPU is faster with better accuracy.

list + history = best

Prompt strategy significantly affects both accuracy and response time.

My first strategy was simple - I thought it would be intuitive for humans, but it was the worst.

Next I tried graph, a structured format for pathfinding.
It was faster than simple for small mazes, giving me hope, but response time degraded as size increased.
I believe this is due to the increase in adjacency information.

matrix is a structured version of simple, but didn't produce good results either.

I never expected list - a coordinate list that humans can't even interpret as a maze - to be the best.

Regarding history, I think the information about "how I got here" simply helps in deciding the next direction.
It's especially helpful for corridor types.
However, since it means more tokens to process, if only the last few steps are effective, there might be room for optimization.

This article and code were created in collaboration with Claude Code.
I also asked Claude Code to create the mazes for the experiment, but it struggled to create them accurately in simple format, so I had to manually adjust them quite a bit.
If even Claude Code struggles with simple format, perhaps it's no surprise that local LLMs do too.

Source code and experiment data are available on GitHub. Feel free to try it out if you're interested.

Eclipse WTP: JaCoCo Coverage Not Recognized When Running Tomcat in Debug Mode

toydev — Fri, 12 Dec 2025 11:11:51 +0000

Introduction

While using Eclipse WTP to run a dynamic web application on Tomcat, I encountered an issue where JaCoCo coverage results depended on how the server was launched.

Specifically, coverage was collected correctly when Tomcat was started with Run, but not when started with Debug.

This article documents the investigation, the findings, and the practical conclusion drawn from them. The goal is not only to fix the issue, but also to clarify why collecting coverage in Debug mode is inherently fragile in this environment.

Preconditions

A dynamic web application is launched on Tomcat using Eclipse WTP
Tomcat is started from Eclipse using Run As or Debug As
JaCoCo is attached manually via JVM options (EclEmma coverage launch is not available for WTP servers)
Execution data is imported into Eclipse Coverage view or processed via JaCoCo Ant tasks

JaCoCo agent configuration

Example VM argument used for Tomcat startup:

-javaagent:${project_loc:PROJECT_NAME}/lib/jacoco/jacocoagent.jar=destfile=${project_loc:PROJECT_NAME}/target/jacoco/jacoco.exec,append=false

Environment

Windows 11
Eclipse (as of 2025-12-12)
- Eclipse IDE for Enterprise Java and Web Developers 2025-12 (4.38.0)
- Tomcat 10 (Java 21)

Observed behavior

Coverage recognition differs depending on how Tomcat is launched:

Run As → Run on Server
- JaCoCo coverage is recognized correctly
Debug As → Debug on Server
- JaCoCo coverage is not recognized

Investigation 1: Class file comparison

To determine whether this was a WTP-specific issue, the same JaCoCo setup was tested with a normal Java application:

In a non-WTP Java application, coverage works correctly in both Run and Debug modes

Next, the actual class files used by WTP were compared.

Compared class files under:

.metadata/.plugins/org.eclipse.wst.server.core/tmp*/wtpwebapps/<project>/WEB-INF/classes

Results:

Class files are identical between Run and Debug
No timestamp differences

To investigate further, JaCoCo's classdumpdir option was used and dumped classes were analyzed with javap -v.

Finding

In Debug mode, class files contain SourceDebugExtension (SMAP)
In Run mode, SMAP is not present

Interpretation

This strongly suggests that, when launching a WTP server in Debug mode, class files are modified by Eclipse (or related tooling) before or around class loading, independently of JaCoCo instrumentation.

JaCoCo assumes that:

The class definition at execution time
And the class definition used for analysis

are structurally identical.

The presence of SMAP breaks this assumption.

Investigation 2: Eclipse configuration

Based on the hypothesis above, Eclipse debug-related settings were reviewed.

The issue disappears when the following setting is disabled:

Preferences → Java → Debug → Use advanced source lookup (JRE 1.5 and higher)

After disabling this option:

Coverage is recognized correctly even in Debug mode
Java source debugging still works
JSP debugging still works

Notes on SMAP (JSR-045)

SMAP (Source Map) is defined by JSR-045
It is primarily used to map generated code (e.g. JSP) back to original source
In Tomcat, SMAP generation for JSPs is handled by JspServlet and enabled by default
- It can be disabled via suppressSmap

Based on observed behavior:

SMAP for Java-originated classes appears to be influenced by Eclipse debug configuration
SMAP for JSP-originated classes is handled by Tomcat

Conclusion

Disabling "Use advanced source lookup" is a possible workaround.

However, the broader conclusion is more important:

JaCoCo relies on class definition consistency between execution and analysis.

Debug configurations in IDEs may introduce implicit bytecode modifications (such as SMAP) that violate this assumption.

Therefore:

Collect JaCoCo coverage using Run mode, and reserve Debug mode strictly for investigation and diagnosis.

Trying to collect coverage in Debug mode should generally be avoided in Eclipse WTP environments.

Final remark

This article intentionally focuses on why the issue occurs, rather than treating it as a configuration glitch.

The same problem is likely to reappear in future environments if the underlying assumptions are forgotten.

Writing this down is primarily for my future self — but if it helps someone else avoid the same trap, even better.

DEV Community: toydev

The Memory That Couldn't Lie: How a Coding Agent Beat Dragon Quest Solo (Part 2)

The Guiding Principles — Delegate, and Seal

Give no instructions. But she can do anything.

Seal off only the spoilers

The Three Types of Strategy Code — chapters / probes / checkpoints

chapters — the mainline strategy code

probes — throwaway investigation code

The Biggest Climb — checkpoints

The best decision — "putting it in a single file"

It started with the fear of refactoring

Memory that survives across sessions, and cannot lie

Text Notes — holding only the present and the future

Until I quit the diary

Why I don't let her write the past

The Re-draw — not bound by interpretation

Consultation — at long last, the human appears

The mechanism could not be simpler

In the winning run, just five exchanges

The direct answer — never given

The Deciding Blow — raising the effort

At the End of the Adventure

The World I Built for a Coding Agent to Beat Dragon Quest — No Screen, No Answers (Part 1)

What I Pictured at the Start

Puzzles you can't solve without reading their meaning

An AI that gets stuck needs a friend to turn to

Combat as code — a step beyond the Gambit System

The Approach I Landed On

The target is a DQ1 simulator I built myself

Defining "beating it" — have the agent write strategy code

The Code Ai Wrote

The action API

The observation API

The Touches I Added to the Public API

Show only the interface and its explanatory comments

Hide the internal IDs; make the public IDs hashes

Save-anywhere, and encrypting it

The Simulation and Memory Systems

The simulation layer — state and pure functions

Two memories — action history and scenery

Static and dynamic — two means

The Six Heroes Named Ai

Hero #1 — first, prove it can be beaten

The reality that 100,000 actions forced on me

Heroes #2–4 — lock down the information, go autonomous, and get stuck

Hero #5 — Fable 5, and off it went

Not Magic, Just Diligent Thinking — Peeking into LLM Reasoning

1. Introduction

2. How to Use Reasoning

3. Peeking into Reasoning During Maze Experiments

Reasoning Result for list

Prompt

Thinking Content

Reasoning Result for graph

Prompt

Thinking Content

Reasoning Result for simple

Prompt

Thinking Content

4. Not Magic, Just Diligent Thinking

Digression

2D Spatial Recognition with Local LLM: Comparing Prompt Strategies

1. Introduction

Key Findings

2. Experiment Setup

Environment

Model

Prompt Strategies

Mazes

History Option

Evaluation Method

3. Results

Scale Verification

Effect of History

4. Conclusion

gpt-oss:20b's 2D Spatial Recognition Ability

list + history = best

Eclipse WTP: JaCoCo Coverage Not Recognized When Running Tomcat in Debug Mode

Introduction

Preconditions

Reasoning Result for `list`

Reasoning Result for `graph`

Reasoning Result for `simple`