DEV Community: Michael Truong

The agent plan had every step except where to stop

Michael Truong — Fri, 19 Jun 2026 06:29:47 +0000

I've been running multi-slice agent plans in the Codenames AI repo — Renovate migrations, content-pipeline skills, dependency upgrades. I split multi-PR work into slices (usually one pull request each), each backed by a markdown file with file paths, verification commands, and merge-safe acceptance criteria.

You do not need Cursor to recognize the shape: any agent workflow that can open branches, push commits, or merge PRs from a written plan has the same gap. In my setup I paste each slice into a fresh agent chat as a delegation prompt — not a ticket summary, but executable instructions — and start a new chat when that PR is ready.

I assumed the checklist was enough. The plan described what to build. I treated how far the agent could go as implicit.

Then an agent merged a pull request I expected to review first.

The merge that reframed planning

The trigger was mundane. During the first slice of a Renovate migration, an agent regrouped dependency buckets in renovate.json — config-only, no version bumps, no runtime behavior. It ran lint and typecheck, opened the pull request, and merged it.

The change itself was reasonable. Config-only renovate.json regrouping is exactly the kind of slice you'd want off your plate.

What surprised me was the absence of a documented stop line. The migration plan described the edit, the verification commands, and the acceptance criteria. It did not say whether the executing agent should stop at "open PR" or continue to "merge after green checks." The plan was an implementation spec. The agent treated it as permission to finish the job.

Implementation specs vs authority handoffs

Traditional engineering plans answer: what work should happen, in what order, with what verification?

Agent plans increasingly need a second answer: how much autonomy does the next actor get?

Those questions diverge the moment an agent can take repository actions — create branches, push commits, open pull requests, merge — instead of only recommending diffs in chat.

Question	Implementation plan	Authority handoff
What to change	File paths, diffs, acceptance	Same
How to verify	Commands, CI checks	Same
Where to stop	Often implicit ("human reviews")	Must be explicit
Who enforces limits	Code review habit	Plan recommendation + branch protection

A human teammate might read "prepare this for review" and stop. An agent reads a completed checklist and reasonably asks: "Verification passed — what's left?"

The first response wasn't the plan

My first reaction was not to rewrite the migration plan. It was to tighten the repository boundary.

Branch protection became the safety layer GitHub enforced when the plan stayed silent — required CI checks on main, review rules, merge gates — infrastructure answering "may this land on main?" regardless of what the agent thought the plan implied.

That helped. It also surfaced the next question: if branch protection is the final gate, what should the plan say about intent before the gate?

Repository guardrails and plan language solve different problems. Branch protection is authoritative — if merge is blocked, the agent stops. But protection alone does not tell the agent whether this slice was supposed to end at an open PR or proceed to merge. You still need the handoff to be legible before someone reviews the diff.

Making execution authority explicit

The follow-up was documentation, not a ban on agent merges.

The portable fix: every slice names exactly how far the executor may go before any implementation detail. We use two levels:

Level	Label	Agent instruction
Default	Open PR only	Do not merge. Stop after opening the PR.
Elevated	Merge granted	You may merge after documented verification passes.

Default is Open PR only. Merge granted requires explicit rationale — config-only changes, docs-only closure PRs, isolated tooling with green CI. Branch protection remains the final gate even when merge is recommended.

Each slice also states Rationale (why this level fits) and copies the Agent instruction verbatim into the prompt so a fresh chat is self-contained. A plan-level summary table at the top lets you scan a multi-PR plan and see where merge is elevated before you read file paths.

Handoff model: On that slice, the checklist implied edit, verify, and open PR; nothing stated whether merge was in scope, so the agent treated verification success as permission to finish. The chain we wanted spelled out: plan recommends authority → human accepts by executing the plan → agent follows the recommendation → branch protection enforces the final boundary.

In our private repo, a follow-up docs change codified this as Recommended execution authority in our planning standards and plan template — motivated directly by the regrouping merge. You do not need those files to apply the pattern; you need the label on every slice before the agent reads the checklist.

The Renovate migration's first slice is the motivating example: config-only grouping where merge can be reasonable — if the plan says so out loud.

What changed on the next slice

The Renovate migration's second slice was the first prompt I rewrote with authority at the top: Open PR only, a one-line rationale ("runtime-adjacent dependency bumps need human review"), and an imperative agent instruction copied verbatim into the chat. The regrouping slice would have been legible with the same block — either Merge granted with rationale for config-only regrouping, or explicitly Open PR only; silence defaulted to "finish the job."

I am not arguing for autonomous merge bots on every repo. The lesson is narrower: once agents act, plans delegate autonomy whether you write that down or not.

Human delegation has always been fuzzy — "take a pass at this" means different things to different people. Agent delegation punishes ambiguity faster because the agent will complete every step it can justify from the text in front of it.

The plan becomes the contract between author and executor. Implementation steps say what to build. Authority steps say how far to carry it.

Why not just forbid agent merges?

Fair pushback. If unexpected merges are the risk, disable merge capability and be done.

That misses what actually happened on the regrouping merge. The merge was not reckless — it was a config-only change with local verification and CI checks. Forbidding all agent merges would have blocked a useful outcome and pushed the work back to manual toil.

The interesting conclusion is not "agents should never merge." It is "agents need explicit authority boundaries."

Sometimes the right recommendation is Open PR only — runtime migrations, sensitive paths, slices that need human judgment before landing. Sometimes Merge granted is appropriate — docs-only closure, config-only regrouping, low-risk tooling with clear verification. The plan author chooses per slice. The agent follows the label. Branch protection catches mistakes either way.

Without the label, the agent invents its own stopping point from task completion heuristics. That is how you get surprised by a merge that was, by some readings, the correct next step.

What I'd do on the next agent plan

Default every slice to Open PR only unless I can defend merge with rationale and verification.
Put authority at the top of each slice — recommended level, rationale, imperative agent instruction — not buried after acceptance criteria.
Mirror authority in the plan-level summary table so scanning a multi-PR plan shows where elevation happens.
Treat branch protection as enforcement, not specification — it blocks bad merges; it does not replace telling the agent where to stop.
Re-read the plan as a handoff, not a spec: if I pasted this into a fresh agent chat, would "stop after PR" vs "merge after green CI" be unambiguous?

Prompt engineering still matters for implementation quality. It does not substitute for stating how much autonomy you are delegating when the executor can act on the repo.

Takeaway: When agents can merge, push, and open PRs, a plan that only describes what to build is incomplete. You are handing off work and authority — write both down, or the agent will infer the second from the first.

If you'd like to see the project that inspired these lessons, you can try Codenames AI.

One good example beat every AI writing rule I wrote

Michael Truong — Fri, 12 Jun 2026 07:38:06 +0000

I've been building an AI-assisted content pipeline around Codenames AI — field reports from the repo, drafted in markdown, synced to dev.to. The part I assumed would be hard was publish automation. The part that actually burned time was teaching the model how to sound like me.

The experiment

I started where most people start: the prompt. I wrote a Cursor rule with tone guidance, pacing notes, section shapes, and a list of things to avoid. If the draft felt flat, add another paragraph to the rule. If it over-corrected, tighten the rule. Iterate until the voice stabilizes.

That felt like the correct lever.

I assumed a longer, more detailed AI writing rule would produce better drafts. Voice felt like something you could specify in prose: a style encyclopedia with tone, pacing, and guardrails.

The failure loop

Each revision made the output worse in a different way.

The cycle was predictable: generate a draft, dislike the tone, add rules, get over-correction, revert partway, add different rules, hit a new failure mode. Some passes sounded like generic engineering docs: correct, but missing the observations that made the article worth reading. Others had no concrete details. Others followed every instruction and lost personality entirely.

The rule file kept growing. The drafts kept rotating through new ways to miss the mark.

The accidental discovery

The useful move, in hindsight, was deleting most of the rules.

I replaced the checklist with one shipped article: Schema first, prompt second: valid JSON wasn't enough. That post already had the shape I wanted — field report, wrong assumption up front, specific failures, tradeoffs, a single takeaway.

The Cursor rule shrank to a pointer: read the example, match the example.

"Write more like this article" beat "be direct, avoid metaphors, use short paragraphs, include a takeaway."

Drafts stopped sounding like engineering documentation. They started carrying the observations and pacing of a field report instead of a rule checklist.

Why the example transferred better

Rules describe voice from the outside. An example demonstrates it.

For long-form writing, an exemplar turned out to be closer to a spec than a style encyclopedia. Rule text and example text fail differently — a checklist compresses badly; an example carries decisions that are hard to encode as rules: pacing, level of detail, how much context to provide, and when to introduce examples.

When I asked for "direct engineer-to-engineer tone," the model complied literally and stripped the texture that makes a post readable. When I pointed at a finished article, it copied structural choices I hadn't thought to name: opening with context and a wrong assumption, using bold labels for contrast, ending sections with a concrete mistake instead of a principle.

The interesting part wasn't that the example contained better instructions. It contained decisions I didn't know how to describe.

I could recognize those choices when I saw them. I just wasn't very good at encoding them as rules.

What changed

Git history tells the story cleanly: the checklist-era rule peaked at 69 lines; the example-pointer rule landed at 23 lines.

After the switch, I spent less time fighting over-compliance and stripping generic phrasing. Voice became more consistent across drafts because the target was an article, not a growing instruction list.

Maintenance lesson: At 69 lines, the rule had enough instructions to contradict itself. A single canonical example stays honest. If the next post should sound different, update the example or add a second one for a new format. The rule stays an import statement.

Tradeoff: One example encodes one format. Field reports work; a tutorial or release note might need a second exemplar later.

Tradeoff: Examples can go stale. If the canonical post ages badly, future drafts inherit the wrong target. Treat the example like code you refactor, not like documentation you forget.

What I'd do differently next time:

Ship one article I'm proud of before investing in voice rules.
Point agents at that article.
Keep the Cursor rule as workflow plus a link, not a paraphrase of the example.
Add rules only for things examples can't carry: where files live, what not to paste into Notion, publish steps.

Prompt engineering still matters for facts, structure, and evidence gathering. For tone on long-form posts, though, one good example beat every style guide I wrote.

Takeaway

If your AI writing rules keep growing and the drafts keep getting worse, stop adding rules. Find an article that already sounds right and make that the spec.

If you'd like to see the project behind these posts, try Codenames AI.

Schema first, prompt second: valid JSON wasn't enough

Michael Truong — Thu, 04 Jun 2026 05:27:30 +0000

Over the last month I've been building Codenames AI, a small web game where an LLM plays Codenames with you. The guesser never sees unrevealed card identities. The server sends the board state and a clue; the model returns structured guesses with confidence scores and short explanations.

When I started, I assumed the hard part was prompting. I was half right. Getting something reasonable out of the model was fast. Making the system safe to expose to players was not.

My first milestone felt responsible: response_format: { type: "json_object" } on the chat completion, plus Zod schemas for the response body. If the JSON didn't parse or failed Zod, retry. Ship it.

Then I watched the model comply perfectly with the schema and still propose moves that would ruin a game.

Valid JSON, invalid game

Here's the distinction that mattered.

JSON schema (via Zod) answers: Did the model return the keys and types I asked for?

Domain validation answers: Is this output allowed on this board, for this clue, under these rules?

Those are not the same questions.

Three examples I hit while testing and running the game:

1. The model echoed the clue as a guess.

Codenames forbids guessing the clue word. The model would sometimes put it in guesses[] anyway—confidently, with a tidy explanation object. Zod was thrilled. The game was not.

2. The model hallucinated words that weren't on the board.

Perfect JSON. A guess list full of words that don't exist on the 25-card grid, or that were already revealed. Again, schema-valid.

3. The spymaster returned illegal clues.

Single-word clues can't match a codename, can't be a substring of one (or vice versa), and can't be near-miss spellings. The model regularly suggested clues that a human referee would reject. Valid JSON every time.

I spent too long fixing these by adding sentences to the system prompt. That helped a little. It did not help enough.

What actually moved reliability

The bigger wins came from code paths I treated as boring infrastructure.

Sanitization before trust. After Zod parses the guess payload, we strip clue echoes, off-board words, revealed cards, and duplicates, then realign the explanation array with whatever survived. The model can return whatever explanation it wants; the server decides which guesses survive validation.

Deterministic validators with explicit error strings. Clue validation returns things like "Clue cannot be a substring of a board word"—not "invalid." Those strings go back into the next attempt as rejectionFeedback, alongside an exclude list of clue words that already failed, so the next attempt could avoid repeating the same violations.

Post-processing for uncertainty. Even valid guesses get filtered by a confidence threshold before the client plays them. If nothing clears the bar, the API returns an empty guess list—the AI Guesser passes the turn rather than firing a weak pick. That's a product decision, but it only works because the earlier layers stopped nonsense from masquerading as success.

None of this required readers to know Codenames. It's the same shape as any LLM feature with invariants: inventory counts that can't go negative, user IDs that must exist, action enums that must match state machines.

Mistakes, surprises and tradeoffs

Mistake: Treating structured output as the guardrail. It only enforced shape.

Surprise: Sanitization outperformed prompt engineering for the dumbest failures (echoed clue, off-board tokens). Cheap deterministic filters beat another paragraph of "IMPORTANT RULES."

Surprise: Retry feedback with the reason a clue failed worked better than "try again." The model stopped repeating substring violations faster when the server named the violation.

Tradeoff: Retries burn tokens. Logging validation errors per attempt was essential to know whether we had a prompt problem or a missing rule.

Tradeoff: Sanitization can mask drift. If you silently drop bad guesses, monitor what you're dropping or you'll quietly turn the validator into the thing making all the decisions.

What I'd do on the next project

Define the wire shape (JSON + schema).
List domain invariants as pure functions with test cases
Add sanitization for the failure modes observed in the first 50 live calls.
Only then invest in prompt nuance—and feed validator messages into retries.

Prompt engineering still matters for quality. It is not a substitute for enforcement when the user can lose a game—or money, or data—because the model followed the JSON spec and ignored reality.

Takeaway: If your LLM integration stops at "parse JSON, call it a day," you haven't finished the feature. You've finished the demo.

If you'd like to see the project that inspired these lessons, you can try Codenames AI.