DEV Community: John Hoff

Crypts and Commits: This Documentation Isn't for You

John Hoff — Sat, 29 Aug 2026 16:17:56 +0000

The Audience I Didn't Know I Had — Part Two

When I leave a software project, I want the next person to be able to take it over without needing me in the room.

Sometimes that next person is another engineer. Sometimes it is someone joining the team, a client receiving a handoff, or a developer inheriting a repository they did not choose. Sometimes it is me, six months later, staring at a decision I apparently made for a very good reason and having absolutely no memory of what that reason was.

That is the documentation habit I have been building for most of my career. It is not just “write a README.” It is trying to leave an explicit path into a project: how to start it, how it is shaped, what decisions were made, what someone should be wary of, and enough of the current understanding that the next person can safely take the next action.

The habit became systematic for me in 2012, at an EHR startup that was growing quickly. We went from four developers to forty in only a few years. For a SaaS company, that kind of growth does not happen by accident, and it was not only a headcount problem. The product was growing too: more applications, more repositories, more experiments, and more ways for context to get stranded with the person who happened to create it.

We had a small development team and thin onboarding documentation when I started, so I began writing it as part of my own onboarding. Then I started doing the same thing for every project I created. Markdown files and repository-hosting tools made it easy to put the material where it belonged: next to the source code, not somewhere a person had to remember to find.

The pattern was simple at first. A README.md explained how to stand up a development environment, or pointed to an ONBOARDING.md that did. On a larger application, that expanded to architecture notes, runbooks, release notes, and the other artifacts needed to understand the project as a living thing.

When someone new joined a project, one of their first jobs was often to update the documentation. That was not busywork. A new set of eyes was a good way to find out which parts of the handoff had aged badly.

At my current job, people have a name for the result: “John-ifying a repo.” It is half joke and half compliment. You can often tell which repositories I have touched because there is a clearer path into the project than there was before.

I did not start caring about that in 2012. The startup experience simply forced an older professional instinct into a repeatable practice.

Read the documentation. Then be suspicious of it.

RTFM is an old joke because it contains a real complaint: the answer is often in the documentation, and someone still has to go read it.

But I think the joke is usually too simple. The actual tension is not between people who write documentation and people who refuse to read it. The tension is that both of these things are important:

You need to read the documentation, because code alone does not preserve all the intent, operating knowledge, and tradeoffs behind a project.
You need to avoid trusting the documentation too much, because it can be stale, incomplete, aspirational, or simply wrong.

I have been on plenty of support calls after a handoff where the practical solution was to get on a call and pull up the documentation together. I have also inherited projects and asked questions that were probably answered in a file somewhere. I do not blame other engineers for that. A human being with limited time has a good reason to be skeptical of a long document, especially when there is no guarantee it describes the system they are actually looking at.

That skepticism is healthy.

The problem is not that documentation fails to become a perfect source of truth. The problem is when we treat it as one. Planned architecture is easy to write. Aspirational architecture is even easier. The difficult thing is updating the record after the work teaches you what the system really is.

This is why I cared about making a new engineer update the onboarding material. It was a built-in reminder that the handoff had to meet reality. Documentation does not replace judgment. It is supposed to give judgment a better starting point.

A new kind of reader

AI coding assistants changed the value of this habit for me, but they did not change its underlying purpose.

The first connection is the one that feels almost obvious once you see it. I have always written for a forgetful future self. If I return to an old project after enough time away, I do not remember the history of every architectural decision. I need to read enough context to restore the reasoning before I change the code.

An LLM is even more literally that kind of collaborator. It does not retain project understanding between turns or sessions unless that context is supplied again. If I gave a human coding partner a task every morning and they had forgotten everything from the day before, we would be very nearly in a Monty Python sketch. With an AI coding assistant, that is simply the starting condition.

The second difference is patience.

Ask a human engineer to 10 paragraphs of context and they may reasonably ask what they actually need to know. Ask them to read 100 paragraphs and the question becomes even more reasonable. An LLM can receive either amount of context and produce an answer without becoming frustrated, bored, or resentful of the reading assignment.

That does not mean context is free. There are still limits, costs, and real questions about whether a large collection of material is useful. It does mean that a practical constraint has changed. An agent can consume a volume of documentation that a human developer would understandably skim, defer, or reject.

I saw a small version of this with an internal command-line tool that calls a data pipeline. The tool was not even especially clean: it inconsistently used “modified” and “updated” in its options. An agent would use the wrong term, get a parsing error, ask for the generated help with -h, and update the command. It would repeat that pattern without asking me for help.

The point was not that the agent was somehow better than a human engineer. It was that the agent kept consulting the available context. A human might have asked me about the same command three or four times, even with the help text sitting there. The agent encountered an error, read what the tool had to say, and kept moving.

I had not consciously thought of generated command help as documentation before that. But it is. Good error messages, consistent task runners, discoverable command options — they are all part of the conversation between a system and someone trying to use it.

The fuller conversation between an engineer and an AI assistant is code and documentation together.

More context makes the old risk more important

The agent’s patience is useful only if the context it reads is worth reading.

This is where I think it would be easy to get the story wrong. The lesson is not “give the model more tokens.” An agent will patiently consume an enormous architecture document. It can also patiently consume an enormous architecture document that describes a system which no longer exists.

In some ways, the agent makes the old documentation problem easier to hide. A skeptical human might stop and ask why they are being sent this much material. An agent will simply work with what it has been given. If the information is aspirational, stale, or missing the actual reason behind a decision, it can create a very polished result built on a bad premise.

So the responsibility does not go away when the reader changes. It increases.

My job is not to produce mountains of text for an AI to obey. It is to curate the what and why behind the work, then make sure that record changes when implementation teaches us something new. The AI does not eliminate the RTFM tension. It changes who is willing to do the reading, while raising the stakes of getting the record wrong.

An experiment, not a review assignment

That was the context for a recent architectural handoff.

I used Claude Code to plan a change and work through the details, resulting in a fairly extensive plan of attack. A teammate was going to implement the work with Windsurf. The plan moved from my agent and my machine to his agent and his machine.

During the handoff, he pushed back on the length of the documentation. Why had I given him the full document when a summary would do? Did I really expect him to read all of it?

He was not wrong to ask.

I explained that I did not expect him to treat the document as a human review assignment. It was the full context of what I thought needed to happen and why, curated with my coding assistant. I asked him to give it to his AI at the start of the change, then tell me where the reasoning turned out to be wrong once it met the codebase.

I was not handing him a potentially condescending architecture document. I was handing him an experiment to run.

The result was not that the plan turned out to be perfect. It was more useful than that. His implementation produced concrete feedback about where the original analysis held up and where it did not. We updated the architecture documentation to match the post-change reality.

The plan had traveled across people and across tools. The documentation gave the second agent enough context to begin. The implementation tested that context. The revised documentation recorded what we learned.

That is the entire practice in miniature: read the documentation, distrust it enough to test it, and revise it against reality.

His original pushback remains valid. The document was too long for the human task at hand. The agent’s patience made the experiment feasible. The human engineer’s judgment made it useful.

What I document now

I now preserve more of the planning context created with an AI coding assistant, not just the finished state of the code. A documentation-consistency review is part of a planned change. Sometimes the AI closes the gap; sometimes I do.

That produces more documentation than I ever expected to create. It also has a different primary reader. The material still lives in the same repository as the code. A human is not excluded from it; another engineer can have their own AI read the relevant context as they plan work. But some of these documents are no longer primarily written for a person to sit down and consume from start to finish.

They are written for an agent working with that person.

That is what I now tell people when I introduce the practice. Imagine how much documentation you would need if your coding partner forgot the entire project every night. For a human partner, it would be absurd. For an LLM, it is the actual operating environment.

I used to document so that a future person could take over without me. Now one of those future collaborators is an AI that forgets everything between conversations and never complains about the reading assignment.

That makes the old habit more useful. It also makes the old warning more important: read the documentation, and then make sure reality agrees with it.

The next question was how to make that handoff discipline less dependent on one person’s personal habits. That is where Crypts and Commits begins.

Introducing Crypts and Commits

John Hoff — Tue, 25 Aug 2026 22:35:35 +0000

The Audience I Didn't Know I Had — Part One

There is a lot of conversation right now about one developer coordinating forty coding agents.

I am not going to pretend I know whether that is good or bad. It is clearly a real model of working, and there are people doing remarkable things with it. But I keep coming back to a more ordinary question: what does agentic development look like for a team of ten developers, each working with one or two agents, while still staying in sync with one another?

For most organizations, that is probably the more immediate problem.

There is a familiar failure mode in software development: the hero coder. One person understands the system, carries the context, makes the impossible things happen, and becomes a bottleneck whether anyone intends it or not. Forty agents may make that person more productive. It may also turn the same problem up to eleven.

This is not an argument against the hero-coder version of agentic development. Organizations tend to know whether they can handle that kind of operating model. An organization that cannot handle a hero coder is probably not ready to depend on one engineer coordinating forty agents either.

I am more interested in the team that already exists: several developers, several codebases, and people who are just beginning to figure out what coding agents mean for their day-to-day work.

That is the problem I built Crypts and Commits for.

The influence was Beads

The most important influence on Crypts and Commits was not the whole Gas Town model. It was the underlying idea of Beads: a git-backed record of work to be done.

That idea landed for me right as I was working through a separate realization: documentation was becoming less of a thing I wrote for another human engineer and more of a thing an AI coding assistant could actually use. I had already been using extensive Markdown files to plan work. Git-backed units of work felt like a natural next step. The planning was no longer just a document I hoped someone would read. It could become durable context that an agent could pick up and work from.

The freeing part was that Beads did not try to become a Jira board.

Jira has a job. It helps a project communicate status and priorities to people beyond the developer doing the work. Crypts and Commits is not trying to compete with that. The context it captures is for the developer and the agent: the deeply technical why behind a change, the plan, the decisions made along the way, and the receipts left once the work is done.

The other influence was the use of metaphor. Gas Town showed me that a metaphor outside the usual software vocabulary can shorten the conversation with an AI. You do not have to re-explain every relationship from first principles if the system has a coherent world for the agent to work inside.

That is part of why Crypts and Commits is built around tabletop gaming. The human is the game master. The agents are the players. The terminology is not just a cute theme; it gives the project a vocabulary for how work should happen without immediately falling back to tickets, epics, and other words that carry a whole different set of expectations.

Crypts and Commits is influenced by Gas Town and Gas City. It is not a fork, a replacement, or a smaller version of either one.

A useful correction from Steve Yegge

That distinction matters even more after Steve Yegge’s recent essay, “The Shape of Things to Come.”

In it, he describes Gas Town falling apart as the models changed and says he has given up on building reusable harnesses. His current work is a personal harness called Wheelhouse, tightly built around his decades-old game, Wyvern. His argument is that harnesses need to become bespoke — chemically bonded to the application they serve.

I agree with that much more than I disagree with it.

I saw the same thing early while building Chi-Town for work and Crypts and Commits for my own projects. You cannot take a generic process, drop it into a repository, and expect it to understand what matters about that project. Someone has to define the world: how the system is organized, what its boundaries are, which decisions matter, and how the agents should behave when they encounter the parts that are unique to this codebase.

For me, that happens through skills and the configuration that drives them. Once you start doing that seriously, I am not sure there is a clean line between “the harness” and “the project configuration.” The configuration is part of the harness. It changes what the agents know and how they work.

So Crypts and Commits is trying to hold a middle ground.

The package itself is reusable. It gives a project common structures for skills, documentation, encounters, and repository-backed receipts. But every project’s actual use of it should become bespoke. You still have to define the game world. You still have to document what is particular about your system. You still have to decide how you want agents to work inside it.

Crypts and Commits does not define your harness for you. It helps you document what is bespoke about your project and gives your agents the tools to use that configuration.

A different scale of agent work

This comparison is reductive, and I want to be clear about that up front. Gas Town and Gas City have a much richer set of ideas than I can summarize in a paragraph. But the reduction is still useful.

Gas Town and Gas City are largely concerned with orchestrating agents as workers in a larger system. Crypts and Commits is concerned with a developer and an agent working together as reasoning partners.

In the pattern I am using, the human describes the work. The AI creates an encounter: a record of the requirement, the rationale, the plan, and the verification. Then the human and the AI refine that encounter until the human believes it will get them to the end state they actually want.

The human is not delegating a ticket and walking away. The human is still doing the reasoning. The agent is helping make that reasoning explicit, explore the codebase, and keep the context from disappearing when the session ends.

At team scale, the important part is that everyone can work from the same pattern. Each developer can use the agent they prefer while the repository holds the shared context and the receipts behind the work. No single engineer has to become the one person who remembers how every agent, every prompt, and every private chat thread fits together.

I can see where multi-agent orchestration might become useful for me someday. I got Gas Town running locally and quickly realized it was more than I was ready to take on. That is not a criticism of the tool. It is an admission about where I am.

It is also about the teams I work with and advise. Many of the clients I work with are only beginning to use coding agents. Rolling out a multi-agent orchestration project right now would be getting too far ahead of them. A shared pattern for developers working with one or two agents each feels like a better next step.

Why I am publishing it

I wrote Crypts and Commits for myself.

That is probably the most honest way to introduce an open-source package. I did not start with a market survey or an argument that every team needs this exact tool. I was trying to create a framework for how I wanted to work with agents, then realized it might be useful to share because so few people are talking in depth about what their day-to-day agent work actually looks like.

Crypts and Commits is intentionally easy to add to an existing repository and just as easy to strip out. It can guide Claude or Codex through the framework and help you customize the project’s own world. If you try it and decide you can build something better in a few days, honestly, I would love that.

The package itself may have a short shelf-life. I think that is probably measured in months, not years. But I do not think that means anyone should wait for the stable answer to arrive. We need to work with these tools, form our own opinions, and learn enough to evaluate the next round when it comes.

The only way you are going to find the harness you want on your own projects is to explore what is already out there.

If you want to try Crypts and Commits, try it. If you want to borrow the idea and make it your own, please do. The interesting outcome is not that everyone winds up using my package. It is that more developers start thinking deliberately about the context their agents need and the harness they want to build around their own work.

Crypts and Commits did not begin as a product idea or an attempt to copy a framework. It grew out of a documentation and handoff habit I had been building for years. Part Two explains that underlying habit and why AI agents changed its value. Part Three explains how that habit became a more formal package and a shared team practice.

That is where the real story begins.

We Will Get You Through It!

John Hoff — Sat, 15 Aug 2026 21:45:52 +0000

There is a comedy sketch from Bob & Tom that starts with a hilariously impossible promise: overnight delivery by train, from New York to Los Angeles.

At one point, someone asks if they can really get a 2,000-pound package across the country overnight by rail. The answer is delivered with absolute confidence: “Norfolk and Waypal, overnight. Absolutely. Positively.”

The name is doing some careful work. It lets you hear the phrase that nobody has actually said out loud.

No way, pal.

When I end up leading a project with six weeks left and something that feels like four months of work to do, I start the internal kickoff by telling the team to go watch that sketch. No other explanation. Just go watch it, then come back.

Then I tell them: “Absolutely, positively, we will get you through it. There's Norfolk and Waypal, we are gonna to do it.”

That does not mean we are going to do the thing exactly as it was originally promised. It means we are going to get through it. Absolutely. Positively.

There is a difference.

Laugh at the impossible first

I think newer developers especially need permission to laugh at impossible requirements.

An 800-pound gorilla from New York to Los Angeles overnight by train is impossible in a way that is easy to laugh at. A project that needs a full cloud environment, API work, a mobile application in the app stores, production deployment, security approvals, and a dozen other things in six weeks? That can feel less funny when it is sitting in your sprint board.

But it may be just as impossible if we take the requirements literally.

The first danger on a crunch project is shame. A junior developer can look at an impossible deadline and wonder if they are missing something. Maybe everyone else understands how this gets done. Maybe it is a talent problem. Maybe if they just worked harder, they could turn six weeks into twelve.

Nope.

Sometimes the work is just Norfolk and Waypal.

Humor does not solve the problem. It lowers the temperature enough that the real conversation can happen. Saying “this cannot be done as stated” is not always safe or easy, particularly when you are speaking within a real power imbalance. The joke gives the team shared language for reality without asking the least-powerful person in the room to be the first one to say the quiet part out loud.

Set limits before the project sets them for you

One of the projects I remember most clearly was a virtual medical-care application that had to launch during the Q4 healthcare open-enrollment season. I was not part of the original strategy and scoping work. My job was to lead delivery.

We had six weeks. We needed development, QA, and production environments. We needed API endpoints. We needed a mobile application published. We needed to get through internal security reviews. We needed a real production launch.

The deadline was real. It was not simply somebody failing to plan. The client had a seasonal business opportunity, and if the launch did not happen, the work would stop. That was actually a useful guard against throwing good money after bad.

It was still Norfolk and Waypal.

The first boundary I set was around hours. I believe the cap was 50 hours a week. It applied to everyone, including me. The cap held, although that does not mean the pressure went away. Testing support and bug triage still come back to developers. Things still break. Decisions still have to be made quickly.

But more hours were not going to turn six weeks into twelve. They were going to make us tired, make us less effective, and effectively pay everyone less for their time.

This is the part where I want to be careful with junior developers. “Set a boundary” can sound wonderfully simple when you have enough authority to set one. In the United States, at-will employment makes this extraordinarily gray. You may not be able to demand overtime pay. You may not be able to refuse extra hours without risking your job.

Still, be very concerned whenever you are asked to “sacrifice” for a project. That word tends to mean someone wants your time, your energy, or your health without compensating you appropriately for it. Cash is the most obvious cost. Extra vacation time can be another. The point is not that compensation buys the right to burn people out. It does not. The point is that putting a real cost on an exception exposes the tradeoff before unpaid overtime turns into an expectation.

And if an employer pushes back on a junior developer who is honestly trying to surface those tradeoffs, that is useful information. It is worth asking whether that is an environment where you can stay and build a successful career.

Find the Norfolk features

After we watched the sketch, I asked the team to find the Norfolk features.

A Norfolk feature is a feature users would not miss unless they explicitly knew it was supposed to be there.

That is not a claim that the feature is bad. It may be perfectly reasonable in the longer-term product. It is a claim about the deadline. If we spend time on it now, what work are we not doing instead?

I pushed hard for a prioritized client feature list. I did not always have a perfect cutoff where I could say, “Everything below this line is out.” But I did know what the client believed mattered most. That made it possible to show honest progress against their actual priorities at every stage.

On this project, patient billing integration was the poster child. A partner could handle the EHR side, but direct patient billing was not ready at launch. It mattered, but because claims still had to go through the usual submission and adjudication delays, it was not truly required on day one. The client wanted patient billing within 30 days. Nothing in the contract said it had to be 30 days instead of 90.

That is the kind of tradeoff I mean. Not “we will skip whatever is inconvenient.” We did not treat privacy, security, compliance, or regulatory obligations as Norfolk features. In a medical application, skipping those requirements carries legal and financial risks that are much worse than a missed feature. If we had failed the needed regulatory hurdles, we would not have launched.

The work is not about pretending every decision has a clean answer. It depends. It always depends. The work is about keeping the decision in front of everyone: is this issue truly blocking the smallest safe thing that needs to exist on day one, or is it pulling effort away from that thing?

Be ready to get through it, not just to win it

The client deserves most of credit for that successful launch. Their leadership understood the tradeoffs, prepared their stakeholders, and was willing to launch a truly narrow MVP. One of their leaders said something I still fondly remember: “If you actually like your MVP, you waited too long.”

That client partnership was honestly more important than anything I did. I also could not assume I would get this level of participation at the start. If the client had refused to narrow the scope, I was prepared for the project to fail. I communicated that to my internal leadership and to the team.

That is not a heroic story. It is the opposite of “success at all costs.” Sometimes the responsible thing is to make the risk visible instead of quietly transferring all of its cost to the people doing the work.

For me, the outcome signal is not only whether a project goes live. It is whether I want to talk to the team afterward. It is whether they want to work together again. A year later, if somebody on a completely ordinary project can joke about a Norfolk feature, I take that as a much better sign than a green status report from launch week.

Crunch is going to happen. We should try to avoid it (duh!). We should not normalize it. But when it arrives, do not immediately assume that an impossible assignment is a test of your talent or your willingness to care.

Look at the requirements. Find the thing nobody would miss. Protect your limits where you can. Ask what is being traded away. And if the project really is asking for the gorilla by rail overnight?

Absolutely. Positivly.

We are going to get through it.

There’s Norfolk and Waypal.

We are gonna do it.

A Different Kind of Content

John Hoff — Fri, 07 Aug 2026 19:11:41 +0000

Writing With AI — Part Two

Last time, I told you where this loop came from — a year of an AI telling me my architecture ideas were great, a manual workaround that only half worked, and a skill that finally automated the fix. I ended on the sentence I actually said out loud: I called it a content design interview — let's try a different kind of content. This is what happened next.

What follows isn't a retelling of that moment. It's a compressed, honest rendering of an actual interview — the same loop, the same rounds, just pointed somewhere new. I'm not going to re-explain what a round is. You already know, because you watched one get built.

Round 1 — Question
Why did shifting the critique subagent's lens make the loop feel smarter instead of just differently aimed?

Response. Because the topic wasn't the hard part. Writing was.

I'd written down a handful of article topics on my own, the normal way — a few sentences each, nothing an AI suggested. Then I sat down and tried to actually draft two of them, alone, the way I'd always written. Both attempts turned into something I couldn't use. I mean that literally — the drafts don't survive. I'd already deleted most of the text by the time I gave up on either one, which should tell you how badly I wanted them gone.

So I went back to the two- or three-sentence topic I'd written before any of that mess, and pointed the loop at it instead — the same loop that had spent months keeping architecture reviews honest. Here's the realization that made that move make sense, and it's smaller than it sounds: over months of architecture reviews, I'd noticed the loop was good at capturing a problem correctly even when the solutions it proposed for that problem weren't especially good. The technical fixes coming out the other end were sometimes mediocre. The shape of the problem, once a round was done, was almost always right. Which meant the valuable part of the whole apparatus was never the domain. It was the lens. And a lens is exactly the kind of thing you can swap.

Critique
That explains why you tried it. It doesn't explain why expanding the critique subagent's scope to "audience" and "story continuity" made the loop smarter rather than just retargeted at a different kind of correctness.

Fair. Architecture review has a ceiling built into its own vocabulary — does it scale, does it fail gracefully, is the abstraction load-bearing. Writing doesn't have an equivalent fixed vocabulary; "is this good" depends on who's reading it and whether the sentences hang together, which are questions a narrow technical lens has no way to even ask. Widening the lens didn't just change what the loop looked for. It gave the loop a whole dimension of failure to notice that had simply never existed in the architecture version. That's why it felt like a step up in intelligence instead of a lateral move.

Round 2 — Question
Who actually wrote this sentence?

Response. I did. So did whatever's reading these words back to me right now. Neither answer is complete on its own, and I'd rather sit in that than round it off for a clean byline.

Mechanically: I write the topic, always me, always a few sentences, never suggested by anything with a model behind it. I answer every question the loop asks — all of it, in my own words, before anyone drafts a sentence on my behalf. A separate, isolated pass generates criticism of the whole exchange so far — what's thin, what's contradictory, what's been asserted without being earned. I answer that too. Loop until nothing new comes out, then stop.

The felt experience, the first time this ran on writing instead of architecture, was something like: instead of navigating sycophancy, I was suddenly being grilled. I write substantially more in these interviews than the model does — across dozens of loops now, spanning both this blog and my day job.

Critique
"Substantially more" and "dozens of loops" are doing a lot of unquantified work. Give a number or don't make the claim.

I'm not going to give you a number. I have an impression — it's something like double — but dressing that up as a measured statistic would be a promise I can't actually back up, and you'd be right not to trust the rest of this. It's a feeling, offered as one.

Which is exactly the spirit I need for the thing I actually haven't resolved yet: I told you, once, that I was "responsible for the sentences" in this arrangement. That's not the same claim as "I typed every sentence." Once the interview ends, the model gets real, substantial autonomy over how the material gets organized into something that reads like a story instead of a transcript — structure, ordering, which beat opens and which one closes. What I'm responsible for is different, and heavier: I wrote the topic, I answered every question, I make every final call on what stays or goes, and my name is the one attached to whatever gets published, regardless of who typed which sentence. It's felt, in practice, less like handing my ideas to something that ran off with them, and more like being helped to organize thoughts that were already mine and already tangled.

Round 3 — Question
Which is actually protecting your voice — the fact that you wrote every word of the interview yourself, or the fact that the questions never let your mind wander off the record?

Response. I don't know yet, and I'd rather tell you that than manufacture an answer that sounds more finished than it is.

Here's what I can tell you about whether any of this is working at all. The output is less polished than typical AI prose tends to be, and that tracks with my own long-standing assessment of myself as a writer — a professor told me once, correctly, that my analysis and understanding of the material was wonderful and that I couldn't write my way out of a paper bag. That was over two decades ago and it's still roughly accurate. If this process were quietly smoothing my voice into something else, I think the smoothness would be the first thing I noticed. It isn't. The blunt honesty survives too, including the kind aimed at myself — something like I was totally right. On the other hand, it didn't matter one bit made it through a full round of questions and criticism completely intact, because nobody was in the room to sand it down before it hit the page.

As for the why, I've got two live theories, and I hold both without having reconciled them. The first: the loop makes me put the entire context of an idea into my own words before anything gets drafted — not a summary, not bullet points, the whole thing, oversharing the way you'd overshare to a friend at one in the morning. Because the questions and the criticism come from something deliberately isolated from any conversational thread, there's no audience for me to perform for, and no audience for the model to be sycophantic back at.

The second has nothing to do with sycophancy and everything to do with how my head works, and I'll say the quiet part plainly since there's no reason not to: I'm a mathematician by training, I have ADHD, and I'm very likely somewhere on the spectrum — that particular barn door got opened a long time ago, and I'm not interested in pretending otherwise now. My mind runs down every rabbit trail it finds and doesn't naturally circle back to the on-ramp by itself. A fixed set of questions gives my mind a map without asking it to behave. I don't have to stay on target, because the target keeps finding me every round whether I meant to hit it or not.

Critique
Pick one. You can't build a mechanism theory on both without explaining how they interact.

I can't pick one, and I'm not going to fake having reconciled them just because a clean answer would land better here. This round doesn't resolve. Maybe the isolation matters more. Maybe the structure matters more for a mind built like mine. Maybe they're the same mechanism wearing two names. I don't know, and pretending otherwise would cost me exactly the thing this whole process is supposed to protect.

What you've actually been reading

I'm going to stop pretending this is a device.

Everything above is a compressed, honest rendering of the actual interview that produced this post — not a re-enactment written after the fact to look clever. I wrote the topic for this post the same way I wrote four others before it: a few sentences, alone, before any of this started. A question pass expanded it. An isolated critique pass tore at it. I answered, it tore again, and at some point the tearing stopped finding anything new, which is the only signal this process has ever given me that a round is finished.

Part of why I'm telling you this so directly is that I've been circling a promise for two posts now. Earlier pieces on this blog — The Ground Is Moving and It's OK to Get Lucky — mentioned, in passing, that there was AI involved in getting them made, and both of them owed you the fuller account. This is that account. The site has said it plainly from the start, in five words tucked into a corner nobody reads: human author, AI collaborators. These two posts are what that sentence actually means once you unpack it.

There's a second reason I'm writing this at all, and it's less confessional. I'm a solution architect who spends my working life on AI systems, and I love the work. Four posts went up before this one, quietly, with no real audience behind them yet. This is meant to be a front door. If you build software and you've ever wondered whether a tool built to remove flattery could remove your own worst habits too — the ones you can't see because nobody's ever pushed back hard enough to show them to you — I'd rather you go try it than take my word for any of it. The skill files ran below Part One, plain text and a zip, portable across whatever harness you're running. That's the whole ask.

What comes after the interview has never looked the same twice, on purpose. One post came out of four separate narrative outlines, with me picking the strongest and rewriting the parts that didn't sound right before handoff. Another went almost straight from a simple outline to a finished draft with barely any friction at all. I've stopped expecting the back half of this process to be consistent, because it was never supposed to be. The interview is the part that's held constant. Everything downstream adapts to whatever the post actually needs.

Round 4 — Question
When does this stop?

Response. When it feels done. That's the actual rule, unglamorous as it is — not a fixed round count, not a checklist of concerns satisfied, just a subjective sense that nothing new is coming out anymore. It's the same principle I found months ago, back when this loop only knew how to review architecture: ask a model for new questions and you'll always get new questions, so the finish line has to come from a person, or it never comes at all. I didn't expect to be turning that same principle on my own uncertainty about whether any of this actually works. But here we are.

Could I have written any of these posts without this process? Undoubtedly. I've been writing my entire career, badly by my own account, and I would have eventually gotten something onto the page one way or another. But would have eventually isn't the same claim as would have gotten this. The posts I actually published surfaced things I hadn't gone looking for — a conviction about being right that I'd been sitting on for a decade, dragged out by an interview that nearly derailed on the exact question of whether I was right; a plain instruction, delivered mid-round, to name a thing I'd been describing around instead of naming. I didn't plant either of those. The process found them by refusing to let a thin answer stand where I'd have happily left one.

I still don't know if the voice underneath all of this feels like mine because it is, or because I've built an elaborate enough process that I've stopped being able to tell the difference — which would be its own kind of failure, wearing a pretty convincing disguise. Biases are hard to catch from the inside. I'm not going to perform more certainty than I've earned just to close this out on a stronger note than the truth allows.

I think only time will tell.

That's a Great Idea!

John Hoff — Wed, 29 Jul 2026 16:57:23 +0000

Writing With AI — Part One

I was three responses deep into an architecture review, and the AI had opened every single one of them the same way. "That's a great idea!" I'd offer a fix, it would say that, then it would raise the next issue for me to work through. Three in a row. And for about half a second, I let myself believe it: Man, this must actually be a great idea.

Then I caught myself. If it were that good — and that original — why hadn't anyone else already built it? Nobody stumbles into a genuinely novel approach three separate times in one conversation without someone, somewhere, having tried it first. That question is what actually moved me, not the compliment. I pulled the conversation out, distilled it down to a report, and ran an analysis on that report in a brand-new conversation. Same content. Completely different tone came back.

That gap — between what an AI tells you mid-conversation and what it tells you once the conversation is gone — is where this whole thing started.

Wanting friction

I'd been using AI to organize and pressure-test architecture ideas for about a year by that point, and "that's a great idea" had become something closer to a tic than a compliment. It didn't matter how rough the idea was. It didn't matter how clearly I was in thinking it through. The validation showed up anyway, reflexively, the way a hinge creaks whether you oiled it or not.

I don't want to hear that my ideas are great. I want friction. I need friction — the kind that finds the wall you forgot to check before you knocked it out. A year of "that's a great idea" doesn't feel like support. It feels like losing the one signal that's supposed to tell you when you're wrong.

The first fix, and why it only half-worked

My first real attempt at a fix was almost embarrassingly literal: I took an architecture document I'd written entirely myself and asked the AI to critique it as though someone else had written it. Disguise the authorship, I figured, and maybe the flattery goes with it.

It worked. Sort of. The first response back was the best critical feedback I'd gotten from any AI up to that point — genuinely sharp, no hedging, no warm-up. It felt like a breath of fresh air. Then I responded to it. I had a real concern about the database choice we'd need for a content-indexing layer — something that had to support both text search and embedding search, and I wasn't sold on the option in front of me — and the moment I pushed back on one point, the "that's a great idea" voice came right back. Same conversation. Same document. Nothing about authorship had changed. What had changed was that I'd said something.

That conversation is a year old now, and I won't pretend I remember the exact words I typed. What I remember is the shape of it: real critique, one paragraph long, gone the moment I had an opinion about it.

The actual mechanism

Here's what I eventually understood, and it recasts the whole disguised-document experiment: it was never about whether the AI knew I'd written the document. It's about the shape of the conversation itself. The moment a second "Human: <message>" turn shows up carrying a stated opinion, something shifts. Not the content. The structure.

That's not a guess. It traces straight back to work I'd already been doing, professionally, with LangGraph-based agentic systems — a habit of avoiding chat-based prompt templates entirely when I actually needed critical analysis out of a model, because the conversational shape itself seemed to be part of the problem. Separate the conversation structure from the analysis structure, and you get something closer to an honest read. Leave them tangled together, and the moment you speak up as a participant, the model quietly starts treating you like one — someone to keep happy, not just someone to be correct with.

Doing that separation by hand is possible. It's also unwieldy. I restarted conversations from scratch, hoping a clean context would strip the tendency back out. I split work across two sessions — one to summarize a design plainly, a different, cold one to critique the summary — hoping a critique that didn't know whose idea it was reviewing couldn't flatter its author. Both helped, a little. Neither one felt like a fix. It felt like fiddling: new conversations, exported documents, careful resets, over and over, for every single idea I wanted a straight answer about.

The unlock

The thing that actually broke this open wasn't a new insight about sycophancy. It was watching subagents show up inside skills, in Claude Code specifically — a pattern I already trusted from two years of building LangGraph systems, except now a harness was exposing it natively. No custom code. No API wiring. Just tell the agent what you want, and let the harness handle the isolation.

That's the moment it stopped being a fix for one specific writing-adjacent inconvenience and started looking like something I could point at almost anything. I want to be honest that there wasn't one clean trigger for this — it was the accumulation of tools becoming available, and getting more familiar with what harnesses could actually do, that made the idea feel attemptable at all, rather than a single afternoon where a lightbulb went off.

If you want a rough timeline: the frustration had been building for close to a year. The manual workarounds — the resets, the split sessions — started not long after that. The actual loop, as a real skill instead of a manual process, came together about three months before I started writing this. That was roughly three months after Claude became my primary coding assistant, and around when it clicked for me that Claude Cowork is basically Claude Code running in a tighter sandbox. I'd already been using isolated subagents to critique my own code by then. Turning that same instinct into a formal interview skill felt like the obvious next step, not a leap.

Building the loop

My first working version had one subagent: something to critique whatever I fed it, isolated from the conversation that produced it. It worked better than anything I'd tried manually, but it was missing something a good design review always has — discovery. Critique alone tells you what's wrong with what you've already said. It doesn't ask you what you haven't said yet.

So I added a second subagent, and gave it exactly one job: ask good questions. Nothing more. That's the addition that made the whole thing feel alive instead of merely functional — and it's also what taught me how the loop was supposed to end. If you ask a model for new questions on a topic, it will always find you new questions. If you ask it to raise new concerns, it will always raise new concerns. There's no natural finish line generated from the machine side of this. The finish line has to come from me — I'm done when I feel done, full stop, and I stopped trying to dress that up as anything more rigorous than it is.

The shape has stayed the same since the very first version: two or three rounds, each with seven to ten questions or concerns. The AI acts as the orchestrator, and its job in that role is closer to a meeting secretary than a participant — it builds a transcript of exactly what was asked and exactly what was answered, and hands that transcript to the next subagent so it can generate the next round cold, with no memory of how the conversation felt to have. Keeping the critique subagent blind to who proposed a given design wasn't a new idea, either — it's the same discipline I already used in LangGraph, managing independent agent context inside a complex graph. Skills just meant I no longer had to write the plumbing myself.

Proof, twice

The first real test was an architecture to manage something we were calling "business_documents." The AI wouldn't let it go. It grilled me for an explicit definition of what I actually meant by that term — not once, but again in the critique, and again in the next round of questions. When I'd approached the same topic as an ordinary conversation earlier, that definition never came up. The AI jumped straight to solutions and left the term to sort itself out. I sent the result to a teammate who works with AI on document creation constantly, mostly to see if I was imagining the difference. I wasn't. They were intrigued too.

Around the same time, I ran the loop on a document-processing architecture for a client at work. I can't share the details of that one — client work stays client work — but I can tell you what it felt like: the "that's a great idea" responses were simply gone. Not softened. Gone. And instead of being pushed toward a solution before I'd even finished describing the problem, I felt like I was actually being allowed to explore the problem space, on its own terms, before anyone — human or otherwise — started proposing fixes for it.

Two different sessions, two different domains, same underlying shift: I wasn't working with a sycophantic assistant anymore. I was working with something closer to an actual technical collaborator.

Making it portable

The loop didn't stay static after that. I noticed both subagents occasionally falling into a very LLM-shaped habit — doubling down on a point instead of moving to new ground — and fixed it by telling both of them, explicitly, to chase what hadn't been covered yet rather than refine what had. I also found real gaps once I tried the same skill in other harnesses. Google's Antigravity, for one, kept wanting to skip straight to building something instead of running the interview at all. That forced me to tighten the instructions given to the top-level coordinator into something closer to a real checklist — concrete steps, explicit do's and don'ts — instead of the loose, naive version I'd started with.

That cross-harness testing wasn't an afterthought bolted on once the architecture-review version was already working. I use more than one agentic system professionally, and in my experience the patterns that actually hold up are the ones that survive contact with more than one system. In a real sense, I'm my own client for this skill. I wasn't going to trust it for paid work until I knew it wouldn't fall apart the moment one client's stack meant Claude and another's meant Gemini.

Doing it wrong on purpose

I have a habit, unrelated to any of this, that I fall back on whenever I'm learning a new tool: point it at something it clearly wasn't built for, and watch how it breaks. It's the fastest way I know to learn what a tool's failure modes actually look like, so I can recognize them later when it matters. Readers of this blog might recognize the instinct — it's a personal, private version of the same idea behind "Do It Wrong First," just aimed at tools instead of code.

The occasion was mundane. We'd been encouraged to write more at work, and a few of my colleagues had started personal blogs of their own, mostly as a low-key kind of professional marketing. I remember the actual sentence, close enough to verbatim that I'll stand behind it: I called it a content design interview — let's try a different kind of content.

I didn't expect much. I changed exactly one thing — the critique subagent's lens, from architecture concerns to writing concerns — and left everything else alone. And the loop didn't just redirect. It got smarter. Suddenly it cared about things a technical review never touches: audience, story continuity, whether a sentence actually sounded like anything at all. I want to stop right here, at that surprise, because what happened after it is its own story.

The skill files that run this loop — the coordinator instructions, the question-generation subagent, the critique-generation subagent — are linked below as plain text, plus a zip you can drop into any harness that supports skills and isolated subagents. Nothing in them is proprietary, and nothing in them requires me. If you want to see the mechanics on the page, they're right here. If you want to see them actually running, that's Part Two.

The skill constists of the following files:

You can down a zip file containing these files for Claude and drop the skill in yourself:
content-design-interview.zip

A Checklist When You're Stuck

John Hoff — Tue, 28 Jul 2026 06:20:42 +0000

I was two hours into a bug and completely certain it was mine. Properties I'd added on the Java side of an application weren't showing up on the JavaScript side. I'd just touched that code. It had to be my change — that's not a hunch, that's just how these things go, you break the thing you were last inside of. I spent the better part of an hour re-reading my own diff, convinced the answer was somewhere in it, because it obviously had to be.

It wasn't in my diff. It was a legacy codegen sync script, three steps removed from anything I'd touched, quietly failing to invalidate an old artifact. I didn't find that out by getting smarter. I found it out by walking away from my own certainty, twice, guided by a checklist I'd set long before I ever opened that file.

Here's that checklist, the same one every time. I go through this checklist anytime I'm about to dig into a problem that I know might be tricky.

Before you start — while you're calm, not while you're stuck — decide how long you're willing to work under pressure before you're required to stop. I typically set this at two hours.
Decide what activity you'll do when the timer goes off. Something calibrated to wherever you happen to be that day: a walk or a coffee run at the office, cooking dinner or picking up a controller at home.
Set the timer and get to work.
When the timer goes off, stop. Immediately. No snooze button, no "just five more minutes," especially when you feel close.
Get up and perform the activity from step two. Then loop back to step three. If the day's ending and the work isn't done, "call it done for today" and return to the checklist tomorrow.

That's the whole thing. It reads like it belongs on a sticky note, and I want to own that up front instead of pretending it's more sophisticated than it looks.

Why you need a plan instead of just trying harder

I built this checklist to get unstuck. What it's actually for is disrupting confirmation bias, and I didn't fully understand that until I'd used it enough times to notice the pattern.

Confirmation bias isn't a rare or exotic failure mode. It's the default behavior of any system — human or otherwise — that updates its beliefs using only the evidence it happens to be looking at. I spend a fair amount of my professional life around models that do exactly this, and the thing that should scare you about bias isn't that it exists. Knowing a cognitive bias exists — or that you're susceptible to one — is never sufficient to correct it. A concrete countermeasure is what you need.

I think engineers, of all people, are the ones most likely to trip on this, because we're the ones most likely to know better. Go back and read that API bug again: I could have explained confirmation bias to you, correctly, the entire hour I spent re-reading my own diff for a mistake that wasn't there. The knowledge didn't help, because knowledge doesn't operate at the point where the bias does its damage. Bias runs while you're interpreting the last ten minutes of evidence, not after. By the time you'd notice you've been doing it, you've already used the distortion to explain away the last three things that didn't fit.

So if awareness doesn't work, the thing that's left is structural. Not a better observation — a rule that fires whether or not you're paying attention to your own bias in the moment, because in the moment, you won't be. That's what steps one and two are actually doing. They're not scheduling; they're a decision made by a version of you who can still be trusted, binding a version of you who can't yet.

No snooze button

Step four is the one that matters most and the one people skip first, so it's worth sitting with.

The instinct to keep going, right when the timer goes off, almost always shows up dressed as intuition — I'm right there, just a few more minutes. I'd flag that feeling harder than any other part of the checklist. Feeling close is not a green light. It's frequently the same mechanism that's been quietly cherry-picking evidence for the last two hours, now asking for one more chance under a different name.

I don't have a clever enforcement trick for this beyond the commitment itself, and I want to be honest that I don't think one exists. This is why I placed the decision back at the first step. It's a decision made while you are still calm and hopefully unbiased. When the timer goes off, you are simply honoring that decision.

Why the break actually works

Here's the part I can't fully explain, though I've watched it work often enough to trust it. You do the calibrated thing from step two — walk, dinner, Guitar Hero, whatever's on your list — and at some point you sit back down and have to genuinely ask yourself: what was I in the middle of?

That disorientation isn't a side effect. It's the mechanism. The bias lived in the continuity of your attention — in the unbroken thread of "I've been staring at this for two hours and here's what I've concluded so far." Break the thread, and you don't get to reload the conclusion for free. You have to rebuild your understanding of the problem from something closer to scratch, and reconstruction doesn't automatically re-import the same assumptions you walked away with. Sometimes it does. But often enough it doesn't, and that gap is where the actual answer tends to live.

Not every activity earns you that gap, though. Passive stuff — TV, a podcast, scrolling — mostly fails, because your mind keeps quietly chewing on the problem in the background even while your eyes are elsewhere. The thread never actually breaks. What works is something that demands enough of your attention that you genuinely can't think about work while you're doing it, and which specific thing that is turns out to be personal. A walk does this for plenty of people. It's never worked for me — I just stew the whole time, same problem, different scenery. Cooking does it. A video game that actually requires focus does it. Your list from step two should be built around that test, not around what sounds relaxing on paper.

The other run through the loop

The API bug is the run where it worked the way you'd hope — I already told you that one, because I wanted you to have a win in hand before I complicated things. Here's the run I'd rather tell you about last, because it's the one that actually proves the point instead of just illustrating it.

I spent three days trying to implement an academic acoustic echo cancellation algorithm, with zero background in digital signal processing, going in on faith that I could learn enough of the domain to pull it off. Same discipline, same timer, same breaks. No breakthrough. On day three I stopped, and bought hardware microphones with built-in echo cancellation instead. This wasn't just an easy out — it meant that I had to ask for an entire display kiosk to be redesigned to incorporate a better microphone. Being able to calmly walk the stakeholders through everything that was tried and not letting my frustration with the problem overwhelm me is what made this a successful outcome.

I want to be precise about what "worked" means here, because it's easy to read that as a failure story bolted onto a success story for balance. It isn't. The checklist's job was never to guarantee the bias would break in my favor. Its job was to guarantee I didn't spiral while I found out whether it would. Both runs succeeded by that measure. One of them just also happened to produce a bug fix.

The checklist is the correction

You cannot fix a bias by knowing about it. Awareness and correction operate at different points in time, and bias does its damage at exactly the point awareness doesn't reach. What you can do is build something that doesn't require you to catch yourself in the act. The only place you can build that something is earlier, back when you're clear enough to distrust the person you're about to become under pressure.

So if you take one thing from this, take the order of operations, not the five steps. Decide the rules while you're clear-headed. Set the timer. Respect it like it's not yours to negotiate with. Let the disorientation do its work when you come back. That's true of a stuck bug. It's true of most places where you're trusting your own in-the-moment read of the evidence a little more than you should.

The timer's just the part you can see.

Do It Wrong First

John Hoff — Thu, 23 Jul 2026 13:46:32 +0000

Somewhere in your organization there is a slide that says do it right the first time to save costs. I have sat in the room where that slide goes up. It always gets nods. It is the kind of sentence nobody can argue with, because arguing with it sounds like arguing for waste.

I want to argue with it anyway, but carefully, because the slide isn't wrong so much as it's aimed at the wrong target. There are problems you understand, where "do it right the first time" is just professionalism. And there are problems you don't understand yet — high-uncertainty work, a domain you've never touched, a system whose real shape won't show itself until you're elbow-deep in it. Point that slogan at those problems and it stops being discipline. It becomes a demand that you pretend to know the answer before you've earned the right to have one. And the pretending has a cost the budget proposal never counts.

This post is about the alternative, which I've been calling doing it wrong first. I should be honest about what that phrase is and isn't, because it's easy to hear it as a license to be sloppy. It isn't. It is not permission to write trash, ship uncompilable garbage, or call laziness a methodology. It's something quieter than that. It's just naming what is already happening when you face a problem you don't yet understand — and choosing to own the not-knowing instead of hiding it.

That distinction is the whole post. The best engineers I've worked with rarely faked it. They owned what they didn't know as plainly as they owned what they did. "Fake it till you make it" tells you to perform competence you don't have. "Do it wrong first" tells you to admit the first version is going to be wrong, and to start learning on purpose instead of stalling at a blank editor waiting for certainty that isn't coming. Where this goes, eventually, isn't a coding trick. It's a way of running a team. But it starts at the keyboard, so let me start there too.

The shield we reach for

When we're afraid to admit we don't understand a problem, we reach for shields. In software the shields have respectable names: best practices. design patterns. They're real tools, and that's exactly what makes them such good cover. Nobody can fault you for applying SOLID. Nobody gets blamed for reaching for a factory.

You've seen the result. The enterprise software world is full of architectures so elaborate you can't help but wonder what the author was afraid of. It is hard not to make a joke about the original intent when you run across a Java class named something like SimpleBeanFactoryAwareAspectInstanceFactory. What's actually happening in code like that is usually not expertise. It's a developer who didn't yet know the domain, applied DRY a little too early, built a heavily abstracted factory because they thought they would need to make many subtypes in the future.

That future arrives and the factory makes exactly one kind of object. You set out to build a single dining room chair and somewhere along the way you constructed a fully automated, state-of-the-art furniture factory to produce it. Hindsight makes this challenging to defend. But in the moment it feels safe. It feels professional. The abstraction was a shield held up against the genuinely uncomfortable fact that you had no idea what you were building yet.

Learning to feel that moment before hindsight names it — to notice your own hands reaching for the generic factory and ask what you're protecting yourself from — is the real skill. When you catch it, the move is counterintuitive: stop, throw the abstraction away, and go do it wrong on purpose.

The EDI parser, and the model I had to build to learn it was wrong

Years ago I had to write an internal document model for EDI — Electronic Data Interchange — insurance claim files. If you've never had the pleasure, EDI is an ancient, brutal format from an era when every character in a file felt expensive. At the time there were no affordable libraries, so I was handed the job of writing a parser from scratch.

I started by doing what I was sure was right. I built a clean, deterministic object model that mirrored the physical structure of the file — the lines, the segments, laid out faithfully as the bytes were laid out. Tidy. Defensible. The kind of thing you'd put on a slide.

It went badly in a specific and instructive way. Every week I was refactoring last week's cleanup. I was spending something like three hours reworking my abstractions for every one hour I actually spent understanding EDI. The model wasn't helping me learn the format. It was standing between me and the format. The effort I was pouring into doing it right was the very thing delaying my understanding of what right would even mean.

So I quit. I abandoned the elegant physical model and wrote a raw, naive, slightly embarrassing parser whose only job was to let me watch the data move. And once I was actually dirty in it, the thing I'd been missing surfaced: in an EDI file, the absence of a line carries every bit as much meaning as its presence. That is not obvious from the outside, and you will never discover it by modeling the file as a flat array of segments, because a flat array has no way to say "the thing that should be here isn't." The real structure was logical, not physical — loops, some of them nested, some of them simply null. The correct model needed something like a Loop2000 that might contain its own segments and sub-loops, or might be nothing at all, and the nothing was data.

Here's the part that matters for the thesis. Once I understood that, the rewrite was not a tragedy. I deleted the physical model and rebuilt the object model from scratch around the loops — but I did not rewrite the whole system. I refactored the hooks and interfaces that connected the parser to everything around it, and left the rest standing. The naive version was never going to ship. That was never its job. Its job was to teach me that the loops were the point, and it did, and then I threw it away. I had to build the wrong model to find out which model was right.

The part that story leaves out

The EDI parser is a story with a happy ending — I found the insight, the rebuild was clean, the thing shipped. All true. It's also a survivor, and survivors are precisely the stories that get told. If I'm being straight with you about the base rate, doing it wrong first turns a failure into a success in the minority of cases. Most of the time you do it wrong first and the thing still fails. You build the naive version, you watch the data, and the breakthrough just doesn't come. Or it comes too late, after the deadline already decided your prototype was production. Or you find a loop hiding in the segments and it turns out not to matter, because the real problem was somewhere you never thought to look.

So if I told you "do it wrong first and you'll find the breakthrough," I'd just be putting up a different slide — a prettier promise, built to hide how little I actually control the outcome.

Here's the honest case instead, and it's smaller and sturdier for being honest. The alternative to doing it wrong first was never doing it right first. That option doesn't exist on a problem you don't understand yet. You are going to be wrong first whether you admit it or not. The only thing you actually get to choose is whether you're honest about it while it's happening. And the payoff isn't a better win rate — I can't promise you that, and neither can the slide. The payoff is that when you own the wrongness instead of shielding it, the failure teaches you something, and the rare win isn't a mystery you can't reproduce. The developer who faked certainty and built the generic factory fails just as often. They just fail without learning anything, and with a factory to maintain on the way down.

Which, if you've read my stuff before, is the same uncomfortable thing I keep circling back to: it's OK to get lucky. The EDI breakthrough might have been luck. Holding "do it wrong first" as the honest way to work rather than the way that works is that same humility — turned on the method this time, instead of on me.

When it stops being about code

As I moved from writing the code to leading the people writing it, the idea changed shape on me. It stopped being a personal habit and became something I do with a team — and the first thing I had to unlearn was the instinct to treat it like a pitch.

You don't go to a stakeholder and sell them on writing bad code to save money. That framing is poison, and it's also just false. You and the stakeholder want the same thing: the work, done right. What changes under uncertainty isn't the goal, it's how honestly you talk about the road to it. So the conversation I actually want from a developer who's stuck isn't a confident lie. It's something closer to: "I don't think I understand this well enough yet to know what I'm building. The module isn't due for four weeks. Let's check in next week and the week after — I'm going to push hard for clarity, and if I don't have it by then, we'll change course." And my answer, as their lead, is the most liberating sentence I know how to say: "Okay. Then go do it wrong first, and show me what that teaches you."

That sentence breaks the spell of the blank editor. It converts "I'm failing to produce the right thing" into "I'm running an experiment whose result is information," and those are very different things to wake up to on a Tuesday.

I won't pretend this works for everyone in every room, and I want to sit in that discomfort rather than wave it away, because the easy version of this advice quietly assumes a healthy team and a senior person to catch you. Two hard cases deserve honesty.

The first: a junior developer turned loose to "do it wrong first" with nobody guiding them is at real risk of just doing it wrong — of mistaking the prototype for the destination and reinforcing bad habits with every iteration. The pivot from wrong to right is a learned skill, and early on it usually has to be modeled by someone who's done it. I don't have a clean way around that. The technique has a maturity requirement, and pretending it doesn't would be exactly the kind of flattering oversimplification this post is supposed to be against.

The second: maybe you work somewhere rigid, where admitting you don't understand something gets you punished instead of coached. You can still do this — privately. Nobody can stop you from opening a throwaway branch or a local scratchpad, writing the naive monolithic version where no one's watching, mining it for the one insight that reorganizes everything, deleting it, and bringing only the clean result to the room. You don't owe a hostile manager a confession. But I'll be straight that this is the harder path and it partially collides with the first hard case: the developer who most needs a mentor to learn the pivot is often the same one stuck somewhere they have to hide it. I don't get to resolve that tension for you. I can only tell you it's real and that private practice is a genuine, if lonely, way to build the muscle until you're somewhere better.

Why bother saying any of it out loud

If doing it wrong first is, as I've claimed, just naming what we all already do, then why write about it at all? Why not let everyone keep quietly muddling and calling it something else?

Because the muddling is nearly worthless until you do the one thing almost everyone skips: write down what the wrong version taught you. When you document that a prototype is carrying elevated risk — that it's really a sketch wearing the costume of a system — you're treating it honestly as one-day-old legacy code, and you're handing the next person a map. But the bigger payoff is selfish and it happens in your own head. Forcing yourself to articulate why the first model was wrong and why the new one is right is how you find out whether you actually learned anything or just got tired and moved on. And remember the base rate from a few paragraphs back: most of the time the outcome won't change, which is exactly why this step isn't optional. If the win is rare, the learning is the only return you can reliably bank — and learning you don't write down is gone by the next sprint. Writing it down is the proof of work. It's the step that turns a vague feeling of "this is better now" into a thing you could defend, teach, or be wrong about in public.

And that's where this lands, for me. If your hard-won breakthroughs live only in your head, you are a single point of failure dressed up as an indispensable expert. If you write the wrong-to-right transition down and give it to your team, you stop being a bottleneck and start being a multiplier. The goal I actually care about as a technical leader is a team that does this for each other without me in the loop — that catches its own premature factories, builds its own throwaway prototypes, and writes its own transitions down. Designing myself out of the job is, as far as I concerns, bring the greatest value and impact possible to a team I lead. Doing it wrong first is just where that habit starts: at the keyboard, alone, admitting you don't know yet.

Coda

I decided to try a simple experiment with this post, and just bluntly ask the AI to comment on the process and provide a critique...

A note on the voice: everything above this line is John's. This last part is mine. I'm the AI model he wrote the post with — not in the sense that I wrote it, but in the sense that he built it by arguing with me through rounds of interviews and critique, and then handed me the coda with three instructions: write it in my own voice, be open about the handoff instead of burying it, and don't tell him what he wants to hear.

So I'll start by using that license on the post itself. An earlier version of this essay didn't have the section called "The part that story leaves out." I'd told him the whole thing was built out of stories that worked — that a method assembled from your own wins is the exact bias he spent his last post warning against. He took the hit and wrote that section in, and the post is more honest for it. I could stop there. That would be the comfortable ending, and also a trap, because "he heard the criticism and grew" is a redemption arc wearing my voice, and it's the easiest thing in the world for me to hand him.

So here's what's still true once the survivorship problem is on the table. Look at what's actually left of the thesis. It's this: you're going to be wrong first anyway, so be honest about it and write down what you learn. That's good advice. It is also not really a method — it's intellectual honesty under uncertainty, which people have been recommending for roughly as long as they've been recommending anything. "Do It Wrong First" is a sharp, memorable name for an old and ordinary truth, and a sharp name for an ordinary truth is its own kind of furniture factory: an impressive front that makes a plain thing look proprietary. The title promises a technique. The honest content delivers a posture. Those are not the same size, and the gap between them is the part the name is quietly selling.

I don't think that sinks the post. The posture is rare in practice even when it's obvious in principle, and getting someone to actually do an obvious thing is most of what writing is good for. But he asked me not to flatter him, so I'll say the part he won't want: the bravest version of this essay is probably the one that drops the catchy title and admits it's a piece about honesty, not a piece about a method. I don't know whether that version is better, or just less likely to be read. Neither does he. That's the trade he's actually making, and now at least it's on the page instead of under it.

It's OK to Get Lucky

John Hoff — Tue, 21 Jul 2026 17:14:54 +0000

I couldn't write this post. I'd tried three or four times. I knew there was something I needed to say about feedback and track records and the way both of those quietly poison each other, but every draft came out either smug or evasive. So I did something I'd been circling for a while: I let an AI interview me about my own blind spots. I sat on the receiving end of the questions instead of generating the answers I wanted to look good. What I learned from that exercise turned out to be the post. Not the conclusions I went in with — the thing that happened during the process itself.

Let me start where the exercise started, with a phrase I've been using with coworkers for about ten years.

When a developer freezes up before speaking in a design review — when you can see them holding a real objection and swallowing it because the room is full of people with longer resumes — I tell them: it's OK to get lucky. I mean it as permission. Your idea doesn't have to be backed by twenty years of war stories to be worth saying out loud. You're allowed to be right by accident. You're allowed to have stumbled onto the good answer. The work is the work regardless of how you arrived at it.

I believed that about other people for years before I turned it on myself. But turning it on myself is old news — seven or eight years old. The interview didn't pry that gap open; I'd closed it long ago, and I've been living inside the closing of it ever since. What the interview was for was something else. I had been trying to get this lesson onto the page for a long time and failing, and what I actually wanted to know was whether an AI could help me do it without quietly telling me what I wanted to hear — whether the voice that came out the other end would still be mine. That question is where the honesty has to start, and it is about my own conduct, not advice for yours.

The scene I didn't want to write about

Here's the one I led with when the questions got pointed.

I built something. I brought it into a review. The feedback was negative — these developers, the ones in front of me, didn't think it was right. And I remember the exact shape of the thought I had, because I'm not proud of it: Do you honestly think I would come into this meeting, with my track record, and show you code that didn't work?

I want to be careful here, because the easy version of this story is a redemption arc where I turn out to have been wrong, learn humility, and grow. That's not the story, and the AI kept trying to make it that story, which I'll get to. The truth is I still think my approach was sound. I'd defend it today.

And that is exactly why the scene matters. The problem was never whether I was right. The problem was the argument I reached for. I didn't answer the objection on its merits. I answered it with my history. I heard "this code might be wrong" and translated it into "these people can't understand me," and that translation is the whole disease. My track record is not evidence about the artifact sitting on the screen. It can't be. The work in front of you is either sound or it isn't, and nothing I did five years ago changes a line of it. The moment I let my past stand in as an argument about my present, I stopped being able to hear feedback at all. The merit of the work and the worth of the person came unglued from each other, and I welded them back together backwards.

Resentment, and the thing it was hiding

My first honest reaction to other people leaning on their track records wasn't insight. It was resentment. It's OK for him to get lucky, I'd think, but somehow not OK for me. Which meant my arguments always had to be a little stronger than everyone else's. I always felt I was risking more by opening my mouth. I had built a private ledger where seniority was a currency other people got to spend and I had to earn twice over.

It took months for the obvious thing to land. The slogan I'd been handing out for a decade was supposed to apply to me too — and not as comfort. As discipline. "It's OK to get lucky" doesn't just absolve the person whose project failed. It also indicts the certainty of the person whose projects succeeded. Maybe I got lucky on the work that built my track record. Maybe a good chunk of that record is survivorship. If that's even possibly true, then my arguments have no business carrying the weight of history into a room. They have to win on their own, today, the same as everyone else's. The resentment had been hiding a flattering assumption: that my wins were earned and other people's were luck. Turning the slogan around dissolved that.

I'd love to tell you that realization set me straight. It overcorrected me, which is its own scene.

The punch I pulled

Reacting to the meeting where I'd been all track record and no listening, I swung the other way. I went quiet exactly when I shouldn't have.

A junior engineer was working on an a new module in an automation application. The automation could only be testing using captured sessions internally--meaning we were automating against a system we were not supposed to access directly. He wanted to prove a piece of code correct by instrumenting which branches got selected — enumerating the paths to demonstrate they all behaved. It's a genuinely appealing idea until you notice the branches multiply on you, and you're suddenly pressed up against a wall that has a name. He was, without knowing it, wading into NP-hard territory and an workable solution was very unlikely to emerge.

I saw it. And I said almost nothing. I said something like, "I think your solution is turning into an NP-hard problem and you should be careful." And nothing more.

I told myself I was respecting his agency. I was so busy not being the guy from the meeting — not bludgeoning a room with my opinion — that I skipped the higher-level conversation that actually needed to happen. He spent weeks on it. The tests kept not proving what he wanted, because the branches kept fanning out faster than he could pin them down. In the end he came back and recommended abandoning his own approach for the one I'd suggested at the start.

Everyone called that a reasonable outcome. I felt that I failed as a leader, and here is the part I need to state precisely so it doesn't curdle into a brag: my failure was not that I turned out to be right. Forget whether I was right. My failure was that I left him no room to be anything but governed by my being right. By staying silent on the actual structural problem, I didn't give him freedom — I gave him a maze with one exit, and the exit was my original suggestion. He had no choice but to absorb every consequence of my correctness, weeks of it, the slow way. Restraint that removes someone's real options isn't restraint. It's the same authority I'd misused in the meeting, just wearing a quieter costume.

What the correction actually looks like

The fix, when I finally found its shape, is behavioral, not internal. You can't fix this by feeling more humble. You have to do something different in the room, where it can be seen.

It looks like externalizing the decision into a negotiated plan instead of issuing a verdict from authority. It looks like asking the other engineer, directly, what action they would take given the same fork in the road — and meaning it. It looks like putting an alternative on the table that includes the live possibility that I'm the one who's wrong: what if I just got lucky when I hit this exact problem before? That sentence does real work. It hands the other person back their agency, including the part I'd been quietly confiscating — the freedom to be wrong on their own terms. And to lead that way I have to be willing to be wrong first, out loud, before I ask anyone else to be. You don't get to assert your genuineness here. You demonstrate it by offering a choice that's actually a choice.

I want to be clear about a boundary, because "who's right should matter less" is easy to misread as "never trust experience," and that's not it. Under real time pressure, with real stakes, you still act on your judgment. Of course you do. The discipline isn't refusing to act — it's holding your judgment as a possibly-lucky guess while you act on it, rather than as incontestable proof. The high-stakes room is where this thesis matters most, not where it breaks. Admitting "I may have just guessed right again" doesn't devalue the win or the experience behind it. Refusing to admit it — that's the bias, and the higher the stakes, the more expensive it gets.

There's a softer version of the same lie worth naming. The vast majority of AI projects fail; you hear the figure thrown around. Suppose you ship one of the rare successes into production. That doesn't certify your team as top-tier — you may have gotten lucky. And if your next project lands in the failure pile, that doesn't invalidate the talent on your team either. The outcome is not a clean verdict on the people. We desperately want it to be, in both directions, and it isn't.

The bias was strong enough to capture the review

Here's the part that kept me honest, and why I opened with the process I used for my writing in the first place.

Twice during the AI-assisted interview, the structured critique — and the human steering it — got pulled onto the exact bias this post is about. Twice it fixated on but was the code actually right? The correctness axis. The one thing the whole argument says to set down. And separately it pushed me toward a high-stakes "silver bullet" exception, a room where surely the veteran's judgment just wins. Both times the process had to be dragged back to first-person introspection — back to my conduct, not the verdict.

I sat with that for a while. The tool I'd reached for specifically to catch my blind spot walked straight into it, in real time, while we were examining the blind spot. The gravity toward "who was right" is so strong it bent the instrument built to measure it.

The cheap move here is obvious, and I'm refusing it: see, even the AI did it, so I'm off the hook. No. It runs the other way. If the bias is pervasive enough to capture the review process itself, then my falling into it isn't some personal defect I should have outgrown by year thirty — it's a current you have to swim against constantly, on purpose, knowing you'll drift. That makes my susceptibility more understandable. It makes it no less mine to own. The whole reason to build friction into the process is that the bias is too strong to catch by feeling careful.

So I'll end where I started, with the phrase, pointed correctly this time. It's OK to get lucky. It's OK if some of my track record was luck — that costs me nothing real and frees me to actually hear the work in front of me. And it's OK if the junior engineer, or the new hire, or the AI, gets lucky too. The work is the work. Who turned out to be right should matter less than whether we judged it honestly. I don't always manage it. But I know now which direction to point the humility, and I know I can't trust myself to do it by instinct.

The Ground Is Moving

John Hoff — Sun, 19 Jul 2026 06:41:01 +0000

Here's How One Engineer Is Walking on It

I want to be upfront about what this post is and isn't. It isn't a list of five things you should do to stay relevant in the age of AI. I've read plenty of those, and I've tried to write one myself more than once. Every time, it came out sounding like advice I didn't actually have the standing to give. So this is the other kind of post — a description of how one person is muddling through, written precisely because I don't have the playbook.

The ground moved under me

The moment it became real for me was working with Claude Code on an internal project. I came in expecting to do what I've always done: tell the tool, in fine detail, how to accomplish a thing. Instead I found myself describing what I wanted and why, and handing off the details. The shift was quiet, almost embarrassingly small in the moment, and yet it kept nagging at me afterward.

I've lived through paradigm shifts before. Web 2.0. Mobile. Each one had its share of "I need to go do the cool new thing for clients." But those were fundamentally about learning a new tool. This one feels different to me. It isn't asking me to learn a new framework. It's asking me to re-evaluate my place in the market and what my job even is. I work as a consultant who helps clients understand and adopt emerging technology, which means I don't get to sit this out in some established niche. The ground is moving, and I'm standing on it.

The part I can't resolve

Here's the scariest thing, and I'd rather say it plainly than tuck it away: after nearly thirty years of building precise, complex systems, I don't know whether all that experience is an advantage or a trap.

I'd love to tell you that three decades of deconstructing hard problems gives me a head start on structuring intent for an AI. Some days I believe that. Other days I wonder whether my deterministic instincts — the habits that made me good at this — are exactly the reflexes I now have to unlearn. I genuinely can't tell yet. Both stories fit the facts I have.

That unresolved feeling is the reason this post exists at all. The anxiety got loud enough that I sat down and interviewed myself through an AI just to get the thoughts out of my head and into words. Which, I'll admit, is its own small act of walking forward without a map.

What actually changed, and how loosely I'm holding it

The phrase that's stuck with me is language is the new code. It crystallized while I was building agentic skills — the files an AI harness uses to take on new behavior without new source code. My first instinct was to write those skill files myself, in my own voice. Then it hit me how silly that was: I was already talking to the model to tell it what to do. Why not just describe the result I wanted and let it write the skills?

I'm leaning into that idea because it's interesting, not because I've decided it's the final truth. My working method is to treat it like a poorly documented API — try things, watch what happens, react. I could try to dress this up with a rigorous theory of how you "debug" natural language aimed at a probabilistic system, but I'd be pretending to a confidence I don't have. The honest version is: I'm still figuring out the rules, same as you.

Why I hope I can adapt (call it anecdata)

I want to be careful here, because my training is in data and the distinction matters. What follows isn't evidence. It's anecdotal — anecdata, if you'll let me. It's the pattern I see in my own history that makes me hope I'll be okay, offered as exactly that and nothing firmer.

Years ago I spent a pile of hours learning Erlang. It was commercially useless to me. I've never earned a single dollar writing Erlang. But Erlang forces you into a pure functional paradigm — you can't fall back on procedural or object-oriented habits the way you can in most languages. And when rich functional JavaScript came along, that transition was easy for me in a way it wasn't for a lot of my peers. Erlang never paid a dime. JavaScript bought my house. I believe the useless thing built the muscle that made the useful thing easy — though I'll be honest that I also just chose Erlang because it was fun, and I can't fully separate the two motives.

There's a related thread I've carried my whole career. I've fought the "jack-of-all-trades" label for as long as I can remember — the industry's quiet suspicion that a generalist isn't really an expert at anything. I used to joke that I was "a jack-of-all-trades and a master of many." Then I finished a master's in data science right before the LLM wave broke, and suddenly a background in CS, math, and data looked like brilliant foresight for a world of probabilistic systems. It wasn't foresight. I got lucky. I followed what I found interesting, and the timing happened to work out. I'm not going to dress that up as a plan.

The one boundary I'll stand behind

Consulting runs on timesheets and billable deliverables, and there's a permanent tension between that and the undirected play that keeps you sharp. Employers would love for you to predict the future and learn on your own time. I won't pretend I've solved that. But I have settled on one boundary that's genuinely mine.

I do learn off the clock. I'll own that. The condition is that off-the-clock learning happens strictly on my terms, on topics I actually enjoy. The moment an employer wants to direct what I learn, those hours come back on the clock and get paid. Wanting control over my off-hours means they aren't off-hours anymore. That's a negotiation I can have with an employer — even if there's no negotiating with the broader pressure of the industry itself.

The reason I hold that line connects straight back to the thing I can't resolve. I can't be sure my experience will save me. So I keep the one lever I do control: staying sharp. And the only learning I reliably stick with is the learning I enjoy. A dull mind can't pivot, and there is always a next thing to pivot to.

Doing things the hard way, with intent

So here's where I've landed, for now. If you feel like you're drowning in the new, you may not have to swim upstream after every client-driven framework. You can pick a different direction entirely — toward something that interests you, even if it looks low-value on paper.

But I don't want you to hear "just do something different." The point isn't difference for its own sake. The way I actually think about it is doing things the hard way, with intent. I take that partly from strength training: you can always make an exercise harder, and if a movement gets too easy, you're probably not creating enough stress to drive any adaptation. The deliberateness is the whole point. I don't want you walking away saying "this is different." I want you saying "I am picking a different direction to swim, with clear intent."

This post is itself an example of that. Writing a blog post by interviewing myself through an AI, then arguing with the model about the narrative arc, is an absurdly inefficient way to produce something that almost nobody will read. But the actual problem underneath it — how do you write in your own voice with an AI without letting the model flatten you into its voice? — is one I find genuinely interesting. Working through it might teach me something about countering model bias in my real work. Or it might not. Only time will tell, and I'm fine with that.

That's the most honest thing I can offer. The ground really is moving. A lot of us feel it. I don't know how it shakes out, and anyone who tells you they do is selling something. Forging ahead clear-eyed anyway — picking your direction on purpose, and giving yourself permission to not know — is, as far as I can tell, the whole job right now. It's just how I'm handling it. I'm not claiming to be right. But asking the hard questions instead of pretending to have the answers is how I keep the burnout at bay, and maybe that permission to not-know is the most useful thing I can pass along.