DEV Community: Daniel Stolf

Code Is the New Server. Specs Are the New Terraform.

Daniel Stolf — Fri, 19 Jun 2026 18:04:50 +0000

Most of us are committing the wrong artifact to version control.

When you commit code today, you're committing the output of a process. The thinking happened somewhere upstream (a Linear ticket, a Slack thread, a notebook, a conversation with the team), and what survived that thinking is the code in the repo. The spec, the trade-offs, the constraints you considered and rejected? Mostly gone. The code is the record. The repo is the source of truth.

We had this exact conversation a decade ago. It was about servers.

Before Terraform, the server WAS the truth. You'd SSH in, install packages, edit configuration files, restart services. Maybe you wrote a runbook. The runbook went stale. The server kept going. When something broke, you debugged the server, not the runbook. The version of truth was the running machine.

Then IaC happened. The .tf file became the source of truth. The server became the build artifact. You stopped backing up servers and started backing up .tf files. When a server drifted from the .tf, you didn't reach for the server to figure out what changed. You ran terraform plan, saw the diff, decided whether to update the .tf or re-apply. The server stopped being something you protected. It became something you could reliably regenerate.

We didn't stop having servers. We stopped versioning them.

AI changes the cost of regenerating code from a description of what it should do.

Not perfectly. Not in every case. But enough that, for a growing set of work, the implementation is cheaper to regenerate from intent than to maintain by hand. And once that's true, the implementation can move into the role the server played: the build artifact. The thing you keep around but don't treat as the canonical source.

What plays the role of the .tf file? The spec. Or whatever you want to call it: the acceptance criteria, the contract, the intent-doc. The thing that says what should be true about the behavior, captured in a form structured enough to regenerate the code from.

That's the thing worth versioning.

If intent is the source of truth, several things shift.

PRs stop being about code diffs and start being about intent diffs. The reviewer asks: did the spec change? does the new spec make sense? does the code match? The interesting conversation moves upstream of the implementation. The downstream check (code matches spec) gets cheaper, often automatable.

Tests come from the spec, not from the code. Acceptance criteria become failing tests before any implementation exists. The code has to satisfy them. This is what test-first really means once code becomes regenerable.

Drift, when it happens, gets repaired differently. If the code stops matching the spec, you don't reach into the code to fix it. You decide whether the spec was right (in which case, regenerate the code) or the spec was wrong (in which case, fix the spec, then regenerate). The same loop terraform plan teaches you.

History and memory attach to the spec. The reasons a decision was made, the alternatives considered, the failure mode that bit you in production last quarter, those belong with the spec they shaped. The code is downstream. It inherits those reasons but doesn't carry them.

The natural pushback is: but code is still code. You still have to read it, debug it, run it, pay for it in CPU cycles. It doesn't go away just because it's downstream.

Right. Servers don't go away under Terraform either. You still SSH in when things break. You still tail logs. You still get woken up at 2am. The change wasn't that the artifact disappeared. The change was where the source of truth lived.

Code in an intent-versioned workflow is the same. It stays in the repo. You can read it. You can debug it. You can ship it. But when you need to change WHAT the system does, you change the spec, and the code follows. When the code drifts from the spec, you investigate why, and the fix lands upstream more often than not. The shift isn't "no more code." The shift is "code answers to something now."

This is what Speccraft is built around.

Specs are first-class versioned artifacts in the repo, not throwaway prompts. Code generation flows from those specs through a structured process. Tests come from the spec before implementation, enforced by a hook that blocks production edits without a corresponding test edit. Decisions, conventions, and scars attach to specs and persist across sessions, so the next contributor (human or model) inherits the rationale, not just the diff.

It's the .tf file for code. The thing you version, the thing you review, the thing you treat as canonical. The code becomes what it should have been all along: the runnable expression of an intent the team agreed on.

What would your repo look like if the spec was the source of truth and the code was the build artifact? Most of us already half-believe this. We just haven't restructured anything to act on it yet.

Have you started treating intent as a versioned artifact in your workflow? Or is the spec still living in a ticket somewhere?

Terraform for AI generated code

Daniel Stolf — Mon, 15 Jun 2026 03:00:00 +0000

Over the past few weeks I've written about the workflows I developed building production systems with AI: spec-driven development, manual RAG, grounding models against hallucinations, the Terraform discipline of plan-before-apply, and what happens when you skip all of that and vibe-code your way into a mess you can't undo.

Every post in this series was pointing at the same thing.

Today I'm sharing it.

Introducing speccraft: a Claude Code plugin for spec-first, test-driven AI development.

The one-line description from the README says it better than I can:

Terraform for AI-generated code. Versioned intent. Auto-injected memory. Cross-model review. Hook-enforced TDD.

Here's what it actually does, in plain terms.

Every change starts with a spec. /spec:new interviews you Socratically (why, what, acceptance criteria_ and writes a versioned spec file directly into your repository. Not a chat artifact. Not a prompt you'll lose at the end of the session. A durable, committed, reviewable document that defines exactly what is about to change before implementation begins.

The spec is the contract. Committing it to the repo makes that contract real.

The spec gets reviewed before code is written. /spec:review routes the spec to auxiliary agents (Codex, Gemini, OpenCode, or a Claude subagent running a critic persona) in parallel, and synthesizes the verdicts. Ambiguity, missing edge cases, untestable criteria: these surface at the spec stage, not after the implementation is already tangled across six files.

Implementation is test-first. Not by convention. By enforcement.

A hook intercepts every edit to a production file. If no sibling test file was edited more recently in the same session, the edit is blocked.

Not warned. Blocked.

Now, I know what you're thinking. We've all tried the "make no mistakes" prompt. We've all been let down.

Turns out it does work, you just have to enforce it with a hook, not a prompt.

Speccraft is how "write tests first" stops being advice and starts being a hard constraint. The model cannot skip the RED step and go straight to GREEN. You can override it (one-time, with a reason that gets logged) but you can't ignore it. The discipline is architectural, not aspirational.

The repo carries its own memory. A small .speccraft/index.md (guardrails, architecture decisions, conventions) is auto-injected at every session start. The model always knows the rules of this codebase. Not because you remembered to paste them in. Because the plugin handles it.

Heavy work is delegated to auxiliary agents. /spec:delegate codex "Generate table-driven tests for internal/health/handler.go": Codex or other agent runs the task, speccraft synthesizes the result. The right model for the right job, orchestrated from a single workflow.

Everything in this series (the Kubernetes Operator built in two weeks, the manual RAG sessions, the hallucination loops, the race condition the model confidently couldn't fix, the Terraform analogy, the vibe-coding mess I couldn't undo), all of it was pointing at the same underlying problem.

AI-assisted development is genuinely powerful. But left undisciplined, it amplifies ambition faster than it delivers correctness. Specs drift from implementations. Memory resets between sessions. Reviews are single-model and single-perspective. Tests get written after the fact, if at all.

Speccraft is the workflow I needed to build the things I was building and couldn't find anywhere else.

So I built it.

It's open source. MIT licensed. Currently Go-first, with Python support already in and more languages on the roadmap.

If you've been building with Claude Code and hitting the same walls, link in the comments.

👉 github.com/DCSTOLF/speccraft

What would you add to a workflow like this? What's the discipline you keep having to re-enforce manually?

The Terraform hook

Daniel Stolf — Fri, 12 Jun 2026 15:00:00 +0000

The most dangerous moment in AI-assisted development isn’t when the model gets something wrong.

Like most things in life, it’s when everything is going right and you stop paying attention.

I’ve been building a complex software project using Claude Code and Codex as my primary implementation engine. The productivity is real. The output quality, when the workflow is disciplined, is genuinely impressive.

But a few times I fell into a different mode entirely.

Prompt after prompt, change after change, no spec, no checkpoints, no Git discipline. Just vibing: chasing a goal with whatever the model suggested, iterating fast, feeling productive.

Until something broke.

And when I went to undo it, I couldn't separate the good changes from the bad ones. They were tangled together across files, across sessions, across prompts I could no longer reconstruct (and even the ones I could would not have helped, because running them again wouldn't reproduce the same code).

I had to roll back work that was correct alongside work that wasn't, with no safe way to recover what I'd lost. I built the bad outcome myself. The model just helped me get there faster.

That experience forced me to design something I’d been resisting: a structured workflow that keeps the implementation grounded and close to deterministic, even when my specs are too ambitious and my instincts are telling me to just ship.

Here’s what it looks like.

Step 1 - Plan: generate a durable spec

A slash command takes the goal and produces a structured specification as a file committed directly to the repository, not a chat artifact, not a temporary prompt. A durable, reviewable, version-controlled document that defines exactly what is about to change and why.

This matters because the spec is now part of the codebase. It can be reviewed by a human, challenged by a subagent running a critic persona, or sent to a different model entirely ( Codex, a local model) for an independent second opinion before a single line of implementation begins.

The spec is the contract. Committing it to the repo makes that contract real.

Step 2 - Decompose: break ambition into steps

Overambitious specs are where agentic workflows go wrong. A goal that sounds coherent at the planning level often requires changes across too many files, too many layers, too many concerns simultaneously (and the model loses coherence trying to hold it all).

The decompose step breaks the spec into sequential sub-specs, each targeting no more than 8 self-contained steps. Dependencies are grouped into step groups. Each step group has a simple validation gate between steps and a full validation gate at the end of the group.

The ambition doesn’t go away. It gets sequenced.

Step 3 - Apply: scoped, gated, recoverable

Implementation is scoped to single files per step. After each step: syntax check, compilation, tests. Only when the gate passes does the next step begin.

No gate, no progress.

This is the part I was skipping when I vibe-coded my way into trouble. The gates feel slow when everything is going well. They feel essential the first time something fails cleanly at step 3 instead of corrupting steps 1 through 7 simultaneously.

If this sounds familiar, it should.

It's the same discipline Terraform enforced on infrastructure a decade ago. You don't run terraform apply on a hunch. You run terraform plan first, you review exactly what is about to change, you confirm it matches your intent, and only then do you apply. The plan is reviewable, the apply is scoped, and the state is always recoverable.

There's a useful way to think about why this matters now. If an agent is a kind of fuzzy compiler (something that turns intent into working code, but non-deterministically) then the plan starts to look like our real source code, and the code itself becomes more like a build artifact. That shifts where the discipline has to live: plan is what you review and version, because it's the thing that actually captures intent. It's the same split Terraform draws between the plan you scrutinize and the state you protect.

We learned that lesson for infrastructure. We're relearning it now for code.

The difference is that a reckless terraform apply might delete a database. A reckless agentic coding session might delete your understanding of what you built - and that's much harder to restore from backup.

The workflow didn’t make me build faster. It made what I built recoverable.

There’s a version of AI-assisted development that is genuinely fast and genuinely reliable, where you can move quickly because you have checkpoints, where you can be ambitious because the ambition is sequenced, where a failure at step 4 doesn’t take steps 1, 2, and 3 down with it.

And there’s a version that feels faster (prompt after prompt, change after change) right up until the moment it isn’t.

I’ve done both. The structured version is slower to start and harder to resist bypassing. But it’s the only one where I can honestly say I know what I built, why it works, and how to reproduce it.

The insight that took me too long to internalize:

Agentic tools don’t just amplify good judgment. They amplify the absence of it too.

The workflow exists to make sure that when I’m too ambitious, too impatient, or too deep in a flow state to notice, the system catches it before I do.

Have you developed guardrails for your own agentic workflows? What forced you to build them?

AI Doesn't Replace Expertise. It Multiplies It.

Daniel Stolf — Wed, 10 Jun 2026 15:00:00 +0000

When I started building my Kubernetes Operator, I wasn't a Go developer.

I had deep Kubernetes expertise. Deep database expertise. Seven years of enterprise platform work. But Go? I was learning it as I went, reading documentation, studying patterns, understanding idiomatic practices for the first time.

The operator shipped in two weeks anyway.

Not because AI replaced the expertise I didn't have. Because the expertise I did have (knowing exactly what the system needed to do, understanding the Kubernetes internals, seeing the database behavior clearly) was the hard part. AI helped me express that expertise in a language I was still learning.

That distinction is everything.

The popular framing right now is that AI replaces developers. Tasks get automated, roles compress, headcount comes down.

I think the framing misses the more important shift.

The layers of the job that AI has eaten so far are the layers that look like translation. Retrieval and synthesis (finding the relevant information and condensing it) got eaten last cycle. Code generation from a known spec is largely solved. Even parts of judgment, the rubric-based review of whether a design covers its failure modes and respects existing conventions, are being distributed across panels of models.

What hasn't been eaten is the upstream question. What should the system do. For whom. Under what constraints. With what failure modes that matter. That question still requires a human who has actually been close enough to the problem to know the answer.

That's expertise. And expertise is the layer that compounds with AI rather than being replaced by it.

The shape of the trade is simple.

The scarce resource was never the code. It was always the knowledge of what to build, for whom, and why. For decades, the cost of translating that knowledge into working software was high enough that most domain experts could not afford the translation, and most engineers were spending their time on the translation rather than the knowledge.

That cost is collapsing.

Which means the people whose advantage is real understanding of a domain (finance, logistics, healthcare, energy, agriculture, anything where the rules are subtle and the stakes are concrete) are about to find themselves with a lot more leverage than they had a year ago. Not because they will become engineers. Because the engineering cost of expressing their expertise just dropped by an order of magnitude.

There is a discipline cost on the other side.

Knowing which models to use for which tasks. Understanding where they fail and why. Building the workflow that separates real productivity gains from impressive demos followed by weeks of cleanup. None of that is automatic, and the gap between "this looks like it works" and "this actually ships and stays shipped" is where most teams underestimate the work.

But the discipline is learnable. And once it's in place, the multiplier on existing expertise is real.

My own journey is a small proof of concept for a much larger argument.

Domain expertise plus AI-assisted development plus the discipline to use both well: that combination can ship production systems faster than anyone expects, in languages you are still learning, solving problems that previously required teams and months.

Retrieval got commoditized. Judgment is becoming commoditized. Expertise is the layer where the human stays in the loop, and the layer where AI works for you instead of around you.

If you have spent years getting close to a real problem, the next decade is going to reward you in ways the last one didn't.

What's a problem you understand deeply that you've never built for because the engineering cost was too high? I'd like to hear what people are about to start building.

I Was the Judgment Layer

Daniel Stolf — Mon, 08 Jun 2026 15:00:00 +0000

There's a popular take right now that the AI coding bottleneck has moved to code review. Models write code faster than humans can read it, so the senior engineer becomes the reviewer of last resort.

The take is half right. The bottleneck moved, yes. Not to code review. Upstream of it, to the spec.

When the AI is fast enough that scaffolding, validation, and boilerplate are basically free, what determines whether the feature is correct is not the code. It's the intent the code was generated from.

If the intent says "webhook handler that updates payment status," the AI builds the happy path beautifully and silently ignores duplicate delivery, idempotency, and the impossible state transitions. The bug was authored at spec time. The reviewer at code time can catch it, but they're catching an artifact of an upstream mistake. That work is slow, expensive, and demoralizing.

So the real question is how to review the spec.

For a while I assumed that was my job, full stop. I'd write the spec, sit with it, ask myself the questions a senior engineer with scars would ask. What's out of scope. What should fail loudly. What's the behaviour on duplicate input.

It worked. It was also slow, and I kept noticing the same uncomfortable thing: I missed the same kinds of cases in review that I missed in authoring. The blind spots overlapped because the reviewer and the author were the same person.

What clicked was treating the spec the way you treat code in a strong engineering team. Put it through reviewers with different priors.

The setup I landed on uses three models.

Claude drafts the spec from a Socratic interview about intent. Acceptance criteria, edge cases, what's out of scope, open questions.

Codex reviews the draft against architecture.md, conventions.md, and guardrails.md. Different training corpus, different defaults, different things it cares about.

OpenCode running Qwen reviews independently against the same rubric. Different corpus again, different cultural priors, different sycophancy patterns.

Each model returns specific objections: missing acceptance criteria, ambiguous edge cases, contradictions with prior decisions, gaps where the spec assumes context it doesn't have.

The reason this works is that the models do not have the same biases.

Claude is confident and thorough, with a recognizable tendency toward completion and a particular sycophancy pattern when the prompt frames the spec as already good.

Codex is trained on a corpus that weighs code patterns heavily, so it surfaces "this spec implies a structure that won't fit your conventions" earlier than the others.

Qwen flags things the other two skipped, often around explicit failure modes and recovery paths.

None of them are right alone. The disagreements between them are the signal.

What I get out of that loop is judgment, applied earlier, by something that is not me and not biased the same way I am.

This is the part that makes me uncomfortable.

For years my edge as an engineer was judgment. Knowing which questions to ask. Knowing what to push back on. Knowing where the bodies were buried.

The retrieval-and-synthesis part of the job got eaten last cycle. I told myself the response was to move up to judgment, and that judgment was durable because it required taste and experience.

The honest update: judgment is also distributable. Not all of it. But the part that looks like rubric-based review (does this spec cover the failure modes, does it conflict with prior decisions, does it respect the conventions) can be parallelized across models with different training data. The result is better than any single human reviewer doing it alone. Including me.

The model wasn't judging. I was the judgment layer.

Now the judgment layer is a panel, and I'm one voice on it.

What's left for the human is smaller, but more concentrated.

Define the rubric. Maintain architecture.md and conventions.md so the panel has something specific to review against. Resolve disagreements when the models conflict. Decide which scars get written into history.md so future specs are reviewed against past pain.

These are still judgment calls. They happen less often than spec review, and each one compounds.

The earlier framing I had, that judgment was the safe layer to retreat to, was wrong. There is no safe layer. There is only the next location where verification has to happen, and the work of being honest when that location moves.

Right now it's at the rubric. Next year it might be somewhere else.

Anyone else running multi-model panels on the review side? What does your panel disagree about most?

Your Test Suite Is Lying To You

Daniel Stolf — Fri, 05 Jun 2026 03:00:00 +0000

Most dangerous moment in AI-assisted development:

The test suite is green.

You ran the agent. It built the feature. It wrote the tests. They all pass. CI is happy. You ship.

Two weeks later, production hits an empty list and crashes.

This is the failure mode I keep watching engineers walk into, and I've walked into it too.

Tests written after the code are not tests. They're documentation of what the code happened to do. Which means when the code has a bug (a silently ignored duplicate webhook, an unhandled empty input, a failure mode the model assumed away) the tests written from that code don't fail. They pass. They lock the bug in as the documented behavior.

The CI signal turns green. The PR gets approved. The spec gets quietly violated.

And the worst part is that the suite isn't broken. It's doing exactly what it was asked to do: confirm that the code does what the code does. The only thing missing is the spec. The thing the tests were supposed to enforce.

Now add agents to the loop.

The agent generates code. The agent runs the tests it generated alongside. The tests pass. The agent reports success. It moves on to the next feature.

There is no point at which a human looks at the test list and asks: "wait, is the spec actually represented here? Did we test for the empty input? Did we test for the duplicate webhook? Did we test what should never happen?"

Without that pause, the suite is just a self-confirming loop. The code says A. The tests say A is correct. The agent reports A is verified. The fact that the spec said B never enters the system.

Tests catch what they were written to catch. Nothing more.

When humans write tests after code, the same pattern shows up. Most engineers will admit this. The difference with AI is volume and speed. A junior writing code-then-tests produces maybe two or three such artifacts a day. An agent produces twenty. The same blind spot scales without the scarring that used to come from finding the bug in production at 2am.

This is also why prompting your way out of it doesn't work. "Write good tests" is advice the model accepts and immediately ignores in practice, because what it was asked to do (make the feature work and verify it) is satisfied as long as the tests pass. The tests passing IS the verification. The model has no way to know the verification should have come from somewhere else.

The only honest fix is the order.

The tests come from the spec, not the code. Acceptance criteria like "returns X on empty input", "rejects duplicate webhook by ID", "fails loudly on unknown status" become failing tests before any implementation exists. The model writes the code to satisfy those tests. The tests didn't come from the code. They came from what the code was supposed to do.

When tests come first, the model can't generate the bug AND the test that ratifies it. The empty input case is already a red bar. The duplicate webhook is already an assertion. The model has to satisfy what the spec required, not what it happened to produce.

In practice, nobody does this voluntarily.

Not because they don't know better. Because writing the failing test first feels slow in a moment when the agent can produce the implementation in seconds. Every TDD article for the last twenty years has acknowledged this part. The skip feels small in the moment. The discipline collapses.

With humans, the skip is at least visible. You can see in the PR that the tests came in the same commit as the code, often even after it. With agents, the skip is invisible. The tests are there. They pass. Nobody notices that the tests are ratification, not verification.

What I've come to believe is that test-first stops being achievable as discipline once the model is fast enough to make the skip free. It has to become structural. Some part of the workflow has to refuse to accept code edits that arrived before their tests. That's the only thing I've seen consistently hold the line.

Whether that's a hook, a CI check, an agent that won't merge, a pre-commit gate: the mechanism matters less than the principle. The model cannot ship code whose verification it also generated. The tests must be older than the code, written from a source the code didn't influence.

The discipline isn't TDD. The discipline is refusing to accept tests that came from the same context as the code they're testing.

In an AI-assisted workflow, that distinction is the difference between a green suite that means something and a green suite that doesn't.

How does your team prevent tests-after-code in AI-generated PRs? Do you trust that distinction is being held?

This Is the Definitive Solution

Daniel Stolf — Wed, 03 Jun 2026 15:00:00 +0000

The most dangerous thing an LLM can say is:

"This is the definitive solution."

I know, because I heard it six times in a row, each time about a different fix for the same bug.

I was building a Kubernetes Operator and hit a race condition in the reconcile loop.

The Operator would update the custom resource status, then immediately get overwritten by a previous state that hadn't been flushed yet. The result was a phase transition loop: the resource cycling through states endlessly, never stabilizing.

I turned to the LLM. Internet search enabled. Full context provided.

The first suggestion was confident and specific. A pattern for handling concurrent reconciliation. Reasonable. I tried it. Didn't work.

The second suggestion went deeper, a sub-function within the controller runtime. Also confident. Also specific. Also wrong.

By the fourth or fifth iteration, the model was descending into progressively more obscure corners of the Kubernetes internals. Each time with the same tone:

"This is a common race condition in Kubernetes Operators. This is the definitive solution."

It wasn't lying. It wasn't hallucinating functions that didn't exist. It was finding real patterns, real APIs, real Kubernetes behaviors and misapplying every single one of them to my specific context, with complete confidence.

Internet search didn't help. It just gave the model more ammunition to be wrong with.

What finally solved it had nothing to do with the model.

I stopped asking. I did the research myself, read through the controller-runtime source, found the specific behavior causing the overwrite, understood the exact sequence of events in my reconcile loop. Then I came back to the model with a clear, grounded explanation of the root cause and explicit instructions for the fix.

It worked immediately.

The model didn't solve the problem. I solved the problem. The model wrote the code.

That distinction matters more than most people want to admit.

Now think about what happens when you add agentic flow.

No human in the loop. The model hits the same kind of problem: a subtle race condition, a context-specific behavior it can't reason about correctly from training data alone. It generates a fix. It runs it. It observes the output. It decides the fix didn't work and tries something deeper.

It does this autonomously. Confidently. Repeatedly.

Each iteration, it goes further into the codebase. Refactoring here. Adding an abstraction layer there. Patching a symptom in one place while introducing a new one in another. Burning tokens at every step.

By the time a human looks at the output, the codebase is a spaghetti mess and the original bug is still there, buried under six layers of confident, well-intentioned, completely misguided changes.

The model didn't fail because it was dumb. It failed because it was confident about something it fundamentally could not know without the right context. And nothing stopped it from acting on that confidence, repeatedly, at speed.

This is the part of agentic AI development that the demos don't show you.

The capability is real. The productivity gains are real. But autonomous execution amplifies both good judgment and bad judgment equally. A model that would have wasted 30 minutes of a developer's time in a chat session can waste 3 hours of compute and leave a codebase significantly worse in an agentic loop.

The solution isn't to avoid agentic workflows. It's to understand where models fail and build human checkpoints, grounding steps, and context injection at exactly those points.

I learned this the hard way, one race condition at a time.

Have you seen an agentic workflow make a problem worse before it made it better? What guardrails have you built?

Opus to Plan. Sonnet to Build. Slash Command to Scale.

Daniel Stolf — Mon, 01 Jun 2026 15:00:00 +0000

The first version of my Kubernetes Operator worked.

It was built for one database platform, for one customer problem, and it did exactly what it needed to do. But it was concrete all the way down: tightly coupled, not extensible, and impossible to test without live infrastructure.

If I wanted to support other database platforms, I'd have to refactor the entire core. Replace concretes with interfaces. Restructure the codebase so logic could be reused across implementations.

This is the part of AI-assisted development that nobody talks about. Not the greenfield build, but what happens next. Taking production code and making it better without breaking it.

Here's the workflow I used.

Phase 1 - Specification with Opus

Before touching a single line of code, I asked Claude Opus to generate a detailed refactoring plan as a Markdown document.

Not a vague outline. A precise spec: which files would be created, which code segments would move where, what the new interfaces would look like, how the abstractions would enable reuse across future database implementations.

I used Opus specifically for this phase: the more capable model, slower and more expensive, but worth it for the reasoning depth required to plan a structural refactor without introducing regressions. The goal was to get the plan right before any implementation began.

I reviewed it, pushed back on specific decisions, asked for alternatives in places where the approach felt wrong. The MD file went through several iterations before I was satisfied.

That document became the single source of truth for everything that followed.

Phase 2 - Implementation with Sonnet in Plan Mode

Once the spec was locked, I switched to Claude Sonnet and engaged Plan Mode, Claude Code's built-in capability to propose a complete implementation plan before executing any changes.

This is a critical step that most people skip.

Plan Mode reads the specification, maps it against the existing codebase, and produces a detailed proposal of exactly what it intends to do (file by file, change by change) before writing a single line. You review it. You approve it. Then it executes.

The spec I'd built with Opus was the input. Plan Mode's proposal was the verification that the model understood the spec correctly. Implementation only started when both aligned.

The refactor ran in one shot. Clean interfaces. Reusable core. Testable without live infrastructure.

Phase 3 - The slash command

Once the architecture was right, I built a custom Claude Code slash command to scaffold new database integrations.

Now, when I need to add support for a new platform, I run the command. It generates the API layer, the controller structure, the interface implementations: all consistent with the refactored architecture, all following the same patterns.

What used to require understanding the entire codebase and carefully threading new logic through it now takes minutes.

The full workflow, in plain terms:

Opus → think deeply, plan precisely, produce a spec

Sonnet + Plan Mode → verify understanding, execute against the spec

Custom slash command → compound the investment, make the pattern reusable forever

Each phase uses the right tool for the right job. Opus for reasoning. Sonnet for execution. The slash command for scale.

This is what AI-native development looks like when it matures past the prototype stage.

The 2-week MVP was speed. This is something different: it's using AI to improve the architecture of something already in production, responsibly, with a verification step between planning and execution.

The spec isn't just a prompt. It's a contract between you and the model. Plan Mode is how you confirm the model understood the contract before it acts on it.

That discipline (spec first, verify before executing, build reusable patterns from every solved problem) is what separates teams that ship consistently with AI from teams that generate impressive demos and then spend weeks cleaning up.

What's your workflow for using AI on existing production code, not just greenfield builds?

I Was the Retrieval Layer

Daniel Stolf — Fri, 29 May 2026 15:00:00 +0000

I once spent half a day debugging code that was completely correct.

The problem wasn't the logic. The problem was that the functions the LLM had written didn't exist.

Not deprecated. Not renamed. Never existed.

Here's what had happened: I caught the model using an outdated API parameter and corrected it. Instead of fixing the issue, it started compensating: hallucinating function names, inventing method signatures, generating plausible-looking code that had no basis in reality. The more I pushed back, the deeper into fiction it went.

That afternoon is why I started doing RAG before the industry had a name for it.

At the time, I was building a Kubernetes Operator using free-tier LLMs (ChatGPT and DeepSeek). No agentic tooling. No memory. No orchestration frameworks. Just a chat window and whatever I could fit into the context.

I had two problems:

Problem 1:
The model didn't know current APIs. Kubernetes controller-runtime, Operator SDK, and Delphix APIs move fast. The model's training data was already stale. Left to its own devices, it would confidently generate code against API versions that no longer existed. When corrected, it would sometimes make things worse.

Problem 2:
The context window ran out. Long sessions degraded. The model would start contradicting earlier decisions, losing track of architecture choices, rehashing solved problems. On a free tier, hitting the limit meant starting over and losing everything.

Here's what I built to solve both:

For the API problem, manual retrieval and injection. Before writing any implementation code for a new component, I would research the relevant documentation myself. Then I'd summarize it (sometimes by hand, sometimes by feeding the raw docs into a separate chat session just for summarization) and inject only the relevant fragments into the working session. Confirmed, current, scoped to exactly what the model needed.

The model wasn't searching. I was the retrieval layer.

For the context problem, session state documents. When a session was getting too long, I'd ask the model to generate a structured Markdown file: current architecture decisions, what had been built, what was left, key constraints and open questions. Then I'd start a fresh session, paste the MD file as context, and continue exactly where I'd left off.

The model wasn't remembering. I was the memory layer.

What I was doing, without knowing it:

Retrieval-Augmented Generation: surfacing accurate, current information and injecting it as context to ground model outputs.

Session state management: the manual precursor to what agent memory systems now handle automatically.

Multi-session LLM chaining: using one model to process and compress information for another, before orchestration frameworks made this trivial.

I didn't invent these patterns. I arrived at them by necessity, the hard way, after a hallucination loop cost me half a day.

That's usually how the best practices emerge.

The tools have improved dramatically since then. But the underlying problem (models that hallucinate on fast-moving APIs, context that degrades over long sessions, outputs that need grounding in verified information) hasn't gone away. It's just more visible now, at scale, in enterprise deployments.

The engineers and companies figuring this out today are rediscovering the same lessons. Usually also the hard way.

Have you hit a hallucination loop that cost you real time? What was your fix?

AI-Native Engineering Story

Daniel Stolf — Wed, 27 May 2026 15:00:00 +0000

In 2020, during the pandemic lockdown, I built a working Kubernetes CSI Driver prototype in a hackathon.

It was good enough to win. But turning it into a production-ready integration took months. Eventually it required a team of 3 additional engineers to get there.

Same person. Same domain. Same company. Months, plus a team.

Fast forward five years.

I built a production Kubernetes Operator (a more complex project) in 2 weeks. Solo. No team allocation. No formal project approval. Just me and a workflow I’d developed almost by accident.

Same person. Same domain. Same company. 2 weeks, alone.

The only thing that changed was how I worked with AI.

Here’s what that workflow actually looked like, before it even had a name:

Step 1 - Spec-driven architecture

I didn’t start by writing code. I used high-reasoning models (ChatGPT, DeepSeek) to think through the design, discussing trade-offs, challenging assumptions, generating a structured Markdown spec with architecture decisions and code scaffolding. The model was my technical co-founder.

Step 2 - Grounding the model

LLMs hallucinate on fast-moving APIs. Before writing any implementation code, I researched and injected the relevant documentation (Kubernetes controller-runtime specs, CRD patterns, Delphix API definitions) directly into context. The model then worked from accurate, current sources. I was doing RAG manually before RAG was a term most people used casually.

Step 3 - Agentic execution

With the spec as a foundation, I used GitHub Copilot to drive implementation, iterating on specific functions, sharing code segments with targeted prompts, reviewing and correcting outputs. Claude Code came later and accelerated the final stretch further.

The result wasn’t just a faster build.

It broke a structural go-to-market deadlock. Without a working solution, customers wouldn’t commit. Without customer demand, engineering wouldn’t prioritize it. By shipping a working MVP first (alone, in two weeks) I generated real demand from a working product.

That converted a churn risk into a US$30M enterprise deal.

The 2020 hackathon wasn’t a failure. The CSI Driver became the technical foundation the Operator was built on. But the contrast matters:

The same engineer. The same problem space. A team and months, versus solo and two weeks.

What changed wasn’t the engineer. It was the workflow.

The practices I stumbled into out of necessity (spec-driven development, context grounding, agentic coding loops) are now what enterprises are actively trying to learn and implement at scale.

The gap between “we’re experimenting with AI” and “we’re shipping with AI” is mostly a workflow problem. It’s not about the model. It’s about how you work with it

What’s the biggest shift you’ve noticed in how your team builds since AI tools became mainstream?