A lot of writing about AI-assisted development focuses on speed.
How fast you can generate code. How many lines per hour. How quickly you can go from idea to pull request.
Speed is real. But it is not the interesting question.
The interesting question is: who is responsible for what?
Get that wrong, and speed becomes a liability. You move faster toward the wrong place, with less awareness of how you got there.
The boundary I use
After four months as a solo engineer building a production system with Claude Code as my primary tool, this is the line I drew:
Human side:
- Requirements — what the system should do, and what it should not do
- Acceptance criteria — what "done" actually means for each unit of work
- Production judgment — whether something is safe to ship
AI side:
- Implementation — writing code that satisfies the criteria
- Code review — flagging issues and suggesting alternatives
- Repetitive testing — generating test cases and running assertions
This is not a rule I found written down somewhere. It is a conclusion I reached by watching what breaks when the line moves in either direction.
Why requirements must stay with humans
AI does not know your domain.
It knows patterns from training data. It can generate plausible-sounding requirements based on what similar systems have needed. But it does not know what your business actually needs, what your users are actually doing, or what your organization can actually support.
When I joined the company, I spent time on the shop floor before writing any code. Not to understand what to build — to understand what not to build. Which parts of the workflow had value precisely because they involved human judgment. Which parts were mechanical. Where systematizing would help versus where it would remove something important.
There is also a more subtle case. Sometimes the right answer from the observation phase is not to build something at all. I once watched a workflow that looked inefficient on paper — manual handoffs, no clear status tracking. The instinct was to systematize it. But two things stopped me. First, the manual handoffs were doing something a system could not: they forced a conversation that kept two people aligned. Second, that part of the workflow existed because of a specific regulatory requirement — one that could change or disappear entirely. Building a system around it would have meant maintaining something with an uncertain lifespan. I left it alone.
The observation phase is where you find out what not to build. AI was not in the room for that.
Why acceptance criteria must stay with humans
"Done" is a business decision, not a technical one.
A feature can pass all its tests and still not be done. Maybe it handles the happy path but leaves error states unclear. Maybe it works in isolation but creates friction in the actual workflow. Maybe it solves the problem that was described but not the problem that exists.
The acceptance criteria is the translation layer between what the business needs and what the system does. Getting that translation right requires the same domain knowledge that requirements do.
When I write a GitHub Issue, I write the acceptance criteria before I ask Claude Code to implement anything. That sequence matters. The implementation is shaped by the criteria. If the criteria is wrong, the implementation is fast in the wrong direction.
Why production judgment must stay with humans
Responsibility cannot be delegated to a tool.
If something breaks in production — data loss, incorrect calculations, a process that runs incorrectly overnight — the question is not "what did the AI do." The question is "who decided to ship this."
That accountability has to live somewhere. It lives with the person who made the call.
A concrete example: a customer-side image leak incident required a rapid audit response. There was almost no historical data to work from. The model's inferences were weak. But with a minimal scope, limited claims, clear caveats, and humans in the loop at every step, the output was still usable. Weak inference plus human judgment is often more than good enough. Strong inference with no human checkpoint is where things go wrong.
When I review a pull request before merging, I am not checking syntax. I am asking: does this match the acceptance criteria, and is this safe to run in production? Those are my questions to answer.
What happens when the boundary slips
The failure modes are predictable.
When AI takes on requirements: The system is built to match a plausible interpretation of the problem rather than the actual problem. This is often invisible until the system is in production and users start working around it.
When AI writes acceptance criteria: "Done" becomes technically correct rather than useful. Tests pass. The feature ships. Something important is still missing, but it is hard to say what.
When AI makes production calls: Nothing fails visibly. But the person who hit "deploy" does not fully understand what they shipped. Over time, this erodes the ability to reason about the system — you have output, but not understanding.
A specific example: I hit a SQL connection error, asked Claude Code to fix it, got a revised version, hit the same error again. Asked again. Got back something close to the original code. The problem turned out to be a version dependency — something that changed after the model's knowledge cutoff. No amount of prompting would have resolved it, because the model simply did not have the relevant information. The fix came from going directly to the latest documentation.
Knowing when to stop prompting and go elsewhere is a judgment call the tool cannot make for you.
What AI handles well, and why
The right side of the boundary is where AI is genuinely powerful.
Implementation, review, and repetitive testing all share a property: the standard is clear before the work starts.
When the acceptance criteria is written, there is an external measure of correctness. AI can write code against that standard, generate tests that check that standard, and flag when code does not meet it. The verifiability is what makes this work.
Boilerplate is the clearest case. Cloud Run configurations, Argo CD manifests, GitHub Actions workflows — these have well-defined correct forms. AI can produce them quickly and accurately. There is nothing to decide; there is something to write.
Error diagnosis is another strong case. A stack trace with a clear error message is a specification of what went wrong. AI can read that specification and suggest what caused it. The interpretation is bounded and checkable.
Fast output in bounded, verifiable tasks. That is where the leverage is.
The practical consequence
Once this boundary is clear, the day-to-day feels different.
I do not ask Claude Code what to build. I decide what to build, write it as a scoped Issue with acceptance criteria, and then ask Claude Code to implement it.
I do not ask Claude Code whether to merge. I ask Claude Code to review the pull request, I read the review, and then I decide.
I do not ask Claude Code if the system is ready for production. I look at the tests, the logs, the behavior in staging, and I make the call.
The AI is fast. The decisions are mine.
Why this matters more as tools improve
As AI tools become more capable, the temptation to move the boundary grows.
The output gets better. The suggestions feel more reliable. The friction of making your own decisions seems unnecessary when the AI gets it right most of the time.
But "most of the time" is not good enough for requirements, acceptance criteria, or production decisions. Those are the moments where being wrong has compounding consequences.
The tools improving does not change what needs human judgment. It changes how much of everything else can be handled without it.
Which means the human role is not disappearing — it is concentrating.
Less time on implementation. More time on what actually needs to be decided.
That is a better arrangement, not a worse one. But it only works if the boundary is held.
Top comments (0)