Over the last year, my day-to-day job has changed in a way I am still trying to understand. I am still an engineer. I still design systems, read code, debug failures, review implementations, and sometimes build things myself. But I write much less code than I used to, and that feels weird.
For most of my career, producing software was the visible evidence that I was doing my job. You had a problem, you designed a solution, you wrote the code, and something that did not exist in the morning existed by the end of the day. There was a very direct relationship between effort and output.
Today, that relationship is disappearing.
A large part of the implementation work around me can now be generated by AI. Not perfectly, not autonomously, and not without supervision, but cheaply enough that the first implementation is increasingly not the hard part. If I need an API endpoint, a migration, a data transformation, a test suite, an integration, or some internal tooling, I can describe the problem, provide enough context, and get a plausible implementation surprisingly quickly.
The expensive part starts afterwards: does it actually work?
Not simply, "does the code run?" or "do the tests pass?" The real question is whether the feature behaves correctly across the range of situations we actually care about. That question has slowly become a much larger part of my job, and it has changed how I think about my own value.
A year ago, I spent much more time thinking about how to implement something. Now I spend much more time thinking about how it can fail. What happens when the model receives something we did not anticipate? When two components disagree? When an apparently good answer contains the wrong evidence? When the evaluator itself is biased? What happens when a model succeeds 95 percent of the time, but the remaining 5 percent contains exactly the failures that matter to the business? What happens when a fallback silently changes provider, region, latency, cost, or behavior? What happens when everything returns HTTP 200 and the system is still wrong?
The implementation is often no longer the difficult intellectual problem. Failure detection is. Failure prevention is. Defining what "good enough to ship" actually means is.
That creates a strange inversion. For decades, software engineering treated testing and validation as something downstream of implementation. First you built the thing, then someone checked whether it worked. In AI systems, I increasingly feel the opposite. Generating the implementation is becoming cheaper, while knowing whether you should trust it is becoming more expensive.
This week, for example, after I take some time off, the team asked me for a new evaluator for a feature. At first glance, that sounds almost like QA work, and I admit that part of me reacts negatively to that idea. I spent years learning how to build software. Am I slowly becoming the person who checks everybody else's work?
But I think that interpretation misses what is actually happening.
The hard part of building an evaluator is rarely writing the evaluator. AI can help enormously with that too. The difficult part is deciding what the evaluator should measure in the first place. What counts as failure? Which failures can be checked deterministically? Which ones require statistical evaluation? Which dataset represents reality well enough? Where are the blind spots? Can the evaluator itself be fooled? Are two supposedly independent checks actually making the same mistake? What threshold is sufficient to release something to production?
Those are not really questions about test implementation. They are questions about the operational definition of correctness. And with AI, correctness is becoming something we have to engineer.
Traditional software gives us a relatively comfortable contract. For the same inputs and state, deterministic software should generally produce the same output. LLMs break that assumption. They generate plausible outputs across enormous input spaces, and they can produce something structurally correct, linguistically excellent, internally coherent, and completely wrong.
So we wrap them in software. We constrain them, validate their outputs, compare signals, build fallback paths, add deterministic checks around probabilistic behavior, create evaluation datasets, measure regressions, and observe production traces. Then, inevitably, we discover a failure mode we did not know existed and modify the system again.
A surprising amount of my work now lives in that boundary. Not creating intelligence, but making probabilistic intelligence reliable enough to become part of a real product.
This also makes me uncomfortable for another reason. If AI can generate the implementation, why couldn't another AI eventually generate the evaluations too? Why couldn't I describe my principles, my distrust, my way of looking for edge cases, and encode all of that into another agent?
The answer is probably that I can. And I should.
If I discover a reliable failure pattern, I want to automate its detection. If I repeatedly perform the same review, I want a system to perform it for me. If a deterministic gate can replace my manual judgment, building that gate is progress. My value cannot depend on protecting tasks from automation, because that would be a losing strategy.
So perhaps the important distinction is not between work humans can do and work AI can do. It is between known problems and unknown ones.
Once a failure mode is understood, it becomes cheaper. We can encode it in a test, build an evaluator, add a policy, or teach an agent to look for it. The difficult part then moves somewhere else. The frontier becomes the next thing we do not yet know how to measure, and that seems to be where more and more of my work is going.
My output is increasingly not code. It is a definition, a constraint, an architecture, a failure taxonomy, a release criterion, or a deterministic guard around something probabilistic. Sometimes the final artifact is twenty lines of code, but those twenty lines may represent two days of thinking about what exactly needs to be prevented.
That changes how productivity feels. If I generated 1,000 lines of production code in a week, I could easily point at what I created. If I spend the same week investigating one subtle failure mode and eventually add a tiny check that prevents it, the visible output looks much smaller. But the economic value might be significantly larger.
The system already knows how to generate. The harder question is whether we can depend on what it generates.
I do not know exactly what this role should be called yet. AI engineer still fits. Reliability engineer fits part of it. Architecture fits another part. Evaluation engineering is clearly becoming important. None of those labels completely captures the transition I am experiencing.
What I do know is that I am moving away from being primarily the person who produces the implementation. I am becoming the person who asks what the implementation must prove before we trust it.
That feels strange because software engineering trained us to identify ourselves with building. Code was craftsmanship, output, and evidence that we were useful. AI is making code abundant, and when something becomes abundant, value usually moves somewhere else.
Maybe the next scarce resource in software engineering is not implementation. Maybe it is the ability to determine when an implementation is wrong before reality does it for you.
The cheaper part is increasingly generated. The expensive part is knowing where it will fail, and making sure it doesn't.
Human concept, nice written by AI.
Top comments (31)
The "evaluator that can itself be fooled" line is where this gets expensive, and I've watched it quietly swallow the budget. Two supposedly independent checks reusing the same ideal answer or the same loss signal will fail together, and you only notice after a bad release. The part I've stopped trusting is the 95%. The tail of cases that break is where the real eval money has to go, and that tail is exactly what the model and I both miss on a first pass. The way I've been keeping myself honest is giving the evaluator one thing it has to actively disprove before it can say pass, a specific falsifier per feature. Without that, "good enough to ship" drifts toward "scores high on a benchmark we wrote."
This sounds smart!
I think the interesting shift is that verification itself becomes an engineering system. Once a failure pattern is understood, we turn it into a test or guard. The real challenge is identifying the failure modes we don’t know to test for yet. That’s where human judgment still seems hardest to automate.
Yes, and I think this creates an interesting loop. The moment human judgment discovers and understands a failure mode, that judgment can be partially automated too. So the valuable part keeps moving. Today I may design the evaluator, tomorrow the evaluator is infrastructure and my attention moves to what it still cannot see.
That loop is probably the most interesting part to me. If every well-understood failure mode eventually becomes infrastructure, then the real engineering frontier keeps moving toward failures we don't yet have a vocabulary for.
I wonder if that also means our evaluation systems need to evolve continuously, rather than being treated as a finished layer. How do you think about knowing when an evaluator itself has become part of the “known problem” and should be automated away?
The interesting shift here is that AI is moving engineering effort from implementation to specification and verification.
The part I would emphasize is the failure taxonomy. Once you can name a failure mode precisely, it can usually be turned into an evaluator, a deterministic guard, or an automated regression test. The real engineering work is identifying the failures that are still invisible to your current evaluation system.
That is also why “the tests pass” is becoming a weaker signal for AI systems. A system can be syntactically correct, operationally healthy, and consistently wrong. Reliability increasingly means designing the evidence required to trust the output, not just generating the output itself.
Agree! I think the failure taxonomy is becoming one of the most important artifacts we produce.
Once a failure has a name and a reproducible shape, it becomes much less interesting. We can encode it, test it, monitor it, and eventually automate most of the response to it.
The difficult part is everything that still sits outside that taxonomy. The failures your current evaluators do not know how to see yet.
And that is why “tests pass” is becoming such a weak statement for AI systems. The pipeline can be healthy, the schema valid, the latency normal, every test green, and the answer still fundamentally wrong.
Increasingly, reliability is not about proving that the software executed correctly. It is about defining what evidence is sufficient to trust the behavior of the whole system.
reidmarlow's falsifier idea deserved more than it got and i think it's the concrete version of your whole argument.
the reason two independent checks fail together is almost always that both were derived from the same artifact. write the test after the implementation, generate the eval from the spec that generated the code, and the second check inherits the first one's blind spot while looking like corroboration. independence is about provenance, not about count.
so the falsifier has to be authored before the thing it's meant to catch exists. one sentence per feature at design time: here is the specific wrong behaviour that would make me pull this. not a success criterion, a named failure. then pass means that particular thing got actively ruled out, rather than nothing objected.
which is also how we'd answer mickyarun. the unit of work stopped being the diff and became the falsifier. how many did i write, how many fired, how many turned out to be the wrong thing to be afraid of. that last number is the interesting one and the only honest measure of judgment i've found, because it's the one that can go badly.
twenty lines representing two days is right, and the reason it's uncomfortable is that nobody has a review process shaped for a twenty line diff that took two days.
I really like your framing of independence as provenance rather than count. Two evaluators are not independent simply because there are two of them.
The falsifier-before-implementation idea is especially interesting because it prevents us from defining correctness after seeing what the system already does. I think there is something very useful there: before building the feature, explicitly name at least one behavior that would make us reject it. That makes the evaluation adversarial by design instead of merely confirmatory.
This is a really interesting perspective on how AI is changing the definition of engineering productivity.
The point that stood out to me most is that implementation is becoming cheaper, while judgment is becoming more valuable. Writing less code doesn't necessarily mean doing less engineering, it can mean spending more time understanding failure modes, defining what “correct” actually means, and building reliable guardrails.
The idea of moving solved problems into automation while focusing human effort on the next unknown problem feels like a strong way to think about the future of software engineering.
Great read! 👏
Thanks!
You guys write code 🙃
rarely...
Long read but I just want to think out loud under this space. How do you tell a child to mark his scripts 🤔
This is the reality at the moment, and it's pretty good.
I think so too. Uncomfortable in some ways, but also interesting. Removing some of the implementation cost lets us spend much more time on the part that was often compressed by deadlines: understanding whether what we built is actually something we should trust.
i think the role name hidden in the post is evidence engineer. implementation is becoming abundant; evidence that is genuinely independent of the implementation is becoming scarce.
one wrinkle: once the evaluator becomes a production artifact, the implementation starts optimizing against it, intentionally or just through repeated tuning. every known failure we encode makes that case cheaper, but also makes the evaluator less independent of the system it scores. a frozen holdout, evaluator version and whether each guard was authored before or after the incident become part of the result.
otherwise the twenty-line guard can be enormously valuable and still depreciate as evidence each time it shapes the next implementation. how are u keeping the evaluator out of the training loop, especially when the same models help write both?
Thanks for the article, Marco and I think it resonates with many of us.
Even before LLMs, Stack Overflow was often our best friend and personally, I’ve lost count of how many solutions I copied and had to adapt because of syntax and dependencies That process sometimes taught us a lot of things for sure and I won't argue against it.
I’m also not entirely sure how to feel about the transformation, but I’m optimistic. The people who are heavily dependent on these tools without understanding are still very much visible. The same is true for engineers who think critically.
Have we lost some of the magic with the boring parts? Maybe a little. But I don’t think the real magic was ever in the clicking or typing. It was in the thought process, understanding how things work under the hood, and the experience needed to know what to trust and what to question. :)
Some comments may only be visible to logged-in visitors. Sign in to view all comments.