Over the last year, my day-to-day job has changed in a way I am still trying to understand. I am still an engineer. I still design systems, read co...
For further actions, you may consider blocking this person and/or reporting abuse
The "evaluator that can itself be fooled" line is where this gets expensive, and I've watched it quietly swallow the budget. Two supposedly independent checks reusing the same ideal answer or the same loss signal will fail together, and you only notice after a bad release. The part I've stopped trusting is the 95%. The tail of cases that break is where the real eval money has to go, and that tail is exactly what the model and I both miss on a first pass. The way I've been keeping myself honest is giving the evaluator one thing it has to actively disprove before it can say pass, a specific falsifier per feature. Without that, "good enough to ship" drifts toward "scores high on a benchmark we wrote."
This sounds smart!
I think the interesting shift is that verification itself becomes an engineering system. Once a failure pattern is understood, we turn it into a test or guard. The real challenge is identifying the failure modes we don’t know to test for yet. That’s where human judgment still seems hardest to automate.
Yes, and I think this creates an interesting loop. The moment human judgment discovers and understands a failure mode, that judgment can be partially automated too. So the valuable part keeps moving. Today I may design the evaluator, tomorrow the evaluator is infrastructure and my attention moves to what it still cannot see.
That loop is probably the most interesting part to me. If every well-understood failure mode eventually becomes infrastructure, then the real engineering frontier keeps moving toward failures we don't yet have a vocabulary for.
I wonder if that also means our evaluation systems need to evolve continuously, rather than being treated as a finished layer. How do you think about knowing when an evaluator itself has become part of the “known problem” and should be automated away?
The interesting shift here is that AI is moving engineering effort from implementation to specification and verification.
The part I would emphasize is the failure taxonomy. Once you can name a failure mode precisely, it can usually be turned into an evaluator, a deterministic guard, or an automated regression test. The real engineering work is identifying the failures that are still invisible to your current evaluation system.
That is also why “the tests pass” is becoming a weaker signal for AI systems. A system can be syntactically correct, operationally healthy, and consistently wrong. Reliability increasingly means designing the evidence required to trust the output, not just generating the output itself.
Agree! I think the failure taxonomy is becoming one of the most important artifacts we produce.
Once a failure has a name and a reproducible shape, it becomes much less interesting. We can encode it, test it, monitor it, and eventually automate most of the response to it.
The difficult part is everything that still sits outside that taxonomy. The failures your current evaluators do not know how to see yet.
And that is why “tests pass” is becoming such a weak statement for AI systems. The pipeline can be healthy, the schema valid, the latency normal, every test green, and the answer still fundamentally wrong.
Increasingly, reliability is not about proving that the software executed correctly. It is about defining what evidence is sufficient to trust the behavior of the whole system.
reidmarlow's falsifier idea deserved more than it got and i think it's the concrete version of your whole argument.
the reason two independent checks fail together is almost always that both were derived from the same artifact. write the test after the implementation, generate the eval from the spec that generated the code, and the second check inherits the first one's blind spot while looking like corroboration. independence is about provenance, not about count.
so the falsifier has to be authored before the thing it's meant to catch exists. one sentence per feature at design time: here is the specific wrong behaviour that would make me pull this. not a success criterion, a named failure. then pass means that particular thing got actively ruled out, rather than nothing objected.
which is also how we'd answer mickyarun. the unit of work stopped being the diff and became the falsifier. how many did i write, how many fired, how many turned out to be the wrong thing to be afraid of. that last number is the interesting one and the only honest measure of judgment i've found, because it's the one that can go badly.
twenty lines representing two days is right, and the reason it's uncomfortable is that nobody has a review process shaped for a twenty line diff that took two days.
I really like your framing of independence as provenance rather than count. Two evaluators are not independent simply because there are two of them.
The falsifier-before-implementation idea is especially interesting because it prevents us from defining correctness after seeing what the system already does. I think there is something very useful there: before building the feature, explicitly name at least one behavior that would make us reject it. That makes the evaluation adversarial by design instead of merely confirmatory.
This is a really interesting perspective on how AI is changing the definition of engineering productivity.
The point that stood out to me most is that implementation is becoming cheaper, while judgment is becoming more valuable. Writing less code doesn't necessarily mean doing less engineering, it can mean spending more time understanding failure modes, defining what “correct” actually means, and building reliable guardrails.
The idea of moving solved problems into automation while focusing human effort on the next unknown problem feels like a strong way to think about the future of software engineering.
Great read! 👏
Thanks!
You guys write code 🙃
rarely...
Long read but I just want to think out loud under this space. How do you tell a child to mark his scripts 🤔
This is the reality at the moment, and it's pretty good.
I think so too. Uncomfortable in some ways, but also interesting. Removing some of the implementation cost lets us spend much more time on the part that was often compressed by deadlines: understanding whether what we built is actually something we should trust.
i think the role name hidden in the post is evidence engineer. implementation is becoming abundant; evidence that is genuinely independent of the implementation is becoming scarce.
one wrinkle: once the evaluator becomes a production artifact, the implementation starts optimizing against it, intentionally or just through repeated tuning. every known failure we encode makes that case cheaper, but also makes the evaluator less independent of the system it scores. a frozen holdout, evaluator version and whether each guard was authored before or after the incident become part of the result.
otherwise the twenty-line guard can be enormously valuable and still depreciate as evidence each time it shapes the next implementation. how are u keeping the evaluator out of the training loop, especially when the same models help write both?
Thanks for the article, Marco and I think it resonates with many of us.
Even before LLMs, Stack Overflow was often our best friend and personally, I’ve lost count of how many solutions I copied and had to adapt because of syntax and dependencies That process sometimes taught us a lot of things for sure and I won't argue against it.
I’m also not entirely sure how to feel about the transformation, but I’m optimistic. The people who are heavily dependent on these tools without understanding are still very much visible. The same is true for engineers who think critically.
Have we lost some of the magic with the boring parts? Maybe a little. But I don’t think the real magic was ever in the clicking or typing. It was in the thought process, understanding how things work under the hood, and the experience needed to know what to trust and what to question. :)
The part of this I have to solve as a CTO rather than as an engineer is what it does to the shape of a team.
The model behaves like a very fast junior that makes almost no typing mistakes and no judgement calls at all. Pointed at a senior, it multiplies them. Used instead of hiring one, it produces a large volume of plausible code and nobody who can tell you why the thing is shaped that way. Both look identical for about two quarters, which is where the trap is.
The second problem is the one you name: output used to evidence itself. Now the most valuable hour someone spends might leave no diff at all, and there is no tool that shows it to me. I have started asking what got prevented this week rather than what got shipped. Better than counting lines, still mostly judgement.
How are you judging your own week now that the diff count stopped meaning anything?
The inversion you name is the job now. Writing got cheap, and the expensive part is deciding whether the thing that looks finished is actually finished. Coding agents make that worse because the first artifact you see is often a confident summary, not the change itself. I have stopped treating "tests pass" or a tidy explanation as the end of the work. The real question is which failure modes the implementation never even tried to cover, and that only shows up when you compare the patch to the cases you care about. Becoming the person who defines "good enough to ship" is not demotion. It is where the accountability sits.
The part that lands hardest is the evidence problem rather than the volume one. Writing code left a daily artifact that proved the day happened, and judging, rejecting and steering leaves you almost nothing to point at by evening, even when it was the harder day.
Curious whether you found something that scratches that itch, or whether you eventually just stopped needing it.
The inversion you describe is exactly what changed in my day to day with coding agents. First drafts of endpoints and migrations arrive cheap, and the expensive work is deciding which failure modes still count as ship blockers when everything returns 200. I spend more time reading what the agent changed and asking how it fails under the cases we care about than typing the first version. Curious whether your team has started writing that failure definition before asking for the implementation, or whether it still shows up after the first green demo.
The shift from how do I implement this to how can this fail is the real story here, and it's a much harder skill to hire for because it doesn't show up in a portfolio of shipped features. Curious whether your team has changed how it evaluates engineers now that the visible output, code, isn't the thing that's actually scarce anymore.
There is a nice operator angle here: the best implementation is often the one that makes a bad state obvious early. A clear signal, an owner, and a reversible response path beat a more sophisticated design that fails silently.
I think this is one of the more interesting changes with AI-assisted development. Writing less code doesn’t necessarily mean doing less engineering.
The value is shifting toward designing the solution, reviewing generated code, understanding trade-offs, testing edge cases, and deciding what shouldn't be built.
If AI can handle some of the repetitive implementation work, spending more time on architecture and problem-solving seems like a pretty good trade-off—as long as we don't stop understanding the code we're shipping.
AI is the reason for everything.
At this point AI is at least very good at giving us new reasons to rethink everything we thought was settled in software engineering.
AI makes coding cheaper; judgment, verification, and responsibility become the real engineering.