the six components land. section 2 is the one i would push on.
a gate needs someone accountable for checking it. automate the check and that person only ever sees the gates that went red, so every gate they see is already a failure. the green ones carry no name. that is exactly where a wrong-but-passing result travels, and in observability it does not look like a gap, it looks like green.
so: when a check is automated, does a named owner stay on the passing path too, or does ownership move to the check itself? and if a green check still produced the wrong outcome, what artefact tells you the gate failed rather than the agent?
I like this distinction between a green check and a correct outcome. Passing checks can give us confidence, but they don't automatically mean we've got it right. I think this is where we need to keep improving what we verify and how we know it's enough.
this is the part i want to pin down, because "know it is enough" is the whole question. the cheap move i keep coming back to: every automated gate needs a known-bad input it is required to fail on, run on the same schedule as the real checks. if that fixture ever comes back green, the gate is broken, not the code behind it. without it a green check can only tell you what it happened to look at, never what it missed. do you keep anything like that aimed at your own gates, or does it only show up when something gets through?
I haven't put that into practice yet, but I really like the idea. Testing the gates with known-bad inputs gives us a way to check whether our safeguards are actually doing their job, rather than just assuming they are. That's definitely something worth exploring.
Good. worth exploring is where these usually stop, so here is the cheapest first test. a known-bad input has to come from outside the gate own list. if every test case was written by whoever wrote the gate, you are confirming the code runs, not that it catches anything. take a gate that went red last month, replay its exact input, and see if it still goes red. if it does not, it was never testing the model.
The useful dividing line is between instructions and gates. Once a requirement is mechanically checkable, it should live outside the prompt. The harder question is defining what evidence allows the workflow to resume after a gate fails.
That's a really good point. Defining what happens when a gate fails is just as important as defining the gate itself. Otherwise, we risk automating the failure without actually improving the process.
I'd make the failure path explicit too: hard policy violations block the workflow, fixable issues get a bounded repair loop, and ambiguous cases go to a human with the evidence attached. Otherwise, a gate is just a red light - people eventually find a way around it instead of improving the process.
The bottleneck moving from production to leftover decisions is exactly what I see. Which checks can actually be a gate is something you only learn after a few mistakes. What the harness doesn't reduce, for me, is how many of those leftover decisions remain. Autonomous loops make it cheap to run several units of work at once, and there are mora and more decisions to make. That's decision-fatigue, not review-fatigue.
I hadn't thought about it quite that way, but you're right. Making each unit of work cheaper doesn't necessarily make the decisions around it cheaper. We might end up moving the bottleneck from reviewing work to deciding what to do next, especially when multiple agents are working in parallel.
That's the queue I keep hitting. Three units all passed their checks, and the remaining work is picking which one is allowed to continue. Reviewing the diffs got cheap. Ranking the next move did not, because that ranking still sits with a person, and it doesn't get cheaper when the units do...
That's an interesting challenge. I wonder if we can delegate some of that prioritisation too, using factors like dependencies, business value, and risk. We'd still need human judgement for the bigger trade-offs, but perhaps we don't need to make every small decision ourselves.
Part one is the honest bit, the tools changed and the way we work didn't. The process has to lose something before it can take anything on. If review, planning and handover all stay exactly as they were, the new tooling just gets bolted on top of the old ceremony. Work out which of those practices was only ever there to catch a mistake the model can't make any more, and drop it.
Sharing tutorials and API guides on omnichannel automation. We help devs build seamless end-user workflows across WhatsApp, Telegram, Meta (FB/IG), SMS, and webchat.
This really resonates. The shift from “how do we get the agent to do more?” to “how do we build a system where we can safely let it do more?” feels like the more important conversation right now.
I especially liked the distinction between instructions and actual gates. A prompt can remind an agent to do something, but if the requirement is objectively testable, making it part of the workflow is much more reliable. And the point about verification being separate from generation is easy to overlook when everything happens so quickly.
The biggest takeaway for me is that AI doesn’t necessarily remove engineering discipline—it makes the gaps in our existing processes much more visible. The idea of starting with one small workflow and letting the harness evolve from real friction feels like a much more practical approach than trying to design a perfect AI process upfront.
Backend developer turned builder. 20 years in software, now building Reporails - instruction governance for coding agents. Writing about context engineering and patterns that make AI agents reliable.
It certainly gets complicated when you look beyond the tools themselves! Getting an agent to produce something is one thing, making the whole workflow reliable and repeatable is a different challenge altogether.
Exactly! And I think the tricky part is spotting how many steps we still rely on people remembering to follow. Making those steps enforceable could remove a lot of unnecessary overhead.
We're a place where coders share, stay up-to-date and grow their careers.
the six components land. section 2 is the one i would push on.
a gate needs someone accountable for checking it. automate the check and that person only ever sees the gates that went red, so every gate they see is already a failure. the green ones carry no name. that is exactly where a wrong-but-passing result travels, and in observability it does not look like a gap, it looks like green.
so: when a check is automated, does a named owner stay on the passing path too, or does ownership move to the check itself? and if a green check still produced the wrong outcome, what artefact tells you the gate failed rather than the agent?
I like this distinction between a green check and a correct outcome. Passing checks can give us confidence, but they don't automatically mean we've got it right. I think this is where we need to keep improving what we verify and how we know it's enough.
this is the part i want to pin down, because "know it is enough" is the whole question. the cheap move i keep coming back to: every automated gate needs a known-bad input it is required to fail on, run on the same schedule as the real checks. if that fixture ever comes back green, the gate is broken, not the code behind it. without it a green check can only tell you what it happened to look at, never what it missed. do you keep anything like that aimed at your own gates, or does it only show up when something gets through?
I haven't put that into practice yet, but I really like the idea. Testing the gates with known-bad inputs gives us a way to check whether our safeguards are actually doing their job, rather than just assuming they are. That's definitely something worth exploring.
Good. worth exploring is where these usually stop, so here is the cheapest first test. a known-bad input has to come from outside the gate own list. if every test case was written by whoever wrote the gate, you are confirming the code runs, not that it catches anything. take a gate that went red last month, replay its exact input, and see if it still goes red. if it does not, it was never testing the model.
The useful dividing line is between instructions and gates. Once a requirement is mechanically checkable, it should live outside the prompt. The harder question is defining what evidence allows the workflow to resume after a gate fails.
That's a really good point. Defining what happens when a gate fails is just as important as defining the gate itself. Otherwise, we risk automating the failure without actually improving the process.
I'd make the failure path explicit too: hard policy violations block the workflow, fixable issues get a bounded repair loop, and ambiguous cases go to a human with the evidence attached. Otherwise, a gate is just a red light - people eventually find a way around it instead of improving the process.
The bottleneck moving from production to leftover decisions is exactly what I see. Which checks can actually be a gate is something you only learn after a few mistakes. What the harness doesn't reduce, for me, is how many of those leftover decisions remain. Autonomous loops make it cheap to run several units of work at once, and there are mora and more decisions to make. That's decision-fatigue, not review-fatigue.
I hadn't thought about it quite that way, but you're right. Making each unit of work cheaper doesn't necessarily make the decisions around it cheaper. We might end up moving the bottleneck from reviewing work to deciding what to do next, especially when multiple agents are working in parallel.
That's the queue I keep hitting. Three units all passed their checks, and the remaining work is picking which one is allowed to continue. Reviewing the diffs got cheap. Ranking the next move did not, because that ranking still sits with a person, and it doesn't get cheaper when the units do...
That's an interesting challenge. I wonder if we can delegate some of that prioritisation too, using factors like dependencies, business value, and risk. We'd still need human judgement for the bigger trade-offs, but perhaps we don't need to make every small decision ourselves.
Part one is the honest bit, the tools changed and the way we work didn't. The process has to lose something before it can take anything on. If review, planning and handover all stay exactly as they were, the new tooling just gets bolted on top of the old ceremony. Work out which of those practices was only ever there to catch a mistake the model can't make any more, and drop it.
This really resonates. The shift from “how do we get the agent to do more?” to “how do we build a system where we can safely let it do more?” feels like the more important conversation right now.
I especially liked the distinction between instructions and actual gates. A prompt can remind an agent to do something, but if the requirement is objectively testable, making it part of the workflow is much more reliable. And the point about verification being separate from generation is easy to overlook when everything happens so quickly.
The biggest takeaway for me is that AI doesn’t necessarily remove engineering discipline—it makes the gaps in our existing processes much more visible. The idea of starting with one small workflow and letting the harness evolve from real friction feels like a much more practical approach than trying to design a perfect AI process upfront.
AI Engineering is the most complex thing to do as of today.
It certainly gets complicated when you look beyond the tools themselves! Getting an agent to produce something is one thing, making the whole workflow reliable and repeatable is a different challenge altogether.
The distinction between an instruction and a gate is crucial. If a step can be skipped without becoming visible, it isn’t truly part of the workflow.
Exactly! And I think the tricky part is spotting how many steps we still rely on people remembering to follow. Making those steps enforceable could remove a lot of unnecessary overhead.