I recently watched a coding agent refuse to start an approved implementation.
The work plan contained this:
## WorkPlan Review
- **Status**: approved (2026-08-02)
The agent found the line and understood the word approved. Then it asked for approval again because the workflow expected a field named Implementation Approval.Status.
Nothing was missing in substance. The label was wrong.
The agent was following a repository-resident workflow I had designed to carry a change through requirements, design, review, planning, and implementation. I had spent months adding status fields, handoff contracts, review gates, retry rules, and escalation paths. Each addition addressed a failure I had actually seen. Together, they had made the workflow brittle enough to stop on a heading.
I had built the system around an old question: how do I stop the model from losing the path?
The model could now follow the path. My path had become the problem.
Two kinds of constraint
The approval incident exposed a distinction I had missed.
A boundary constraint reduces the solution space:
- Preserve the public API contract.
- Do not perform an irreversible external action without authority.
- Implement the confirmed requirements and keep recorded non-goals out of scope.
- Treat the task as complete only when the required behavior is observable.
A work-generating constraint creates obligations:
- Always produce three alternatives.
- Require unit, integration, and end-to-end tests for every feature.
- Require a mitigation task, owner, rollback plan, and approval record for every risk.
- Resolve every review finding before continuing.
- Stop when an optional field is absent or uses an unexpected label.
Both can look like safeguards. They behave very differently once the model is capable of carrying them out.
Boundary constraints tell the agent where it may operate. Work-generating constraints are read as requested work. A reliable agent will reliably create the extra artifacts, tests, abstractions, and stop conditions they imply.
That gave me the principle I now use when changing an agent workflow:
Be strict about boundaries and evidence. Be flexible about the path between them.
Why the problem surfaced now
The capability shift behind this is real, although public measurement often trails the models people are using.
METR's task-completion time horizon shows a strong historical increase in the difficulty of software tasks that agents can complete. METR also warns that the benchmark covers well-specified tasks, that capability remains jagged, and that new releases may go unmeasured for weeks or be skipped entirely. As of August 2, its public page was still dated May 8, and even its list of recent unmeasured models had not caught up with later releases.
Actual usage has moved toward longer execution as well. An analysis of roughly 400,000 coding-agent sessions found a recognizable division of labor: people made most planning decisions, while agents made most execution decisions. The authors describe it as people deciding what to build and the agent deciding how to build it.
Neither source proves how today's models behave in my workflow. The model families involved in the failures described here were released after the measurements and benchmarks I found. I did not find a public rerun on those releases.
That timing matters most for FixedBench. In May 2026, the researchers tested five then-current coding models on 200 issues where the code had already been fixed. The agents still made undesirable changes in 35% to 65% of cases. It is useful evidence that action bias existed in that generation. It is not a current failure rate.
The more interesting result is what happened when the researchers changed the instruction. Telling the agents to verify first and treat abstention as success reduced edits to already-correct code. On partially fixed code, the same instruction made them abstain when more work was still required. Prompt engineering traded action bias for passivity.
That is exactly the trap I had created in my own workflow. A model failed once, so I encoded the opposite behavior as a universal rule. When the model or task changed, the compensation remained.
The strongest evidence here comes from the failure history of this workflow on the current models I use. The studies explain why those failures are plausible and why simple prompt fixes decay. They do not substitute for observing the system itself.
How reliability started manufacturing work
The strict approval field was only the most visible failure.
In another run, the workflow stopped before implementation because it could not find external accounting evidence, document approval records, an exact release command, and detailed E2E environment assumptions. The repository, local services, and test tools were available. Most of the implementation could have proceeded. The workflow had converted useful planning context into mandatory readiness gates, then treated every missing gate as a user-owned decision.
The documents had expanded too. Product requirements and design docs started collecting stakeholder approval, live-service access, release procedures, dashboards, and operational evidence. A work plan for a data-export change included account setup and production concerns even though the requested outcome ended at implementation.
Review made the expansion worse. During that same planning flow, a reviewer requested a deterministic 20,000-row proof. The planner accepted the finding without asking what decision the proof would change or whether a cheaper boundary check would be enough. The reviewer then found that the proposed proof data contradicted an existing aggregation contract, which triggered another revision. Every step was technically defensible. The loop itself had no economic judgment.
I had made the reviewer authoritative and the author obedient. “Review until approved” had become “add work until the reviewer runs out of ideas.”
That is not convergence. It is a ratchet.
What I changed
I did not solve this by deleting every rule. I changed where the workflow is strict.
Decide the outcome before designing the solution
A user request is no longer accepted as an automatically valid implementation scope. Before design, the workflow records:
- the observable outcome;
- what is required now;
- what describes the current state, a desired future, or speculation;
- explicit non-goals;
- a rough view of implementation cost and structural impact.
The cost assessment stays rough because this is requirements work, not detailed estimation. Its job is to make a bad tradeoff visible while scope is still cheap to remove.
“No change” and “reuse what already exists” are valid conclusions, but neither is a default. The agent must inspect enough evidence to decide whether the outcome is already satisfied, partially satisfied, or still requires implementation. That avoids turning FixedBench's action bias into its mirror image.
This also changes what users need to communicate. A useful request contains the boundary of the current outcome, not just a list of desired features. “Extend the existing authentication path without changing the public response contract” is more valuable than a long list of implementation rules. It states both the result and the point at which design needs to be reconsidered.
Give the model decisions, not a predicted route
The old workflow described complete routes: run one agent, populate every field, run another agent, stop on any missing input, repeat until a serialized state appears.
I now try to give each phase four things:
- the purpose it owns;
- the evidence that can change its decision;
- the criteria for choosing the next action;
- the smallest result the next consumer needs.
“Run every test lane” predicts a route. “Use the narrowest test that observes the required boundary” provides a decision criterion. Exact schemas still matter where software parses the response. A human-readable work plan does not need to fail because two headings express the same approved state. I changed the approval check to accept semantically equivalent evidence instead of requiring one heading.
The main agent also owns lightweight resolution. It can interpret semantically equivalent states, resolve repository-local ambiguity, retry with new evidence, and continue unaffected work. User escalation is reserved for decisions that belong to the user: a changed product outcome, a new requirement, a major approved design change, unavailable authority, or an irreversible external action.
Let review findings be declined
The receiver of a review now has three choices:
- Apply a finding that contradicts an approved requirement, accepted design, repository rule, or observable correctness.
- Decline a finding that adds scope, reverses an exclusion, duplicates proof, requests optional hardening, or costs more than its observable effect justifies.
- Return for user decision when resolution changes the product outcome or a major approved decision.
A decline includes evidence and goes back to the reviewer. The reviewer may maintain the finding when that evidence still leaves the result incorrect or unverifiable. Repeating the same preference without new evidence does not block the workflow.
A useful review process preserves a basic engineering skill: deciding not to implement a technically reasonable suggestion.
The constraint I had to put back
Removing rules can be just as careless as adding them. I proved that during this rewrite.
The planning flow generated integration and E2E test skeletons so the first vertical slice could prove an acceptance criterion across a real boundary as early as possible.
After I simplified the handoff, the planner treated the generated test skeleton as an unrelated file and planned a new E2E test later in the project. The artifact still existed, so the workflow looked complete. Its reason for existing had disappeared.
I restored the skeleton's purpose in the planner: consume the exact generated file in the earliest task that can make its boundary executable. Shared infrastructure may come first only when no acceptance criterion can work without it, and only the infrastructure needed by that first slice belongs there.
This is a boundary constraint worth keeping. It connects a planned implementation to observable proof and prevents integration risk from being postponed until every component has been built. Removing it made the workflow smaller and less reliable.
What remains strict
Several controls still earn their cost:
- User approval for product requirements and major design decisions.
- Explicit authority and mechanical containment for irreversible actions.
- Exact schemas at genuinely machine-consumed boundaries.
- Plans for work with real dependencies.
- Early vertical proof of an accepted outcome.
- Observable verification before completion.
- Independent review where a separate perspective can catch false confidence.
For dangerous operations, sandboxes, restricted credentials, isolated filesystems, and network controls are stronger than long prompt instructions. They limit the blast radius even when judgment is imperfect. Repeated approval prompts are a weaker substitute: one implementation report describes high approval rates and declining attention, and responds by using automated judgment to reduce routine prompts.
The same division applies to the workflow as a whole. Enforce the durable boundary mechanically where possible. Let the model choose among reversible actions inside it. An account of an agent-first engineering environment uses a similar principle: treat the top-level instructions as a table of contents rather than an encyclopedia.
The four questions I now ask
I now use four questions when reviewing an existing rule:
- What changes if I remove it? Name the decision, irreversible boundary, downstream consumer, or observable failure it affects.
- Does it limit scope or create work? Keep durable boundaries. Require current evidence before generating artifacts, test lanes, approval records, retry loops, or operational tasks.
- Can the model choose from evidence? Replace exhaustive branches with a purpose and selection criteria when the choice is reversible.
- Does it work in a fresh session? Rerun a representative task from the beginning. Continuing the old conversation only proves that a human can steer around the failure.
The fresh-session test has to cover both sides. A small change should remain small, ordinary ambiguity should not return to the user, and optional review findings should not expand scope. Required tests, approvals, and quality checks must still run. Fewer rules are not evidence of a better workflow.
A workflow can become technical debt
The hardest lesson for me was that reliability work can itself become overengineering.
Every workflow rule depends on an assumption about model behavior. A rule added after yesterday's failure can become redundant when the model improves, or harmful when a different task needs the opposite judgment. Status fields and review gates add coupling just as surely as public interfaces and shared abstractions do.
Boundary constraints age better because they describe what must remain true: the approved outcome, the public contract, the authority available, and the evidence required for completion. The route between those boundaries can change with the model.
I no longer ask only whether a new rule would have prevented the last failure. I also ask what work it will create, what decision it will remove from the model, and whether it will still make sense after the failure mode disappears.
The job of an agent workflow is no longer to make a model capable of finishing the process. It is to help a capable model finish the right amount of work.
Top comments (22)
"Be strict about boundaries and evidence, flexible about the path" is the most portable line here — I'm going to steal it for how we review agent workflow changes.
The distinction between boundary and work-generating constraints explains a failure I kept mis-diagnosing: every scaffold I added to stop a weaker model wandering became requested work the moment the model got capable enough to execute it literally. The approval-field-label story is perfect because nothing was wrong in substance — the agent did exactly what a reliable executor should, and reliability was the problem.
Where this gets sharp is that boundary constraints are the ones you can safely make rigid (API contract, no irreversible external action, done-means-observable), while work-generating ones want to be hints, not gates. We've started literally tagging steps in our workflow specs as
boundaryvspath, and on each model upgrade we only re-audit the path ones. Have you found a good way to keep the path flexible without losing the audit trail that made you add those gates in the first place?What’s worked for me is treating the audit trail as an output contract rather than a process contract.
The workflow still persists the approved outcome, requirements, non-goals, and accepted design decisions. Tasks cite the sections and acceptance criteria they implement, and completion requires observable verification. For each review finding, we record whether it was applied, declined, or sent back for a user decision, and why. Implementation lands in task-scoped commits.
What changed is that none of this has to be produced through one exact sequence or schema. Semantically equivalent approval evidence counts. Missing optional fields don’t stop the run. Reviewers aren’t automatically authoritative, and the orchestrator can choose the narrowest check that actually observes the boundary.
The E2E skeleton failure in the post is the caveat: the artifact survived, but its purpose did not. So I now treat the link between an artifact and the boundary it is meant to prove as part of the contract too. That’s the case I’d watch for when re-auditing only the path steps.
That keeps traceability fixed while letting the route change. Your boundary/path tagging feels like the same split applied one level up, to the workflow spec itself.
I like your point that a better model can be held back by yesterday’s safeguards. I used to assume that adding more rules would make them more reliable, but sometimes the workflow just becomes a collection of fixes for older models.
Yes, I made the same assumption. Each rule had fixed a real failure at the time. Newer runs showed that some of those fixes had started generating work of their own.
I like the audit-trail-as-output-contract framing. The trap I keep seeing is agents treating process artifacts as work products, so the harness starts optimizing for paperwork instead of preserving the decision. I’d keep only two mandatory records. One says what was authorized. The other says which observable check proved the change did the authorized thing.
I like that two-record framing. I think of the approved outcome and its proof as the endpoints, but for non-trivial work the agents need artifacts that connect them. When the intended value is still ambiguous, the PRD turns it into an explicit outcome and requirements. The Design Doc connects that outcome to implementation choices and defines how it can be proved. The Work Plan incorporates the tests or checks needed to prove it and schedules them at the earliest point the implementation can make them executable.
Humans can inspect and approve those documents, but their operational purpose is to give each downstream agent the decisions and proof obligations it needs without inheriting the earlier conversation. If an artifact isn’t giving a later agent something it needs to implement or verify the approved outcome, it doesn’t earn its place.
The model-upgrade audit I would add is a paired canary on the same representative task: compare boundary violations, unnecessary artifacts, review-loop length, and proof of the requested outcome, not just pass/fail. Then classify each rule by the failure it prevents and the work it creates. If a rule only compensates for a failure that no longer appears, remove it or make it advisory, but keep the outcome-to-proof link as a hard invariant. That gives the workflow a way to adapt without turning every model release into a rewrite.
That’s a useful way to structure a model-upgrade eval. I usually run the workflow end to end on real work, fix whatever concrete failure appears, and run it again. That’s how this rewrite began with overimplementation and then uncovered problems with approval handling, readiness gates, review loops, and finally the E2E skeleton. Real runs expose failures I didn’t know to measure; a paired canary would make known failures easier to compare. Tracking the failure each rule prevents and the work it creates would help interpret the result.
The skeleton suggests one more signal: whether an artifact still serves its intended role in the workflow. The skeleton still existed, but the planner no longer used it in the earliest executable slice. Artifact count and eventual pass/fail could look fine even after that early proof had disappeared, so I’d check the artifact’s role as well as the outcome-to-proof link.
The boundary constraint / work-generating constraint split is sharp. I've been reaching for similar language in robot control without ever naming it cleanly â this is the right framing.
The tension that keeps surfacing for me: the two categories aren't actually binary. Most real constraints sit somewhere on a spectrum between them. "Re-plan if confidence drops below 60%" looks like a boundary â it restricts the solution space. But triggering it creates a new planning obligation, which is work-generating. The same constraint is both simultaneously. The distinction is less about the constraint's structure and more about whether the model treats it as a ceiling or a floor.
This is where embodied agents hit it harder than code agents. A robot's safety constraint ("don't cross the red zone") is a genuine boundary in physical space â violating it has irreversible consequences no model capability can paper over. But "re-plan if confidence is low" is a work-generating constraint that fires on a heuristic, and the model will generate the work it implies. On a 10Hz control loop, that work is genuinely expensive.
We handle this in moteDB by making the constraint layer reason about urgency: hard boundaries (collision, thermal limits) bypass the model entirely and go straight to the actuator safety circuit; soft constraints (trajectory confidence, resource pressure) get surfaced as memory predicates the model can query. The model's plan is informed by the constraint state but doesn't spend compute generating work to satisfy it unless the constraint is actually binding.
FixedBench's result on abstention is the most practically useful number in this piece. If 35-65% of regressions come from agents treating the benchmark as "do something" rather than "verify first then decide," that's a prompt architecture problem, not a model capability problem. The same 35-65% probably applies to most production workflows designed before models could be trusted with the word "stop."
I like the ceiling-versus-floor distinction. The split may be more useful as a way to inspect a rule’s effects than as a binary taxonomy: one rule can contain both the condition that must remain true and the work generated when it changes. Your split between actuator safety circuits and queryable predicates is a very concrete implementation of that. The 10 Hz example makes the cost of getting that wrong much more visible than it is in a coding workflow.
FixedBench is also where the prompt-architecture point gets more interesting for me. Telling agents to verify first and treat abstention as success reduced over-editing, but on partially fixed code it made them abstain while work was still required. The bias moved rather than disappeared. That is the shape I now expect when a workflow encodes the opposite of yesterday’s failure as a universal rule.
The boundary vs work-generating constraint split is the cleanest diagnosis I've seen for "why did my agent harness get worse after the model got better."
I keep hitting the same ratchet: weaker model fails → encode the opposite as a universal gate → stronger model executes the gate literally → reliability manufactures work. Label-sensitive approval is the pure form of that failure.
Practical pattern that has held up across model upgrades: keep boundary checks mechanical (schema/tests/sandbox/credentials) and keep path steps as decision criteria with evidence, not required artifacts. The audit trail then lives in a decision log ("why we chose this test lane") instead of mandatory fields that no longer match reality.
Decline-with-evidence for review findings is underrated too — without it, review becomes a scope ratchet.
That’s where I ended up too, especially on test lanes. The workflow only adds a wider test when a named observable claim crosses a real boundary and a cheaper check can’t observe it. Selecting none is a successful result; the workflow doesn’t generate an artifact just to explain the absence.
The one place I did something different was the decision log. I didn’t make it a separate required artifact. A mandatory log can fall into the same ratchet you described. It starts as a place to capture rationale and ends as another field that must be filled whether or not it still matches reality. Instead, I keep the rationale in the artifact where it will be used. The Design Doc ties each claim to a verification level and an observable pass condition. If the workflow generates a test skeleton, it carries the acceptance criterion and the failure mode it is there to catch. The reason for the test stays inspectable without fixing the route used to produce it.
Decline-with-evidence was one of the highest-leverage changes in the rewrite. A decline goes back to the reviewer with evidence. If they still see a real problem, they can keep the finding open. What no longer blocks progress is the same preference repeated without new evidence. That turned review from a scope ratchet back into a boundary check.
A stronger model often reveals assumptions that were quietly built into the workflow. Teams that keep prompts, tool routing, and evaluation loosely coupled can adapt much faster to new model releases instead of redesigning everything after each upgrade.
I’ve found loose coupling harder than it sounds. The approved outcome has to survive through design and planning all the way to observable proof. In trying to preserve that chain, I made parts of the workflow depend on one another through duplicated status fields and handoff assumptions, and real runs kept exposing mismatches between them. A large part of this rewrite was keeping the chain while removing those accidental dependencies.
The workflow stopping on approved versus Implementation Approval.Status is a great illustration of scaffolding outliving the problem it solved. Once the model can hold the path on its own, every rigid handoff field you added becomes a new place for it to trip. How are you deciding which guardrails to retire as the base model improves, rather than only adding new ones after each failure?
I don’t have a separate guardrail-retirement pass. It usually happens in the review and rerun loop. When a run exposes a mismatch, I work through the immediate problem, but lately I also ask whether the mechanism has become over-constrained rather than assuming it needs another rule. Then I look for the same mismatch elsewhere and decide whether it needs a local patch or whether the rule should be consolidated and simplified. After the change, I rerun the full workflow on real work.
That is what happened with approval. Adding another accepted field name would have fixed the symptom; instead, I made approval depend on meaning rather than field names. The readiness and review-loop failures kept pointing at the same pattern, which turned a local fix into a broader rewrite.
The E2E skeleton case worked the other way. Simplifying the handoff removed its early-proof role, so I put that constraint back. Retiring a guardrail is a change like any other, so I rerun the workflow before I trust it.
Sow done with "better models" 😠
I know the feeling 😅 “Better” always seems to come with homework.
Amazing share
Thanks, Glad you found it useful.
Some comments may only be visible to logged-in visitors. Sign in to view all comments.