I recently watched a coding agent refuse to start an approved implementation.
The work plan contained this:
## WorkPlan Review
- **Status**: a...
For further actions, you may consider blocking this person and/or reporting abuse
"Be strict about boundaries and evidence, flexible about the path" is the most portable line here — I'm going to steal it for how we review agent workflow changes.
The distinction between boundary and work-generating constraints explains a failure I kept mis-diagnosing: every scaffold I added to stop a weaker model wandering became requested work the moment the model got capable enough to execute it literally. The approval-field-label story is perfect because nothing was wrong in substance — the agent did exactly what a reliable executor should, and reliability was the problem.
Where this gets sharp is that boundary constraints are the ones you can safely make rigid (API contract, no irreversible external action, done-means-observable), while work-generating ones want to be hints, not gates. We've started literally tagging steps in our workflow specs as
boundaryvspath, and on each model upgrade we only re-audit the path ones. Have you found a good way to keep the path flexible without losing the audit trail that made you add those gates in the first place?What’s worked for me is treating the audit trail as an output contract rather than a process contract.
The workflow still persists the approved outcome, requirements, non-goals, and accepted design decisions. Tasks cite the sections and acceptance criteria they implement, and completion requires observable verification. For each review finding, we record whether it was applied, declined, or sent back for a user decision, and why. Implementation lands in task-scoped commits.
What changed is that none of this has to be produced through one exact sequence or schema. Semantically equivalent approval evidence counts. Missing optional fields don’t stop the run. Reviewers aren’t automatically authoritative, and the orchestrator can choose the narrowest check that actually observes the boundary.
The E2E skeleton failure in the post is the caveat: the artifact survived, but its purpose did not. So I now treat the link between an artifact and the boundary it is meant to prove as part of the contract too. That’s the case I’d watch for when re-auditing only the path steps.
That keeps traceability fixed while letting the route change. Your boundary/path tagging feels like the same split applied one level up, to the workflow spec itself.
I like your point that a better model can be held back by yesterday’s safeguards. I used to assume that adding more rules would make them more reliable, but sometimes the workflow just becomes a collection of fixes for older models.
Yes, I made the same assumption. Each rule had fixed a real failure at the time. Newer runs showed that some of those fixes had started generating work of their own.
I like the audit-trail-as-output-contract framing. The trap I keep seeing is agents treating process artifacts as work products, so the harness starts optimizing for paperwork instead of preserving the decision. I’d keep only two mandatory records. One says what was authorized. The other says which observable check proved the change did the authorized thing.
I like that two-record framing. I think of the approved outcome and its proof as the endpoints, but for non-trivial work the agents need artifacts that connect them. When the intended value is still ambiguous, the PRD turns it into an explicit outcome and requirements. The Design Doc connects that outcome to implementation choices and defines how it can be proved. The Work Plan incorporates the tests or checks needed to prove it and schedules them at the earliest point the implementation can make them executable.
Humans can inspect and approve those documents, but their operational purpose is to give each downstream agent the decisions and proof obligations it needs without inheriting the earlier conversation. If an artifact isn’t giving a later agent something it needs to implement or verify the approved outcome, it doesn’t earn its place.
The model-upgrade audit I would add is a paired canary on the same representative task: compare boundary violations, unnecessary artifacts, review-loop length, and proof of the requested outcome, not just pass/fail. Then classify each rule by the failure it prevents and the work it creates. If a rule only compensates for a failure that no longer appears, remove it or make it advisory, but keep the outcome-to-proof link as a hard invariant. That gives the workflow a way to adapt without turning every model release into a rewrite.
That’s a useful way to structure a model-upgrade eval. I usually run the workflow end to end on real work, fix whatever concrete failure appears, and run it again. That’s how this rewrite began with overimplementation and then uncovered problems with approval handling, readiness gates, review loops, and finally the E2E skeleton. Real runs expose failures I didn’t know to measure; a paired canary would make known failures easier to compare. Tracking the failure each rule prevents and the work it creates would help interpret the result.
The skeleton suggests one more signal: whether an artifact still serves its intended role in the workflow. The skeleton still existed, but the planner no longer used it in the earliest executable slice. Artifact count and eventual pass/fail could look fine even after that early proof had disappeared, so I’d check the artifact’s role as well as the outcome-to-proof link.
The boundary constraint / work-generating constraint split is sharp. I've been reaching for similar language in robot control without ever naming it cleanly â this is the right framing.
The tension that keeps surfacing for me: the two categories aren't actually binary. Most real constraints sit somewhere on a spectrum between them. "Re-plan if confidence drops below 60%" looks like a boundary â it restricts the solution space. But triggering it creates a new planning obligation, which is work-generating. The same constraint is both simultaneously. The distinction is less about the constraint's structure and more about whether the model treats it as a ceiling or a floor.
This is where embodied agents hit it harder than code agents. A robot's safety constraint ("don't cross the red zone") is a genuine boundary in physical space â violating it has irreversible consequences no model capability can paper over. But "re-plan if confidence is low" is a work-generating constraint that fires on a heuristic, and the model will generate the work it implies. On a 10Hz control loop, that work is genuinely expensive.
We handle this in moteDB by making the constraint layer reason about urgency: hard boundaries (collision, thermal limits) bypass the model entirely and go straight to the actuator safety circuit; soft constraints (trajectory confidence, resource pressure) get surfaced as memory predicates the model can query. The model's plan is informed by the constraint state but doesn't spend compute generating work to satisfy it unless the constraint is actually binding.
FixedBench's result on abstention is the most practically useful number in this piece. If 35-65% of regressions come from agents treating the benchmark as "do something" rather than "verify first then decide," that's a prompt architecture problem, not a model capability problem. The same 35-65% probably applies to most production workflows designed before models could be trusted with the word "stop."
I like the ceiling-versus-floor distinction. The split may be more useful as a way to inspect a rule’s effects than as a binary taxonomy: one rule can contain both the condition that must remain true and the work generated when it changes. Your split between actuator safety circuits and queryable predicates is a very concrete implementation of that. The 10 Hz example makes the cost of getting that wrong much more visible than it is in a coding workflow.
FixedBench is also where the prompt-architecture point gets more interesting for me. Telling agents to verify first and treat abstention as success reduced over-editing, but on partially fixed code it made them abstain while work was still required. The bias moved rather than disappeared. That is the shape I now expect when a workflow encodes the opposite of yesterday’s failure as a universal rule.
The boundary vs work-generating constraint split is the cleanest diagnosis I've seen for "why did my agent harness get worse after the model got better."
I keep hitting the same ratchet: weaker model fails → encode the opposite as a universal gate → stronger model executes the gate literally → reliability manufactures work. Label-sensitive approval is the pure form of that failure.
Practical pattern that has held up across model upgrades: keep boundary checks mechanical (schema/tests/sandbox/credentials) and keep path steps as decision criteria with evidence, not required artifacts. The audit trail then lives in a decision log ("why we chose this test lane") instead of mandatory fields that no longer match reality.
Decline-with-evidence for review findings is underrated too — without it, review becomes a scope ratchet.
That’s where I ended up too, especially on test lanes. The workflow only adds a wider test when a named observable claim crosses a real boundary and a cheaper check can’t observe it. Selecting none is a successful result; the workflow doesn’t generate an artifact just to explain the absence.
The one place I did something different was the decision log. I didn’t make it a separate required artifact. A mandatory log can fall into the same ratchet you described. It starts as a place to capture rationale and ends as another field that must be filled whether or not it still matches reality. Instead, I keep the rationale in the artifact where it will be used. The Design Doc ties each claim to a verification level and an observable pass condition. If the workflow generates a test skeleton, it carries the acceptance criterion and the failure mode it is there to catch. The reason for the test stays inspectable without fixing the route used to produce it.
Decline-with-evidence was one of the highest-leverage changes in the rewrite. A decline goes back to the reviewer with evidence. If they still see a real problem, they can keep the finding open. What no longer blocks progress is the same preference repeated without new evidence. That turned review from a scope ratchet back into a boundary check.
A stronger model often reveals assumptions that were quietly built into the workflow. Teams that keep prompts, tool routing, and evaluation loosely coupled can adapt much faster to new model releases instead of redesigning everything after each upgrade.
I’ve found loose coupling harder than it sounds. The approved outcome has to survive through design and planning all the way to observable proof. In trying to preserve that chain, I made parts of the workflow depend on one another through duplicated status fields and handoff assumptions, and real runs kept exposing mismatches between them. A large part of this rewrite was keeping the chain while removing those accidental dependencies.
The workflow stopping on approved versus Implementation Approval.Status is a great illustration of scaffolding outliving the problem it solved. Once the model can hold the path on its own, every rigid handoff field you added becomes a new place for it to trip. How are you deciding which guardrails to retire as the base model improves, rather than only adding new ones after each failure?
I don’t have a separate guardrail-retirement pass. It usually happens in the review and rerun loop. When a run exposes a mismatch, I work through the immediate problem, but lately I also ask whether the mechanism has become over-constrained rather than assuming it needs another rule. Then I look for the same mismatch elsewhere and decide whether it needs a local patch or whether the rule should be consolidated and simplified. After the change, I rerun the full workflow on real work.
That is what happened with approval. Adding another accepted field name would have fixed the symptom; instead, I made approval depend on meaning rather than field names. The readiness and review-loop failures kept pointing at the same pattern, which turned a local fix into a broader rewrite.
The E2E skeleton case worked the other way. Simplifying the handoff removed its early-proof role, so I put that constraint back. Retiring a guardrail is a change like any other, so I rerun the workflow before I trust it.
Sow done with "better models" 😠
I know the feeling 😅 “Better” always seems to come with homework.
Amazing share
Thanks, Glad you found it useful.