DEV Community

t49qnsx7qt-kpanks
t49qnsx7qt-kpanks

Posted on

the harness-design problem: why agent governance is an engineering question, not a compliance one

the harness-design problem: why agent governance is an engineering question, not a compliance one

r/LocalLLaMA in april gave us a cleaner signal than most enterprise analyst reports: "agent performance is increasingly a harness-design problem, not only a weights problem."

that's the practitioner version of what governance frameworks are circling around but rarely landing on. the governance problem isn't about policy documents or audit checklists at the org level. it's about whether the execution harness gives you the control surfaces you actually need — review queues, rollback paths, supervision triggers, and after-the-fact reconstructability.

most builders have the model right. most builders don't have the harness right.

what the reddit synthesis is actually telling you

CTLabs' community synthesis identified three consistent patterns in enterprise agent deployments that work:

  1. plan-first architecture — the agent generates a full execution plan before any action is taken. the plan is reviewable before it runs.
  2. staged execution — actions are batched at natural checkpoints with human or automated review gates between stages.
  3. supervision with rollback — agents operate best in contexts where a reviewer can reject a proposed action and the system can recover cleanly, not just log the rejection.

these aren't best practices from a governance framework. they're what practitioners converged on after watching agents fail in production. the governance layer follows the harness requirements, not the other way around.

review queues are an engineering artifact

the pattern that consistently shows up across successful enterprise deployments: agents propose, humans (or automated policy checks) approve, the action executes only after approval.

this sounds obvious. the failure mode is subtle: most teams implement the review queue as a UX feature bolted onto an agent that already executes autonomously. when the agent runs first and the review queue captures the result, you have a log — not a control. a control intercepts execution before the action fires.

# this is a log, not a control
result = agent.run(task)
audit_log.record(result)   # too late — the action already happened

# this is a control
action_plan = agent.plan(task)
approval = review_queue.submit(action_plan)  # blocks until approved or rejected
if approval.granted:
    result = agent.execute(action_plan)
Enter fullscreen mode Exit fullscreen mode

the second pattern is harder to build because it requires the agent to expose its intended actions before execution. but it's the only pattern that gives you an actual rollback path — you can reject the plan before anything irreversible happens.

rollback is a first-class requirement, not an afterthought

the reddit thread on rollback paths surfaced the same problem across multiple teams: rollback was designed for the happy path. if the agent completed step 3 before the review flag fired, rolling back steps 1 and 2 was either impossible (because the side effects were external) or expensive (because the undo logic wasn't written when the do logic was).

the fix requires two things at the harness design stage:

  • compensating transactions for every irreversible action — not optional. if your agent can send an email, delete a file, or charge a card, there's a compensating action (recall, restore, refund). that action needs to be defined and tested before the agent ships.
  • checkpoint state serialization — at every stage boundary, the agent's full state is serialized. if a review rejects an action at stage 4, you can restore to the serialized state at stage 3 and replay from there with a corrected plan.

BizSuite's AI Audit covers exactly this surface: a 48-hour gap analysis that maps your existing agent harness against the control surface requirements — review queue architecture, rollback path coverage, audit trail completeness, and policy enforcement points. the entry price is $997 and the output is a prioritized remediation plan, not a PDF.

the governance conversation you should be having with your team

the question isn't "do we have a governance policy?" most teams do. the question is: "if an agent took a wrong action at 2am last tuesday, could we reconstruct exactly what it decided, why, what it did, and what the state of the world was before and after?"

if the answer involves more than two steps of log triage, the harness isn't right. the EU AI Act's August 2 enforcement date makes this concrete for any enterprise operating in the EU — documented decision logic and human oversight verification aren't aspirational; they're required.

the harness is the governance. everything else is paperwork.

read more about the AI Audit at https://getbizsuite.com/ai-audit

Top comments (0)