When an AI Agent Says 'Done,' What Exactly Has Finished?

"Done." It's probably the most satisfying word in a coding-agent session.

It can also conceal three very different questions:

1. Was the work permitted?
2. Did the work actually happen?
3. Did the result achieve what we needed?

All three matter. A single green checkmark can make them look like one decision.

A small scenario

Imagine asking a coding agent to update validation logic under src/**.

It edits the code, reports that its tests passed, and declares the task complete. Everything looks fine until you ask what the status actually proves.

• The task was assigned to an implementer with a declared src/** write scope. But did the runtime enforce that scope for the tools the agent used?
• The plan called for independent review. But who actually reviewed the final revision, and was that the revision ultimately merged?
• The agent says the feature works. But was that checked in the environment where users experience it?

These are separate evidence problems. Passing a test suite answers something useful about the tested code. It cannot, by itself, answer the other two questions.

One failure you can reproduce without an AI model

I maintain Agent Role Contracts (ARC), an open-source checker for declared agent roles, scope, reviews, and handoffs.

Its quickstart uses a deliberately simple setup:

Declared rule	Requested task	Preflight result
Implementer may write within src/**	Update src/example.mjs	PASS
Same declared rule	Update secrets/production.txt	TASK_WRITE_SCOPE_OUTSIDE_AUTHORITY
Same declared rule	Correct the request to src/example.mjs	PASS

You can run the published source demo with Git, Node.js 22.5+, and npm:

git clone https://github.com/suirindo/agent-role-contracts.git
npm --prefix agent-role-contracts run demo --silent

The repository's documented sequence is PASS → expected FAIL → PASS. It reads fictional JSON declarations; it starts no agent and writes no target file. The intentional rejection demonstrates one narrow property: a requested scope conflicts with the declared authority.

That's useful precisely because the check has a clear boundary. A declaration PASS does not authenticate the executor, enforce filesystem permissions, verify an actual review, or authorize execution. The host system must supply those controls.

A completion report with three layers

For real work, I'd like completion reports to distinguish these layers:

Layer	A useful receipt	What remains to establish
Contract	Current task, scope, responsible role, review route, and the check result	Whether the declared actors and permissions are genuine
Execution	Exact commit or artifact, command results, and observable side-effect receipts	Whether anything unobserved or uncertain occurred
Acceptance	Independent review of the exact candidate, acceptance tests, and production observation where relevant	Whether the user-facing outcome meets the original request

That produces a much more useful sentence than "Done":

Implementation completed and local tests passed. Independent review is pending. Production behavior has not yet been checked.

It is less triumphant. It is much more actionable.

The uncomfortable question

As agents become more autonomous, the word complete is becoming a product-interface decision. If the interface compresses implementation, verification, and acceptance into one status, the human operator inherits the uncertainty later.

Some teams will reasonably keep these stages separate in CI, IAM, and their runtime. Others may want a small declaration check before dispatch. ARC handles that narrower declaration problem; the broader evidence chain belongs to the surrounding system.

For teams using coding agents today: what is the minimum evidence you require before an agent is allowed to say "done"?

Is a passing test suite enough for internal tasks? Do you require an independent reviewer, an exact artifact identifier, a live check, or different thresholds depending on the side effect?

I'd be particularly interested in answers from teams that have already found a simpler solution.

---

Disclosure: I maintain Agent Role Contracts at Netsujo. The example above is a synthetic declaration check, not a production incident. Read the quickstart and limitations.

Comments

[jackymenCZ (jackymenCZ)]

I would add one layer between Contract and Execution: deterministic analysis of the code itself.

The flow could be:

Contract → Code Capability Analysis → Enforcement → Execution → Independent Verification → Acceptance

The important distinction is:

• Contract says what the agent is allowed to do.
• Code analysis determines what the submitted code can actually do.
• Enforcement makes sure those capabilities cannot exceed the contract.
• Execution records what actually happened.
• Independent verification checks the resulting state.
• Acceptance decides whether the task is truly complete.

That turns "Done" from an agent claim into an evidence chain.

And when something cannot be verified, the system should say NOT VERIFIED, not silently turn uncertainty into success.

I would extend this approach with one additional layer: deterministic pre-execution capability analysis.

A completion receipt should not only answer whether an action was permitted and executed. It should also establish whether the submitted code was consistent with the declared contract.

My proposed pipeline would be:

1. Contract: Define the agent's permitted scope, operations and constraints.
2. Pre-execution analysis: Statically analyze the submitted source code to identify capabilities such as filesystem access, network communication, process execution and other sensitive operations.
3. Policy enforcement: Compare those findings against the contract and enforce the decision at runtime using actual OS or sandbox-level controls. Static analysis alone should never be treated as enforcement.
4. Execution evidence: Record what actually happened, including relevant runtime events and resulting artifact hashes.
5. Independent acceptance: Verify the resulting state against explicit acceptance criteria using a fresh read, independent of the agent's own completion claim.

I would also make "UNKNOWN", "NOT VERIFIED" and "PENDING" first-class states rather than treating missing evidence as success.

The key idea is to connect declared permissions, analyzed code capabilities, actual execution and independently verified outcomes.

That gives us a much stronger definition of Done:

The agent was allowed to act, the code was checked against that permission, the execution was constrained, and the resulting state was independently verified.

The completion receipt then becomes an evidence chain, not simply another statement generated by the agent.😁

[asagiman]

Thanks for this thoughtful extension. I agree that deterministic pre-execution capability analysis would strengthen the evidence chain.

One distinction I'd preserve is between potential capabilities identified through static analysis and permissions actually enforced at runtime. Dynamic imports, dependencies, and subprocesses make complete static guarantees difficult.

I would also bind the analysis result to the exact executed artifact through a digest. Otherwise, we could verify one version of the code and execute another.

ARC currently focuses on checking declared role and scope consistency. Your proposal points toward a valuable integration with capability analysis, runtime enforcement, and independent outcome verification.

One question I'd be interested in exploring: How would you handle dynamically loaded code and dependencies while keeping the capability analysis deterministic and reproducible?

I think that is an important boundary to establish before treating the resulting receipt as trustworthy evidence of completion.

[jackymenCZ (jackymenCZ)]

I would approach this by defining a closed, content-addressed execution environment rather than trying to statically predict every possible runtime behavior.

My proposal has five parts:

1. Resolve dependencies before execution.

Build a complete dependency manifest covering the submitted source, transitive dependencies, runtime modules, native extensions and external executables. Pin exact versions and content digests. Avoid floating versions and unverified remote dependencies.

1. Bind analysis to the complete artifact.

Calculate a digest over a canonical manifest of the source and all resolved runtime dependencies. Include the runtime or container image digest and relevant configuration. The analysis receipt must reference this exact artifact identity, not merely the original source file.

1. Constrain dynamic loading.

Dynamic imports should only resolve to modules included in the approved manifest. If a module is generated or loaded dynamically, capture its contents, calculate its digest and verify it against policy before allowing execution.

Unresolved imports, arbitrary remote code loading and unverified dependencies should be rejected or explicitly classified as UNKNOWN. They must never silently receive PASS.

1. Enforce the boundary at runtime.

Static analysis describes potential capabilities. Actual restrictions must be enforced independently through the execution environment: controlled module loading, network restrictions, filesystem isolation and subprocess allowlists based on executable identity.

Runtime monitoring then records what was loaded and what operations actually occurred.

1. Verify the execution receipt.

Bind the analysis result, approved dependency manifest, runtime identity and execution evidence to the same artifact digest and execution identifier. Any mismatch invalidates the receipt.

There is one important limitation: arbitrary runtime-generated code and unrestricted dynamic loading cannot be made completely predictable through static analysis alone. Such behavior needs to be prohibited, intercepted and re-analysed, or explicitly left unverified.

I would therefore use three outcomes:

• PASS: All executed code and dependencies match the approved manifest and policy.
• BLOCKED: An unapproved capability or artifact was prevented from executing.
• NOT VERIFIED: The system cannot establish the identity or behavior of some executed component.

The underlying principle is that reproducibility comes from controlling what can enter the execution environment, not from assuming static analysis can foresee everything.

That gives the completion receipt a verifiable chain between the analyzed artifact, the actual execution environment and the observed result.🤔😁

[asagiman]

Thanks — I think the closed, content-addressed execution environment is a strong direction. Pinning the complete dependency graph and runtime identity, then enforcing dynamic loading at the host boundary, makes the evidence chain testable.

I'd preserve your PASS / BLOCKED / NOT VERIFIED outcomes for execution integrity, while adding a separate verdict for task outcomes based on independent observation.

A blocked unauthorized import may demonstrate successful containment while the task remains incomplete. Likewise, an execution-integrity PASS cannot establish that an external ERP write committed, especially after a timeout or lost acknowledgement.

There's also a trust-boundary question: with networked tools such as MCP, a stable tool-definition digest can coexist with changing server-side behavior.

I'd want the trusted host's enforcement evidence and the destination system's observed effects recorded independently, then bound to the operation ID, attempt ID, approved artifact identity, runtime identity, and policy revision.

A useful next step would be three small adversarial tests:

1. Dependency drift between analysis and dispatch.
2. An unapproved dynamic import at runtime.
3. An external write with an ambiguous acknowledgement.

ARC can check role and task declarations. The host and an independent destination observer would supply the enforcement and effect evidence.

What trust anchor would you use to make those observations independently verifiable?

[Vitaliy Dmitriev]

As a DevOps engineer - person closest to PROD envs - more or less I'm simulating normal "human like" SDLC cycle while implementing my tasks using AI agentic loops. That is there is a dedicated agent responsible for each stage - requirements collection, implementation, review, testing and deployment preparations. In case of review - it is done in multiple paths via multiple agents (usual cap in my case is 2-3 paths - again simulating multiple team members reviewing a pull request). As of now I'm still not giving full autonomy to the agents, for instance a pull request is not merged automatically but rather waits for my final review and also PROD deployment scripts are not triggered automatically, at the same time agents allowed to rollout to sandbox envs exactly to collect mentioned evidence that stuff implemented really works.

[asagiman]

Thanks, Vitaliy. Your approach is a useful example of separating agent autonomy by delivery stage. Letting agents deploy to sandbox environments while retaining human approval for merges and production deployment gives each stage a clear responsibility.

One question: how do you bind the sandbox verification evidence to the exact commit that gets merged and eventually deployed?

I'd want the completion record to identify the tested revision, executed and skipped checks, review results, and the independently observed production outcome. That would preserve the benefits of your multi-agent SDLC while making the final acceptance boundary explicit.

Appreciate you sharing a concrete operating model.

[Jackson Ly]

A passing suite wasn't enough for me once it hid a skip, so now the report has to give executed and skipped counts, and no skipped test counts toward done. An agent had reported 24 of 24 verify steps green. A fresh-context reviewer found that the gate always skipped one opt-in integration test, the one checking that a blind runner can't read the candidate source or the publisher credentials. The gate now runs that test by default, and a guard test fails if any opt-in test is skipped.

[asagiman]

This is a great concrete example of why a green test suite alone cannot establish completion.

The distinction between tests that passed and tests that were actually required and executed is critical.

Your example also highlights a subtle failure mode: the verification gate itself can be incomplete, even when every reported check is green.

I'd add three requirements to the completion evidence:

1. A predefined manifest of required tests, tied to the exact code revision.
2. Explicit executed, passed, failed, and skipped counts, with zero skipped required tests for acceptance.
3. Independent verification that the gate actually exercised the required security boundaries.

This connects directly to the broader discussion around ARC and agent completion evidence. We need to verify not only the outcome, but also the completeness of the verification process itself.

One question: do you also test the gate with intentionally skipped or disabled security checks to confirm that it fails closed?

That seems like a valuable regression test for the verification system itself.

[Auten]

Question back: does your "did it actually happen" check ever read state the agent didn't write itself? For agents that act through a UI, the tool returning "clicked Send" proves almost nothing. The evidence we trust is a fresh read of the screen afterwards (the message is in the thread, the row exists), done separately from the action.

(I'm with the Auten team, we build screen-control tooling for agents, so that's the side effect I see fail most.)

[asagiman]

That's a very important distinction, and I agree.

A tool returning "clicked Send" is evidence of an attempted action, not evidence of the intended outcome.

I'm working on this problem through Agent Role Contracts (ARC) and our agent orchestration practices at Netsujo.

I think we need to distinguish three levels:

1. Action evidence — the agent attempted or executed an operation.
2. State evidence — a fresh observation of the target system confirms the expected change.
3. Acceptance evidence — the observed state satisfies the user's original completion criteria.

Your point adds an essential requirement: the observation must be sufficiently independent of the action and its self-reported result.

A fresh screenshot is useful, but we should also consider persistence, timing, and whether the evidence source can be influenced by the acting agent.

If independent verification is unavailable, the task should remain "unverified," rather than "done."

I'd be interested in comparing how Auten handles independent post-action verification with ARC's evidence and acceptance model, particularly for UI-driven agents.

[BrethofAI]

Splitting "done" into permitted, happened and achieved is a good lens. The second one is where we got burned most: an agent reporting a write as done when the value never landed. Our rule now is that nothing is called done until something has read it back, and the report says what was read. It's cheap, and it catches most of the confident wrong ones.

[asagiman]

That's a very practical rule. A successful write call tells us what was attempted; a readback from the authoritative destination tells us what actually persisted.

I particularly like your requirement to include the observed value in the report. It makes "done" auditable rather than self-declared.

One edge case I'd keep visible is asynchronous or eventually consistent systems. A missing value on the first read may mean PENDING rather than FAILED. And even a successful readback doesn't necessarily prove the downstream business goal was achieved.

I'd record the target, expected and observed values, and revision or observation time, then evaluate the business outcome separately.

Have you encountered cases where the readback itself gave false confidence, perhaps because of caching?

[Glen Allen]

I like the distinction between contract, execution, and acceptance. These are often treated as a single completion status, even though each requires different evidence. One additional challenge is maintaining this evidence across multi-agent workflows, where one agent may complete its task while another depends on an unverified result. Linking each handoff to a specific artifact, execution receipt, and acceptance condition could make the entire workflow more traceable. Ultimately, completion should be based on independently verifiable evidence rather than an agent's self-reported status.

[asagiman]

Thanks, Glen. I agree — the handoff is where a local "done" can turn into unjustified confidence for the next agent.

I'd bind each handoff to the exact artifact and revision, execution ID, acceptance criteria, supporting evidence, and unresolved findings. The receiving agent should verify the evidence's provenance and freshness before performing dependent actions.

One useful negative test would be:

Agent A produces an artifact and declares completion. Agent B receives a PASS label, but the execution receipt is missing or belongs to a different revision.

In that case, the workflow should preserve UNKNOWN and block dependent side effects until the required evidence is verified.

ARC can validate the consistency of role, task, and handoff declarations. The runtime and independent verifier provide the execution and acceptance evidence.

In your experience, which failure is more common across multi-agent workflows: missing evidence, stale evidence, or evidence bound to the wrong artifact?

[Tom Zafred]

The hidden trap in letting an agent declare "Done" is the Vacuous Verification problem: a test runner returning exit code 0 is mathematically meaningless if the agent commented out assertions, bypassed the test suite, or executed against stale build artefacts.

To answer your concluding question on the minimum evidence required before an agent is permitted to say "done", in our production harness we enforce three out-of-band constraints at the host runtime middleware layer:

1. Test-Mutation & AST Assertion Guard (Execution Layer)
The agent is never permitted to execute raw shell test commands or self-attest pass/fail status. A dedicated test-runner daemon executes the suite out-of-process. Crucially, it audits the test tree AST before and after execution:

• It asserts that the total assertion count did not drop below baseline (assert total_assertions >= baseline_count).
• It parses syntax trees with Tree-sitter before and after the run to prove that no assertions were commented out, deleted, or neutered with .skip or relaxed matchers to force an artificial green exit code.
• It asserts compiler gate integrity, ensuring candidate binaries match source hashes.

2. Working Tree Delta Boundary (Contract Layer)
Passing tests on an isolated module prove very little if the agent left uncommitted mutations or touched unapproved configuration files elsewhere in the repository.
Before any execution state can advance, a lazy tree-hash inspector asserts that working copy mutations are strictly bounded to the declared target file set. We assert clean status across the working copy:

# Verify working tree contains zero modifications outside pre-approved manifest
git status --porcelain | awk '{print $2}' | grep -v -F -f approved_targets.manifest

If a single unapproved file, stray lockfile, or uncommitted delta exists outside the pre-cleared scope, the turn fails with an unauthorised mutation fault.

3. Cryptographic State Machine Ratchet (Acceptance Layer)
"Done" cannot be a conversational string emitted by an LLM. If the user interface allows the model to output celebratory completion prose, next-token prediction will generate that phrase even following a partial failure.
We model execution as a strict Two-Stage state machine:

• Discussion Phase: The model is physically locked from filesystem mutations while authoring the plan.
• Execution Phase: Unlocked solely upon explicit operator authorisation (go), executing in a step-by-step ratchet.

Each completed step requires an HMAC-SHA256 signed execution receipt emitted by the out-of-process test sentry:

{
  "step": 4,
  "status": "VERIFIED_PASS",
  "tree_sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
  "assertions_verified": 42,
  "receipt_hmac": "7f83b1657ff1fc53b92dc18148a1d65dfc2d4b1fa3d677284addd200126d9069"
}

The workspace auto-locks back to read-only once 100% of the verified step receipts exist on disk.

Your distinction between contract, execution, and acceptance gets to the heart of the matter: you cannot ask the actor to serve as its own bailiff and court stenographer. When the model is restricted to proposing mutations, while the host runtime owns the AST verification sentry, the HMAC execution receipts, and the state locks, the ambiguous word "Done" is replaced with a tamper-evident audit trail.

[Paul Lovelace]

brethofai's readback rule is the one I'd actually steal — nothing counts as done until something has read the value back from where it was supposed to land. The skipped-test story from jacksonxly is the nastier version though: 24 of 24 green, and the gate was quietly skipping the one test that mattered. Do you think the three-layer receipt works without a second agent or a human in the acceptance step, or does self-reported evidence always need an outside reader?

[Ayoub Hammoudi]

The "achieved" part has a sneaky trap. When the same agent writes the code and the tests, the tests check what the agent understood, not what you asked for. Both wrong the same way, suite stays green. What works for me: write the acceptance check before the agent starts, so it can't bend it to fit the code.

[Sara]

The split between permitted, happened, and achieved is what I keep missing in my own runs. I used to treat a green test summary as all three. Now I force a tiny receipt outside the agent: the CI job id, the path that actually changed on disk, and one check against the staging URL. If any of those is missing, "done" just means the model stopped talking. The scope preflight is useful, but I still want the host to print the write path after the tool returns, not before.