TL;DR
When an AI agent changes few files, runs few tests, and says "Done", is it really done? Not always. Things may still break in unexpected places.
I prefer to define a strict definition of "Done", which I call the Task Completion Protocol. It usually lives in AGENTS.md and, in its simplest form, looks like this:
- Run the repository's real lint and test commands.
- Fix failures and rerun the checks.
- Report the exact evidence before claiming completion.
This does not replace CI or code review. It moves a basic verification gate earlier, before a broken state reaches the next agent session, the pull request, or you.
Video Walkthrough
If you prefer a visual walkthrough, this video shows the concept in action. I run the same task twice: without a completion protocol and with it - this demonstrates how explicit definition of "done" helps. I also show few tricks and techniques for making agents follow the protocol.
The Agent Says "Done." The Repository Disagrees.
You have probably seen some version of this:
Agent:
Done. Implemented the fix and updated the tests.
Terminal:
$ make test
... FAIL
The implementation might even look correct. Perhaps the agent ran one focused test, inspected the diff, and produced a tidy summary. Then you run the repository's normal checks and find a lint error, a failing test in a different place, or any other error that was not caught by the agent.
This gets worse as AI-assisted workflows grow to involve multiple agents. In an interactive session, you may spot the failure and fix it yourself. In a multi-step workflow, one worker reports success and the next starts from a broken state. These problems accumulate quickly, and the final state may be badly broken.
The Repository Needs to Define What "Done" Means
Yes, this is obvious for humans: make some changes, run tests and lint, fix issues, maybe check few other things, and only then say "Done". Agents need it written down, and AGENTS.md is the perfect place for it. I call this section the Task Completion Protocol.
Here is a compact version you can adapt to your repository:
## Task Completion Protocol
Before reporting completion, classify the task as coding or non-coding.
For coding tasks:
1. Run lint: `your command to run lint`.
2. Run tests: `your command to run tests`.
3. Check whether commands, workflows, or architecture changed. If so, update AGENTS.md.
4. Report the commands run, their results, test coverage, and whether AGENTS.md changed.
5. Do not report the task as complete while a required check is failing.
For non-coding tasks:
1. Summarize the findings or actions taken.
2. Confirm that the requested deliverable was produced.
3. Do not run unrelated lint or test commands unless requested.
My Go backend template has a real version of this protocol. It uses make lint and make test. Your protocol should name the exact commands and any specific checks that matter.
Don't just say "Run the tests". This still leaves the agent guessing. It is best to name the exact commands the agent must run.
Classification and Hierarchy Are Important
"Always run everything" may work in a small repository. In a monorepo, it quickly becomes expensive and sometimes absurd.
A documentation change does not need the entire test suite. A CSS change may need visual verification, not a set of backend tests. The same applies to Terraform, Helm, and similar tools.
At minimum, I classify tasks as coding or non-coding. We don't want to run tests if we only changed some docs, skills, or similar files.
The hierarchy makes sense as well. Each area can have its own AGENTS.md file with its own completion protocol. Each type of change may require different checks and commands.
The root AGENTS.md can define project-wide behavior if relevant. More specific files can define what completion means there. Typically I have something like this:
AGENTS.md
├── apps/web/AGENTS.md
├── apps/backend/AGENTS.md
└── deploy/terraform/AGENTS.md
And I make sure to keep the root AGENTS.md minimal.
The Reporting Part Also Matters
Do not let the agent "guess" what to report. Require an explicit format. I typically use something like this:
Lint: pass - make lint
Tests: pass - make test
Coverage: 87.42% - this one is very important
AGENTS.md: no changes needed
Status: complete
And it is not really for you to read (though I do sometimes). It is more for the agent. A strict reporting format gives the agent more "motivation" to actually run your checks. Otherwise it may just "guess" the outcome.
The dynamic part matters too. I typically require coverage reporting. If you don't collect it (I hope you do), ask for test duration or something else the agent can't guess. I haven't seen a model report this reliably without running the checks.
This Does Not Replace CI or Review
The completion protocol is a local behavioral contract. CI remains the hard and deterministic gate.
You still want deterministic checks in the pipeline, and you still need to review and validate the change. Passing tests do not prove the requested behavior is correct, the architecture is sensible, or the security model survived the refactor. But it gives you a solid baseline:
- The repository is "green".
- The agent used the team's canonical checks.
- Failures were caught sooner instead of passed forward.
The Harness May Make It Harder
One frustrating caveat: repository instructions are not always the highest-priority instructions an agent receives. The harness may add constraints that shape command execution, testing behavior, and final reporting.
If an agent repeatedly ignores a clear completion protocol, the wording in AGENTS.md may not be the problem. Ask your model which instruction is shaping the verification behavior and where it came from. I use Agent Diagnostics Mode for this kind of investigation.
Start Small
You do not need a giant governance document. Start with the checks you already expect a responsible human contributor to run. Extend your completion protocol as the project grows.
This helps me a lot and I hope it will help you too!
Top comments (1)
The coverage number as a "can't guess" forcing function is the detail that actually makes this stick in practice - we ran into exactly this: "Tests: pass" was often fabricated in final summaries, but once we required coverage percentage with branch count, the model started running the checks rather than inferring their success. The harness caveat at the end is worth emphasizing more - when Claude Code is the harness, it has its own system-level behavioral contract that can shadow AGENTS.md instructions, which means the protocol has to also be surfaced at the session level, not just the file level. The classification tree (coding/non-coding, then area-specific) scales well; we added a third category for data pipeline changes that require a dry-run diff, since tests pass but the output schema may still shift.