Everyone's at Level 1, and it unlocks nothing

Chapter 15 - A maturity model sorted by what depends on what, not by what's cheap
The Agentic Development Lifecycle - @praveen govindaraj

Most teams adopting AI coding agents start by creating an AGENTS.md file. They document coding conventions, explain the architecture, and tell agents which tests to run.
It's useful—but how do you know the agent actually follows those instructions?

In a study of 2,853 software repositories, 90.6% had context files, while only 0.9–3.2% had hooks that can run checks during an agent's editing workflow.
Almost everyone had shared instructions. Very few had mechanisms to enforce or observe agent behaviour.
We've mistaken the easiest step for the first step on a mandatory maturity ladder.

1. Maturity Is About Feedback Loops A context file can tell an agent to run tests, but it cannot guarantee that tests run. Instructions describe expected behaviour; they don't enforce it. Maturity is about which feedback loops a team can actually close. Consider a simple workflow:
2. An agent changes a file.
3. A type checker runs.
4. The check detects an error.

5. The workflow blocks the change and provides feedback.
   Now imagine recording failures, identifying recurring problems, and introducing controls to prevent them.
   That's the difference between detecting individual failures and learning systematically.
   The maturity model defines five levels:
   • Level 0: Individual AI-assisted development.
   • Level 1: Shared context files and conventions.
   • Level 2: Blocking checks, sensors, and execution traces.
   • Level 3: Controls introduced in response to recurring failures.
   • Level 4: Reusable harness templates adopted by other teams.
   Each level represents a capability the organisation can demonstrate—not simply another tool it has installed.

6. The Conventional Ladder Is Backwards
   The usual progression sounds logical: document conventions, automate them, collect evidence, improve controls, and share the resulting system.
   But does every capability depend on the previous one?
   Consider a blocking type check. It can run during an agent's editing workflow without an AGENTS.md file.
   The context file might help the agent understand errors, but it isn't a technical prerequisite for the check.
   Level 1 is not a prerequisite for Level 2.
   So why is Level 1 so common?
   Because it's cheap. Writing a Markdown file takes minutes. Building reliable checks requires integration, maintenance, and decisions about what should block a change.
   We've confused the order in which practices are adopted with the order in which they depend on one another.
   If your team is stuck polishing context files, it may be time to move sideways rather than keep climbing the assumed ladder.

7. Start With a Blocking Check
   For many teams, a practical next step is integrating a reliable computational check into the agent's editing workflow.
   Start with something relevant to your codebase, such as a type checker or suitable test suite.
   Where appropriate, a failed check should prevent the change from proceeding until the problem is addressed.
   Compare these approaches:
   Instruction: "Remember to run the type checker."
   Enforcement: The workflow runs the type checker and blocks the change if it fails.
   The first depends on the agent following instructions. The second introduces an independent mechanism for detecting a class of failure.
   Checks aren't perfect, but a useful blocking check changes what the system can demonstrate.

8. Checks Need Evidence
   Blocking checks help prevent failures, but they don't automatically help teams learn from recurring problems.
   Suppose an agent repeatedly introduces the same defect. Your team adds a check to catch it.
   Months later, is the problem still occurring? Is the check effective? Are other failures going unnoticed?
   Without useful records, these questions are difficult to answer.
   That's why Level 2 includes sensors and trace capture. Sensors observe what happens; traces preserve execution information and outcomes for later investigation.
   At Level 3, teams use this evidence to introduce controls in response to recurring failures.
   They should also be able to remove controls when they're no longer justified.
   A team that only adds safeguards may accumulate unnecessary complexity.
   Maturity means learning from evidence, not endlessly adding restrictions.

9. Level 4 Is an Organisational Decision
   At Level 4, teams create reusable harness templates that other teams can consume.
   This can prevent everyone from rebuilding the same development infrastructure independently.
   But reuse becomes harder when teams have different frameworks, deployment models, and service architectures.
   A template proves its value when a team that didn't create it can actually use it.
   Level 4 requires the organisation to limit the variety of service architectures it supports.
   This isn't just a tooling decision. It's an architectural and management decision about how much variation the organisation is willing to maintain.

10. What Maturity Doesn't Buy You
    A mature development lifecycle can detect mechanical failures faster and more cheaply. It cannot guarantee that the team is building the right software.
    If requirements are incomplete or misunderstood, a system can pass its checks and still produce the wrong outcome.
    Automated tests verify encoded expectations; they don't guarantee those expectations capture everything users need.
    Better tooling also doesn't automatically make high-impact changes safe to delegate. Engineering judgment remains essential.
    Maturity improves how failures are detected and handled. It doesn't eliminate the need to decide whether the system is doing the right thing.

11. Assess Your Team
    Answer these eight questions using evidence from your repositories and workflows:

12. Do you have shared, version-controlled agent context?

13. Can checks block changes during the agent edit cycle?

14. Do those checks produce observable outcomes?

15. Are execution traces and sensor outcomes stored somewhere queryable?

16. Have you introduced a control in response to recurring failures?

17. Have you removed a control and documented why?

18. Has another team consumed a harness template it didn't create?

19. Do you measure harness coverage against drills that actually fired?
    Answer based on what your team can demonstrate today—not what it intends to implement.
    Assess maturity for a particular class of work, too. A low-risk internal tool and a high-impact service may require different levels of control.

Final Thoughts
You don't need to perfect every context file before introducing automated checks. Nor must every capability follow a prescribed sequence.
Identify the feedback loop your team is missing. For many teams, that means starting with a blocking check and a trace store.
Don't optimise for climbing the ladder. Optimise for closing the next useful feedback loop.

Read the Full Chapter
📖 Chapter 15: The Maturity Model
Next: Chapter 16 — Team Topologies and the Economics of Review.
Your turn: Which maturity level is your team at, honestly? Are you still improving context files, or have you started closing the feedback loops that let your system observe, learn, and improve?