We noticed delivery volume in the current phase had reached twice that of the early build phase, without any rise in repeated repair. Here’s how we’re testing whether completion, rather than implementation quality, has become the main constraint, and what we need to measure to find where unfinished changes accumulate.
Encouraging, certainly. Reproducible? Not yet.
We don’t have the raw counts, phase boundaries or cohort observation windows behind that comparison. And the completion figure was harder to ignore: only 52.2% of changes started during the reporting period were delivered. The underlying counts weren’t available, nor was the final state of the remaining work. It may have been awaiting review, blocked, rejected or still in progress.
Our working hypothesis is that completion, rather than implementation quality, is now the main constraint. The evidence fits that explanation, but it doesn’t prove it. The delivery system isn’t finished, and we still can’t say where unfinished work accumulates.
The gate is holding, but work may be waiting behind it
First-pass acceptance was 91.7%, and no change entered a second repair pass. We don’t have the raw counts or cohort dates for either measure.
Repeated repair would be a clear sign that implementation, review criteria and repository context were out of step. We didn’t see that pattern.
We did see consequential findings. Automated reviewers raised every recorded finding. Correctness accounted for 93.8% of them, and 59.4% were classified as blocking. The finding counts weren’t provided.
These weren’t cosmetic objections. They concerned whether a change behaved as intended and whether it was safe to progress. Even so, we can’t conclude that review caused the lower delivery rate.
Part of the problem is that the workflow states aren’t defined precisely:
- Started
- Delivered
- Accepted
- Repair pass
- Blocking
A blocking finding isn’t necessarily a repair pass. It may stop acceptance while a change remains under review, waits for an engineering decision or depends on work outside the implementation. It becomes a repair pass only when the workflow sends the change back for implementation and records that return as a repair cycle.
That distinction could explain the apparently odd combination of blocking findings and no second repair passes. A finding might hold a change before acceptance, redirect it for more work or expose missing context that has to be resolved elsewhere.
Strong first-pass acceptance, no repeated repair and a lower delivered share are consistent with review constraining flow rather than creating churn. They still don’t prove it. Commercially, though, this is the preferable failure mode: unresolved work stays visible inside the delivery system rather than appearing later as operational risk.
The next improvement isn’t to relax review. It’s to shorten the path from a valid blocking finding to a clear engineering decision and verified resolution.
To test the hypothesis, we need the final disposition of each finding and the time each change spends in every blocked state.
Two readiness gaps stood out
Repository readiness checks passed at 92.6%. Again, some important context is missing: the number of checks, repositories and assessment dates.
Several controls were consistently present:
- runtime pinning;
- lockfiles;
- continuous integration;
- architecture mapping;
- one-command checks;
- testing guidance;
- business rules;
- secret controls.
That gives coding agents a more deterministic environment and reduces dependence on undocumented local knowledge. A repository that exposes its checks through one command, describes common failures and provides the right tools is easier to change reliably.
The aggregate figure concealed two concentrated gaps. Agent tooling and default-branch protection each passed for 33.3% of the assessed scope, although neither the scope nor the denominator was defined.
They cover different parts of delivery. Agent tooling determines whether an agent can inspect, test and check a repository through supported interfaces. Default-branch protection determines whether the review and evidence path can be bypassed.
Improving tooling alone would increase execution capability without ensuring that the required checks remain binding. Tightening branch protection alone could enforce a process that some repositories can’t complete efficiently.
So we now treat them as a pair. Repositories need accessible checks and supported tools, while protected integration paths keep those checks in force. Readiness isn’t merely a documentation exercise; it’s part of delivery capacity.
The planning numbers describe different groups
Approved plans covered 34.8% of changes during the reporting period. A separate phase comparison showed coverage rising from none in the early build phase to 66.7% in the current phase. More implementation now begins with an explicitly reviewed account of scope, approach and risk.
Those percentages still can’t be compared directly:
- 34.8% covers changes in the reporting period.
- 66.7% covers changes in the current phase.
- None covers changes in the early build phase.
Without dates, counts or cohort boundaries, we can’t tell how the reporting period overlaps with the current phase. We can’t reconcile the figures numerically either.
There was another result that deserved care. First-pass acceptance was 80% for changes with an approved plan and 100% for changes without one. The underlying counts weren’t available.
Does that mean planning harms delivery? The data doesn’t support that conclusion.
Plans are selected through a risk-sensitive gate, so planned work is likely to carry more uncertainty or consequence than work allowed to proceed directly. Comparing those groups without controlling for risk would confuse selection with effect.
Model routing follows the same risk distinction:
| Risk or function | Routing |
|---|---|
| Classification at any risk | gpt-5.4-nano |
| LOW-risk planning and implementation | gpt-5.6-sol |
| STANDARD-risk planning and implementation | claude-sonnet-5 |
| HIGH-risk planning | claude-opus-5 |
| HIGH-risk plan challenge | gpt-5.3-codex |
| CRITICAL_FINANCIAL planning and implementation | claude-opus-5 |
| CRITICAL_FINANCIAL plan challenge, review and business review | gpt-5.6-sol |
| Escalation at any risk | claude-opus-5 |
The source doesn’t include the evaluation process, selection criteria or fallback behaviour behind these assignments. The table describes the configuration, not the engineering reasoning.
The useful principle is the separation of responsibility. Classification, planning, implementation and independent challenge shouldn’t collapse into one judgement.
What we’re changing next
First, we’ll make the reporting reproducible. That means recording raw counts, cohort dates, state definitions and observation windows for delivery, acceptance, findings, repair passes, readiness and planning. Every percentage should trace back to a named population.
Then we’ll follow blocking correctness findings through the workflow. For each one, we’ll record its state, elapsed blocked time and final disposition. That should separate undelivered changes into rejected, awaiting review, externally blocked and still in progress.
We’ll also prioritise agent tooling and default-branch protection together, then reassess the same defined repository scope using the same criteria.
Approved-plan coverage can increase according to risk rather than one threshold being applied indiscriminately. We should assess its effect only when planned and unplanned cohorts can be compared consistently.
The current results suggest the quality gate is holding as delivery volume rises. What they don’t show is where the other 47.8% of started work ended up. That’s the next measurement we need: where each change stopped, how long it stayed there and how the story ended.
Originally published at AFT Group Engineering Insights.
Top comments (0)