Commit-Bound Evidence and Human Acceptance in an Internal Software Factory

You will learn how commit-bound evidence, independent review and human acceptance combine to control code changes in an internal software factory. The factory generates code changes, but it does not yet provide an auditable record of why a specific change should be accepted.

Early coding agents received work without an agreed plan, acceptance gate or independent review. The implementing agent also reported completion. Acceptance depended on that account rather than evidence tied to the resulting commit.

An agent’s account of its own work is not evidence that the resulting software meets the requirement.

The proposed control path combines an approved plan, continuous integration evidence bound to a commit, independent review and a human acceptance decision. The design is clear. The available records do not show the complete path operating.

Acceptance workflow

APPROVE PLAN AND ACCEPTANCE CONDITIONS
    |
    v
IMPLEMENT CANDIDATE
    |
    v
RUN RELEVANT CONTINUOUS INTEGRATION CHECKS
    |
    v
VERIFY EVIDENCE AGAINST REVIEWED COMMIT
    |
    |-- evidence absent, malformed or mismatched --> REJECT
    |
    v
INDEPENDENT REVIEW
    |
    |-- accepted --> HUMAN ACCEPTANCE DECISION
    |
    `-- rejected --> REPAIR
                        |
                        |-- budget remains --> IMPLEMENT CANDIDATE
                        |
                        `-- budget exhausted
                                |
                                v
                    HUMAN JUDGEMENT | REPLAN | CHANGE SCOPE

Automated review does not make the final acceptance decision. An accepted review passes the candidate to a person.

Assessment limits

The available material does not define:

• The assessment period
• The codebase estate
• Eligibility rules
• The complete record population
• Representative control invocation records
• A complete anonymised delivery trace

This assessment uses the following classifications:

Classification	Threshold
Specified	Expected behaviour and failure handling are documented
Reported in use	Records state that the control is used, but do not include representative invocation records
Deployed	Invocation records show the control running within a defined scope
Operating effectively	Records show the control running across that scope, handling failures as specified and producing assessable evidence
Untested hypothesis	A proposed design choice lacks comparative evidence

A statement that a control is used does not meet the threshold for deployment. The available material does not support classifying any control as deployed or operating effectively.

No concrete commit-binding protocol or anonymised control trace is available. Neither can be reconstructed from partial statements without inventing missing events.

Mandatory controls and design hypotheses

The acceptance path depends on mandatory controls. Model-family diversity is separate. It is an untested review design choice, not an acceptance control.

Type	Control	Current assessment
Mandatory	Approved plan and acceptance conditions	Reported in use, with non-universal coverage
Mandatory	Continuous integration evidence bound to the reviewed commit	Specified
Mandatory	Fail-closed evidence verification	Specified
Mandatory	Review separate from implementation	Reported in use
Mandatory	Human acceptance decision	Specified in the workflow
Mandatory	Bounded repair and terminal outcomes	Specified
Supporting	Classified review findings	Reported in use
Untested hypothesis	Different-model-family review	No comparative defect-detection evidence

The records state that implementation and review use an agreed plan. They also state that the reviewing agent uses a different model family from the implementing agent, and that findings are classified by severity and category.

Invocation records, finding counts and a defined assessment scope are absent.

The plan is the contract

Implementation starts from an agreed plan and specification. The plan defines:

• Intended behaviour
• Affected boundaries
• Validation approach
• Acceptance conditions

Implementation and review use the same plan as their reference.

A precise plan can still contain an unresolved product decision. If ambiguity reaches implementation, an agent must stop or infer intent. An inferred answer may produce coherent code while addressing the wrong requirement.

Product clarification therefore happens before implementation. Business rules, exceptional cases and acceptable trade-offs belong in the specification. A person decides before code is written.

The factory must implement an approved interpretation of the requirement, not merely produce a plausible change. Acceptance conditions must exist before implementation begins. Current records are insufficient to measure how consistently this happens.

Evidence must belong to the reviewed commit

A plan defines the target. It does not prove that a candidate meets it.

The specified gate requires evidence from the relevant continuous integration checks. Evidence counts only when independently verified and matched to the exact commit under review.

A test summary copied into an agent response is insufficient. It may:

• Refer to another revision
• Omit a failed check
• Describe an environment that cannot be reproduced

The specified response is fail-closed. Missing, malformed or mismatched evidence causes rejection.

A concrete commit-binding protocol is not documented. Missing design details include:

• How the gate obtains the reviewed commit identity
• How evidence carries the tested commit identity
• How the identities are compared
• How the evidence producer is authenticated
• Which systems may produce or store trusted evidence
• How evidence reuse across revisions is prevented
• How verification failures are recorded

Until these interfaces and trust boundaries are documented, commit binding remains a specified control.

Minimum auditable evidence record

An auditable record needs enough data to link the approved requirement, candidate, checks, review and disposition.

Record area	Minimum fields
Plan	Plan identifier, approved version, approval state and acceptance conditions
Candidate	Repository reference and candidate commit identity
Continuous integration	Checks invoked, result for each check, tested commit identity and evidence provenance
Verification	Candidate identity, tested identity, match result, verification result and failure reason where applicable
Review	Reviewer identity or type, reviewed commit identity, decision and classified findings
Repair	Attempt number, repair-budget state and resulting candidate identity
Disposition	Human decision and final state: accepted, replanned or scope changed

These fields define an assessable record. They do not imply that such a schema is implemented.

The minimum state transitions are:

PLAN_APPROVED
    -> CANDIDATE_CREATED
    -> CHECKS_COMPLETED
    -> EVIDENCE_VERIFIED
    -> REVIEW_COMPLETED
    -> HUMAN_DECISION

EVIDENCE_REJECTED
    -> REPAIR_STARTED
    -> CANDIDATE_CREATED

REVIEW_REJECTED
    -> REPAIR_STARTED
    -> CANDIDATE_CREATED

REJECTED_WITH_BUDGET_EXHAUSTED
    -> HUMAN_JUDGEMENT
    -> REPLANNED | SCOPE_CHANGED

Each transition needs a recorded source state, destination state, candidate identity, reason and time. The available material does not show these transition records.

Independent review

The reviewing agent uses a different model family from the implementing agent. Rejection requires a stated engineering reason rather than a general expression of confidence.

Review findings are classified by severity and category. Counts by classification, reviewer type, period and scope are unavailable.

All recorded review findings in the current phase came from automated reviewers rather than people. However, the records do not define the phase or provide the underlying counts.

People still:

• Define intent
• Approve plans
• Resolve ambiguity
• Decide whether a reviewed candidate is acceptable

Different-model-family review may reduce shared assumptions, blind spots or interpretations of the specification. No comparison with same-family review shows that it detects more faults. It therefore remains an untested hypothesis.

Repair needs an enforceable state machine

A rejected candidate can return to implementation, but it cannot circulate indefinitely. A repair budget provides the intended boundary.

REJECTED
    |
    |-- budget remains --> IMPLEMENTATION
    |                         |
    |                         v
    |                    EVIDENCE CHECK
    |                         |
    |                         v
    |                       REVIEW
    |
    `-- budget exhausted --> HUMAN JUDGEMENT
                                  |
                                  |-- REPLAN
                                  `-- CHANGE SCOPE

The budget is a safety control, not a measure of engineering effort. Assessing it requires:

• The configured repair limit
• A record of each repair attempt
• The budget state at each rejection
• The terminal state reached after exhaustion

Those records are unavailable. Bounded repair remains a specified control.

The codebase is part of the control system

A controlled workflow cannot compensate for an environment that agents cannot reproduce.

Readiness requirements cover:

• Written engineering conventions
• A pinned runtime
• Dependency locking
• An architecture map
• Testing guidance
• Common failure modes
• Consistently invocable checks

Agent tooling and default-branch protection are reported as the least consistently satisfied areas. The assessed estate, scoring method, dates and underlying counts are not available.

Where readiness differs between codebases, the same workflow can provide different levels of assurance. Each undocumented convention or environment-specific step pushes a decision back into inference and makes acceptance harder to audit.

Measurement remains incomplete

Previously reported percentages and the delivery-activity comparison are omitted because they cannot be reproduced from the available records.

Measure	Reporting status	Evidence required
Readiness controls passing	Not independently reportable	Checklist, passing count, assessed count, scoring method, measurement date and estate coverage
Approved-plan coverage	Not independently reportable	Eligible work-item definition, approved-plan count, eligible count, sample size and period
First-pass acceptance	Not independently reportable	Accepted candidate count, assessed count, period, eligibility rules and treatment of withdrawn or replanned work
Verified-evidence coverage	Not reportable	Eligible delivery count, verified-evidence count and documented verification mechanism
Delivery activity change	Not independently reportable	Activity unit, phase dates, phase durations and rules establishing comparability
Review findings	Not reportable by classification	Counts by severity, category and reviewer type, with period and scope

First-pass acceptance means acceptance through the configured evidence and review path. It does not mean code perfection.

Implementation priorities

The work should proceed in dependency order.

1. Define assessment scope

   • Set the assessment period.
   • Define the codebase estate and eligible work.
   • Identify the complete record population.

2. Standardise plans

   • Make approved plans and acceptance conditions universal.
   • Record product clarifications as durable decisions.

3. Specify commit-bound evidence

   • Define the evidence fields.
   • Document producer authentication.
   • Document identity matching, storage and trust boundaries.
   • Define rejection behaviour for absent, malformed, stale or mismatched evidence.

4. Record control execution

   • Store verification results and failures.
   • Store review decisions and classified findings.
   • Retain the candidate identity at every stage.

5. Enforce bounded repair

   • Record the configured limit.
   • Record each repair transition.
   • Ensure budget exhaustion produces a recorded terminal outcome.

6. Produce an end-to-end trace

   • Use an existing delivery record.
   • Do not reconstruct missing events.
   • Include plan approval, commit identity, continuous integration provenance, verification, review, repair attempts and final disposition.

7. Establish reporting

   • Define metric numerators, denominators, periods, eligibility rules and exclusions.
   • Report verified-evidence coverage and review findings only when the required records exist.

8. Address codebase readiness

   • Strengthen agent tooling.
   • Strengthen default-branch protection.
   • Record the assessment method and codebase coverage.

What this means

The acceptance design has the required control points, but the records do not show the complete path operating against a specific commit. The immediate task is to define the evidence protocol, record state transitions and produce an end-to-end trace without reconstructing missing events.

---

Originally published at AFT Group Engineering Insights.