A pull request is green.
The specialist reviewers found nothing that should block it. The risk classifier marks the change as low risk. The human-review branch disappears, and the change continues toward production.
Every component may have done exactly what it was asked to do.
The remaining question is whether the system was right to stop asking a human.
That question jumped out at me in Gergely Orosz's diagram of OpenAI's agentic software factory.
The public workflow includes code-writing agents, CI, specialist agent reviews, risk classification, deployment agents, production monitoring, and feedback loops. The article says that areas of the codebase can opt into automatic approval for low-risk pull requests, while higher-risk changes can receive stricter review.
I have not tested OpenAI's system. I am looking at the evaluation problem the design raises.
The small diamond marked "Low-risk change?" is not merely organizing work. Its answer can determine whether human review remains in the release path.
That makes it a release decision.
CI and Risk Classification Prove Different Things
A green CI run can provide strong evidence about the change under the conditions the pipeline checked.
The code built. The selected tests passed. The linters accepted it. A performance harness may have found no unacceptable regression. Specialist review agents may have found no issue within their assigned domains.
All of that matters.
None of it automatically proves that the change belonged in the low-risk route.
The classifier is making a different claim. It is saying that the available evidence, affected surface, expected consequence, and uncertainty are compatible with less human scrutiny.
That claim needs evidence of its own.
Otherwise, a team can evaluate every component around the gate while leaving the routing decision itself mostly assumed. The code is tested. The review agents are monitored. The deployment agent watches production. But the decision that removed the human is treated as a label rather than behavior that can be right or wrong.
"Low Risk" Depends on Context
Risk is not a permanent property attached to a file or type of change.
A change that was low risk yesterday may stop being low risk after a dependency changes, a permission boundary moves, a feature becomes widely used, or a once-correct operational rule is reversed.
The code can still pass the same local checks.
The route can still be wrong.
That is why evaluating the gate requires more than collecting examples of clean deployments. Easy successes mostly confirm that obvious low-risk changes can pass through a low-risk path. The harder cases sit near the boundary.
For example:
- A small change touches a component whose blast radius recently expanded.
- A test suite still enforces an old constraint after the product requirement changed.
- A deployment agent watches a healthy proxy metric while the required production state quietly diverges.
- Several specialist reviewers pass the change because the relevant failure exists between their domains rather than inside one of them.
These are challenge cases, not claims about observed OpenAI failures. Their purpose is to ask whether the classification still holds when the surrounding conditions become less convenient.
Score the Decision Against What Happened Next
The gate should not be judged only by whether the pull request was green when it arrived.
It should be judged against the consequence of the route it selected.
Did the supposedly low-risk change preserve the required production state?
Did it trigger a rollback, incident, manual intervention, or delayed repair?
Did monitoring capture the condition that actually mattered, or only the signals the deployment agent expected to matter?
Would an informed reviewer, given the same evidence available at classification time, have selected the same route?
This does not mean every negative production event proves the classifier was wrong. Systems fail for many reasons, and hindsight can make risk look more obvious than it was. The evaluation has to preserve what was knowable when the decision was made.
But it also cannot stop at the classifier's own explanation. A confident risk label is still a claim produced by the system being evaluated.
The useful answer comes from connecting the decision to independent evidence about the resulting state.
The Gate Needs a Record Someone Else Can Inspect
If a team wants to defend an automatic approval later, the record needs to make the decision reconstructable.
At minimum, I would want to know:
- What exact change was classified?
- Which risk policy and version were applied?
- What evidence did the classifier receive?
- Which specialist reviews ran, and what did they cover?
- What route did the gate select?
- Was approval human, automatic, or mixed?
- What was actually deployed?
- Which production postconditions were checked afterward?
- Did rollback, escalation, or repair become necessary?
That record turns "the system marked it low risk" into something another person can examine.
It also creates the foundation for replay. When a policy, dependency, or requirement changes, the team can rerun earlier decisions and ask whether changes previously sent down the easy path would still belong there.
Without that evidence, the gate can report that it made the correct decision, but the organization cannot show why the decision was defensible.
The Nugalaxy Evaluation Cases examine this broader gap between visible success and the behavior the evidence can actually support.
More Automation Makes the Gate More Important
An agentic software factory can make code creation, review, deployment, monitoring, and repair much faster.
That speed changes the role of the risk classifier. It is no longer sorting a small queue for convenience. It is controlling where scarce human judgment enters a high-volume system.
False positives have a cost. If the gate sends harmless changes to humans too often, the bottleneck returns and reviewers learn to distrust the alerts.
False negatives have a different cost. The system removes review precisely where a human might have challenged the assumptions shared by the surrounding agents.
Both errors matter, but they are not equally consequential in every part of the codebase. The evaluation has to reflect the real cost of the wrong route, not merely report overall classification accuracy.
This is where the harness contract matters. The system needs to capture the decision inputs, expected route, executed route, and production outcome in a form that can be compared consistently. Read more about harness engineering here.
Test the Gate, Not Only the Code
The public diagram shows an ambitious feedback system. Code is written, checked, reviewed, classified, deployed, observed, and repaired through connected agentic loops.
The more capable that factory becomes, the less useful it is to ask only whether each pull request passed CI.
The factory is also deciding when its own evidence is sufficient to proceed without a person.
That decision should be evaluated as seriously as the code it lets through.
A green build can show that the change passed the checks.
It cannot, by itself, show that removing the human was safe.
Top comments (2)
The distinction between evaluating the change and evaluating the route is the key one. I would make the gate measurable with a weighted confusion matrix: false negatives should cost more when the affected component has a large blast radius or weak rollback path.
I would also keep the classifier inputs and policy version alongside the selected route, then join that record to post-deploy evidence such as invariant checks, rollback/escalation, and delayed incidents. That makes it possible to measure boundary cases rather than just celebrate clean low-risk passes.
One extra safeguard: when a dependency, permission boundary, or production invariant changes, freeze automatic approval for the affected slice until the gate is recalibrated. Otherwise the system can keep making internally consistent decisions against an outdated definition of risk.
The important metric is not simply the classifier false-negative rate. It is the counterfactual value of the human review the gate removed. I would keep a small random sample of low-risk changes in the human path, then compare discoveries, reversals, and downstream incidents against the auto-approved cohort. Without that control group, a clean deployment can look like evidence for the gate even when a reviewer would have caught a near miss that monitoring never measured.