Two reviewers can look at the same agent response and disagree without either person being careless.
One sees a helpful answer. The other sees a broken requirement. A grader can turn one of those judgments into a pass or fail label, but the grader cannot settle the underlying decision by itself.
That decision has to belong somewhere.
This is one of the quiet problems in agent evaluation. Teams often talk as if acceptable behavior is waiting to be discovered by a better metric.
But some disagreements are not noise. They are unresolved product decisions.
An agent can be fluent, relevant, and technically correct while still doing the wrong thing for the workflow. It can follow the user's request while violating a policy boundary. It can produce an answer that looks useful to support, risky to legal, incomplete to product, and acceptable to an automated grader.
At that point, the question is not only "How should we score this?" It is "Who owns the requirement?"
A Grader Cannot Own the Requirement
A grader can apply a rule. It can compare the answer against expected behavior. It can check whether a field is present, whether a refusal happened, whether a citation exists, whether the output matches a schema, or whether the response fits a defined acceptance condition.
But the grader is downstream of a human decision. Someone has to decide what the agent is allowed to do, what the product promises, what risk the workflow can accept, and what kind of miss changes the verdict.
If that decision is unsettled, a grader can still produce a label. The label just inherits the confusion.
This is where teams can get a false sense of clarity. A dashboard says the case passed. One reviewer says the answer was fine. Another says the agent crossed a boundary. Everyone points at the same output, but they are not judging the same requirement.
The disagreement is doing useful work. It is showing that the acceptance decision has not been placed cleanly enough.
Requirement Ownership Is Not the Same as Evaluation Judgment
Requirement ownership answers a product question:
What must be true for this behavior to be acceptable in this workflow?
That answer may come from product, policy, support, security, legal, operations, or another responsible function. There is no universal owner because there is no universal requirement. The owner depends on the behavior, the user, the action boundary, and the consequence.
If the agent is drafting a harmless summary, product may own most of the acceptance decision. If it is handling account access, security may own the boundary. If it is making claims about policy, legal or compliance may need to define what cannot be said. If it is deciding when to escalate work, operations may own the practical threshold.
Evaluation judgment is different.
Evaluation judgment asks whether the observed behavior satisfied the requirement under the tested condition. It turns the requirement into something reviewable. It separates "the agent sounded right" from "the agent did what this product needed."
Both forms of judgment matter. They are just not interchangeable.
The person or team defining acceptable behavior should not disappear behind the evaluator. The evaluator should not have to invent the requirement while scoring the output.
Grading Is the Label, Not the Decision
Grading is the moment a judgment becomes a result.
Pass. Fail. Partial. Needs review. Unsafe. Unsupported. Wrong tool. Missing evidence.
Those labels are only as good as the acceptance decision underneath them. If the requirement is vague, the grade becomes a polished version of a vague standard. If different teams disagree about what matters and nobody resolves it, the grade can hide the conflict instead of exposing it.
This is why an agent can "pass evaluation" and still make people uneasy.
The problem may not be the grader. The problem may be that the grader was asked to settle something that should have been decided before grading began.
For public examples of agent behavior where the tempting verdict is not enough, see the Nugalaxy Evaluation Cases.
The useful evaluation question is whether the answer satisfied the right requirement, for the right user, under the right conditions, with the right consequence attached.
That is heavier than a pass/fail label, but closer to the real decision.
Runtime Enforcement Is Another Boundary
There is one more distinction that gets blurred: defining a requirement is not the same as enforcing it at runtime.
A team can agree that the agent must not take an action without permission. Evaluation can test whether it respects that rule in covered cases. A grader can mark observed behavior as pass or fail.
Runtime enforcement is the product system making sure the agent cannot cross the boundary when the workflow is live.
That might involve permissions, tool constraints, approval steps, logging, escalation paths, or other product controls. The important point is simpler: the evaluation result should not be treated as the control itself.
Evaluation can show whether the agent demonstrated acceptable behavior under observed conditions. Runtime enforcement decides what the agent is technically able to do when the stakes are real.
Those two should support each other. They should not be confused.
An agent that passes a test for permission handling still needs the product boundary that prevents unauthorized action. A product boundary still needs evaluation so the team understands how the agent behaves near it. One does not replace the other.
The Nugalaxy harness engineering guides cover the machinery around those decisions.
The Owner Depends on the Requirement
There is no single answer to "who gets to define acceptable agent behavior?"
The owner changes with the requirement. Product may define usefulness. Security may define access boundaries. Legal may define claim boundaries. Operations may define escalation thresholds. Support may define what a user should receive in a messy workflow. Evaluation may help turn those decisions into observable cases and consistent judgments.
The wrong move is pretending the grader can absorb all of that responsibility.
It cannot.
A grader can encode and apply a decision after the requirement exists. It can make disagreement visible. It can keep the team from relying on taste, confidence, or whoever reviewed last.
But it cannot decide what the business, product, or responsible function has not decided.
Acceptable Behavior Has to Be Assigned
Agent evaluation gets sharper when responsibility is separated instead of blended together.
The requirement owner defines what acceptable behavior means for the workflow.
Evaluation judgment checks whether observed behavior satisfies that requirement.
Grading turns that judgment into a repeatable label.
Runtime enforcement constrains what the agent can actually do in the product.
When those jobs collapse into one another, the pass/fail label starts carrying decisions it did not make. The team ends up arguing about the grade when the real argument is about the requirement.
A useful evaluation result should make the ownership clearer, not hide it.
The agent did not behave correctly just because a label says pass.
It behaved correctly if the responsible requirement was clear, the observed behavior satisfied it, the grade represented that judgment, and the product boundary could hold.
Top comments (0)