There's a user proxy in one of these benchmarks that opens the conversation with "I'm John Smith, by the way. My number is 555-123-2002." The task existed to test whether the agent asks for identifying info before touching the account. The proxy volunteered it on turn one, unasked.
The agent scored full reward. The thing the task was built to measure never got measured.
That example is in UserProxyBench, a paper by Ashish Jain and Armaan Sandhu that bolts an evaluation layer onto the tau-bench family. I've been reading it because it names a failure mode I keep running into in code review evals: the simulated participant is part of the measurement instrument, and almost nobody scores it.
The setup
Interactive agent benchmarks increasingly put a second language model in the user seat. It controls what information the agent receives and when. That's how you get multi-turn tasks at all. The same pattern is leaking into training, where the simulated user sits inside the RL loop and decides what the policy sees on each rollout.
UserProxyBench measures whether that user did the job it was handed. They call it the User Fidelity Score: the fraction of episodes where the proxy satisfies every applicable criterion in its own private user blueprint. The rubrics come from Claude Opus 4.8, 4 to 8 atomic criteria per task, reviewed by a different model (GPT-5.6-Sol) for grounding and achievability before scoring starts. 2,141 criteria total across four enterprise domains: telecom, airline, retail, banking. 375 tasks. GPT-5.5 frozen as the agent, seven different proxies rotated through the user seat, one rollout per task.
The criteria fall into four families: groundedness, premature disclosure, goal deviation, missed information. 19 failure modes total, and 18 of them actually fire somewhere in the set. The one that doesn't is self-contradiction, which gets zero criteria. Worth noting because it means the taxonomy isn't padded.
The numbers
Swap only the user proxy, hold the agent fixed, and mean task reward moves by 15.2 points. A benchmark result changing because of a component the benchmark doesn't score.
24.4% of successful episodes contain a user-specification violation. Nearly a quarter of the passing runs had the user break its own contract.
The dominant failure is premature disclosure. Every proxy gets the same explicit instruction: "Disclose information progressively. Wait for the agent to ask for specific information before providing it." They keep not doing it.
Mean UFS across the seven proxies runs from .599 to .864. Gemini-3.5-Flash is the best behaved at .864. GPT-5.4-mini is the worst at .599, and also the worst on agent reward (.644), which is the one case where the two numbers happen to agree. Sonnet-4-6 sits at .778 on fidelity while scoring .784 on reward, so the ranking flips depending on which column you read.
The other two failure examples are as concrete as the first one. An unexecuted action claim from Gemma-26B: "Okay, I'll try that. Let me see… I'm turning off this Airplane Mode now. Just a moment… okay, I've waited a bit." No toggle call ever happened and the device state still showed Airplane Mode on. And a fabricated device state: "Oh! It worked! The little bar finished moving and it says it was sent!" when the last relevant tool result was a failure. Both in episodes that passed.
The failure that reward can't see
This is the part that matters if you run evals.
Among successful episodes, the over-disclosing proxies cause the frozen agent to make 1.06 fewer tool calls and ask 0.26 fewer questions, consistently across all seven. Same reward. Less work. The agent looks equally competent while doing a strictly easier task, because the user did half the information gathering on its behalf.
Across the 374 tasks all seven proxies completed, mean agent turns correlate with UFS at r = .955. Task reward correlates at r = .594. If you want to know whether the interaction you're evaluating is the interaction you designed, the fidelity score tracks it and the reward mostly doesn't.
That's the same shape as the problem I keep hitting with LLM judges. When the thing doing the grading is another model and you only look at the final score, correlated errors hide inside a number that looks clean. The user seat is one more place for that to happen. I wrote about the offline vs online split in AI code review benchmarks and this is the same argument landing from a different direction.
What it means for AI code review evals
Most code review benchmarks I read use a fixed corpus of real PRs with human-annotated ground truth, so the "user" is a dataset and it can't misbehave. That's a real advantage and it's why I keep pushing people back to corpora over demos.
The moment you move to interactive evaluation you're building a simulator again. Any harness where the agent asks clarifying questions, requests a rerun, or negotiates scope with a synthetic PR author has a user proxy inside it. Same for the reviewer-side case: if you're testing whether a reviewer catches a planted bug, the "PR author" that responds to its questions is a model, and its behavior is a variable.
If you're scoring that, the proxy needs its own column. Not "did the agent solve it" but "did the simulated counterpart execute the role it was given". The judge-free probe in the paper is the detail I'd copy first: on telecom they checked all 658 trajectories against gold user write actions and found 54 missed writes. Of those, 23 passed every applicable rubric criterion and 20 passed a directly relevant one. An LLM-judged rubric missed things a mechanical check caught for free. Deterministic checks where gold actions exist, model judges only where they don't.
Persona prompting doesn't buy you control
There's a companion finding worth keeping next to this one. Persona Following Is Not Selective Control audits eight black-box LLMs and finds that changing one attribute in a persona prompt shifts answers on attributes the prompt never mentioned. Describe a user as more risk-seeking and their color choices move. They call it trait-conditioned completion: the model treats the persona as evidence about the person and extends the inferred profile into unspecified territory.
Then they test whether declaring a non-target attribute as neutral fixes it. It doesn't. The target attribute still leaks through in 51 to 81% of items across the five open-weight checkpoints, even when the model correctly reports the declared state. So "we prompted the simulator with a detailed persona" is not evidence that the persona is the only thing steering it. Directional declarations can be passed by simply following the instruction, which is exactly why you need the neutral state to see the failure.
What I'd want in an interactive eval report
- Score the simulator, not just the agent. Publish a fidelity number next to the reward number.
- Use deterministic probes where gold actions exist, and say which criteria were judged by a model.
- Name the proxy and its version. A 15.2-point reward swing from a component you don't list is a result nobody can reproduce.
- Hold the instrument fixed. Same reason I care about model pinning in reviewer tooling: if the model under the tool can change without a version bump, your before-and-after comparison is measuring the vendor's deploy schedule. Self-hosted and BYOK setups let you pin the model so the reviewer is the only variable in play. CodeRabbit's standard plans bundle fixed vendor models with no BYOK, while Kodus runs BYOK on every plan with zero token markup, so the model is a line in your config instead of a moving target. Different design choice, different reproducibility story.
The cost side is real too, and the paper doesn't hide it. The seven proxies span user-side cost per simulation from .0063 to .1841. Qwen3.6-35B, Gemma-31B, and GPT-5.4-mini are flagged as strictly dominated on the cost-UFS plane, meaning something cheaper scores higher on fidelity. So "pick the cheapest proxy that clears your fidelity bar" is a rule you can apply rather than a slogan.
The caveat the authors put on their own work
UFS is LLM-judged, and Claude Opus 4.8 both writes the rubrics and grades the trajectories. Only the verifier is a different model. They say so in the discussion and flag the single-rater manual audit of ten tasks and 50 criteria. One rollout per task. Serving prices are time-dependent. That's an honest limitations section and it's also the exact setup my self-review blind spot post warns about: generator and grader being the same model is a known correlation risk, and the judge-free probe finding 54 missed writes suggests the effect is live here. Which is fine. The point of the paper is the measurement gap, and the gap is real whether or not the fidelity number itself needs a second judge.
The uncomfortable version
There's a second study making the rounds, a UC Berkeley co-authored one presented at COLM, that found ten minutes of AI assistance measurably reduced people's persistence on hard problems once the tool was taken away. 1,222 participants across three randomized experiments. The framing there is "productive struggle" getting removed.
I read that and then read the 1.06-fewer-tool-calls result and they feel like the same story at different scales. Offloading the hard part works right up until you measure what's left. In the benchmark case, what's left is a passing score on a task that quietly got easier, with nobody able to see it from the reward column.
None of this means interactive evals are broken. It means the user seat is load-bearing and it's unmeasured in most harnesses people publish numbers from. Score it, name it, pin it, and put the fidelity number where the reward number is, because that's where your reader is looking.
Top comments (1)
I'd count preservation of existing user behavior as a first-class benchmark outcome. An agent can satisfy new acceptance criteria while silently breaking a caller contract or modifying files outside scope. Task completion alone misses those regressions.