Last Friday, NVIDIA's research team published a result that should change how every backend team budgets its AI work. On ARC-AGI-3, a benchmark where agents play long-running games with no instructions, Claude Opus 5 scored 30.2% on its own. Wrapped in NVIDIA's agent system, called AVO, the exact same model scored 100%. All 183 levels, across all 25 public environments, using 12% fewer actions than the previous best system.
Nobody retrained the model. Nobody swapped in a bigger one. The NVIDIA team's conclusion was blunt: "system design, not model capability alone, can unlock frontier-level long-horizon performance."
That result did not arrive alone. Earlier this month one developer showed he could improve 15 different LLMs at coding in a single afternoon by changing exactly one thing: the edit tool in his harness. DeepSeek shipped a developer preview of its own harness, which hit 745 points on Hacker News. ZCode built one specifically for GLM-5.2. Lilian Weng wrote a whole essay on harness engineering for self-improvement. Somewhere between prompt engineering and context engineering, a distinct discipline crystallized this year, and it is called harness engineering.
Full disclosure: I have not run AVO, and the benchmark numbers I cite are NVIDIA's, verified against their engineering blog. But I run my own AI agent infrastructure, I have built agent harnesses around Spring Boot services for the past year, and the architecture NVIDIA describes maps almost one-to-one onto what a Java team can build in an afternoon. That is what this article is: the AVO architecture translated into plain Java you can run, extend, and ship behind your existing services.
What the Harness Actually Is
The harness is everything around the model. In NVIDIA's words, it is the agent system that "determines how the model receives context, uses tools, maintains state, responds to feedback, recovers from failure, and sustains progress over long-running tasks." The model generates text. The harness decides what text the model sees, what it is allowed to do, what happens when it fails, and what survives between steps.
If you have ever written a while loop that calls an LLM, parsed its output, fed errors back in, and tried again, congratulations, you have written a bad harness. The difference between a bad and a good one is now measurable in the tens of percentage points, which is more than most model upgrades give you.
AVO's numbers make the components concrete. In GPU-kernel optimization work, AVO autonomously explored over 500 directions, committed 40 kernel versions, and produced kernels up to 10.5% faster than FlashAttention-4 on DGX B200 systems. That is not a model being clever once. That is a system letting a model be slightly clever hundreds of times in a row without dying. The NVIDIA team summarized it as: memory determines what survives, tools determine what actions are possible, feedback grounds progress, and recovery allows work to continue beyond a single model invocation.
Those four things, memory, tools, feedback, and recovery, are your build list. Let's build them in Java.
The Architecture in Four Classes
A harness is a loop with guardrails, not a framework. You do not need LangChain, and you do not need Spring AI for the core (though I'll show where Spring plugs in). You need Jackson on the classpath and four small components:
- Tool, the contract every capability implements, with structured input and output
- AgentState, the scratch memory that survives between steps
- AgentLoop, the supervisor that runs steps, validates results, and recovers from failure
- One real tool, because a harness is only as good as its worst tool
Here is the core. It is deliberately plain Java 21 so it runs anywhere, including that legacy service nobody wants to touch.
public record ToolResult(
boolean success,
String feedbackForModel, // what the model sees on failure
JsonNode data // structured output for the harness
) {
public static ToolResult ok(JsonNode data) {
return new ToolResult(true, "", data);
}
public static ToolResult fail(String feedback) {
return new ToolResult(false, feedback, JsonNodeFactory.instance.nullNode());
}
}
public interface Tool {
String name();
String description(); // this IS the model's documentation
JsonNode schema(); // JSON Schema for the arguments
ToolResult invoke(JsonNode args, AgentState state);
}
The single most important design decision is hiding in that ToolResult record: structured output from every tool, and failure feedback written for the model, not for the logs. This is the exact insight from the developer who improved 15 models in an afternoon. He found that harnesses like Claude Code leak raw output and return errors like "String to replace not found in file," an error so common it has its own GitHub megathread. When he made tools return structured data and model-readable feedback instead, every single model got better, because models were no longer wasting tokens parsing noise or guessing at what went wrong.
Now the state, which is what NVIDIA means by "memory determines what survives":
public final class AgentState {
private final Map<String, String> facts = new ConcurrentHashMap<>();
private final List<String> transcript = new CopyOnWriteArrayList<>();
public final AtomicInteger stepsUsed = new AtomicInteger();
public final AtomicReference<Instant> deadline = new AtomicReference<>();
public void remember(String key, String value) { facts.put(key, value); }
public String recall(String key) { return facts.get(key); }
public void log(String entry) {
transcript.add(Instant.now() + " | " + entry);
if (transcript.size() > 200) {
transcript.removeFirst(); // bounded context, always
}
}
public String recentTranscript(int lastN) {
return transcript.subList(
Math.max(0, transcript.size() - lastN),
transcript.size())
.stream().collect(Collectors.joining("\n"));
}
}
Two deliberate constraints here. The transcript is bounded, because unbounded context is how agent bills explode and quality collapses. And facts are separate from the transcript, because a long-running agent needs to recall "the migration is on staging, credentials are in vault path X" without replaying 200 log lines to find it. AVO does a fancier version of this with persistent memory; the principle, not the mechanism, is what you are copying.
Finally, the supervisor loop, the part most homegrown agents get wrong by not writing it at all:
public final class AgentLoop {
private final LlmClient llm; // your provider client
private final Map<String, Tool> tools;
private final int maxSteps;
private final Duration timeBudget;
public AgentLoop(LlmClient llm, List<Tool> tools,
int maxSteps, Duration timeBudget) {
this.llm = Objects.requireNonNull(llm);
this.tools = tools.stream()
.collect(Collectors.toMap(Tool::name, t -> t));
this.maxSteps = maxSteps;
this.timeBudget = timeBudget;
}
public JsonNode run(String objective, AgentState state) {
state.deadline.set(Instant.now().plus(timeBudget));
String currentGoal = objective;
while (state.stepsUsed.incrementAndGet() <= maxSteps) {
if (Instant.now().isAfter(state.deadline.get())) {
state.log("TIME BUDGET EXHAUSTED, stopping");
break;
}
LlmResponse response = llm.call(
systemPrompt(), currentGoal,
toolCatalog(), state.recentTranscript(30));
if (response.wantsToolCall()) {
Tool tool = tools.get(response.toolName());
if (tool == null) {
// recovery: unknown tool becomes feedback, not a crash
state.log("UNKNOWN TOOL: " + response.toolName());
currentGoal = objective
+ "\n\nThe tool '" + response.toolName()
+ "' does not exist. Available: " + tools.keySet();
continue;
}
try {
ToolResult result = tool.invoke(response.toolArgs(), state);
state.log("TOOL " + tool.name()
+ (result.success() ? " OK" : " FAILED"));
currentGoal = result.success()
? objective + "\n\nLast tool output:\n"
+ result.data().toPrettyString()
: objective + "\n\nTool failed, fix and retry:\n"
+ result.feedbackForModel();
} catch (Exception e) {
// recovery: exceptions are feedback, not death
state.log("TOOL THREW: " + e.getMessage());
currentGoal = objective
+ "\n\nTool threw: " + e.getMessage()
+ "\nAdjust your arguments and retry.";
}
} else {
state.log("FINAL ANSWER produced");
return response.finalAnswer();
}
}
return JsonNodeFactory.instance.textNode(
"Budget exhausted. Partial transcript:\n"
+ state.recentTranscript(10));
}
private String toolCatalog() {
return tools.values().stream()
.map(t -> "- %s: %s\n args: %s"
.formatted(t.name(), t.description(),
t.schema().toPrettyString()))
.collect(Collectors.joining("\n"));
}
private String systemPrompt() {
return """
You are an autonomous agent working toward one objective.
Use tools one step at a time. When a tool fails, read the
feedback carefully and adjust your arguments. When the
objective is met, produce the final answer as JSON.
""";
}
}
Read the failure paths again, because they are the whole point. Unknown tool, tool exception, tool failure: each one becomes structured feedback that re-enters the loop instead of crashing the run. That is NVIDIA's "recovery allows work to continue beyond a single model invocation," in forty lines. The step limit and the wall-clock deadline are there because an agent that cannot stop is not autonomous, it is a runaway bill.
Where Spring Plugs In
Everything above is a POJO, which is exactly why Spring fits it well. The harness core has no framework dependency, so wiring it into a Spring Boot service is configuration, not surgery:
-
Expose the loop as a bean with per-request
AgentState, never a singleton state, since two concurrent requests sharing scratch memory is the kind of bug you find at 2 AM -
Implement tools that hit your real stack: a
CustomerLookupToolwrapping your@Service, aSqlQueryToolbehind yourDataSourcewith a read-only user, anHttpToolfor internal APIs - Persist the transcript to whatever logging stack you already run; the state object hands you a clean string for it
-
Put a
@RestControllerin front if you want async job semantics: POST an objective, return a job ID, poll for the final answer
The one Spring-specific warning I will give from experience: do not let tools share a transactional context with the loop. Run each tool invocation in its own transaction via TransactionTemplate, or a failing agent step will roll back things you did not want rolled back.
The Checklist I Would Tape to the Monitor
Most teams do not need a better model first, they need a harness audit. When you look at your own agent code, check these in order:
- Structured tool output: do tools return data the model can parse, or raw strings it has to guess at?
- Failure as feedback: when a tool fails, does the model see why, in its own language, or does the loop just crash or retry blindly?
- Bounded context: is there a hard cap on what the model sees per step, or does context grow until quality and cost both degrade?
- Persistent facts: can the agent recall earlier findings without replaying the full transcript?
- Hard budgets: is there a step limit and a time limit that actually stop the run?
- Worst tool first: which of your tools has the highest failure rate? That one tool is your AVO gap. Fix it before you upgrade any model.
That middle item deserves its emphasis back in the source: the developer behind the 15-model improvement found that Grok 4 failed 50.7% of its patch attempts and GLM-4.7 failed 46.2%, not because the models were weak, but because the harness handed them a tool format they did not speak. Fix the tool, and "bad" models became good ones overnight.
The Parting Perspective
There is a version of the next two years where every team chases the leaderboard and rewrites integrations every time a new model drops. NVIDIA just showed the other road: keep the model, rebuild the system around it, and collect capability gains that no model upgrade in the same period would have given you. For Java shops especially, that is good news. Loops, state, transactions, supervision: we have been building those for twenty years. The harness is the part of AI engineering we are already good at.
I write about Java, Spring Boot, and AI every week. Subscribe, it's free.
Have you built an agent loop around an LLM in Java? What broke first, the tooling or the context? I would genuinely like to hear where your harness leaks.
Sources: the NVIDIA AVO engineering post (ARC-AGI-3 and GPU-kernel numbers), Can Bölük's harness writeup (edit-tool results across 15 models), and Hacker News coverage of the DeepSeek Harness developer preview. The Java code is mine and runs as written with Jackson on the classpath; adapt the LlmClient interface to whichever provider you use.
Top comments (0)