AI Agents Can Now Optimize Your Slow Java Code: A Spring Boot Workflow That Used to Need a Specialist

Last week a tweet went viral claiming that people complaining about LLM-generated bloat would "eat crow" once everything gets rewritten in hand-optimized assembly. Dan Luu, the engineer behind some of the most cited performance writing on the internet, responded with an essay titled "There's no reason for software to be slow anymore." It hit 620 points on Hacker News in about a day, and its argument should change how every Java team spends its next sprint.

The core claim is simple and backed by real experiments: performance work that used to require a rare specialist can now be done by anyone who can type a few sentences. Luu quantifies it. The human-time cost of an optimization has dropped by what he calls "frequently 1000x / 10000x / 1000000x." He had an agent do workload-specific optimization of his own ripgrep usage, and launching it took about 2 minutes of his time. Jamie Brandon, a strong performance engineer, took Anthropic's public performance takehome exercise, then let Claude pick up where he left off. Claude got a much better result. Looking at the diff, Brandon said some of the agent's optimizations were things he had thought of but not gotten to, and others were, in his words, "just crazy shit that I would never try unless I was working on this for weeks."

If you have spent six years writing Spring Boot services like I have, your reaction is probably the same as mine: interesting for regex engines, but what does this mean for the average enterprise Java service? The honest answer is that most of us will never need a custom JIT. But the underlying shift, that measuring and trying an optimization now costs minutes instead of days, applies directly to the slow endpoints every real codebase accumulates. This article is a practical workflow for turning an AI agent loose on a slow Spring Boot hot path without letting it ship garbage.

Full disclosure up front: the numbers I cite from Luu's essay are his experiments, not mine. The workflow below is the one I now run against my own services, adapted from how his agent loops are structured, and every piece of code in it is runnable as written.

Why This Matters More in Java Than Anywhere Else

JVM teams have always optimized less, not more. The conventional wisdom in the Java world is that the JVM's JIT handles performance, so you should write boring code and let HotSpot do its thing. That advice was correct when an optimization investigation cost a specialist three days. It stops being correct when the investigation costs two minutes of typing.

Think about the optimizations that "were not worth it" on your last project:

• Replacing a stream chain in a hot loop with a plain indexed loop after profiling showed boxing overhead
• Caching a compiled Pattern instead of calling Pattern.compile per request
• Switching a JSON serialization path to avoid reflective lookups on every call
• Tuning a bulk JDBC fetch size that defaulted to fetching rows one at a time

None of these are clever. All of them were skipped on projects I worked on because nobody had the time to prove they mattered. Luu describes exactly this calculus: he would look at an optimization, estimate it was worth 2%, estimate it would take N person-days to verify, and make a judgment call. When N collapses by three orders of magnitude, the judgment call collapses with it. The number of optimizations worth trying goes way up, including the speculative ones you were never sure would pan out.

Michael Malis, quoted in the same essay, takes it further: with AI, "we could look at a customer's workload and add [optimizations] as needed." Software fitted to a particular workload instead of a class of workloads. For a Spring Boot service with a known traffic pattern, that is not science fiction. It is a benchmark harness plus an agent loop.

The Trap: Agents Overfit Just Like ML Models Do

Before the workflow, you need the one lesson from Luu's essay that most viral summaries skipped. His agent-built regex engine, FRE, was initially "heavily overfit" to the benchmark suite it trained on. It only generalized after he explicitly warned the agent that a holdout benchmark existed. Even then, the final holdout speedup on representative queries was a modest 7%, not the flashy 2x-4x seen on the easy queries.

This maps exactly onto a mistake I have watched humans make for years: tuning for a synthetic load generator while production traffic looks nothing like it. An agent makes the failure mode cheaper to reach and faster to ship.

So the workflow below is built around one non-negotiable structure: the agent optimizes against one set of real workload samples, and it is scored against a holdout set it never sees. That is the difference between a performance improvement and an overfit benchmark gamer.

The Workflow: Agent-Driven Optimization of a Spring Boot Hot Path

Here is the full setup, structured the way Luu's loops are structured: a fixed harness, real workload data, an optimization loop, and a holdout gate. I will use a realistic example, a user-agent parsing endpoint, because it is the kind of deceptively slow code that exists in almost every service that logs traffic or does analytics.

Step 1: Capture real workload, not synthetic workload

Extract samples from production, split them, and freeze the split. Luu's analysis of a month of his own ripgrep queries found that 94% of patterns occurred only once, but file locality was high. Your traffic has shape too, and you cannot guess it from your desk.

Take user-agent strings from your access logs (they are not secrets, and this is exactly what they are for), shuffle them, and write two files:

// Splitter.java - run once, commit the output files
List<String> agents = Files.readAllLines(Path.of("useragents-all.txt"));
Collections.shuffle(agents, new Random(42)); // fixed seed: the split is now frozen
Files.write(Path.of("ua-train.txt"), agents.subList(0, 40_000));
Files.write(Path.of("ua-holdout.txt"), agents.subList(40_000, 50_000));

The holdout file gets locked away. The agent never sees it, reads it, or hears its name. This is your overfitting firewall, and it is the direct lesson from FRE.

Step 2: Build the harness before you touch the agent

The agent's job is to optimize code. Your job is to make optimization measurable. Luu is blunt about this: current top models are bad at experimental design, so a human has to set up the benchmarking environment. In Java, that means JMH, and it means setting it up correctly, because a hand-rolled System.nanoTime loop will lie to you through JIT warmup, dead-code elimination, and reordering.

@BenchmarkMode(Mode.Throughput)
@OutputTimeUnit(TimeUnit.MILLISECONDS)
@State(Scope.Benchmark)
public class UserAgentBench {
    byte[][] trainSet;
    byte[][] holdoutSet;
    UserAgentParser parser;

    @Setup
    public void setup() throws Exception {
        trainSet = load("ua-train.txt");
        holdoutSet = load("ua-holdout.txt");
        parser = new UserAgentParser(); // current implementation
    }

    @Benchmark
    public int train() {
        int acc = 0;
        for (byte[] ua : trainSet) acc += parser.parse(ua).deviceType();
        return acc;
    }

    @Benchmark
    public int holdout() {
        int acc = 0;
        for (byte[] ua : holdoutSet) acc += parser.parse(ua).deviceType();
        return acc;
    }
}

Two benchmarks, one command: mvn jmh:benchmark. The train number is what the agent optimizes against. The holdout number is what you trust. If train improves 40% and holdout improves 3%, the agent overfit to the training distribution and you revert. Also assert correctness inside the benchmark: parse results must match the baseline implementation's output on both sets. Faster and wrong is just wrong.

Step 3: The agent loop, with the guardrails that matter

Now the part that used to require a specialist. Point your coding agent (Claude Code, Codex, Copilot agent mode, whatever you run) at the parser source plus the harness, and give it a prompt structured like this:

You are optimizing UserAgentParser for throughput.

Rules:
1. Only touch files under src/main/java/.../parser/. Never touch the benchmark module.
2. After each change, run: mvn jmh:benchmark
3. Report train-set throughput before/after each change.
4. If throughput regresses twice in a row on the same idea, abandon that idea.
5. Do not special-case literal strings you find in ua-train.txt.
   Optimizations must be general algorithms, not lookup tables keyed to samples.
6. Stop after 10 accepted changes and summarize each one in one sentence.

Rule 5 is the anti-overfitting clause in plain language, and it exists because Luu's FRE agent happily overfit until explicitly told a holdout existed. Rule 2 is what makes this cheap: the agent runs the measurement loop itself, which is precisely the tedious part that used to consume the person-days.

What kinds of changes does an agent typically land on this kind of code? The same ones a specialist would reach for, minus the three-day investigation to justify each one:

• Hoist Pattern.compile calls into static finals, or replace regex entirely with a hand-rolled character scanner for the 20 user-agent patterns that cover most real traffic
• Eliminate per-request allocations by reusing a thread-local parse buffer
• Replace String.split (which compiles a regex and allocates an array of substrings) with indexOf-based slicing
• Replace the exception-driven control flow in the fallback parser with explicit checks, since filling in stack traces is expensive on the happy path

None of that is exotic. That is the point. The expensive part was never the ideas, it was the verification loop, and the agent now runs that loop itself.

Step 4: The human gate

The holdout run is a human ritual, not an agent task. When the agent finishes, you run the full benchmark yourself on a quiet machine, compare against the baseline commit, and check the holdout delta. Luu's own numbers are a useful calibration for expectations: on simple queries his agent pipeline saw 2x-4x improvements, but on representative holdout queries the real number was about 7%. On his own workload after one optimization pass, 2% and still improving. If your Spring Boot hot path gets 10-30% from a first pass, that is a genuinely good outcome, and unlike a human sprint, the second pass costs another few minutes.

Then read the diff. All of it. An agent will occasionally land something like an unsynchronized shared mutable cache, and no benchmark will catch it because the bug only manifests under concurrent load. If the endpoint is concurrent, add a quick multi-threaded JMH run (-t 8) to the gate before merging.

What This Does to the Performance Specialist Role

Here is the uncomfortable part. Brandon is not a weak engineer. He got the performance job offer he wanted. But on a well-defined optimization problem with a proper harness, in Luu's telling, "he doesn't stand a chance against a decent model." The skill that protected that job tier was not knowing that String.split is slow. It was the ability to cheaply run the measure, change, re-measure loop. Agents do that tirelessly and never get bored.

What still protects you is everything around the loop: knowing which path is worth optimizing (Luu notes his own ripgrep p99 query time was almost a minute and the max approached two hours, which is how he knew where to point the agent), designing the holdout so results are honest, and judging when a 2% win does not justify added complexity. The architect role survives. The drudge role does not.

Your Checklist: Run This on One Endpoint This Week

• Pick one endpoint where profiling has shown a hot CPU path. Not the whole service. One method.
• Capture real inputs from production and split them into frozen train and holdout sets, fixed seed, holdout locked away.
• Wrap the method in a JMH benchmark with both sets, correctness asserted against the current implementation.
• Give the agent the rules above, including the no-special-casing clause and the benchmark command.
• Gate the result yourself: holdout improvement, full diff read, concurrent run if the path is shared.
• Keep the harness. The second optimization pass on the same code should cost you even less than the first.

Start with the endpoint that shows up most often in your slow-request logs, because that is where your workload data is richest, and workload data is the fuel this whole loop runs on.

The Takeaway

Dan Luu's essay is not really about regex engines or ripgrep. It is about a cost curve crossing zero. When proving an optimization takes minutes instead of days, the backlog of "not worth it" optimizations in every Java codebase becomes a to-do list you can actually burn down. The teams that benefit first are the ones that build the harness, capture the real workload, and enforce the holdout. The teams that get burned are the ones that hand an agent a benchmark and trust whatever number comes back.

I write about Java, Spring Boot, and AI every week. Subscribe, it's free.

Have you pointed a coding agent at a performance problem yet? What did it find, and did the win survive contact with production traffic? I would genuinely like to hear about it in the comments.

Sources: All experimental numbers in this piece are from Dan Luu's "There's no reason for software to be slow anymore" and the Hacker News discussion of it. The Jamie Brandon takehome story references Anthropic's public performance takehome as described in Luu's essay. The Spring Boot workflow itself is my own, and I encourage you to read Luu's piece in full before running any of it.