DEV Community

Cover image for I Asked AI Models to Optimize a Java Hot Path. Then I Audited Their Benchmarks.
Emerson Vieira
Emerson Vieira

Posted on

I Asked AI Models to Optimize a Java Hot Path. Then I Audited Their Benchmarks.

I wanted to answer a simple question:

When can an O(n) algorithm be a better choice than O(1) in a real system?

Not in a textbook example, but in something closer to a production payment system.

That question turned into an experiment involving three AI models, Java 21, JMH, several implementations of the same operation, and some unexpected results.

The most interesting part wasn't which algorithm won.

It was discovering that an AI model can write a benchmark, execute it, explain the results convincingly, and still make a mistake that compromises the experiment.

Here's what happened.

The problem: a payment authorization hot path

Imagine a fintech processing around 100,000 card authorizations per second, distributed across multiple service instances.

During authorization, the system needs to determine whether the merchant's MCC (Merchant Category Code) is blocked for the card's financial product.

MCCs identify merchant categories, such as restaurants, supermarkets, or gambling establishments.

Our hypothetical system has a few characteristics:

  • Java 21, running on a long-lived JVM.
  • Between 4 and 12 blocked MCCs per product.
  • A hard business limit of 12 entries.
  • 96% of queries return "not blocked".
  • The remaining 4% are uniformly distributed across blocked codes.
  • Configurations are loaded into memory and published through immutable snapshots.
  • No database or network calls during the lookup.
  • The operation executes once per authorization.

The goal was to implement this lookup efficiently without sacrificing correctness, readability, or thread safety.

Here's the straightforward solution:

public final class BlockedMccs {
    private final int[] codes;

    public BlockedMccs(int[] codes) {
        if (codes.length > 12) {
            throw new IllegalArgumentException(
                "Maximum of 12 blocked MCCs"
            );
        }

        this.codes = codes.clone();
    }

    public boolean contains(int mcc) {
        for (int code : codes) {
            if (code == mcc) {
                return true;
            }
        }
        return false;
    }
}
Enter fullscreen mode Exit fullscreen mode

It's a linear search.

O(n).

At first glance, using a HashSet might sound better. Its average lookup complexity is O(1).

But there is a detail that changes the problem: n never exceeds 12.

It doesn't matter whether the fintech has 10,000 customers or 50 million. Each product still has at most 12 blocked MCCs.

That's where asymptotic complexity stops telling the whole story.

Three AI models, one engineering problem

I gave the same technical problem to three models:

  • Gemini 4 Neon Extra High
  • Claude Opus 5 Max
  • GPT 6.2 Sol Ultra

I wanted more than working Java code.

I asked them to consider memory layout, allocations, CPU cache behavior, JVM optimizations, concurrency, and the actual impact of the lookup on the service.

I also asked for a JMH benchmark and for a clear distinction between performance hypotheses and measured results.

The approaches differed.

Gemini produced a detailed technical analysis and proposed JMH benchmarks, but didn't execute them.

Opus went further. It created a project, implemented 13 variants, ran a custom benchmark harness, investigated unexpected results, and eventually executed JMH.

GPT also ran JMH and, in a separate implementation from scratch, tested multiple list sizes and product counts.

One methodological detail matters here: an earlier GPT run used the same workspace as Opus, which could have exposed it to existing code. I didn't treat that as an independent experiment. I requested a fresh implementation in a separate project.

This wasn't a controlled ranking of AI models. Their execution workflows differed. I was interested in how they approached engineering decisions, not declaring a winner.

And the results were worth investigating.

The algorithms

The implementations compared four main approaches.

1. Linear scan — O(n)

Iterate through a small primitive array until the MCC is found.

The advantages are straightforward: contiguous storage, no boxing, no per-lookup allocation, and very little implementation complexity.

For 12 entries, the worst case requires 12 comparisons.

2. Binary search — O(log n)

Sort the MCCs when loading the configuration, then use:

Arrays.binarySearch(sortedMccs, mcc) >= 0;
Enter fullscreen mode Exit fullscreen mode

This reduces the number of comparisons, but introduces dependent decisions about which half of the array to inspect next.

For very small arrays, fewer comparisons don't necessarily mean less CPU time.

3. Bitmap — O(1)

A bitmap represents membership using individual bits.

If the application contract restricts MCCs to the range 0–9999, a long[157] is enough to represent the entire domain.

The lookup is essentially:

boolean contains(int mcc) {
    return mcc >= 0
        && mcc < 10_000
        && (words[mcc >>> 6]
            & (1L << (mcc & 63))) != 0;
}
Enter fullscreen mode Exit fullscreen mode

Instead of searching through blocked codes, it calculates the position of one bit.

The trade-off is memory.

Twelve integers contain 48 bytes of data, excluding the array header. The bitmap reserves 1,256 bytes of data per product.

4. HashSet — O(1) on average

The familiar Java collection.

It offers expected constant-time membership checks, but requires a hash table and boxed Integer values.

Whether a lookup allocates an Integer depends partly on the value and whether the JIT can eliminate the allocation.

All four approaches are reasonable candidates.

The benchmark would tell us how they behaved under the selected workload.

Benchmark environment

I ran the primary experiments on my personal laptop.

Component Specification
Machine Acer Predator Helios 300, PH315-54
CPU Intel Core i7-11800H
Cores / threads 8 / 16
Base frequency 2.30 GHz
RAM 32 GB
Operating system Windows 11 Home, 64-bit
Java HotSpot 21.0.12.1
Benchmark framework JMH 1.37
Power Connected to AC power

The laptop remained plugged in during the benchmarks.

That doesn't eliminate CPU frequency scaling, thermal effects, scheduler noise, or background activity. It just removes battery-only operation as a variable.

At nanosecond scale, these details matter.

The primary experiments were conducted in separate runs on the same machine. Their absolute timings shouldn't be treated as a perfectly controlled head-to-head comparison.

First results: the bitmap won

The first extensive investigation came from Opus.

It initially expected the linear scan to be highly competitive because the arrays were so small.

After running its benchmarks, that prediction didn't hold.

For one product with eight blocked MCCs, JMH reported approximately:

Implementation Time
Bitmap 1.73 ns
Primitive open-addressing hash 2.19 ns
Sorted linear scan with early exit 2.84 ns
HashSet 3.38 ns
Simple linear scan 4.40 ns
Binary search 7.26 ns
IntStream.anyMatch 22.09 ns

These results use the synthetic retail-concentrated miss distribution. They include benchmark input-handling overhead and are not isolated CPU instruction timings.

The bitmap was the fastest.

But look at binary search.

An O(log n) algorithm was slower than O(n).

It performed fewer comparisons, but the operations required to select the next element were more expensive in this measured workload.

This is exactly why Big O alone isn't a performance benchmark.

Another surprise was HashSet<Integer>.

In an earlier custom harness, Escape Analysis appeared to eliminate almost all the allocation overhead.

Under JMH, the same kind of lookup allocated approximately 16 bytes per operation.

The model initially proposed a JVM explanation for the discrepancy, but the exact JIT mechanism wasn't proven.

That distinction matters: observing different behavior doesn't automatically establish the reason for it.

And then I found a more serious problem.

The benchmark itself had a bug

Opus had also written a multi-product benchmark to investigate whether the larger memory footprint of the bitmap would eventually hurt performance.

Its results suggested that the bitmap remained faster even with thousands of products.

That seemed convincing.

But while reviewing the code, I found that the workload was generated incorrectly.

The benchmark independently generated a sequence of products and a sequence of MCCs.

When generating a supposed hit, it selected a blocked MCC from a randomly chosen product.

Later, the lookup could use a completely different product.

Conceptually, the bug looked like this:

int product = randomProduct();

int mcc = randomBlockedMcc(
    randomProduct() // Not necessarily the same product!
);

boolean blocked = rules[product].contains(mcc);
Enter fullscreen mode Exit fullscreen mode

A value intended to produce a hit could become a miss.

As the number of products increased, the effective hit rate collapsed.

The original workload was supposed to contain 4% hits.

In the archived multi-product results, the observed rate was approximately:

Products Observed hit rate
1 3.516%
8 0.439%
64 0.098%
512 0.098%
4,096 0.098%

The workload had changed from roughly 96% misses to almost 99.9% misses.

That doesn't mean the bitmap results were worthless.

It means the experiment was no longer measuring the workload it claimed to reproduce.

And that's an important distinction.

A benchmark can produce stable, repeatable numbers while measuring the wrong thing.

Rebuilding the workload

The fresh GPT implementation addressed this problem directly.

Instead of generating products and MCCs independently, it generated queries associated with specific products.

The correct relationship is:

int product = randomProduct();

int mcc = shouldHit()
    ? randomBlockedMccFor(product)
    : randomAllowedMccFor(product);
Enter fullscreen mode Exit fullscreen mode

The new experiment generated 76,800 queries per scenario, containing exactly 3,072 hits — 4% of the total.

The hits were distributed equally among the blocked MCCs of each product.

All queries were generated before measurement, and every implementation received the same inputs.

The benchmark compared:

  • 4, 8, and 12 blocked MCCs.
  • Uniform and concentrated miss distributions.
  • One product, plus a separate 128-product scenario.
  • A single benchmark thread.
  • Three forks in the reported runs.
  • JMH average time in nanoseconds per operation.
  • Allocation behavior using the JMH GC profiler.

This was still a microbenchmark, not a full authorization service.

But the workload was substantially better controlled.

The independent results

Here are the results from the fresh GPT implementation.

All timings are in nanoseconds per operation.

Uniformly distributed misses

Blocked MCCs Linear Binary Bitmap HashSet
4 4.14 9.86 2.05 7.42
8 5.60 13.56 2.03 8.27
12 6.26 13.95 2.02 11.62

One product per scenario.

The results show a useful pattern.

The bitmap remained around 2 ns as the number of blocked codes increased.

The linear scan became more expensive, as expected, although not in a perfectly linear relationship with the number of elements.

Binary search performed poorly relative to the other approaches.

Concentrated retail misses

For eight blocked MCCs:

Implementation Time
Bitmap 1.99 ns
Linear scan 5.41 ns
Binary search 8.28 ns
HashSet 10.74 ns

Changing the input distribution significantly affected some approaches.

The bitmap remained fast.

What about multiple products?

The independent test also measured 128 products with eight blocked MCCs each.

Distribution Linear Bitmap
Uniform misses 5.53 ns 2.43 ns
Concentrated misses 5.48 ns 2.43 ns

This time, the workload preserved the intended 4% hit rate.

The bitmap was still faster.

It would be premature to generalize this to thousands of products, multiple threads, or real production memory pressure. Those scenarios require additional tests.

Still, the evidence from these local runs clearly favored bitmap lookup speed.

So why recommend the slower algorithm?

Here's where the experiment became an engineering question rather than just an algorithm comparison.

Despite the benchmark results, the models generally recommended starting with the simple int[] implementation.

At first, that sounds contradictory.

Why choose an algorithm that takes around 5–6 ns when another takes around 2 ns?

Because performance is only one part of the decision.

Consider the maximum observed difference of approximately 4.2 ns per lookup.

At 100,000 lookups per second:

4.2 ns × 100,000 operations/s
= 420,000 ns/s
= 0.00042 CPU-seconds per second
Enter fullscreen mode Exit fullscreen mode

That's approximately 0.042% of one CPU core, assuming the measured difference translates into saved CPU time.

Across the whole service, that is a very small potential saving.

It also doesn't mean the end-to-end authorization p99 would improve by a proportional amount.

Actual p99 latency depends on many other factors: database access, network calls, queues, garbage collection, contention, and resource saturation.

The linear array has some practical advantages.

It's compact, straightforward to audit, doesn't require a fixed numerical domain, and needs no derived lookup structure.

The bitmap is not particularly complicated either, and it also makes zero allocations during lookup. Its main trade-offs are the larger per-product memory footprint and the need to validate the bounded MCC domain.

If we had millions of repeated lookups per authorization, the decision could change.

If we had just a handful of products and strict latency requirements measured in nanoseconds, the decision could change.

If a production profiler identified this method as a significant CPU consumer, the decision could change.

The benchmark identifies the faster implementation under the tested conditions. It doesn't automatically make the production decision for us.

To make that decision, I'd want actual product cardinality, representative transaction traces, CPU profiling, and an end-to-end load test.

What the AI models got right — and wrong

The most interesting part of the experiment was watching the models handle evidence.

Gemini gave a strong theoretical explanation, but some performance predictions remained untested.

Opus built a more extensive investigation. It discovered that some of its original assumptions were wrong, corrected earlier conclusions about allocation and benchmark behavior, and ultimately relied on JMH.

But its multi-product workload contained an error that compromised part of the analysis.

The fresh GPT implementation produced a more controlled workload and explicitly verified the hit distribution. It explored fewer algorithms than Opus, but covered the full range of 4, 8, and 12 blocked codes.

None of this makes one model universally better than another.

What it shows is that there are several different skills involved:

Generating code is not the same as benchmarking code.

Benchmarking code is not the same as designing a valid experiment.

And producing valid measurements is not the same as making a good engineering decision.

LLMs can help with every one of those steps.

But they can also make mistakes at every one of them.

One model initially attributed an unexpected performance difference to interface dispatch. Later JMH results undermined that explanation.

Another conclusion about boxing changed when the measurement environment changed.

And a benchmark that looked perfectly reasonable had a subtle bug in its workload generator.

The problem wasn't that the code failed to compile.

It compiled.

It ran.

It returned numbers.

Those numbers even supported a plausible technical explanation.

That's what makes this kind of mistake dangerous.

What I'd do before shipping

If this were going into an actual payment authorization service, I'd do a few more things before making the final decision.

First, verify the domain contract. Are MCCs guaranteed to remain within the expected numerical range? How are unknown or invalid values handled?

Second, preserve snapshot consistency. An authorization should capture one immutable configuration version and use it throughout the decision. An atomic replacement inside one JVM doesn't automatically synchronize configuration versions across a distributed cluster.

Third, run the benchmark using representative traffic rather than synthetic distributions alone.

Fourth, profile the entire authorization flow. Optimizing a 5 ns lookup means very little if the real bottleneck is somewhere else.

Finally, document the choice and the conditions under which it should be revisited.

A simple implementation with a known performance trade-off is often preferable to a more elaborate one whose additional speed has no measurable operational value.

But simplicity shouldn't become an excuse to ignore evidence either.

Final thoughts

I started this experiment because I wanted to understand when O(n) could be a better choice than O(1) in a financial system.

I ended up learning more about benchmarks, JVM behavior, and AI-assisted engineering than I expected.

The results weren't exactly what I initially anticipated.

The bitmap consistently won the local lookup benchmarks.

The linear scan beat binary search despite having worse asymptotic complexity.

And the most significant error wasn't in the lookup algorithm at all.

It was in the code generating the test workload.

That's probably the biggest lesson I'll take from this experiment.

A benchmark tells you what was faster under the conditions you measured. Engineering determines whether that difference matters. And critical thinking determines whether you measured the right thing in the first place.

AI can be a useful partner in that process.

But the ability to generate a convincing technical explanation is not the same as having evidence that the explanation is correct.

Top comments (1)

Collapse
 
salparvez profile image
Sal Parvez | ML Systems RI •

The line that stays with me: a benchmark can give stable, repeatable numbers while measuring the wrong thing. The hit rate sliding from 4% to about 0.1% as products grew is the kind of bug nobody catches by reading the summary, only by checking what the inputs actually were. I come from construction estimating, and it's the same failure as a clean takeoff run on the wrong drawing set: every number adds up and none of them describe the job. Printing the observed hit rate next to each result, like you did in the table, is a cheap guard. I'd make it a standing rule for any agent-written benchmark: report the measured workload, not only the intended one.