DEV Community

Gan
Gan

Posted on AI-assisted

AI Says This 20% Gap Can Never Be Closed. Can You Prove It Wrong?

Ten years ago I worked at a mobile game company. To speed up our gateway, I wrote enif_protobuf — Erlang has no official protobuf runtime, so I built a C core + NIF that raised gateway throughput several times over pure Erlang code (today's Erlang table still shows ~3.9x encode / ~2.3x decode vs pure-Erlang gpb).

Recently, with AI help, I ported that same dynamic design into pbwire: one C core, 23 language bindings, each benchmarked against its official counterpart. All numbers are in BENCHMARKS.md.

One rule: stay dynamic. Load a standard FileDescriptorSet at runtime, no per-message generated code, and bytes must match protoc exactly before anything is timed.

It won almost everywhere. Java (pure Java, no JNI) does 1.13x / 1.95x on the 74KB tier. The single loss: 136B encode at only ~0.7x (~75ns vs ~58ns against protobuf-java).

Why insist on dynamic?

  1. Generated code pins the schema at compile time — every .proto change means regenerate, recompile, restart. No hot updates for a gateway.
  2. The output is big, unreadable, slow to compile, and pollutes the repo.
  3. Business needs annotations and custom field behavior; generated classes are hard to extend.
  4. Some languages have no official runtime at all (Erlang back then) — a shared dynamic core is the only reusable answer.

So my codegen emits only thin models, and all wire encoding goes through the same core. Winning this cell with a generated writer would mean losing the product.

The last 10 days: small wins, no breakthrough

I ran many intensive rounds over the past 10 days. A few small optimizations landed, but nothing fundamental — the 136B cell never moved.

The ledger is in perf-attempts: 8 targeted attempts all failed, and JNI is rejected too (small-tier encode at just 0.54x of pure Java). AI's verdict is a structural floor: stripping all byte writes still leaves 23.3ns of per-field skeleton (22 fields, ~1.06ns/field), while the official writeTo() is straight-line code with no such loop. The second wall is HotSpot's inlining ceiling.

Help wanted

Full context is in the help doc — please don't retry what's already there. I have no preset direction in mind; any idea is welcome as long as it keeps the dynamic constraint and comes with A/B numbers.

Repro (needs JDK 17+ and protoc; the baseline is stock protobuf-java 4.36.2, auto-downloaded from Maven on first run, matching the pinned protoc 36.2):

make java-bench
# A/B: build two class trees with ./build.sh, then run alternately
bindings/java/benchmarks/ab.sh /tmp/base out/classes
# Only if you want a newer-than-Maven baseline, build it yourself:
# export PROTOBUF_JAVA_JAR=/path/to/protobuf-java.jar
Enter fullscreen mode Exit fullscreen mode

A well-evidenced "0.7x is the floor" is equally welcome — it closes the issue. AI says this 20% can never be won back. Can you prove it wrong?

Top comments (0)