DEV Community: Nikolai Fedotenko

I built a C++ optimizer that trusts neither me nor the LLM

Nikolai Fedotenko — Sat, 31 Oct 2026 18:45:16 +0000

Tracing JITs have a trick that AOT-compiled C++ never gets to use: they watch the values flowing through hot code, specialize for them, and guard the assumption with a cheap check. V8 and LuaJIT do this at runtime. I wanted it for a C++ trading system, ahead of time.

So I built pgspec — a small pipeline that profiles a binary on recorded workload, finds the values that dominate at runtime, and regenerates the build so those values become compile-time constants on the hot path. Guarded, with a generic fallback, and with an LLM in the loop proposing specializations.

The most important design decision turned out to be this: nothing in the pipeline is trusted — not the rule-based heuristics, not the LLM, not even my own benchmark. Every proposal has to survive a bit-exact differential test and beat the baseline on a benchmark before it ships. That gate ended up rejecting my code, my benchmark methodology (twice), and one of the LLM's ideas. Here's the tour.

The pipeline

profile (value histograms)  →  proposer (rules | Claude)  →  codegen (specialized.hpp)
        →  bit-exact differential test  →  benchmark gate  →  miss-rate monitor

The hot path is a policy template. The generic spec reads everything from the order at runtime:

struct GenericSpec {
    OrderType type(const Order& o) const      { return o.type; }
    uint32_t  lot_size(const Order& o) const  { return o.lot_size; }
    double    tick_size(const Order& o) const { return o.tick_size; }
};

template <class Spec>
inline Result process_order(const Order& o, Spec spec) {
    const uint32_t lot = spec.lot_size(o);
    const uint32_t lots = o.qty / lot;     // real division with GenericSpec…
    const uint32_t odd  = o.qty % lot;
    // … fee switch on spec.type(o), notional on spec.tick_size(o) …
}

A profiling build records per-site value histograms — crucially, the joint distribution of value tuples, not per-field marginals. Two fields can each look bimodal while only two of their four combinations ever co-occur, and a guard is a conjunction, so coverage is a property of the joint distribution. The analyzer turns the top tuples into generated specs:

// AUTO-GENERATED
struct Spec0 {
    [[gnu::always_inline]] static inline bool guard(const Order& o) {
        const unsigned g0 = (unsigned)(o.type == OrderType::Ioc);
        const unsigned g1 = (unsigned)(o.lot_size == 100u);
        const unsigned g2 = (unsigned)(o.tick_size ==
                                       std::bit_cast<double>(0x3f847ae147ae147bULL));
        return (g0 & g1 & g2) != 0u;
    }
    constexpr OrderType type(const Order&) const      { return OrderType::Ioc; }
    constexpr uint32_t  lot_size(const Order&) const  { return 100u; }
    constexpr double    tick_size(const Order&) const { return std::bit_cast<double>(0x3f847ae147ae147bULL); }
};

After inlining, qty / 100 becomes a magic-number multiply, the fee switch folds away, and the tick multiply is a constant. The guard is deliberately branchless — each compare materializes as a flag, combined with bitwise &, so the whole thing costs one heavily-biased branch. (The naive && chain compiled to three data-dependent branches and cost 14% on workloads that drift from the profile.)

The correctness contract is brutal and simple: the specialized binary must produce bit-identical results to the generic one on any input. Guards select which code path, never what it computes. The pipeline enforces this with checksums over both the profiled-like workload and an adversarial one, built with -ffp-contract=off so FP folding stays IEEE-identical.

The LLM in the loop

The fun part. The analyzer has two proposers behind one contract: a rule-based one (top-k joint tuples) and Claude. The model receives the hot-path source plus the profile — including exact bit patterns of observed doubles — and returns specs as structured JSON, under a contract that says, roughly: whenever all your guard terms hold, every accessor must equal the field exactly, or the differential test will reject you.

The trust boundary is explicit:

Every expression the model returns must tokenize against a whitelist (fields of o, enum values, numeric literals, std::bit_cast<double>(0x…ULL), comparison/arithmetic operators). I tested it with system("rm -rf /") in a guard term — rejected before codegen.
Whatever passes is compiled, differential-tested bit-for-bit, and benchmark-gated like any other spec.
Any failure — no key, invalid JSON, rejected expression, model refusal — degrades to the rule proposer with a warning.

The live run exceeded expectations. The model returned the two correct hot tuples with exact bit patterns, deliberately stopped at two specs ("the remaining ~5% is spread over 12+ tuples at ~0.42% each; each would cost a dispatch branch for well under 1% gain"), warned that upstream-computed tick sizes might differ by 1 ulp from the profiled bits so the guard would safely miss, and suggested checking that via the hit counters — it had noticed the monitoring macros in the source. Its "what I'd try next" list included hoisting dispatch per burst. More on that below, because that idea got rejected — by measurement, not by me.

This is the same conclusion the community keeps arriving at from the code-review side — see "From Vibe-Coding to Verification" by Tamiz Uddin and "Why AI-Generated Code Needs Independent Verification" by pangxueyuan2creator: LLM output is a proposal, and deterministic verification is what makes it shippable. pgspec just applies that to performance work, where "correct" isn't enough — it also has to be faster, and that's an empirical question.

What measurement actually taught me

Every one of these surprised me, and every one came from the pipeline disagreeing with my expectations.

1. My first benchmark lied. The checksum was a serial FNV chain (xor → mul → xor), whose ~5-cycle latency was the critical path. It hid the hot-path cost entirely and "proved" specialization made things slower. Independent accumulators fixed it. The benchmark is part of the product.

2. macOS core scheduling produced a 1.9x phantom. Identical binaries differed by 1.9x between pipeline runs — P-cores vs E-cores. The fix: interleave the binaries' runs and take min-of-N, so scheduling noise hits everyone alike.

3. Throughput-bound loops punish this optimization; latency-bound loops reward it. With independent iterations, Apple Silicon's out-of-order core hides udiv latency in an otherwise-idle divider unit, while the magic-multiply replacement competes for shared ALU ports: 0.74x. Put the division on a serial dependency chain (position feedback into the next order's size, like real inventory/risk logic) and the same transform gives 1.17–1.19x. Same code, opposite verdicts.

4. Generated code pushes surrounding code past inliner thresholds. With two specs, the hot-loop lambda silently stopped being inlined, which forced the running position through a store-load on the serial chain. It looked exactly like "multi-spec doesn't pay" (0.84x) until the disassembly showed the outlined call. One always_inline later: 1.17x, with two specs beating one.

5. Partial specialization is a branch-predictor hack. A spec that pins only type == Ioc (99% of orders) and keeps lot/tick as runtime reads made the adversarial workload 1.4–1.6x faster: the guard acts as a predictable pre-classifier, splitting one unpredictable 4-way switch into two heavily-biased paths. The rule proposer can't express this; the LLM contract can.

6. The LLM's per-burst idea lost — and the gate caught it. Hoisting the guard cascade to the head of each instrument burst sounds great, but run membership still has to be verified per order, so you save almost nothing and pay run bookkeeping: 1.13x vs 1.19x for the plain cascade. The idea implicitly assumes a feed-provided dispatch key. Which led to…

7. The dispatch-strategy crossover is a function of spec count. With a real instrument id, a generated switch jumps straight to the instrument's spec and the guard only verifies. At 2 specs the cascade wins (the switch adds nothing); at 6 Zipf-weighted instruments the keyed dispatch wins (1.10x vs 1.06x, and 11% faster than the cascade in throughput) because cascade cost grows with N while the switch is O(1). Nobody chooses the strategy: the pipeline builds all three binaries and the gate picks per profile.

The part I'd defend in a design review

The pipeline ends with an accept/reject gate and a production monitor:

Gate: a specialization (or dispatch strategy) that doesn't beat the alternatives on the primary workload is reverted. It has, so far, rejected a throughput regression, a mis-inlined build, and an LLM-suggested structural change. The proposer suggests; measurement decides.
Monitor: the generated header embeds the profiled generic-path share; a -DPGSPEC_MONITOR build (~2% overhead, zero when compiled out) counts live guard hits and flags drift when the live miss-rate exceeds twice the expectation. On the profiled workload: 4.98% live vs 5.00% expected. On a shifted one: 95.8%, flagged. A shipped specialization is a bet that the profiled distribution holds — and the day it breaks is exactly the volatile day you care about, so the fallback path has to stay fast, not merely correct.

What's next

Instrument-aware LLM proposals (the model already sees the instrument column; it should emit the keyed dispatch table too), Intel PT/LBR sampling instead of instrumented profiling builds, and — once compilers catch up — C++26 reflection (P2996) to generate the probes and Spec structs from annotated types instead of an external Python codegen.

Repo: github.com/nfedotenko/pgspec. The whole thing is ~1200 lines including the pipeline, and every number above is reproducible with ./pipeline.sh.

If you're into this corner of the world, TickForge by Tapesh Chandradas is a nice hand-written counterpart — the manual discipline pgspec tries to automate.

I built a localization pass that never lets an unreleased game script leave the laptop

Nikolai Fedotenko — Wed, 14 Oct 2026 21:40:12 +0000

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 2: Side Hustle

What I Built

locsmith is a command-line tool that pre-translates a game's string table with an open-weight model running locally, then checks every line before it can break the build.

I work in game development, and the localization step is where small studios get hurt. The script of an unannounced game is under NDA, so pasting it into a cloud chatbot is off the table. An agency pass on every content update costs more than many indie teams make. And raw machine translation breaks things in ways that only show up in QA: a {price} placeholder that became {цена}, a </color> tag that disappeared, or a button label that's twice as wide as the button.

locsmith handles the boring 80% before a human translator sees the file:

It batches strings to a local model with context hints ("Blacksmith NPC, gruff but friendly") and the glossary terms relevant to that batch.
It validates every result: placeholders and rich-text tags must match the source exactly (reordering is allowed), line breaks are preserved, per-string max_len is respected, and glossary terms are used.
If a string fails, it goes back to the model with the specific problem ("lost placeholder {price}", "too long: 32 > 28") and is retried. Strings that still fail are marked review instead of silently shipping.
Reruns only translate new or changed strings, so it fits a daily-build workflow. A non-zero exit code means "human needed", which lets it gate a CI step.

Who would pay: solo devs and 2–10 person studios shipping to Steam in several languages. They'd pay for a one-time license or a small subscription that turns "we'll localize after launch" into "every build has a draft translation and a list of the 20 lines a human must look at".

Demo

One command against the bundled sample (13 strings, English → Russian, with a glossary):

$ python3 locsmith.py samples/strings.csv --to ru -g samples/glossary.ru.csv
translated 13/13

gemma4:12b @ http://localhost:11434/v1
strings: 13  ok: 11  needs review: 2  auto-fixed on retry: 0  cached: 0
time: 200.2s  (15.40s per new string)
  REVIEW hud.stamina_low: too long: 41 > 24 chars
  REVIEW shop.buy: too long: 32 > 28 chars
written: samples/strings.ru.csv

The output CSV is ready to drop into the build; the two review rows are flagged, not shipped:

key	source	target	status	issues
shop.buy	Buy for {price} Aether Shards	Купить за {price} Осколков Эфира	review	too long: 32 > 28 chars
shop.not_enough	You need {0} more Aether Shards.	Вам нужно еще {0} Осколков Эфира.	ok
loot.picked	You picked up %d x %s	Вы подобрали %d x %s	ok
quest.reward	[b]Reward:[/b] {xp} XP and {shards} Aether Shards	[b]Награда:[/b] {xp} ОЗП и {shards} Осколков Эфира	ok
npc.smith.greet	Back again, Warden? ...	Снова ты, Страж? Кузница не спит, и я тоже.	ok

Every placeholder ({price}, {0}, %d, %s, {xp}, {shards}, {key_dodge}), rich-text tag (<color=#ff4040>, [b]) and \n survived, and the glossary terms came through inflected — Осколок Эфира → Осколков Эфира, Warden → Страж. The only two flags are genuine UI-length overflows a human (or a shorter rephrase) needs to resolve.

Code

nfedotenko / locsmith

Local, private, validated pre-translation of game strings with an open-weight model (Gemma via Ollama)

locsmith

Local, private pre-translation of game strings with an open-weight model.

Indie teams can't send an unannounced game's script to a cloud API (NDAs, leaks) and can't afford an agency pass on every build. locsmith runs a first translation pass on your own machine then validates every string so the output is safe to drop into the build:

placeholders and tags survive untouched: {0}, {name}, %s, %1$s, <color=#f00>, [b], \n
UI length limits (max_len per string)
glossary terms (character names, currencies) used consistently; inflected languages supported via stems, strict for exact names
broken strings are sent back to the model with the exact problem, up to --retries times; the rest are flagged review
reruns only translate new or changed strings

Works with any OpenAI-compatible endpoint: Ollama, llama.cpp llama-server, vLLM, LM Studio. Standard library only.

Gemma 4 and other reasoning models "think" before…

View on GitHub

How I Built It

Model: Gemma 4 (gemma4:12b, Q4_K_M) served by Ollama on a MacBook Pro (Apple M1, 16 GB). The tool talks to any OpenAI-compatible endpoint, so llama.cpp's llama-server, vLLM or LM Studio work too. Switching models is a single flag.
Reasoning off by default. Gemma 4 is a thinking model: left alone it spent ~1,700–3,000 tokens reasoning about a 13-string batch before emitting any JSON, which on a 16 GB laptop meant minutes per batch. locsmith sends reasoning_effort: none, which drops the same batch to ~310 generated tokens with identical output quality — about a 10x cut. --reasoning low|medium|high turns deliberation back on when you want it.
Zero dependencies: a single Python file using only the standard library, so a studio's build machine doesn't need another package manager.
Validation is the product. A single regex tokenizer recognizes {0}/{name}, printf (%s, %1$s, %.2f), HTML-like tags (<color=#f00>), BBCode ([b]) and literal escapes. Source and translation are compared as multisets, so a language can move %s before %d but can't lose or invent one.
Glossary in inflected languages: in Russian, "Aether Shard" can appear as осколок, осколков or осколками. Exact matching produces endless false alarms, so by default each glossary word is matched by its stem. Proper names can be marked strict instead.
Tests cover the validators, JSON recovery from chatty model output, the retry loop and the incremental cache.

Results on the bundled sample (13 strings, en → ru, gemma4:12b via Ollama):

passed validation on the first try: 11 / 13
auto-fixed on retry: 0
left for human review: 2 (both genuine UI-length overflows, not model mistakes)
time: 200 s total, ~15 s/string on a 16 GB M1 — memory-bound, since a 12B model barely fits; the first-pass batch of all 13 runs in ~31 s warm, and the retries for the two length failures dominate the rest. A machine with more RAM, or a smaller Gemma, is considerably faster.

Tiger Data: asking the catalog a question

The translator stays on the laptop. After a pass, a loc lead still needs a dashboard question: which Russian UI strings actually overflow max_len, and which keys carry glossary terms? That is ordinary SQL, not another LLM call, so I loaded the public sample catalog onto a Tiger Cloud TimescaleDB service (locsmith-dev: PostgreSQL 18.6, TimescaleDB 2.30.2) through the Tiger MCP and asked it there.

The overflow query returned in ~200 ms and agreed with locsmith's review flags — independently, without dumping rows into chat:

key	Russian	max	chars	over by
`hud.stamina_low`	Мало выносливости! (+ color tags)	24	41	+17
`shop.buy`	Купить за `{price}` Осколки Эфира	28	31	+3
`achv.first_blood`	Первая кровь	20	12	fits
`ui.play`	Играть	12	6	fits
`ui.settings`	Настройки	14	9	fits

A join against the glossary table showed coverage too: Aether Shard → Осколок Эфира on shop.buy, shop.not_enough, quest.reward; Stamina on the HUD and tutorial; Warden (strict) on the blacksmith greeting; Hollow on the achievement description.

The unreleased script never goes to Tiger. Only keys, lengths, status, and glossary stems — the same metadata a studio already puts in a loc dashboard. TimescaleDB is still Postgres here; hypertables would matter later for per-build translation runs as time series. That split is the product: Gemma translates privately, Tiger answers the catalog.

Why Does Open Innovation Matter?

For this product, an open model is a requirement, not just a preference.

Data never leaves the machine. An unannounced game's dialogue, item names and plot twists are what the NDA protects. With a local open-weight model, the leak surface is the developer's own laptop. I can say that to a studio with a straight face. "We send it to an API, but they promise not to train on it" doesn't pass a publisher's legal review.
The unit economics work. A mid-size indie game has tens of thousands of strings, and every patch adds more. With local inference, the marginal cost of a rerun is electricity. That's what makes "translate every build" a feature instead of a budget meeting.
No lock-in, room to specialize. Games have their own vocabulary and tone. With open weights, a studio can swap in a bigger model for the story and a fast one for UI, or fine-tune on its previously approved translations. Nobody can deprecate the model under them mid-production.
Works offline. Build machines, trains, conventions, game jams.

The trade-off is that a local 12B model makes more mistakes than a frontier API. That's exactly why the validation and retry loop exists: the tool doesn't trust the model, it checks it. And why the catalog lives in Postgres after the pass: the questions a loc lead asks ("what overflowed this build?") should be SQL, not another prompt.

Prize Categories

Best Use of Gemma
Best Use of Tiger Data