We Shipped an Optimization Our Own Benchmark Said Not To

Pooja — Mon, 07 Sep 2026 01:46:20 +0000

Here's the number first: we added a flag that parallelizes the slowest part of our log query tool across 8 goroutines, ran it against a real 76.3MB file, and it finished in 7.558 seconds. The same query, same file, one worker: 6.535 seconds. Eight workers lost to one by 16%.

We shipped the flag anyway.

logq is a command-line log query tool — filter, group, count, percentile, and time-window a JSONL/logfmt/plain-text file the size of your disk, from one static binary, zero third-party dependencies. Built for Track B (Parsers & Data Formats) of the Zero Dependency Hackathon.

At a glance

-j 8 (8 parallel workers) measured 16% slower than -j 1 on a real 76.3MB file — shipped anyway, disclosed the moment it was found

Replaced tidwall/gjson (10,420 known importers) with ~150 lines on top of stdlib's own encoding/json decoder

A three-valued MISSING/null/false evaluator — the single design decision we're proudest of, and it isn't a package replacement at all

Added CI two days before submission; it failed for a reason we never fully diagnosed, so we removed it rather than leave an unexplained red badge

We're citing exact commit hashes and timestamps throughout this post instead of describing things from memory, because STDLIB.md, this project's own dependency ledger, is built the same way: "this ledger is built incrementally... so the git log itself is the evidence trail for the rationale." If a claim below has a hash next to it, git show <hash> in the public repo is where it actually came from.

The bug most log tools ship anyway

Most query tools treat "this field doesn't exist on this record" and "this field exists and is JSON null" as the same case — or they don't think about the distinction until a record shaped differently from the rest crashes the whole run. logq treats MISSING, null, and false as three genuinely different states, all the way through the evaluator, and it's the first real design decision in the project's history: commit e04368a (Value/Record model, MISSING vs Null) and 8849137 (the three-valued evaluator, with a generated truth-table matrix of tests) land ten minutes apart, twenty-eight minutes into the whole build — before there was a JSON decoder, before there was a CLI, before there was anything to actually filter yet.

	any comparison (`==`, `>=`, ...)	`exists(field)`
field absent	`false` — never an error	`false`
field present, `null`	`false`, except it matches the `null` literal	`true`
field present, `false`	evaluates normally	`true`

The practical effect: a filter query never crashes partway through a file just because one record has a different shape than the rest. It also never quietly tells you null "equals" missing, which is the kind of wrong answer that shows up with no error message attached — the worst kind, because nothing tells you to go check.

The package with 10,420 reasons to exist

tidwall/gjson has 10,420 known importers on pkg.go.dev — checked live twice, a day apart, so the number in this post wouldn't be stale by the time anyone read it. People reach for it because Go's own encoding/json, decoded the default way into map[string]any, does two things quietly: it decodes every number to float64 (losing precision past 2^53 — a snowflake ID or trace ID comes back a different, confidently-printed number, no error), and Go maps have no defined iteration order, so a re-printed record's fields land in a different order than the source line.

Commit 09dd14c, four minutes after the truth-table commit above, replaces both properties with the same tokenizer gjson itself sits on top of: encoding/json's Decoder.Token(), streamed into a hand-written ordered map, with json.Number instead of the float64 default. About 150 lines. Not a smaller gjson — the same stdlib primitive gjson wraps, pointed at the two specific things gjson exists to fix. Full trade-off table in STDLIB.md.

It's not a strictly better trade, and BENCHMARKS.md says so directly: gjson can extract one path from a large object without decoding the fields around it. This decoder builds the full ordered record regardless of what the query touches. For a query reading one field out of thirty, gjson wins. Written down, not hidden.

We shipped an optimization our own benchmark said not to

-j N exists because the obvious complaint about a single-threaded aggregation stage is "why isn't this parallel." Commit a29a587, twenty-three and a half hours into the project, makes it parallel: stats' per-group aggregation shards across N goroutines, routed by a hash of the group key so a group's records always land on the same shard — no round-robin, no cross-shard merge beyond a trivial flush at the end.

Correctness got verified in that same commit: byte-identical output between -j 1 and -j 16 over a 2,000-record fixture. Speed didn't. That's worth being honest about — the flag shipped on the strength of being correct, not on the strength of being measured. The obvious assumption, more workers means faster, rode along unexamined for two and a half more hours, until commit 58da5cf — a release-readiness sweep whose own message admits it bundles ten different things into one sitting, benchmarking included — actually ran the numbers:

workers) takes 7.558 seconds — 16% slower, because -j only shards aggregation while JSON decoding, the real
bottleneck, stays
single-threaded" width="800" height="448">

Eight workers, 16% slower than one, on a real 467,715-line file. The root cause isn't subtle once you look at what -j actually touches: it shards the aggregation math, the count/sum/percentile bookkeeping per group. It does nothing to JSON decoding, which is the part that actually dominates per-line cost. So -j N spends real wall-clock time on channel dispatch and cross-goroutine coordination to parallelize the cheap 20% of the work, while the expensive 80%, decode, stays exactly as single-threaded as it was at -j 1.

The honest options at that point were: quietly remove the flag before anyone benchmarks it, quietly leave it in and hope nobody does, or publish the number and ship it anyway. We did the third one, in the same commit that found the regression, not a follow-up two weeks later titled "fix docs." The flag stays because a future version that actually shards decoding will make -j worth having — and when that lands, this number is the baseline it has to beat, not something quietly removed from the history first.

The CI we couldn't explain, so we removed it

CI landed in that same sweep commit, 58da5cf: matrix build across Linux/macOS/Windows, a coverage gate, -race, fuzzing, a reproducibility check. It sat in the repo for a full day without ever actually running.

It ran for the first time two days later, triggered by commit d098ba7 — a docs-only push, README and a slide deck, no Go code touched. build passed on all three OSes. test failed on all three. race failed too, and there was no way to see why: the repository wasn't reachable from the sandbox this project was built in, no way to pull the actual failing log output.

We spent the next hour doing what was actually reachable instead: downloaded the exact Go 1.23.4 toolchain CI resolves to and ran the full build/vet/test suite with it locally — clean. Cross-vetted for Linux and macOS explicitly, for the first time ever rather than just cross-compiling for them — clean. Checked the one binary fixture in the repo for line-ending corruption, a real risk given this dev machine's Windows git config — untouched. Every avenue actually reachable came back clean, and the real CI failure stayed unexplained.

Commit 0ab77e6 is the decision that came out of that hour, 55 minutes after d098ba7 by the log's own timestamps: remove the workflow entirely rather than leave a red badge nobody could explain, or worse, quietly disable just the failing jobs so it would go green without the underlying question ever getting answered. No CI beats a badge that's lying, even by omission.

Watch it run

Numbers on a page are one thing. Here's the compiled binary, live: the same three queries from the top of this post, the -j regression, gzip and logfmt input handled transparently, and a typo caught with a positioned "did you mean" suggestion instead of a stack trace.

The numbers, together

Filter throughput:      19.24 MB/s in-process (20K lines) / ~15-16 MB/s real 76.3MB file
Memory across 10x scale: 344,272 -> 353,968 bytes (9,696-byte delta, ~48MB envelope)
Tests:                   746 passing, 14 packages, 89.6% line coverage (internal/*)
Fuzzing:                 10.4M executions against the parser, zero crashes
Reproducible build:      two builds, identical SHA-256 hash
-j 8 vs -j 1:             16% SLOWER on a real 467,715-line file — shipped and disclosed anyway

All of it reproducible from a clean checkout: make bench, make soak-manual, make cover, make repro-check, documented in full in the README. None of it is estimated, and none of the numbers above were softened for this post — they're the same ones sitting in the repo's own markdown files, checked against git log before a word of this was written.

We're Team Vanquisher — a team entry for Track B of the Zero Dependency Hackathon (Aug 28–31, 2026). Every hash above is checkable in the public repo with git log --format="%h|%ad|%s" --reverse. Source under MIT.