Two of my own numbers almost contradicted each other, and I nearly published both without noticing.
While finishing geo-sentinel, a spatial telemetry pipeline that ingests simulated GPS pings and checks them against geofence boundaries using Uber's H3 index instead of database-side polygon queries, I had a benchmark showing over 350,000 lookups per second. I also had a live Grafana dashboard from the actual running pipeline showing roughly 100-125 pings per second. Both numbers were real. Put next to each other without context, they looked like a bug, or worse, like I didn't know which one was true.
The setup
The core idea: pre-rasterize geofence polygons into H3 hexagonal cells once, then checking whether a GPS ping has crossed a boundary becomes an in-memory hash set lookup instead of a database query. A naive approach runs ST_Within against PostGIS on every single ping. At any real volume, that's the bottleneck, not the logic itself.
So I built both paths side by side. A Go producer simulates vehicles moving along real road coordinates, publishing pings through Redpanda. A consumer resolves each ping to an H3 cell and checks it against the geofence set in memory. And a separate standalone benchmark runs the same comparison with no network, no broker, no I/O at all, just the raw algorithmic cost of one approach against the other.
Why two different throughput numbers both being true isn't a contradiction
The standalone benchmark measures something specific: how fast can this machine resolve a coordinate to an H3 cell and check set membership, with nothing else in the way. That number came back over 350,000 lookups per second, between 547x and 791x faster than the equivalent ST_Within query, measured the same way, on the same hardware, same sample size.
The live pipeline measures something different: how fast pings actually move from a simulated producer, through a message broker, into a consumer, end to end. That number sat around 100-125 pings per second in the demo configuration, not because the H3 lookup suddenly got slow, but because the producer itself wasn't generating pings any faster than that. The bottleneck in the live demo is the simulated fleet's ping rate, not the evaluation logic the whole project is actually about.
I nearly wrote the README as if these were one continuous story, start with the big number, end with the dashboard, and let the reader assume they were the same measurement scaled down. They're not. One proves the algorithm is fast in isolation. The other shows what a deliberately-throttled demo pipeline looks like end to end. Conflating them would have meant the strongest evidence in the whole project quietly didn't match the thing it was supposed to be evidence for.
What actually changed
Nothing about the code. The fix was entirely in how the two numbers get presented: labeled separately, scoped explicitly, with a line stating plainly that the live demo's throughput is bounded by the producer's configured rate, not the H3 engine. The benchmark result stands on its own. The dashboard stands on its own. Neither one pretends to be the other.
What this project didn't try to be
This isn't a production fleet-tracking platform, and a hand-rolled H3-plus-Redpanda pipeline skips a lot that a real system would need, backpressure handling at scale, geofence updates without a full restart, multi-region replication. What it does prove, with numbers that actually mean what they say, is that moving spatial lookups off the database and into memory is a real, measurable win, and that the win is worth stating precisely rather than letting an impressive number do more talking than it earned.
Top comments (0)