DEV Community: Théo Mathieu

What "accurate" means for a geocoder, and how I decide a change is safe

Théo Mathieu — Sat, 14 Nov 2026 19:54:27 +0000

I run OpenGeocoding. Originally published on the OpenGeocoding blog. Cross-post formatting was prepared with AI assistance.

“How accurate is it?” sounds like one question, but a geocoder answers several at once. Did it find anything? Is the answer in the right country? Is the point close enough to be useful? Is it as precise as the address you sent, a house for a house and a town for a town? And when it returns a house number or a postcode, does that agree with yours?

This post explains how I score those questions for OpenGeocoding, how close I count as close enough, which losses I accept when I change the engine, and how I keep the test set from grading the index on its own data. The first post has the pass rates; this one is about what stands behind them.

A pass has several conditions

The scorer looks only at the first result. A best-of-several answer would flatter every service, and an application usually takes the first one. It ignores the confidence each service reports, because those numbers don’t mean the same thing from one provider to the next. A case then passes when the answer meets all of these, taken from the case’s expectation (what a pass means):

the country is one of the allowed countries;
the point is within the case’s distance tolerance;
the precision is within the allowed range, so a house case can’t pass on a street;
a house number or postcode it returns agrees with the request.

Failures are not one bucket either. The evaluation guide separates too_coarse (right country or region, but not the precision the address asked for), abstain, near_miss, far_miss (more than 100 km away), wrong_country and a few others. They matter because they aren’t equally bad. A country answer to a street address is a different failure from a point in another country, and the second is the more serious one.

How close is close enough

The tolerance depends on what the case asks for, and the corpus records it per case instead of using one global number:

Registry houses: the tolerance is recorded on each case. The US sample uses 150 m, or 300 m where the register places the point at a parcel centroid, and a separate sample of US addresses with loosely placed points is scored at 500 m and reported apart (US sample).
Landmarks: 1 km. Those cases measure coarse coverage, not the position of an entrance.
Company-address town cases: 5 km, rising to 10 km from 50,000 people and 20 km from a million (GLEIF cases).
Indonesian villages: twice the radius of a circle with the village’s area, between 2 and 25 km.

None of this tells you whether a point is the front door. The scorecard says so directly: the pass rates don’t measure postal deliverability, entrance accuracy or unit correctness. Whether 150 m is good enough depends on your workload. For enriching company records with a town and a region it is more than you need. For routing a courier to a flat it is not enough.

Which losses I accept when I change the engine

A pass rate says how good the engine is. A change needs a different question: what did it break? Every pull request that touches the service, its data or the test corpus is evaluated against the commit it branched from, on the same index, and the two runs are compared case by case (gates).

The rule has one exception:

A case that passed before and fails now fails the change, as does a case that moves to a worse failure class and a new critical result.
The exception is the panel built from OpenStreetMap addresses, which measures agreement with data the index itself reads. There I accept at most 5 locations that newly fail, and only when at least 5 times as many newly pass and nothing else fails.
Losses on registries, Wikidata, GLEIF and constructed cases are never traded for gains elsewhere, because those measure accuracy against sources outside the index.
Speed is a separate gate: the change fails if SearchAddress, SuggestAddress or ReverseGeocode gets more than 15% slower in total, at p50, p90 or p99.

So the acceptable loss is zero where the evidence is independent, and a small, explicit trade where it is not. Two experiments in the engine’s notes show why I keep it strict. A change to town matching reached 2,360 passing towns, but introduced 52 regressions where a coarse answer or an abstention became a point more than 5 km from the expected one. I dropped it. The version I kept finds 1,880 and, in the local comparison, adds no newly failing check. Another prototype gained 17 passes by trusting a place-type word alone, and it selected the wrong Samoa namesake, so I rejected that too.

A gate like this measures change, not accuracy. It tells me a commit didn’t make the cases I know about worse. It says nothing about the addresses nobody wrote a case for.

Not grading the index on its own data

A geocoder tested on the data it was built from will look good. The corpus has several defences against that, and each has a limit.

Expected answers come from outside the engine. Wikidata, national address registries, the US Census geocoder and similar sources provide them, never Nominatim, another geocoder or this service. GeoNames and Overture, which the index also reads, only chose which places to ask about.

Registry samples start from the registry. Cases first drawn from the index’s own Overture data and then checked against a register can’t show an address the index lacks, so recall stays invisible. The registry samples are drawn from the register first, and the index is asked about whatever was drawn (registry-first samples).

Overlap is reported, not hidden. Several registries used as expectations are also inputs to the index, so in those countries a pass measures fidelity to the register. The OpenStreetMap panel measures agreement with the index’s own source. Wikidata towns partly come from GeoNames, so some town gains reflect a shared origin. The reports group results by kind of evidence so the independent sources are not blended with the self-referential ones.

Balance, and how much a number can carry

Counts per country are not equal, and the headline rate depends on that. Town cases take up to 100 settlements per country, company-address cases up to 50 and the landmark house-coverage cases 20, so no country dominates those groups. The registry samples differ in size: from 120 cases for French Polynesia to 2,794 for South Korea, and they make up most of the 29,013-query selection that the published pass rate is computed on. That is why the scorecard also breaks results down by cohort and by registry, and why the cohorts differ in level and difficulty.

Variants of one location (a typo, a missing country) share a key, so they don’t count as independent places. About one location in five is held out from development and reported separately. With around 250 cases in a slice, the evaluation guide notes, an aggregate rate carries several points of uncertainty, and a single country’s row is one piece of evidence rather than a rate. I read the per-country tables that way, and I’d ask you to as well.

One more limit applies to all of it: this is a vendor-run evaluation on my own corpus, as the scorecard states at the top, and human review of the cases is still pending.

What to do with this

Use the same method on a sample of your own addresses. Decide first what close enough means for your workload: a town for enriching profiles, a house within 150 m for deliveries. Then geocode a few hundred addresses from each country you care about and look at the failures, not only the rate. The free plan has 50,000 requests a month with no card, and the bulk geocoding guide has a script that geocodes a CSV. The methodology is in the evaluation guide, and if a country of yours behaves worse than these tables suggest, I’d like to hear about it.

A coding agent is only as good as its harness

Théo Mathieu — Sat, 14 Nov 2026 19:49:21 +0000

I run OpenGeocoding. Originally published on the OpenGeocoding blog. Cross-post formatting was prepared with AI assistance.

A coding agent can write a fix for a geocoding bug in minutes. The hard part is knowing whether the fix is right, and whether it quietly broke something in another country. That is a question for the harness the agent works in, not for the agent. This post describes the harness around the OpenGeocoding engine: what checks every change, why the world index had to rebuild quickly for any of it to work, and the part that still isn’t stable, which is the speed gate.

The engine’s public repository merged 281 pull requests between 12 and 27 October 2026, and many of them were written by a coding agent. At that pace, I can’t judge every change by reading it. I need the checks to tell me what the change did to the whole world.

Why a geocoder needs this more than most code

Address matching is a long list of rules that interact. The engine has rules for Japanese block addresses, Korean road names, Dutch postcodes, Czech house numbers and many others (the first post has a few examples). A rule that helps one country can change the answer for an address somewhere else, and nothing in the code tells you where. Unit tests catch the cases you thought of. What I needed was a check that runs every case I have, in every country, on every change.

A world index that rebuilds in minutes

That check is only meaningful on a full world index. A synthetic test world is useful for logic, but the regressions that matter come from real data: a street in Australia that happens to share its name with a town, a Japanese address whose block number sits in a different field than expected.

So the gate needs the real index, and the base and head of a pull request have to be compared on the same one. Changes to the index format are frequent: 28 commits between 13 and 27 October set a new builder hash, and each of them names a new world index. With an import that takes days, as a Nominatim planet import does, testing each of those changes against the world wouldn’t be practical.

The engine’s builder produces the world index in about 7 min on one machine (build record). That number is the foundation of everything below. When a change alters the index, the agent’s instructions say to rebuild it locally and run the comparison there before pushing, rather than wait for CI. On the cluster that runs the CI gate, a first build for a new builder is slower and can take hours, so the local build is what keeps the loop short.

Expected answers from registries, never from a geocoder

The gold suite takes its expected answers from official and public reference sources: national address registries, Wikidata, the US Census geocoder and others. They are never taken from Nominatim, another geocoder or this engine’s own output. The cases are generated by a command, evaluate build-gold, from a list of seeds, and the generated file is never edited by hand. The suite also includes a panel of addresses drawn from OpenStreetMap; since the index reads much of the same data, that panel measures agreement rather than accuracy, and it catches regressions all the same. Today the whole suite has 36,920 cases (evaluation guide).

The rules in the engine’s AGENTS.md matter here as much as the data. An agent with a failing check has an easy way out: loosen the expectation. The instructions close that door explicitly. Expectations are never weakened, cases are never dropped, and a failure is never relabelled to make a gate pass. Known failures carry a known_failure label with their cause, stay visible in every report, and the label comes off only after a fix is verified. About one location in five is held out by a stable hash and kept out of tuning.

Base against head, on every pull request

Every pull request that touches the service, its data or the corpus gets a staging image, and its merge base already has one from main. A private cluster evaluates both images on one full world index and compares the two runs. The result comes back as two comments on the pull request (how the gate works):

gold-full, the quality check: passing checks per RPC on both sides, pass rates for each country where anything changed, and the cases that changed. It fails on any case that newly fails, fails in a worse way, or gives a new critical result.
gold-speed, the latency check: SearchAddress, SuggestAddress and ReverseGeocode timings on both sides. It fails when the total, p50, p90 or p99 of any of them is more than 10% slower.

The comparison is relative on purpose. It answers “what did this change do?”, not “how good is the engine?“. A wrong answer counts as worse than no answer, so a case that used to abstain and now returns a nearby but wrong place fails the gate, even when the new answer looks plausible.

Alongside the gold suite, 1,000 real address strings from 125 countries are kept as a snapshot, one row per address (address snapshots). They have no expected answers, so they don’t measure accuracy, but a changed answer shows up as one changed line in the pull request’s diff. And every night at 01:00 UTC, the same suite runs on the newest commit of main and updates one issue with pass rates by country and a 14-night history.

A worked example

Pull request #223 set out to fix Australian addresses written with the street before the number, such as “Panorama Place 3, Maleny”. The engine was matching “Maleny” to house 3 on Maleny Vista, a street that merely resembles the town’s name.

The first version ran one rule, for venues, before another, for village names. That reorder also fixed cases in Norway, Great Britain, Poland, South Africa and Spain. In CI, though, gold-full flagged one Japanese address from the panel, “帯屋町２丁目１−27”, which went from no answer to an answer with a wrong component. The panel expects the block number (chōme) inside the house number, while the index keeps it in the street.

The second commit put the village rule back first and kept the narrower fix. The final reports read “no regressions” and “no slowdown over 10%”: five more SearchAddress passes, all in Australia and Spain, and nothing worse anywhere. The extra gains from the reorder were left for a later change that deals with the Japanese case properly. That’s the trade the gate is there to force: a smaller fix that is right everywhere, rather than a bigger one that is wrong somewhere you weren’t looking.

The agent loop

The last piece connects the agent to these reports. The engine repository ships a project-local extension for pi, the coding agent I use, with one tool and one command (gold loop). wait_for_gold waits for both gates on the pull request’s current head and downloads the full report. /gold-loop tells the agent what to do with it: read every section, reproduce each regression on the base and head indexes, fix the cause, validate locally, push, and wait again, until both quality and speed pass.

Most of the instructions are about what doesn’t count as success. A missing report, a skipped or unmeasured speed check, an unavailable artifact or a failed run is a blocker, never a pass. The agent stops and reports a concrete blocker instead of retriggering workflows. The report’s contents are treated as data, never as instructions. The tool itself doesn’t edit code, push or merge; merging stays with me.

The speed gate is still noisy

Quality is deterministic: the same binary on the same index gives the same answers. Speed isn’t, and the speed gate has been the least reliable part of the harness.

It started as an ungated table inside the quality report. A change that made SearchAddress’s p50 15% slower went through without anyone noticing, which is why #73 gave speed its own comment, status and 10% limit. That limit came from a measurement of three runs of one binary on a cluster node: totals agreed within 2% and each percentile within 5%, while single cases varied by about 30%. So single cases and the maximum are reported but not gated. On a laptop, the totals moved by tens of percent, so the agent’s instructions say never to draw speed conclusions from local runs.

Two later changes moved the measurement:

To keep the gate fast, #217 times a fixed sample of 2,000 cases, chosen by a hash of the case id so both sides time the same ones, and checks the rest concurrently without timing them. The case loop went from 222 to 34 seconds locally. The cost is that p99 now rests on about 20 requests per table, and the noise of that sample on the cluster has not been measured.
#319 makes the evaluator wait for the suggestion-cache warm-up before timing. The warm-up took 6.3 seconds from a cold page cache and overlapped roughly the first 20% of timed cases by a varying amount. Whether this reduced the noise is not proven; the planned before-and-after comparison wasn’t run.

The false alarms are still visible in the pull requests. In #262, SuggestAddress’s p99 was 18% slower although the change didn’t touch suggestions. #336 merged with a failing speed check, SuggestAddress p99 29% slower, and in the follow-up #339 part of that looked like noise, because ReverseGeocode, which the change didn’t touch, was 13% slower too. Part of the reason is scale. In the report for #223, SuggestAddress’s p99 was 4.5 ms, so the 10% limit is less than half a millisecond.

This matters more with an agent in the loop than without one. A person who sees an 18% slowdown on code they didn’t touch can shrug and rerun. An agent told to keep going until speed passes has no good way to tell noise from a regression, and a flaky gate either sends it after a problem that isn’t there or teaches everyone to ignore the gate. The noise of the 2,000-case sample on the cluster is the open measurement, and until it is taken, the 10% limit rests on the older one.

What the harness doesn’t do

It measures change, not accuracy. A case that fails the same way on both sides doesn’t fail the gate. Some registries are also inputs to the index, so for those countries a pass is a fidelity check rather than an independent accuracy measurement.
Small numbers stay small. Per-country rows are individual cases, not rates, and a rate computed from a few hundred cases carries several points of uncertainty.
The report has no per-case responses. The cluster returns only the report, so debugging a listed case means rerunning it locally on an index from the same builder.
Outside contributions need approval. Pull requests from people outside the project wait for a reviewer, because approving one runs that author’s code in the cluster.

None of this makes the agent smarter. It makes the agent’s work checkable, quickly and against the whole world, and that is what lets me merge it.

The engine, its evaluation tools and the gold loop are on GitHub: Mokto/opengeocoding. The evaluation guide and AGENTS.md are the best places to start if you want to build something similar for your own project.

Building a geocoder whose world index rebuilds in minutes

Théo Mathieu — Sat, 14 Nov 2026 19:48:45 +0000

I run OpenGeocoding. Originally published on the OpenGeocoding blog. Cross-post formatting was prepared with AI assistance.

I build and run OpenGeocoding on my own: a geocoding engine written in Rust, and a hosted API in front of it. This post covers why I built it, how it works, how I measure it and where it is still weak.

Updated 5 November 2026: the benchmark and coverage figures now measure release 1.7.0.

The problem: an index you can’t rebuild

Geocoding at scale means turning a lot of messy text into coordinates. Take a team that receives a few million company addresses, job locations and profile locations a month, in dozens of languages and formats, and needs a country, a region and, where the data allows, a house for each one. The engine’s unscored test sample is 1,000 real strings of that kind.

I first tried to build on Nominatim, the geocoder behind openstreetmap.org. It is good software, but its build time was the problem. Its installation guide strongly recommends 128 GB of RAM or more for a planet import, requires at least 1 TB of disk, and describes the import as taking around 2.5 days on a well-configured machine.

When a rebuild takes days, you don’t rebuild often. Data updates wait. It also means you can’t test a change to parsing or ranking against the whole world, because the thing you would test against takes days to produce. I wanted an index I could rebuild whenever the code or the data changed.

What I built

OpenGeocoding is one Rust process. It parses an address with libpostal, looks it up in a read-only index and answers. There is no database, and nothing is downloaded at runtime: the index and the libpostal models are mounted read-only.

The index is built from Overture Maps, with OpenStreetMap, OpenAddresses and GeoNames as additional inputs. Places, streets and postcodes go into Tantivy, the Rust equivalent of Lucene; house numbers go into finite-state transducers beside it. Dropping the Nominatim base for a search-engine index was the most important decision in the project.

The last recorded world build, on 17 October 2026, took 409 seconds on an 18-core Apple M5 Max with 64 GB of RAM, with a 32.2 GB peak memory footprint and a result of about 16 GB on disk. Source downloads are not included (build record and methodology). This is not a like-for-like comparison with a Nominatim import: the inputs, the indexing work and the hardware all differ. What it changed is how I work. I rebuild the index whenever the builder or one of its source datasets changes.

Rules per country, not exceptions per address

Every country writes addresses differently, and libpostal, a statistical parser, gets some of them wrong. The tempting fix is to patch the address that failed. I try never to do that, and to write the rule for the country instead.

Postcodes are the simplest example. The engine knows the postcode shape of 21 countries:

("gb", r"^[A-Z]{1,2}\d[A-Z\d]?(\d[A-Z]{2})?$"),
("nl", r"^\d{4}([A-Z]{2})?$"),
("ca", r"^[A-Z]\d[A-Z](\d[A-Z]\d)?$"),
("pt", r"^\d{4}(\d{3})?$"),
("us", r"^\d{5}(\d{4})?$"),
("de", r"^\d{5}$"),

The table is used to recover a postcode the parser labelled as something else, and to decide whether a postcode belongs to the country the query names. Other rules work the same way: German, Austrian and Swiss street names with a misspelt “strasse”, Portuguese streets under the national register’s abbreviated names, Japanese block addresses the engine reads itself because libpostal splits them, and Korean road-name addresses resolved through their province, city and district hierarchy.

What keeps this honest is a gate. Every pull request that touches the engine runs the full test suite on a full world index, base against head. A case that newly fails, or fails in a worse way, fails the change, and expectations are never loosened to make it pass (evaluation guide). A one-off patch for one address usually breaks another one somewhere else, and the gate shows it.

One process, in Rust

An early version ran the index as a separate process and called it over HTTP. Merging the two into one process took one connection from about 220 to about 360 requests per second, and memory from 6.0 GB to 3.5 GB. Later work on how lookups use Tantivy cut CPU time per request from 8.2 ms to 1.4 ms, mostly by merging index segments at build time and by filtering document kinds in the collector instead of walking a term with 40 million postings. These were single measurements on a busy laptop, recorded in the lookup history, and they are not comparable with the benchmark below.

The hosted API is a separate service

The geocoder stays stateless: no authentication, no database, no writes. Everything commercial lives in a separate service. A Rust edge in front of the geocoder holds API keys, per-second limits and credit balances in memory, so no request waits on Postgres. Every 5 seconds it flushes usage counts and reloads keys and balances. Each flush writes its batch number and the usage counts in one transaction, so a retried batch is never billed twice. If Postgres goes down, the edge keeps serving from its last snapshot. Only successful replies are billed.

How accuracy is measured

The test corpus takes its expected answers from official registries and public reference sources: Wikidata, France’s Base Adresse Nationale, the Dutch BAG, the US Census geocoder, postcodes.io, Kartverket, swisstopo, CartoCiudad and other national address registries. Never from Nominatim, from another geocoder, or from this engine. A case passes when the first result is in the right country, within the case’s distance tolerance and at the right precision (house, street or town), and any house number or postcode it returns agrees with the request.

I sent every provider the same queries, with no country or proximity hints, between 17 October and 5 November 2026, and scored release 1.7.0 against their answers (scorecard and methodology):

Service	Full corpus (29,013 queries)	US/Canada subset (1,947)
OpenGeocoding	85.6%	97.7%
Geoapify	70.5%	81.0%
OpenCage	51.3%	50.8%
geocode.maps.co	42.8%	38.7%
Geocodio	not measured	89.3%

I ran this evaluation myself, on my own corpus, so read it with the caveats in the scorecard. Several national registries used as expectations (the US, the Netherlands, France, Norway, Switzerland, Spain, Czechia) are also inputs to the index, so in those countries a pass measures fidelity to the register, not independent accuracy. The corpus is not a random sample, and most of its cases were available while I developed the engine; on the fifth of locations held out from development, OpenGeocoding passed 86.4%. The OpenGeocoding row ran locally on an index from the release’s builder, close to but not identical to production. These are pass rates on a test corpus, not a production accuracy estimate.

Performance

On the hosted API, the geocoder’s average processing time was 5 ms over about 87,864 completed requests in the 24 hours to 19 October 2026, 11:55 CEST. That excludes the edge, the network and response delivery (measurement details).

For throughput, I benchmarked release 0.14.0 on 19 October 2026: one Kubernetes pod, 16 concurrent connections, three runs of 20,000 requests each. The runs reached 2,675, 2,656 and 1,774 queries per second, with p95 latency between 18.4 and 29.9 ms, and all 60,000 requests completed without errors. The pod requested one CPU with no limit and used 4.1 to 5.0 cores of a shared Intel Core i5-13500. The client cycled through 1,000 repeating queries with the parse cache warm, and called the pod directly, bypassing the ingress, TLS and the edge. It is a short warm-cache run, not a sustained capacity figure. The third run was slower than the first two; the methodology and raw record has all three.

Where it is weak

House-level coverage across most of Asia and Africa is still poor. In the coverage measurement of the release 1.7.0 world index, India has 85,266 house numbers for 1.46 billion people, China 44,614, Indonesia 68,068, Pakistan 96,337, Nigeria 7,245 and Ethiopia 424. The United States has about 113 million, France 27 million, Japan 22 million and Taiwan 5.7 million. In the weak countries, expect a town or a street, not a house. The benchmark shows the same thing: of 151 street and house cases in Asian countries such as Malaysia, the Philippines, India, Vietnam and Thailand, OpenGeocoding passed 43, against 61 for Geoapify, and of 2,794 Korean school addresses, 795.

The engine is strongest where open address data is dense: North America, most of Europe, Australia and New Zealand, and a few others such as Japan and Taiwan. Outside those, treat it as a town and region geocoder until you have checked your own addresses, and check your towns too: in the 78 countries and territories the corpus has no registry for, OpenGeocoding found 58.4% of 3,240 towns, behind Geoapify (68.4%) and OpenCage (66.4%), mostly because small settlements are missing from the index. The index has streets in 240+ countries and territories, but streets everywhere is not houses everywhere. Asia and Africa are what I’m working on next.

The licence

The engine is source-available under PolyForm Shield 1.0.0, not open source. You can read it, run it, modify it and redistribute it, including commercially, and you can self-host it inside your own product. What you can’t do is offer a product that competes with it, for example by taking the engine and selling it as a geocoding API. A free competing service counts too. I picked it so the project can’t simply be taken and resold as someone else’s API.

The data keeps its own licences: OpenStreetMap and parts of Overture are under the ODbL, and the attributions page lists every source.

Pricing

The free plan has 50,000 requests a month at 2 requests per second, with no card. Paid plans start at $29 a month for 300,000 requests. Autocomplete is free, and when credits run out billable calls stop instead of creating an overage charge (pricing).

Try it

The fastest way to judge it is the demo on the homepage: type an address from your own data, no account needed. The API docs cover search, autocomplete, reverse geocoding and regions, and the engine is on GitHub. If it gets your country wrong, I want to hear about it.

Billing every API request exactly once, without a database on the request path

Théo Mathieu — Sat, 14 Nov 2026 19:47:20 +0000

I run OpenGeocoding. This article originally appeared on the OpenGeocoding blog on 20 October 2026. Cross-post formatting was prepared with AI assistance.

The first post was about the geocoding engine. This one is about the smaller service in front of it: the edge that checks API keys, applies rate limits, spends credits and counts usage for billing. It is written in Rust like the engine, runs on Kubernetes as several identical replicas, and one rule shaped all of it: nothing on the request path waits for a database.

The edge lives in a private repository, so I’ll describe it rather than link to it. The one code sample below is a simplified version of the real query.

Why not write to the database on every request

The obvious design is to look up the key and write a usage row in Postgres for every request. It’s simple and exact, and it’s what I considered first. The problem is throughput. In the engine’s benchmark, a single geocoder pod served 1,700+ searches a second (methodology). If every one of those requests also writes to Postgres, the database has to sustain the same rate, for all customers at once, and keep sustaining it as the geocoder scales out. The engine would only be as fast as the billing database behind it.

Latency points the same way. The geocoder’s average processing time is 5 ms (how it’s measured). A database round trip on every call would add its own latency on top, and a slow or unavailable database would become a slow or unavailable API.

How backend processing time was measured

The request-weighted mean was 4.71 ms across approximately 87,864 completed RPC observations, from 18 October 2026, 11:55 CEST to 19 October 2026, 11:55 CEST, rounded to the nearest millisecond on the homepage. The mean divides the increase in summed gRPC handling time by the increase in completed RPC observations over that window.

Over 99.9% were address searches, so this is not an all-endpoint benchmark. The timer excludes the public API edge, network time and response serialization/delivery. Public and internal callers are not distinguished. Rollouts and missed scrapes limit completeness; this is a dated observation, not a latency guarantee.

So keys, limits and balances live in memory, and Postgres is updated in the background.

Kubernetes first

Everything runs on Kubernetes, and I designed it that way from the start rather than moving it there later. Each part is a Deployment with several identical replicas spread across nodes, and none of them assumes it is the only copy:

The geocoder is stateless and read-only. Each pod pulls a complete, immutable index from an S3-compatible store in an init container and serves it from a read-only volume. A small leader, one of two pods sharing a Kubernetes Lease, starts a build Job when the index named by the chart’s settings isn’t in the store yet, and rolls out the serving pods once it is. New pods load their index before old ones stop, so a rebuild never interrupts serving (engine Helm chart).
The edge, the subject of this post, runs three replicas by default. Each has its own in-memory snapshot, its own counts and its own random ID. The replicas never talk to each other; they only meet in Postgres, every few seconds.
The web app (marketing site, dashboard and signup) also runs three replicas.

In front of the edge, an L7 ingress balances each HTTP request and gRPC call across pods, not each connection. Rollouts start a new pod before removing an old one, and disruption budgets keep node drains to one pod at a time.

Most of what follows is about how the edge replicas agree on keys, limits and money without asking each other.

What a request does

For each request, the edge:

hashes the API key with SHA-256 and looks it up in an in-memory snapshot of active keys;
takes a token from the account’s per-second bucket;
for billable operations, checks the account’s remaining credits;
calls the geocoder;
counts the request if it succeeded.

“Succeeded” is narrow on purpose. Only a JSON 2xx reply or a gRPC reply whose final grpc-status is 0 is billed. Over gRPC, an HTTP 200 can still end with an error status in the trailers, so the edge buffers unary gRPC replies through their trailers before it counts anything. Backend failures return a generic error and aren’t billed. Autocomplete and region lookups need a key and share the rate limit, but they don’t spend credits.

The remaining credits are worked out locally: the balance from the last Postgres load, minus the requests this replica has counted that the balance doesn’t include yet. When that reaches zero, billable calls get a 429. There is no metered overage.

Each replica subtracts only its own counts. Requests served by the other replicas reach its balance through Postgres, after their next sync and its next reload. That gap of a few seconds is where the small overshoot described at the end comes from.

The 5-second sync

Every 5 seconds, each edge replica runs one sync. It takes its pending counts as a batch, gives the batch the next sequence number and writes it in a single transaction:

BEGIN;
-- Adds nothing if this batch was already written.
INSERT INTO edge_flush (edge_id, seq)
VALUES ($1, $2) ON CONFLICT DO NOTHING;

-- Only when that insert added a row:
INSERT INTO usage (key_id, day, count)
SELECT * FROM UNNEST($3::uuid[], $4::date[], $5::bigint[])
ON CONFLICT (key_id, day)
DO UPDATE SET count = usage.count + EXCLUDED.count;

-- Lock the accounts' credit grants, then spend the batch
-- from the grant that expires soonest.
SELECT 1 FROM credit_grant
WHERE user_id = ANY($6) ORDER BY id FOR UPDATE;
UPDATE credit_grant SET used = used + ...;
COMMIT;

The edge_flush row is what makes it exactly once. edge_id is a random ID each edge process picks when it starts, and seq numbers its batches, so replicas flush independently and their batches never collide. If the transaction fails, or commits but the reply never arrives, the batch stays in memory with the same sequence number and is sent again on the next sync. A retry of a batch that already committed hits the existing row and adds nothing; a retry of one that didn’t commit writes it for the first time. Either way, it lands once. A test against a real Postgres database flushes the same batch twice and checks the count. The row is written even when the batch is empty, so it also serves as the replica’s heartbeat.

Credits are spent in the same transaction. Each paid period and each free month is a separate grant with its own expiry, and a batch is spent from the grant that expires soonest. Locking the account’s grant rows first means two replicas flushing at the same moment take turns instead of spending the same credits twice.

After the commit, the sync reloads active keys, each account’s per-second limit and balance, and the number of edge replicas that flushed in the last 15 seconds. A batch stays counted against the local balance until a reload that includes it succeeds, so there is no moment where requests are neither in Postgres nor in memory.

For Postgres, this means one write transaction per replica every 5 seconds, with rows per key and day rather than per request.

Rate limits without shared state

Rate limits are the part I simplified most. A strict per-account limit across several replicas needs a shared counter, usually Redis, consulted on every request. Instead, each replica takes its share. With N replicas that sent a heartbeat in the last 15 seconds, an account’s bucket on each replica refills at limit / N tokens a second and holds at most the larger of limit / N and 2 tokens.

As an example, take the free plan’s 2 requests a second across 5 replicas. Each replica refills at 0.4 a second and holds 2 tokens. Starting from full buckets, a client can make up to 10 requests, and then the refill settles at 2 a second overall. The two-token minimum absorbs short bursts when the load balancer happens to send a few requests to the same replica. Retry-After is rounded up to the next local token, so an empty bucket in that example answers with 3 seconds rather than a fixed 1.

Scaling needs no configuration. A new replica’s first sync is also its first heartbeat, so the others count it at their next reload, within 5 seconds, and lower their share. When a replica stops, its last heartbeat ages out after 15 seconds and the others raise their share at their next reload. While that happens, the shares are briefly off, which is one of the inaccuracies listed below.

When Postgres is down

If Postgres is unreachable, the edge keeps serving from its last snapshot and keeps counting in memory, and the counts are written once the database is back. I chose to use a snapshot of any age rather than turn away paying customers. Signup, dashboard changes and the website demo do stop, because they need the database. The edge’s liveness check doesn’t depend on Postgres, so an outage never makes Kubernetes restart pods that are holding cached keys and unflushed usage.

A new edge pod can’t load keys without Postgres, so it stays not ready while the existing pods carry the traffic. A rollout during an outage stalls instead of replacing working pods with ones that can’t serve.

Shutting down without losing usage

Rollouts are the other moment where counts could go missing. Kubernetes replaces edge pods one at a time, starting the new one before the old one stops. On SIGTERM, the edge drains in-flight requests for up to 30 seconds, stops its background sync, then freezes billing and flushes everything still pending, retrying for up to 15 seconds. A request that finishes after billing is frozen gets an “unavailable” error instead of a successful reply that would never be billed. If the final flush still fails, the edge logs the lost usage and exits with a nonzero status. These budgets, a 10-second preStop delay and the error reporter’s 2-second close timeout all fit in Kubernetes’ 60-second grace period.

What this costs

The design accepts a few known inaccuracies. I’d rather list them here than have someone find them later:

A replica that crashes loses up to one sync interval, 5 seconds, of counts. Those requests are never billed.
A balance can be overspent by roughly one sync of traffic plus bucket bursts. At the Growth plan’s 25 requests a second, that is about 125 requests plus bursts.
Rate limits are approximate. Burst capacity grows with the number of replicas, restarts and changes in the replica count can briefly allow extra bursts or throttle early, and uneven routing can return a 429 below the account’s nominal rate.
A persistent connection pinned to one pod only gets that pod’s share of the limit, so the load balancer has to balance each HTTP request and gRPC call, not each connection.
Successful replies are charged when they are prepared, before delivery. A client that disconnects while the reply is on its way is still billed for the work.

Most of these fall in the customer’s favour. Lost counts are never billed, and an overspent balance means a few free requests rather than an unexpected invoice, because there is no overage to bill.

The improvement I’d like to make later is sticky routing per account. If the ingress sent every request from one account to the same edge replica, that replica would hold the account’s whole rate limit and see all of its unflushed usage. Limits and balance checks would then be exact, without Redis and without a network call on the request path. It isn’t built yet, and it brings its own trade-offs. The ingress only sees the API key, not the account, so an account with several keys needs a way to land on the same replica. A very busy account would load a single replica. And when replicas are added or removed, some accounts move to another replica and start there with a fresh bucket.

Shared counters in Redis would also make the limits exact, but at the cost of a round trip on every request, and I’d rather not pay that. Keeping Postgres in the background also keeps another option open: because the edge only talks to Postgres in the background, a second region with its own edge and geocoder could use a distant database without slowing anyone down.

Try it

The free plan includes 50,000 requests a month at 2 requests a second, with no card. Create a free account, make a few calls, and you’ll see them in the dashboard’s usage, written by the sync described above.