DEV Community

New Way Capital Advisory
New Way Capital Advisory

Posted on

I normalised 51 million property transactions from 16 government registries — here's the bug that nearly ruined it

Property registry data is open in many countries. HM Land Registry publishes every sale in England and Wales. The Dubai Land Department publishes transactions. US counties publish deeds. It's all free, and any developer can download it this afternoon.

So why did I spend a year building an API over data anyone can get?

Because of a bug I want to describe, since I think it generalises well beyond property.

The failure that doesn't raise an error

Early on I ran a query for a well-known residential district and got a few hundred transactions back. Reasonable-looking number. Prices in a plausible range. Nothing in the response suggested a problem.

The real figure was over a hundred thousand.

The registry records that district under its official administrative name. Everyone — buyers, agents, listings, the developer writing the query — calls it something else. Both names exist in the data. The common one is attached to a small fraction of the records. So a natural query returns real transactions, correctly formatted, from the right city, and gives you a picture of the market that's wrong by two orders of magnitude.

There's no exception to catch. No empty result to check. No status code. Your test suite passes, because what came back is valid data. You ship, and every number downstream is quietly built on 0.5% of reality.

This class of bug — plausible, silent, structurally invisible — is the actual reason property data is hard. Not acquisition. Acquisition is a download.

And it isn't one district. It's every market, with its own conventions: postcodes that changed, towns absorbed into other towns, buildings recorded under a developer's project name instead of the name on the door, transliteration variants. Each one produces a confident wrong answer.

What that means at 16 markets

Once you've found this in one registry, you have to assume it in all of them. That turns into:

  • Ingest per source. Sixteen formats, sixteen cadences. Some publish bulk monthly. Some expose only the current year and expect you to have kept the rest. Some have historical records sitting in an unusable state.
  • Normalisation onto one schema, so /v1/comps?market=uk and market=dxb answer the same question the same way despite the sources agreeing on nothing.
  • Resolution that survives naming drift, so the common name and the registry name reach the same records.
  • Geocoding every transaction, because radius search is table stakes and no registry hands you coordinates.

Building one market is a weekend. Building sixteen is a company. That gap is the whole product.

What I ended up with

Over 51 million recorded transactions, 16 markets, 7 countries — UK, France, Ireland, Singapore, Taiwan, Dubai, and ten US metros. Recorded sale prices from official registries. Not listings, not estimates.

curl -H "X-RapidAPI-Key: $KEY" \
-H "X-RapidAPI-Host: property-comparable-sales.p.rapidapi.com" \
"https://property-comparable-sales.p.rapidapi.com/v1/comps\
?market=nyc&zip_code=10003&radius_miles=1&months=12&limit=3"

{"results": [{"sale_date": "...", "price": ..., "property_type": "...", "distance_km": ...}]}

Endpoints: /v1/comps, /v1/stats, /v1/trends, /v1/valuation, plus CSV and PDF export.

Two exist specifically because of the problem above:

  • /v1/resolve — give it free text, it tells you which market the address belongs to before you spend a call on it
  • /v1/coverage — every market, its source, its geography, and how current it is, returned by the API rather than buried in docs

That second one matters. Registries publish on their own schedules and some lag badly. Rather than smooth over it, the API reports its own recency so you know what you're standing on. I'd rather lose a signup than have someone build on stale data without being told.

It's tested, and the tests are the docs

Every market's published example runs against both /v1/comps and /v1/valuation as a standing regression — 16 for 16 against live data. A broken documented example is invisible to me and infuriating to you, so it's checked continuously rather than trusted. The service has been through full-restart testing with everything recovering unattended, and it's been serving production traffic since launch.

The part I didn't expect

The developers doing the most interesting work with this are building agents. So there's an MCP server — the same data exposed as tools, so a model can call it directly instead of someone hand-assembling HTTP requests:

github.com/Tianning-lab/property-comps-mcp-server

That shift is worth naming. Models made building an application dramatically cheaper and left the data problem exactly as expensive as it always was. If you're building something that reasons about property, the reasoning layer is now the easy half.

Try it

Docs and live playground: api.nwc-advisory.com/docs
Listing: rapidapi.com/NWCA/api/property-comparable-sales

Built in Switzerland. Python, FastAPI, SQLite.

If you hit a market I don't cover or a place name that doesn't resolve, tell me — that second one is the bug I care most about, and I can't find them all myself.

Top comments (0)