DEV Community

Devil Scrapes
Devil Scrapes

Posted on

UK Police's Crime API Returns 404 When the Answer Is Zero

Quick answer

data.police.uk, the UK government's keyless street-level crime API, does not answer "no data for this month" with a 200 and an empty list. It answers with an HTTP 404. A client that treats every non-2xx status as a failure turns a completely normal, expected outcome — the most recent month or two hasn't been published yet — into a crashed run. The UK Police Street-Level Crime Data Scraper tells the two apart on purpose, and exports category, month, street name, coordinates, outcome status and crime ID for up to 5 points and a 6-month window at $2.70 per 1,000 rows.

Why does a "no data" month return 404 instead of an empty array? 🧱

Most JSON APIs signal "your filter matched nothing" the same way regardless of why it matched nothing: 200 OK, body []. data.police.uk doesn't. Ask it for crimes-street/all-crime?lat=...&lng=...&date=2026-08 before that month's crime file has actually been published — the source has a real 1-2 month lag behind the calendar date — and you get back a 404, not a 200 with an empty list.

That single status code has to carry two completely different meanings for a scraper that runs on a schedule:

  • A genuine 404 — this (location, month) pair simply has no crime file published yet. Nothing is broken. This is the expected shape of the last month or two of any date range you ask for.
  • Every other non-2xx429 because you're leaning on it too hard, 5xx because the source is unhappy, a malformed body you can't parse. These are real failures and should say so.

Folding the first case into a generic "non-2xx means the run failed" branch is exactly the mistake this Actor is built to avoid. Our own fleet has hit its mirror image before: 42 FAILED customer runs, on other Actors, from treating "the search legitimately found nothing" as an error rather than a valid answer — and it once got a different Actor delisted outright. So the client here handles 404 as its own explicit branch, immediately, with no retry — it logs the location and month at WARNING, returns None to mean "zero rows for this pair," and lets the run keep going to the next (location, month) combination. A 429 or 5xx still retries with backoff and honours Retry-After; anything else still raises. Only the 404 gets special-cased, and it's special-cased on purpose.

What does one query point actually cost you? 🌐

The other thing a naive integration misses: data.police.uk's search radius isn't a parameter you tune, it's fixed server-side at roughly a mile around the point you send, and a single request downloads that entire month's crime file for the point before any filtering happens on your end. There's no category narrowing at the API layer beyond picking a category-specific endpoint instead of all-crime, and no way to ask for "just give me theft." That's why this Actor bounds the two dimensions that actually control cost — up to 5 points and a 6-month span per run — rather than pretending it's an unbounded national mirror. Each (point, month) pair is one full-file fetch; keeping the fan-out small keeps the bill predictable.

Underneath both of those, the ordinary defences still apply: rotating Chrome/Firefox/Safari TLS fingerprints on every request, exponential backoff with a 30-second cap on retriable statuses, and Pydantic-validated rows — outcome_status and persistent_id normalised to null when the source sends an empty string instead of omitting the field, so you never have to guess whether "" means unresolved or means broken.

What you get per row

Field Example
category anti-social-behaviour
month 2026-06
street_name On or near Bedford Street
latitude / longitude 51.510389 / -0.124157
outcome_status null when unresolved
persistent_id null when the source sends ""
crime_id 136061732
search_lat / search_lng the input point that produced this row

Set 1-5 lat/lng points, a crime category (or all-crime for every category), and a month_from/month_to range up to 6 months wide. Export as JSON, CSV or Excel from the run's dataset.

FAQ

Does this need an API key or proxy?
No. data.police.uk is a keyless UK government API — this Actor makes direct requests, no proxyConfiguration field to configure.

Why did my run return zero rows?
Either the point/month pair genuinely had no recorded crime, or the month is too recent for the source's publish lag and answered 404. Both finish as a SUCCEEDED run with a status message naming exactly what was searched — never a failed one.

Can I search more than 5 points or a longer date range?
Not in v1. The bound keeps runs fast and their cost predictable — each point/month pair is one full-file download, not a filtered query.

Where does the data come from?
The UK Police public Street-Level Crimes API, the same open dataset published by UK police forces.


Built by Devil Scrapes. We publish the traps we hit, because the ones that return a status code you didn't expect are the ones that cost real runs.

Top comments (0)