EPA ECHO Reports a Rejected Search as HTTP 200

Quick answer

EPA ECHO's Clean Water Act registry is free and keyless, but you cannot fetch a facility roster from it with one request — the API answers a search with a query ID, not rows, and it reports a rejected search as an Error object inside an HTTP 200. The EPA ECHO CWA Facilities Scraper runs ECHO's two-call protocol correctly and flattens every regulated facility in a state — name, address, registry ID, county, EPA region, permit status — into one row at $3.05 per 1,000 facilities.

The two things that break a naive ECHO client 🧱

The search call doesn't return facilities. cwa_rest_services.get_facilities answers with a QueryID and a match count. The rows live behind a second, separate call — cwa_rest_services.get_download?output=CSV&qid=<QueryID> — which streams the entire result set in one CSV body with no server-side pagination, no offset, and no page parameter to loop over. A client written against the shape most REST APIs use will happily parse the first response, find a count where it expected records, and either report zero facilities or invent a pagination loop that ECHO has no concept of. This Actor runs both calls in sequence, carries the query ID across, and parses the CSV as the single payload it actually is.

A rejected search arrives as HTTP 200. When ECHO refuses a query — most commonly because it wasn't filtered enough to be worth running — it returns a normal 200 whose JSON carries Results.Error.ErrorMessage. Nothing in the status code says anything went wrong. A client that branches on response.status_code alone treats the refusal as a successful empty result and reports "no facilities in this state," which is a wrong answer that looks exactly like a right one. This Actor inspects the Results body and raises on Error, so a refused search fails loudly instead of quietly claiming a state has no regulated dischargers. The download call gets the same treatment in reverse: if it hands back an HTML error page instead of a CSV body, that's caught as a transport failure rather than parsed as zero rows.

Underneath both, the run still has to survive an ordinary day on a federal API — 429s and 5xxs under load, and the occasional malformed CSV line. Transient responses are retried with capped exponential backoff honouring Retry-After, across rotating Chrome/Firefox/Safari TLS fingerprints; a single unparseable CSV row is validated, logged and skipped rather than aborting the batch; and a genuinely empty state/name combination finishes as a clean SUCCEEDED run, not a false failure.

The judgement call: our own gates passed a scraper that never ran 🚧

This Actor is also the reason we now check something we previously assumed.

Its first cloud run lasted five seconds, wrote zero rows, emitted not one application log line, and was recorded SUCCEEDED, exit code 0. It had passed every gate we own: 40 passing tests, ruff clean, pyright clean, input-prefill verified, scaffold-stub check clean.

The cause was that two entry paths had drifted apart. apify run executes python -m src, which runs src/__main__.py — correct, so every local test passed. The Dockerfile's CMD executes python -m src.main, and src/main.py had no __main__ guard, so the container imported the module, defined every function, called none of them, and exited 0. The scraping code was entirely correct. It was simply never invoked.

Nothing static could see that, and nothing local could reproduce it. Under pay-per-event billing this is the expensive shape: it charges the start fee and returns nothing — the same failure as a scraper that scrapes nothing, arriving through a different door. So we added verify_entrypoint_runs.py, which proves the module named in CMD actually calls its entrypoint, and wired it into the pre-commit gate.

Writing it taught its own lesson. The first version matched strings and demanded a __main__ guard, and it reported 19 healthy Actors as broken — including our most-used listing, which has a perfectly valid top-level asyncio.run(main()) and needs no guard at all. A module run as __main__ doesn't need a guard; it needs a call. The rewrite uses an AST walk instead of pattern matching, and now passes 180 of 180 Actors while still catching the broken one.

We would rather tell you that than let you assume a green test suite means the container runs.

What you get per row

Field	Example
facility_name	ACME MANUFACTURING CO
source_id	RI0001234
registry_id	EPA Facility Registry Service (FRS) ID
street_address / city / state / county	full postal breakdown
epa_region	01
epa_system / statute	CWA / NPDES
permit_status_desc	Effective, Administratively Continued
scraped_at	ISO 8601 UTC timestamp

Set a 2-letter state (required — ECHO rejects an unfiltered query), optionally narrow with a facilityName substring, and export as JSON, CSV or Excel.

Useful for ESG and site-diligence work, permit-status monitoring across a jurisdiction, sizing regulated dischargers by county, and lead-gen for environmental consultancies.

👉 EPA ECHO CWA Facilities Scraper on Apify