Quick answer
A demo input that returns rows today can return zero rows next month, and nothing in your test suite will notice. We caught it in the USGS Earthquake Data Scraper the day it was built: the prefilled example query hard-coded endtime to the build date. It worked perfectly. It would have quietly become a broken first-run demo for every new customer a week later.
The fix took one line. Finding it took knowing that "the tests pass" and "the product works" are different claims.
The prefill is a product surface, not a form ๐
On Apify, the input schema's prefill is what a customer sees the first time they open your Actor. They press Start. Whatever happens next is your entire first impression.
So a prefill has a job that no validator checks: it must return data. Our own verifier, verify_input_prefill.py, confirms the prefilled payload validates against the input model. That is a genuinely useful check and it is completely blind to this failure โ a payload can be perfectly well-formed and match nothing at all.
We have paid for that gap before. An earlier Actor over the openFDA device endpoint shipped with every filter prefilled, so the demo run ANDed applicant + device name + product code + advisory committee + device class + a date range into a query that matched nothing. Cloud QA reported SUCCEEDED but produced 0 rows. Green pipeline, dead demo.
The rule we landed on: prefill ONE realistic filter plus a range, and check the exact prefilled payload against the live API before shipping. Not a similar payload. That one.
The time-boxed variant is sneakier ๐ฐ๏ธ
A wrong-filter prefill is broken immediately, so you find it immediately. A date-boxed prefill is worse, because it works on the day you write it and decays silently afterwards.
endtime = 2026-09-02 # build day: 623 events.
# four weeks later: whatever happened to fall in a stale window.
The obvious fix is to compute the window at runtime. The better fix, if the API allows it, is to stop bounding the future at all โ so we asked the live service what it does with an endtime that hasn't happened yet:
GET /fdsnws/event/1/query?format=geojson&starttime=...&endtime=2028-01-01
-> HTTP 200, real data, capped at now
USGS simply clamps a future endtime to the present. So the prefill now ends in 2028 and the demo cannot go stale. That answer was not in the docs โ it came from asking the server.
Rows existing is not proof of delivery ๐งพ
The other habit worth stealing: after the cloud smoke test came back SUCCEEDED, 3 rows, we did not stop there.
We have been burned by exactly that signal. A different Actor once returned three perfectly-shaped rows from a blocked page โ a nav link, a company's generic listing page, and the search URL echoed back as a result. Status SUCCEEDED. Row count above zero. Every dashboard green. The rows were furniture.
So the check is no longer "did rows arrive" but "could only a genuine row look like this". For this one we took an event ID out of our own dataset and asked USGS about it directly:
| Field | Our row | USGS |
|---|---|---|
| magnitude | 5.6 | 5.6 |
| place | 159 km WSW of Abepura, Indonesia | 159 km WSW of Abepura, Indonesia |
| coordinates | 139.3456, -3.2239, 69.052 km | 139.3456, -3.2239, 69.052 |
| significance | 482 | 482 |
| MMI / alert | 4.376 / green | 4.376 / green |
Same event, field for field, from a source that has never heard of our Actor. That is delivery evidence. "The run succeeded" is not.
What the Actor actually does ๐
USGS's FDSN event service is the authoritative global seismic catalogue โ the same feed behind the USGS earthquake map. It answers in nested GeoJSON: coordinates as a positional [lon, lat, depth] array, timestamps as epoch milliseconds, and a properties bag where most fields are legitimately null.
This Actor flattens all of that into one typed row per event:
- magnitude and magnitude type, place, ISO-8601 event time
- longitude, latitude, depth in km โ unpacked from the coordinate array
- significance score, PAGER alert level, tsunami flag
- community felt-reports (CDI/MMI), review status, canonical event URL
Filters cover the full useful surface: time window, magnitude and depth bounds, and both bounding-box and radius search โ most rival scrapers stop at a magnitude-and-date box.
Optional fields come back as null when USGS itself doesn't report them, never as a missing key and never as a crashed row. That matters more than it sounds: a schema that rejects a null kills the whole write on the first sparse record, and a fully-populated test sample hides it perfectly.
Who this is for
- Insurance and catastrophe risk โ event feeds by region and magnitude band.
- Infrastructure and logistics โ did anything shake near these coordinates.
- Research and journalism โ reproducible historical windows without writing a GeoJSON unpacker.
The honest limitations ๐ง
- USGS caps a single response at 20,000 events; very wide windows need splitting.
- The
/countpre-flight and per-event detail endpoints are out of scope for v0.1 โ every field the detail endpoint would add to a list view is already on the row. - Long-tail FDSN parameters (
eventtype,catalog,contributor) aren't exposed yet.
Pricing
$0.05 per run plus $0.002 per event โ about $2.05 per 1,000 events. A run that finds nothing costs the start fee and nothing else.
โ USGS Earthquake Data Scraper on Apify
Built by Devil Scrapes. We handle the pagination, the retries, the null-shaped edge cases and the parameters the docs got wrong, so you get a flat table instead of a weekend.
Top comments (0)