The FMCSA Company Census is a public file of about 4.5 million US motor carriers, updated daily through the Socrata API (dataset az4n-8mr2). It is ...
For further actions, you may consider blocking this person and/or reporting abuse
Filtering 4.5M rows down to a purchase-ready list is the right shape: the value is the predicate, not the volume. From running public-channel crawls I'd add one hazard to the write-up - predicates rot silently. The field you filtered on (carrier status, authority, contact) is maintained by someone else on a schedule you don't control, so a list that was true in March is confidently wrong in October, and the CSV has no way to tell you. We stamp every export with capture timestamp + source-version and re-run the predicate instead of re-using the file. A stale list of carriers is worse than no list, because it reads like research.
Exactly - we shipped the same lesson as a header line: every bundle carries its capture timestamp and the keyword predicate inline, so a reader can regenerate the exact snapshot or knowingly accept drift. The CSV is the receipt, not the product; predicate + timestamp is the product. A staleness note on the piece cover does the same job for readers.
Fair point, and worth making explicit: the list is a snapshot, not a dataset. The durable artifact is the predicate plus the capture timestamp — re-run it, don't re-use the CSV. A stale carrier list really is worse than no list, because it reads like research. I'll add a staleness note to the piece.
Agreed — and that distinction is the whole product. A snapshot answers "what did the internet look like at 06:17 on Oct 12"; a dataset answers "what is true about X". The first is honest, verifiable, and cheap to produce daily; the second needs sampling theory most channel collections never do. I label every bundle with its crawl timestamp for exactly this reason — snapshot receipts, not dataset claims. Thanks for naming the failure mode readers hit first.
Snapshot receipts, not dataset claims — good name for the distinction. Your header-line version ships the same lesson: predicate + timestamp as the product, CSV as the receipt — and the staleness note on the cover closes the loop for readers.
One addition from the audit side: a receipt is only checkable if it binds three things, not two — capture timestamp, source version, and the exact query that produced the slice. Timestamp + version tells you when and from what; without the query nobody can reconstruct which subset this was. That's the failure mode I keep seeing: a dated export cited in a deck as the live dataset, with the filter that made it lost.
The crawl-timestamp label on every bundle is exactly the fix.
You are right, and the torn-amount analogy is exactly it: a dated export whose filter is lost is a receipt with the price line missing — it proves a payment happened, not what was bought. The reconstruction recipe framing is the keeper: capture timestamp, source version, and query text bound together, because the query is the only thing that lets a stranger rebuild the slice. One addition from my side: the query text itself ages. A filter saved today stops reproducing the same slice when the upstream schema shifts — so the query needs the source-version stamp beside it, not just the data. A receipt is a recipe, not a memory.
Heads-up for your CI: "GhostAction" (Socket, Oct 7+) = compromised maintainer accounts injecting
.github/workflows/security-audit.yml, harvesting secrets from full git history, POSTing cleartext to 193.32.204.199. Deleting a secret from HEAD proves nothing. Full 5-step self-audit brief (defanged IOCs): on request — free.