Oaida Adrian

Posted on Aug 13

I Built a Canada Product Recalls & Safety Alerts Scraper That Reads Open Government Data

#webscraping #python #opensource #tutorial

I Built a Canada Product Recalls & Safety Alerts Scraper That Reads Open Government Data

Every few weeks another consumer product gets recalled — a battery that overheats, a fan that catches fire, a vehicle part that fails. For compliance teams, importers and retailers, staying on top of those announcements is a job in itself. Canada publishes every recall on a government website, but the site has no JSON API and no RSS feed. It is plain HTML, served by Drupal.

So I built an Apify actor that reads it directly — no API key, no login, no proxy budget.

The niche, and why build our own

The product-recall scraper niche on Apify is small but telling. The reference implementation that dominates it is dromb/canada-recalls-safety-alerts — a reliable, well-shaped actor that turns Health Canada's recall database into clean JSON. When I looked at the niche, that actor was the reliability benchmark. It worked, it had a sensible schema, and it had traction.

Our house rule is simple: when a niche is worth entering, build our own implementation rather than depend on someone else's actor — same source, same schema, same promises, our code. That way the data pipeline we sell isn't hostage to another developer's maintenance schedule. This post is the story of that build.

The data source

The database lives at recalls-rappels.canada.ca (part of the open.canada.ca family) — Health Canada's official recall, alert and safety advisory database covering consumer products, vehicles, food, drugs and health products. It's Drupal, which means:

No public JSON API. The structured data exists, but you have to parse it out of HTML.
A full-text search index at /en/search?search_api_fulltext=<keyword>.
Detail pages at /en/alert-recall/<slug> with structured fields (Drupal machine-names).
A polite 418 for plain script clients — it wants a browser User-Agent.

The whole design flows from those four facts: one source, one job, HTML scraping, and a browser User-Agent.

First contact: the 418

The very first request to the search page came back 418 I'm a teapot. Not a 403, not a captcha — a teapot. The site is configured to serve plain-scripted clients nothing. The fix is one line of honesty:

HTTP_HEADERS = {
    "User-Agent": (
        "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 "
        "(KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36"
    ),
    "Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8",
    "Accept-Language": "en-CA,en;q=0.9,fr-CA;q=0.8,fr;q=0.7",
}

The search page also 301-redirects to itself, so redirects must be followed. Once you look like a browser and follow redirects, the site is completely open — no login anywhere.

The flow

Run a full-text search against the Drupal search index (/en/search?search_api_fulltext=<keyword>).
Page through the results (page=,0,N pagination).
Visit each recall detail page (/en/alert-recall/<slug>).
Parse the structured Drupal fields into clean records.
Apply the category and date post-filters, cap at maxItems.

Archived recalls are excluded — the search surface excludes them, and the reference actor behaves the same way, so every row is an active recall.

Detail fetches are paced (DETAIL_DELAY_SECONDS = 0.25) with exponential backoff on 4xx/5xx — it's a government site, be polite.

The schema — reverse-engineered from the reference

The hardest part wasn't scraping. It was matching the reference actor's schema exactly, so that anyone using the community actor can switch to ours with zero changes. I pulled a 50-item sample of the reference's dataset and reverse-engineered the semantics field by field.

The final record has 17 fields:

{
  "recallId": 82426,
  "title": "Portable Clip-On Rechargeable Mini Fan recalled due to fire hazard",
  "dateUpdated": "2026-08-12",
  "recallClass": null,
  "category": "Household items",
  "summary": "Fire hazard",
  "affectedProducts": "Portable Clip-On Rechargeable Mini Fan",
  "companies": "Consumer product safety",
  "hazard": null,
  "issue": "Fire hazard",
  "whatToDo": "Consumers should immediately stop using the recalled product and return it to CTG Brands Inc. for a refund...",
  "archived": false,
  "status": "active",
  "language": "en",
  "sourceUrl": "https://recalls-rappels.canada.ca/en/alert-recall/portable-clip-rechargeable-mini-fan-recalled-due-fire-hazard",
  "dataSource": "open.canada.ca",
  "rawSourceType": "recall_record"
}

The interesting semantics, all verified against the reference dataset:

category strips the top-level taxonomy prefix. The site's raw value is Consumer products - Electronics; the reference emits just Electronics. Multi-item categories join differently (Light Truck & Van - SUV stays as-is because the top-level prefix doesn't apply).
summary and issue always mirror each other — the last - segment of the issue-type label.
companies is the issuing organization from publisher metadata (Consumer product safety, CFIA, TC), not the manufacturer.
recallClass is Type I/II/III for health-product recalls, Class 1/2/3 for food recalls, else null.
hazard is always null in practice — the field exists in the reference schema but the live pages don't populate it.
archived is always false and status always active, because the search surface excludes archived recalls.

Getting this right was a weekend of diffing: parse a page, compare with the reference row for the same recall, adjust, repeat across consumer/vehicle/drug/CFIA recall types.

Filters

All input fields are optional — {} returns the most recent active recalls:

Field	Type	Description
`keyword`	string	Full-text search term (`battery`, `croissant`, `vehicle`). Empty = browse latest recalls.
`category`	string	Case-insensitive partial match against the parsed category (`Electronics`, `Household items`, `Vehicles`).
`dateFrom`	string	Only recalls last updated on or after this date (`YYYY-MM-DD`).
`maxItems`	integer	Maximum records to extract (default 50, max 500).

The keyword filter runs server-side through Drupal's search index; category and dateFrom are post-filters on the parsed records. The dateFrom filter is what makes an incremental feed possible: run it weekly with your last-run date, and you get only the new recalls.

The smoke test

The cloud smoke run that proved the actor:

{
  "keyword": "battery",
  "category": "Electronics",
  "dateFrom": "2026-06-01"
}

Result: SUCCEEDED, 4 well-formed Electronics items, all past the date filter, all schema keys exact with native types:

recallId	Title	Date
82213	Super Off-Road power bank recalled due to fire hazard	2026-08-12
82139	Wyze Cam v3 recalled — battery may overheat	2026-08-06
82210	Arizer vaporizer battery recalled	2026-08-08
82150	Steambow AR-6 recalled — arrow rest may break	2026-07-10

Every field parses, every type is native (recallId is an int, archived a bool — not strings), and the reference comparison is clean.

Pitfalls worth naming

Three Apify-specific traps bit during this build:

Input schema requires editor on every property. Apify's input schema validation rejects any field without an editor — add it to all four inputs or the push fails.
actor.json dataset fields are string-only, and enforced on push. Declaring storages.dataset.fields with type: "integer" fails at build with Schema validation failed. The fix is to drop the storages block entirely — native types are then stored, exactly like the reference actor.
run-sync-get-dataset-items?timeout=N sets the RUN's timeout, not the API wait. A 20-page crawl needs N >= 480. And the sync socket read can time out client-side while the run keeps going — poll /v2/actor-runs/{id} and fetch defaultDatasetId when that happens.

Run it

The actor is darknezz/canada-recalls-safety-alerts. From the API:

curl -X POST "https://api.apify.com/v2/acts/darknezz~canada-recalls-safety-alerts/run-sync-get-dataset-items?token=YOUR_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"keyword":"battery","category":"Electronics","dateFrom":"2026-06-01"}'

Or with the Python SDK:

from apify_client import ApifyClient

client = ApifyClient("YOUR_TOKEN")
run = client.actor("darknezz~canada-recalls-safety-alerts").call(
    run_input={"keyword": "battery", "category": "Electronics", "dateFrom": "2026-06-01"},
)
for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(item["recallId"], item["title"])

Use cases

Compliance monitoring — weekly cron on a category (electronics, toys, food) and alert on new recalls via Telegram or webhook.
Importer / retailer screening — check a product line against active recalls before ordering or listing.
Safety trend analysis — pull all recalls in a category over a window and analyse hazard mix, issuing agency and product types.
Data enrichment — join recall IDs against internal product SKUs for a live exposure report.

Limitations, honestly

HTML scraping — the parser targets stable Drupal machine-names; a site redesign would need a rebuild.
English only — records come from the /en/ surface; French (/fr/) records are not covered.
Archived recalls excluded — matching the reference actor; historical records are not returned.
Politeness pacing — very large result sets (500+ items) take proportionally longer by design.

Why this pattern wins

One source, one job. The Canadian government publishes this data for public consumption; the actor just makes it machine-readable at a predictable price. No third-party aggregator, no licensing ambiguity, no API key to rotate — and because the schema matches the established reference exactly, the actor slots into pipelines that already consume recall data.

The teapot was the only wall. Everything else was parsing.