Building a Weather-Triggered Tire Advisory Pipeline: Ingesting Environment Canada Data to Drive Maintenance Alerts for Calgary Drivers
I run the technical side of a small Calgary tire business, and for the past two winters I have been iterating on a pipeline that watches Environment and Climate Change Canada (ECCC) open data and decides, automatically, when a weather situation is relevant enough to tires that a driver should hear about it. Not "it might snow" — everyone in Calgary owns a window. The pipeline exists for the narrower, more actionable events: the first sustained cold stretch that makes winter rubber matter, a chinook swing that silently pulls four or five PSI out of every tire in the city, freezing rain, summer hail cells, and the deep-cold snaps where TPMS dashboards light up across town for reasons that are physics, not punctures.
This is a design writeup, not a product pitch. I am going to cover the ECCC data surfaces worth ingesting, the normalized schema I landed on after two bad first drafts, the advisory rule catalogue with actual thresholds, the hysteresis and deduplication machinery that stops the system from flapping, how I backtested rule precision and recall against past seasons, and the delivery layer with rate limiting and audit logging. All numeric thresholds shown are design examples from my configuration — tune them against your own data before trusting them.
Why Calgary Weather Is a Legitimately Hard Input
Calgary sits at roughly 1,045 metres elevation on the dry side of the Rockies, and its weather has three properties that make naive alerting embarrassing.
First, chinooks. Warm, dry downslope winds can raise the air temperature 15 to 25°C within hours. Calgary averages somewhere in the range of 20 to 30 chinook days per winter depending on how you count them. A rule that fires on "temperature crossed 0°C" will fire constantly from November through March and train every recipient to ignore it.
Second, variance around the seasonal transition. The historical median for the first hard freeze lands in early-to-mid October, but the spread is wide: I have replayed years where the first sustained sub-7°C stretch arrived in late September and years where it held off until November. A calendar-based reminder ("it is October 15, think about winter rubber") is wrong by weeks in either direction more often than it is right. The 7°C figure itself is the industry-standard crossover point where winter tread compounds begin outperforming all-season compounds — that is the physical basis for the winter tire timing guidance we publish, and it is the number the pipeline hunts for in forecast data.
Third, hail. Calgary sits inside what meteorologists informally label hailstorm alley. Convective cells between June and September produce short-notice severe weather warnings, and the tire-relevant advice ("get the vehicle under cover; inspect sidewalls afterward") has a useful lead time measured in tens of minutes. That pushes part of the architecture toward push-based ingestion rather than polling.
So the input signal is genuinely volatile, the false-positive cost is recipient fatigue, and the false-negative cost is a driver on all-seasons in the first -15°C week. Those constraints shaped everything below.
The Three ECCC Data Surfaces and When to Use Each
ECCC exposes its open data through several distinct mechanisms, and the pipeline uses three of them for different jobs.
GeoMet OGC APIs for structured queries
The GeoMet platform (api.weather.gc.ca) serves OGC API - Features collections over plain HTTPS with JSON responses. The collections I lean on:
-
climate-daily— quality-controlled historical daily observations (max/min/mean temperature, precipitation) per station. This is the backbone of backtesting. -
climate-stations— station metadata, used once to pin the Calgary International Airport station (climate identifier 3031094 in my configuration) as the canonical observation source. - CAP alert collections — active weather alerts as GeoJSON features with polygon geometry, which lets you do a point-in-polygon test against a Calgary coordinate instead of string-matching region names.
A representative request for historical dailies:
GET https://api.weather.gc.ca/collections/climate-daily/items?
CLIMATE_IDENTIFIER=3031094
&datetime=2023-09-01/2024-04-30
&sortby=LOCAL_DATE
&limit=1000
&f=json
Paginate with offset, respect the numberMatched field, and cache aggressively — historical dailies are immutable once quality control settles, so I store them locally and never re-fetch a closed month.
Citypage XML for the human-aligned forecast
The citypage weather product is the same forecast Calgarians see on official channels, published as XML per site under dd.weather.gc.ca/citypage_weather/. Calgary is site s0000047 in Alberta's directory. The payload includes current conditions, seven-day forecast periods with textual summaries and numeric highs/lows, and issue timestamps.
Why use a legacy-feeling XML product when model data exists? Because advisory text ultimately reaches humans, and the citypage forecast is the forecast a recipient can verify against every other weather surface in their life. If my pipeline says "sustained cold arriving Thursday" and their phone's weather app disagrees, trust erodes. Anchoring rules to the public forecast keeps the system's claims checkable.
MSC Datamart AMQP for push latency
The Datamart also publishes an AMQP 0-9-1 message feed (the xpublic exchange on dd.weather.gc.ca, anonymous credentials, topic-based routing) announcing every new file the moment it lands. The Sarracenia project wraps this, but a plain pika consumer works fine for a narrow subscription. I subscribe to two topic patterns: the Alberta citypage updates and the CAP alert directory.
import pika
AMQP_URL = "amqps://anonymous:anonymous@dd.weather.gc.ca:5671"
TOPICS = [
"v02.post.citypage_weather.xml.AB.#",
"v02.post.alerts.cap.#",
]
def start_consumer(on_announcement):
params = pika.URLParameters(AMQP_URL)
conn = pika.BlockingConnection(params)
ch = conn.channel()
q = ch.queue_declare(queue="", exclusive=True, auto_delete=True)
for topic in TOPICS:
ch.queue_bind(exchange="xpublic",
queue=q.method.queue,
routing_key=topic)
ch.basic_consume(queue=q.method.queue,
on_message_callback=on_announcement,
auto_ack=False)
ch.start_consuming()
Each announcement message carries the path of the new file; the consumer fetches it over HTTPS, hands it to the parser, and acknowledges. The division of labour that emerged: AMQP for anything latency-sensitive (alerts, forecast refreshes), scheduled HTTPS polling as a belt-and-suspenders fallback every 30 minutes, and GeoMet queries for historical and station data. If the AMQP connection drops overnight — it happens — the poller means the worst case is a half-hour-stale forecast, not a silent outage.
Normalizing Everything Into One Typed Schema
The first draft of this system passed raw parsed XML dictionaries around, and every rule reimplemented unit handling and missing-value checks. The second draft fixed that with a small set of typed models that every ingestion path must produce. Rules only ever see normalized records.
from dataclasses import dataclass
from datetime import datetime, date
from enum import Enum
from typing import Optional
class Source(Enum):
CITYPAGE = "citypage"
GEOMET_DAILY = "geomet_daily"
CAP_ALERT = "cap_alert"
@dataclass(frozen=True)
class ForecastPeriod:
site_id: str # e.g. "s0000047"
issued_at: datetime # UTC, from the feed's own timestamp
period_start: date # local Calgary date the period covers
is_overnight: bool
temp_high_c: Optional[float]
temp_low_c: Optional[float]
pop_percent: Optional[int] # probability of precipitation
condition_codes: frozenset[str] # normalized: {"freezing_rain", ...}
text_summary: str
source: Source
@dataclass(frozen=True)
class Observation:
station_id: str
observed_at: datetime
temp_c: Optional[float]
wind_kmh: Optional[float]
station_pressure_kpa: Optional[float]
source: Source
@dataclass(frozen=True)
class HazardAlert:
cap_id: str # CAP message identifier, natural dedup key
event_kind: str # normalized: "freezing_rain", "hail", ...
severity: str # CAP severity: minor/moderate/severe/extreme
onset: Optional[datetime]
expires: Optional[datetime]
covers_calgary: bool # result of point-in-polygon test
raw_headline: str
Two normalization decisions earned their keep:
Condition codes are a controlled vocabulary. Citypage text summaries are free-form English ("Periods of freezing drizzle changing to snow near noon"). A mapping layer reduces each summary to a set of canonical codes — freezing_rain, freezing_drizzle, snow, rain, thunderstorm, hail_risk — via ordered keyword rules with an explicit unmapped bucket that gets logged and reviewed weekly. Rules never regex the prose directly. When ECCC rewords a summary format (it has happened), the blast radius is one mapping table, not every rule.
Timestamps are stored in UTC, bucketed in local time. Forecast periods and advisory windows are meaningful to humans in America/Edmonton local time, including the daylight-saving transitions. Everything persists as UTC; every rule that reasons about "days" converts through the timezone explicitly. The one early bug I shipped in this area — a winter-window evaluation that shifted a day at the November DST boundary — is why the conversion now lives in exactly one function with a regression test pinned to the 2023 transition dates.
The persistence layer is PostgreSQL. Trimmed DDL for the two hot tables:
CREATE TABLE forecast_snapshot (
id BIGSERIAL PRIMARY KEY,
site_id TEXT NOT NULL,
issued_at TIMESTAMPTZ NOT NULL,
period_start DATE NOT NULL,
is_overnight BOOLEAN NOT NULL,
temp_high_c NUMERIC(4,1),
temp_low_c NUMERIC(4,1),
pop_percent SMALLINT,
condition_codes TEXT[] NOT NULL DEFAULT '{}',
text_summary TEXT NOT NULL,
ingested_at TIMESTAMPTZ NOT NULL DEFAULT now(),
UNIQUE (site_id, issued_at, period_start, is_overnight)
);
CREATE TABLE advisory_event (
id BIGSERIAL PRIMARY KEY,
rule_id TEXT NOT NULL,
dedup_key TEXT NOT NULL UNIQUE,
fired_at TIMESTAMPTZ NOT NULL DEFAULT now(),
window_start DATE,
window_end DATE,
evidence JSONB NOT NULL,
state TEXT NOT NULL DEFAULT 'pending'
CHECK (state IN ('pending','delivered','suppressed','expired'))
);
The UNIQUE constraint on forecast_snapshot makes re-ingestion of the same issuance a no-op (ON CONFLICT DO NOTHING), which matters because the AMQP consumer and the fallback poller will regularly both fetch the same file. The evidence JSONB column stores the exact snapshot rows a rule saw when it fired — indispensable later when a recipient asks why they received something, and equally useful when they ask why they did not.
The Advisory Rule Catalogue
Each rule is a pure function from (normalized data, rule state) to (optional advisory, new rule state). The catalogue, with my current design-example thresholds:
| Rule ID | Trigger summary | Arm threshold (design example) | Disarm / reset | Season gate |
|---|---|---|---|---|
winter_window |
First sustained sub-7°C forecast stretch | ≥5 of next 7 daily highs < 7°C | Re-arms next Aug 1 | Sep 1 – Dec 15 |
chinook_swing |
Large fast temperature rise or reversal | ≥18°C swing within 36 h in forecast or obs | 72 h cooldown | Oct 15 – Apr 15 |
pressure_drop |
Cold arrival implies meaningful PSI loss | Forecast mean drops ≥15°C vs trailing 7-day obs mean | 96 h cooldown | Year-round |
freezing_rain |
Freezing rain / drizzle in forecast or CAP | Condition code present with PoP ≥ 60%, or CAP severity ≥ moderate | Clears when window passes | Year-round |
hail_risk |
Severe thunderstorm watch/warning covering Calgary | CAP event mapped to hail-capable kinds | Clears on CAP expiry | May 1 – Sep 30 |
deep_cold |
Extreme cold with TPMS side-effects | ≥2 consecutive forecast lows ≤ -25°C | Ends when forecast lows > -20°C (hysteresis gap) | Nov 1 – Mar 31 |
Season gates are not cosmetic. They cut the evaluation surface — hail_risk simply cannot fire in January — which both prevents absurd advisories and shrinks the false-positive space the backtests have to measure.
A few of these rules deserve a closer look, because the interesting engineering hides in their edge cases.
The winter-window rule and its off-by-one-season traps
The intent: detect the first stretch of the autumn where the forecast says daytime highs will sit below 7°C for most of a week, because that is when compound physics starts favouring winter tread. Drivers running all-weather rubber (the mountain-snowflake-rated year-round category) get a differently worded advisory than drivers on pure all-seasons, whose compound is the one actually going hard below the crossover.
The naive implementation — "count forecast days with high < 7" — flapped badly in my 2023 replay. A single chinook-warmed Wednesday in an otherwise cold week resets the count, the advisory un-fires, then re-fires Friday. Three fixes:
- Quorum, not streak. Require 5 of 7, not 7 consecutive. Chinooks poke one-day holes in cold stretches constantly; a quorum tolerates them.
-
Fire-once latching. Once
winter_windowfires, it stays fired for the season. There is no natural "un-trigger" for "winter is coming" — the state resets on August 1. - Persistence requirement. The quorum must hold across two consecutive forecast issuances at least 12 hours apart. A single aggressive model run at 4 a.m. that walks back by the afternoon issuance never reaches recipients.
def eval_winter_window(periods: list[ForecastPeriod],
state: RuleState) -> tuple[Advisory | None, RuleState]:
if state.latched or not in_season(date.today(), SEP_1, DEC_15):
return None, state
week = daytime_periods(periods)[:7]
cold = [p for p in week if p.temp_high_c is not None
and p.temp_high_c < 7.0]
if len(cold) < 5:
return None, state.observed(False)
if not state.held_for(hours=12): # persistence across issuances
return None, state.observed(True)
advisory = Advisory(
rule_id="winter_window",
window=(week[0].period_start, week[-1].period_start),
evidence={"cold_days": len(cold),
"highs": [p.temp_high_c for p in week]},
)
return advisory, state.latch()
</code-omitted-helpers-are-boring>
(That closing pseudo-tag is a joke; the helpers are ordinary date filters.) The practical effect of the three fixes in the 2023-24 replay: fire count dropped from 6 to 1, and the single firing landed on October 11 — four days before the first snowfall that season reached the ground. Design-example replay figures, but they sold me on latching.
One more subtlety: the rule reads forecast data, but the season's observed dailies from climate-daily are what the backtest later uses to decide whether the firing was "correct." Keeping those two roles separate in code — forecast triggers, observations judge — kept the evaluation honest.
Chinooks, the ideal gas law, and where the PSI actually goes
The pressure_drop and chinook_swing rules share a physical core, so let me do the worked math once, properly. Tire pressure follows the ideal gas approximation well enough for advisory purposes: at fixed volume and air mass, absolute pressure scales with absolute temperature.
P2_abs = P1_abs × (T2_K / T1_K)
The trap everyone hits: gauges read gauge pressure, and the scaling law applies to absolute pressure. In Calgary the distinction is bigger than at sea level, because at ~1,045 m the ambient pressure is around 88.5 kPa ≈ 12.8 PSI rather than 14.7.
Worked design example. A driver sets tires to 35.0 PSI (gauge) on a mild chinook afternoon at +5°C. A cold front ends the chinook and the overnight temperature reaches -25°C — an entirely ordinary Calgary reversal.
T1 = 5°C = 278.15 K
T2 = -25°C = 248.15 K
P1_abs = 35.0 + 12.8 = 47.8 PSI (absolute)
P2_abs = 47.8 × (248.15/278.15) = 42.6 PSI (absolute)
P2_gauge = 42.6 − 12.8 = 29.8 PSI
Loss ≈ 5.2 PSI over a 30°C drop → ~1 PSI per 5.8°C
That is the familiar rule of thumb — roughly 1 PSI per 5-6°C — derived rather than asserted. Note what it means in practice: a tire that read a healthy 35 in the warm reads under 30 in the cold, without any leak, and 29.8 is below many door-placard specs. The vehicle placard and the sidewall marking are different numbers with different meanings, which trips up enough people that we wrote a plain-language sidewall marking explainer for exactly this confusion; the placard is the inflation target, the sidewall figure is a maximum.
The pressure_drop rule turns the derivation into a trigger: compute the trailing 7-day mean observed temperature from climate-daily plus recent hourly observations, compare against the forward 3-day forecast mean, and if the drop is ≥15°C, emit an advisory whose body includes the computed estimated PSI loss for a placard-typical inflation, clearly labelled as an estimate. Recipients respond much better to "expect roughly 3-4 PSI lower readings by Thursday" than to "cold weather affects tire pressure," which is weather-app wallpaper.
-- Trailing observed mean vs. forward forecast mean (design example)
WITH trailing AS (
SELECT AVG(mean_temp_c) AS obs_mean
FROM climate_daily_local
WHERE station_id = '3031094'
AND local_date >= CURRENT_DATE - INTERVAL '7 days'
),
forward AS (
SELECT AVG((temp_high_c + temp_low_c) / 2.0) AS fc_mean
FROM latest_forecast
WHERE period_start BETWEEN CURRENT_DATE + 1
AND CURRENT_DATE + 3
)
SELECT obs_mean, fc_mean,
(obs_mean - fc_mean) AS drop_c,
(obs_mean - fc_mean) >= 15.0 AS should_consider_firing
FROM trailing, forward;
should_consider_firing, not should_fire — the SQL is a candidate filter; hysteresis state in the application layer makes the final decision.
The chinook-specific angle deserves a sentence more. A chinook rise also matters: air warms, pressure climbs back, and a driver who topped up during the cold snap is now a couple PSI over placard, then under again when the chinook collapses. The chinook_swing advisory therefore talks about re-checking after the swing settles rather than chasing the gauge mid-swing, and it links the general inflation-habits material on pressure-aware tire care rather than urging immediate action. Advising inaction at the right moment is an underrated advisory type.
Freezing rain and hail: the CAP-driven rules
These two rules are structurally simpler because ECCC has already done the meteorology; the pipeline's job is filtering and framing. CAP alert XML arrives via the AMQP subscription within seconds of publication. Processing steps:
- Parse the CAP envelope; extract event type, severity, urgency, onset/expiry, and the polygon.
- Point-in-polygon test against a fixed Calgary coordinate set (I use five points — downtown plus four quadrant centroids — so an alert clipping only the far northwest still registers).
- Map the event to the internal vocabulary. Freezing rain and freezing drizzle map directly. For hail, ECCC issues severe thunderstorm watches and warnings whose text frequently mentions hail size; the mapper marks any severe-thunderstorm CAP as
hail_riskand elevates confidence when the text matches hail-size patterns. - Frame the tire-relevant angle. For freezing rain: stopping distances, the limits of any tread compound on wet ice, and a pointer to our winter traction primer for the compound background. For hail: move the vehicle under cover if feasible, and afterwards inspect sidewalls and tread for bruising — with a note that impact damage is often repairable when caught early if it is in the tread face rather than the sidewall.
The cap_id field gives natural deduplication: CAP updates reference their predecessors, so an updated warning replaces rather than duplicates the advisory. More on the general mechanism below.
Deep cold and the TPMS false-alarm problem
Below about -25°C, two things happen that generate confused drivers. First, the pure physics above: a tire inflated to placard at 0°C is down roughly 4-5 PSI at -30°C, which is enough to cross the typical TPMS trigger threshold of 25% below placard on some vehicles. Second, direct-TPMS sensor batteries and radio electronics behave worse in extreme cold, so intermittent sensor-fault indications (as opposed to low-pressure warnings) become more common.
The deep_cold advisory therefore has an unusual goal: pre-empt panic. It fires when the forecast shows two consecutive lows at or below -25°C, and its content explains that a TPMS lamp on the first brutal morning most likely means "cold air is denser," what the difference between the low-pressure symbol and the sensor-fault symbol is, and that the correct response is to check with a gauge and add air — not to assume a puncture. For drivers who cannot comfortably do that at -30°C, the advisory mentions that our mobile service unit operates through cold snaps across the usual coverage zones. That is the sum total of self-reference this rule carries; the advisory is 90% physics explainer by word count, deliberately.
The -25°C arm / -20°C disarm pair is the hysteresis gap doing visible work. Calgary cold snaps frequently hover near the threshold; a symmetric threshold would toggle the advisory state daily.
Hysteresis: Designing Rules That Refuse to Flap
Flapping is the failure mode that kills advisory systems, so it gets its own machinery rather than per-rule improvisation. Every rule runs inside a shared state wrapper with four orthogonal anti-flap controls:
Split thresholds. Arm and disarm conditions differ by a deliberate gap (fire at -25°C, clear at -20°C; fire at ≥18°C swing, ignore swings under 12°C while active). The gap width is a tunable per rule, chosen from the backtest's flap counts.
Minimum hold time. Once armed, a rule cannot disarm for a configured period (12-72 hours depending on the rule) even if conditions momentarily recede. Forecast issuances wobble run to run; the hold time absorbs the wobble.
Persistence-to-fire. Conditions must hold across N consecutive evaluations before the first firing, as in the winter-window rule. This trades a few hours of latency for a large reduction in retracted advisories, and for every rule except the CAP-driven pair the trade is obviously right. CAP rules skip persistence — ECCC has already applied its own issuance discipline, and freezing-rain lead time is precious.
Cooldown after firing. After emitting, a rule sleeps for a per-rule window regardless of input. A chinook that oscillates for five days produces one advisory, not five.
The wrapper is small enough to show nearly whole:
@dataclass
class RuleState:
armed_since: datetime | None = None
last_fired: datetime | None = None
latched: bool = False
consecutive_hits: int = 0
def observed(self, hit: bool) -> "RuleState":
n = self.consecutive_hits + 1 if hit else 0
armed = self.armed_since if hit else None
if hit and armed is None:
armed = utcnow()
return replace(self, consecutive_hits=n, armed_since=armed)
def held_for(self, hours: int) -> bool:
return (self.armed_since is not None
and utcnow() - self.armed_since >= timedelta(hours=hours))
def in_cooldown(self, hours: int) -> bool:
return (self.last_fired is not None
and utcnow() - self.last_fired < timedelta(hours=hours))
State persists to a rule_state table keyed by rule_id, written transactionally with any advisory_event insert, so a process restart mid-evaluation cannot double-fire. That transactional pairing — state transition and event emission committing together or not at all — is the single most load-bearing line of design in the whole system.
Deduplication and Idempotency Keys
Hysteresis prevents rapid re-firing from the same rule instance; deduplication prevents the same logical event from producing multiple advisories through different paths. The two overlap but are not the same problem. Sources of logical duplicates I have actually observed:
- The AMQP consumer and the fallback poller ingesting the same forecast issuance (handled at the snapshot layer by the unique constraint).
- A CAP warning being updated four times during one freezing-rain event, each update a new file.
- A restart replaying an evaluation against data that already produced an advisory.
- Two rules describing one physical event — a sharp cold arrival can satisfy both
chinook_swingandpressure_drop.
The mechanism is a deterministic dedup key per advisory, unique-constrained in the database:
def dedup_key(rule_id: str, window_start: date, window_end: date,
site_id: str) -> str:
basis = f"{rule_id}|{site_id}|{window_start.isoformat()}|{window_end.isoformat()}"
return hashlib.sha256(basis.encode()).hexdigest()[:24]
The key deliberately excludes the firing timestamp and the evidence payload: two firings about the same window are the same advisory no matter when the pipeline noticed. For CAP rules, window_start/window_end come from the CAP onset/expiry, and the references chain in updated CAP messages collapses updates onto the original key with the evidence column appended rather than a new row inserted.
Cross-rule duplication gets a coarser tool: a suppression matrix. If pressure_drop wants to fire while a chinook_swing advisory for an overlapping window is already in delivered state, it writes its event with state='suppressed' — recorded, auditable, never sent. The matrix is a static config of (suppressor, suppressed, max window overlap) triples, currently three entries. I resisted making it clever; three static rows have covered every real collision in two seasons.
Backtesting Against Past Seasons
Threshold numbers without evaluation are vibes. The backtest harness replays historical data through the exact production rule code — same functions, same state wrapper — with a clock abstraction so utcnow() follows the replay.
Data. For observations, climate-daily gives clean ground truth back decades. Forecasts are the harder half: ECCC does not archive citypage forecasts in an easily queryable way, so I have been archiving every issuance myself since instrumenting the pipeline (which now covers two full winters), and for older seasons I approximate "what the forecast plausibly said" by applying a lag-and-noise model to observed dailies. Findings from the approximated seasons get a lower evidence tier and never justify a threshold change on their own.
Labels. Precision and recall require ground truth for "an advisory should have existed." I hand-labelled event windows per season from observed data plus ECCC's public seasonal summaries: the actual first sustained cold stretch, each ≥15°C 36-hour swing, each freezing-rain event, each hail event with confirmed Calgary impact, each ≤-25°C snap. Labelling five past seasons took an evening per season. It is boring work and it is the only reason the numbers below mean anything.
Metrics. Per rule, per season: true positives (fired within the labelled window or within its defined lead margin), false positives (fired with no matching label), false negatives (label with no firing), plus two operational counts — flaps (fire/clear/fire on one label) and lead time (hours between firing and label onset).
Design-example results from the replay of the two fully-archived winters, after the current thresholds settled:
| Rule | Labels | TP | FP | FN | Precision | Recall | Median lead |
|---|---|---|---|---|---|---|---|
winter_window |
2 | 2 | 0 | 0 | 1.00 | 1.00 | ~5 days |
chinook_swing |
41 | 33 | 6 | 8 | 0.85 | 0.80 | ~14 h |
pressure_drop |
9 | 8 | 3 | 1 | 0.73 | 0.89 | ~30 h |
freezing_rain |
6 | 6 | 2 | 0 | 0.75 | 1.00 | ~4 h |
deep_cold |
4 | 4 | 1 | 0 | 0.80 | 1.00 | ~2 days |
Reading that table honestly: chinook_swing misses one swing in five, almost always the ones the forecast itself underpredicted — an input ceiling, not a rule bug. pressure_drop runs the loosest precision, and I accept that because its advisory is the cheapest to receive (a gauge check costs a minute). The tuning principle that fell out of a season of fiddling: set each rule's threshold according to the cost asymmetry of its errors. Winter-window false negatives are the most expensive outcome in the whole catalogue, so that rule runs the most sensitive quorum the flap budget allows. Hail false positives are nearly free ("move the vehicle under cover" wastes five minutes), so hail_risk forwards every severe-thunderstorm CAP without second-guessing.
One meta-lesson: keep the backtest harness runnable in CI against a frozen fixture season. Twice, an innocuous refactor changed rule behaviour (a timezone slip; a quorum boundary going from < to <=), and the fixture replay caught both before production did.
Delivery: Rate Limiting, Quiet Hours, and the Audit Trail
An advisory in pending state enters the delivery layer, which owns four responsibilities and refuses to learn anything about meteorology.
Per-recipient rate limiting. Token bucket per recipient: capacity 3, refill one per 48 hours (design-example values). When the bucket is empty, lower-priority advisories drop — recorded with state='suppressed' and a suppression_reason — while the two highest-priority kinds (freezing_rain, hail_risk) may overdraw the bucket into a bounded deficit. The priority ordering is static config, reviewed seasonally.
Quiet hours with priority override. Nothing routine sends between 21:00 and 07:30 local; it queues for morning. Imminent-hazard kinds override, since a 22:10 hail cell does not wait for breakfast.
Channel fan-out. Email and SMS behind a common interface. Every message body is generated from the advisory evidence plus a template per (rule, audience) pair, and every message ends with the same two fixed elements: a plain-language "why you received this" line derived from the evidence JSON, and an unsubscribe path. Recipients opted in explicitly through the notification-preferences step of our online reservation flow; the pipeline maintains no other recipient source, and the seasonal-changeover reminder list from our changeover information page feeds the same consent store rather than a parallel one. Consent lives in one table with one meaning.
Audit logging. Every delivery attempt writes an append-only row:
CREATE TABLE delivery_audit (
id BIGSERIAL PRIMARY KEY,
advisory_id BIGINT NOT NULL REFERENCES advisory_event(id),
recipient_hash TEXT NOT NULL, -- HMAC of recipient id, not the id
channel TEXT NOT NULL,
attempted_at TIMESTAMPTZ NOT NULL DEFAULT now(),
outcome TEXT NOT NULL, -- sent / bounced / rate_limited / quiet_hours
provider_ref TEXT,
body_sha256 TEXT NOT NULL
);
Storing body_sha256 instead of the body keeps recipient content out of the audit table while still letting me prove exactly which template revision a given person received — the rendered bodies live in short-retention storage keyed by that hash. The audit table answered a real dispute last winter: a recipient was sure they had never been warned before a freezing-rain morning; the trail showed a bounced outcome from their expired address, which turned a trust complaint into an address update.
The whole delivery layer is intentionally dumb. Every interesting decision — should this exist, is it a duplicate, is it worth a human's attention — happens upstream where it can be backtested. The layer that touches humans only schedules, throttles, renders, and records.
Failure Modes Worth Designing For
A grab-bag of production lessons that did not fit elsewhere, each earned the annoying way:
-
The AMQP connection dies silently. Heartbeats help but do not catch everything. The real safeguard is an end-to-end freshness check: if the newest
forecast_snapshot.issued_atis older than 8 hours, page me — citypage forecasts refresh far more often than that, so staleness means ingestion is broken regardless of which component lied. - ECCC file formats drift. Rarely and usually announced, but the parser treats any unmapped structure as a soft failure that quarantines the file and alerts, rather than a crash or — worse — a silent partial parse.
-
Station data has holes.
climate-dailyrows can arrive late or flagged. Every aggregate over observations carries a minimum-coverage requirement (≥5 of 7 trailing days present) and rules degrade to forecast-only evidence, noted in the evidence JSON, when coverage fails. - Clock discipline. The replay clock abstraction leaked into production once, freezing a rule's idea of "now" at process start. Rule code now receives time as an argument; nothing below the entrypoint imports a clock.
- Threshold drift review. Once a season, the labelled-event set gets extended and every threshold re-justified against the updated table. Thresholds that survive because nobody looked are just superstition with a config file. The general tire-knowledge material we maintain gets the same seasonal review pass, for the same reason.
What I Would Tell You to Build First
If you are standing up something similar — for tires, HVAC, roofing, agriculture, anything where weather drives maintenance urgency — the ordering that would have saved me a month:
- Archive forecasts from day one, even before you have rules. The observation history is free from GeoMet forever; the forecast history only exists if you save it, and every serious evaluation question needs both.
- Build the normalized schema and the labelling workflow before the rules. Rules are an afternoon each once the data is typed and the ground truth exists; they are endless if every rule is also a parser and its own judge.
- Make hysteresis shared infrastructure, not per-rule cleverness. The four controls above cover every flapping pattern I have met.
- Wire the audit trail before the first real recipient. Retrofitting provenance is miserable; recording it from the start is one table.
The pipeline is a few thousand lines of Python, one PostgreSQL database, and a handful of static config tables. Nothing in it is architecturally exotic. Its value comes from the specificity of the domain rules — the 7°C compound crossover, the 1 PSI per 5-6°C derivation at Calgary's altitude, the arm/disarm gaps sized to chinook behaviour — and from the discipline of measuring the rules against seasons that actually happened. Weather data is abundant and free; restraint is the part you have to build yourself.
KMJ Tire is an independent Calgary tire and oil-change business. Everything above describes internal tooling; thresholds and results are design examples from our configuration and replays, not universal constants. The public-facing side of the winter-timing rule lives on our seasonal changeover page.
Top comments (0)