A customer reported that a profile screen showed a blank subscription tier. The SDK compiled, unit tests passed, CI was green on the branch that shipped. At runtime the code read profile.plan and got undefined, because the server had stopped returning that field two releases earlier.
We had committed the generated TypeScript types straight into the repository. Nobody regenerated them, so the committed snapshot still described an API the server no longer served. Typechecking proved the code matched our own frozen copy of the contract; it said nothing about what the live endpoint returned.
Why a checked-in snapshot freezes
When generated types are committed, the generator stops running as part of the build. They become a source file a developer edits by hand during review and an artifact that drifts silently. A spec change upstream — a field renamed, marked deprecated, or dropped on a v2 — never reaches the repo, because nothing re-reads the spec.
The assumption behind committing was trustworthiness: the types are stable, the diff stays reviewable, and contributors do not need the generator toolchain installed. That tradeoff buys convenience at the cost of freshness. The file is now a photograph of the contract taken on merge day, not the contract itself.
Why typecheck cannot defend against a dropped field
The type proves compilation, not the response. TypeScript checks that code reads plan against a type that declares plan?: Plan. It never calls the server. A field declared optional in our stale snapshot and a field absent from the live response both typecheck identically.
No runtime assertion reads the payload. The code assumes the field exists because the type says it may. Without a guard or a schema validation on the response, a missing value flows straight through to a UI that renders an empty label.
Review trusts the generated artifact. A reviewer sees well-formed types, sees that the SDK compiles, and moves on. The generated file looks authoritative, so nobody asks where it came from or whether it matches today's spec.
The drift accumulates quietly. A single dropped field produces no error on the client until a user reaches the exact screen that reads it. It is not a build failure, not a test failure, not a lint warning — just a value that arrives as undefined in production traffic.
The cheap habits that catch it before merge
Regenerate from the live spec as a build step, not a hand commit. Keep the generator in CI so committed types are a byproduct of a fetched spec, not a manually maintained source. When the spec moves upstream, the regenerate step fails or shows a diff a reviewer must look at.
Pin the spec version the SDK targets. If the generator consumes a local or fetched spec, record which commit or tagged release it corresponds to, so a mismatch between the checked-in types and the live API has a visible reference point instead of an invisible gap.
Replay recorded requests against the spec. Capture real request/response pairs from staging or a sandbox and run them against the current spec; a field the server no longer sends shows up as a missing required property or a schema mismatch instead of surfacing in production.
Validate optional fields at the boundary. Even with generated types, assert that a critical optional value is present where the feature depends on it, or normalize a missing value to a documented default. The type allowing a field does not mean the wire always carries it.
When the contract is the source of truth
The cleanest position is to stop treating generated types as the contract and keep the spec as the source of truth. Replay recorded traffic against the local spec before merge so a dropped operation or field appears as a red diff on screen, in front of the reviewer, rather than as an empty value after deploy. That local-first loop is what we built into Powerduck: fetch the spec, run real recorded calls against it, and let drift show in the diff instead of waiting for a customer report.
What I check now
- Are these types generated in CI from a fetched spec, or committed by hand and never regenerated?
- Does any check replay real responses against the current spec, or does typecheck stand in for the contract?
- Does the code guard an optional critical field, or does it assume the wire always carries what the type allows?
Green typecheck means the code matches a frozen photograph. It does not mean it matches the live API.
Top comments (1)
Typecheck passing only proves the code matches the frozen copy, not the live server. Moving generation into CI so committed types come from a fetched spec is the habit that actually prevents this drift.