Data Contracts: Ending the "Someone Changed a Field Upstream" Outage

Every data team has lived this outage. A dashboard breaks — or worse, doesn't break, and just starts showing subtly wrong numbers. Days later you trace it to an upstream team who renamed a column, changed a unit, or started sending null where they used to send 0. Nobody did anything wrong: they owned that system and had no idea a pricing model three teams away depended on it. That's an undeclared dependency, and the fix is to declare it — with a data contract.

What a data contract actually is

A data contract is an explicit, enforced agreement between a data producer and its consumers about a dataset's schema, types, semantics, and guarantees. Not a wiki page — an executable spec that's checked in CI and at runtime, so a breaking change is caught at the source instead of discovered in production downstream.

# contract: policy_events.v1
owner: policy-admin-team
schema:
  policy_id:      { type: string, required: true, unique: true }
  premium_cents:  { type: integer, required: true, min: 0 }   # cents, not dollars
  currency:       { type: string, enum: [GBP, USD, EUR] }
  effective_date: { type: date, required: true }
guarantees:
  freshness: "< 24h"
  null_rate: { premium_cents: 0.0 }
sla: "breaking changes require a major version + 2-week notice"

Why this matters more in insurance than most places

The dependency chains are long and consequential. A field in the policy-admin system feeds a rating model, which feeds a regulatory return. A silent change at the top can surface as a mispriced book or a compliance miss — discovered weeks later, by a regulator or an actuary, not by an alert. The cost of an undeclared dependency scales with how far the blast radius travels, and in insurance it travels far.

What enforcement looks like

1. Producer-side CI check. The contract lives with the producing pipeline. A schema-incompatible change fails the build. You cannot merge a breaking change without bumping the version.
2. Runtime validation at the boundary. Data is validated against the contract as it lands. Violations (a new null, an out-of-range value, a renamed field) are rejected or quarantined, not silently propagated.
3. Versioning with notice. Breaking changes = a new major version and a migration window, so consumers move deliberately instead of being surprised.
4. Ownership. Every contract has a named owner. "Who do I talk to about this field?" always has an answer.

The mindset shift

Data contracts turn "we'll find out when it breaks" into "it can't break without someone explicitly agreeing to break it." It's unglamorous — schemas, CI checks, version numbers — and it's exactly the discipline that separates data platforms that scale from ones that firefight every sprint.

Full write-up with the insurance framing:

Data Contracts: Ending "Someone Changed a Field Upstream" →

From IntelliBooks' series on the data foundation under insurance AI.

Are you enforcing contracts in CI, at runtime, or both? What broke badly enough to make you adopt them?