DEV Community

Revin
Revin

Posted on

I replayed 48 commits of one OpenAPI file: 11 of them removed a required field after clients were live

Before I wrote software for a living I signed drawings. Steel structure, Autódromo de Interlagos. A beam went up only after a drawing that had a revision index in the corner: rev. 0, rev. 1, rev. 2, each one with a date and a name next to it. Wanted the beam two meters to the left after the concrete was already down? That was rev. 3. And rev. 3 had a price and a new date before anyone touched a bolt.

Then I moved into software and watched the scope of one service change twice in the same week with no revision index anywhere.

There is a question on Software Engineering Stack Exchange asking why the IT industry cannot deliver large, faultless projects quickly as in other industries. 543 points, a little under 124,000 views, 31 answers. I read most of them. They blame requirements, estimation, physics, hiring, management. What I did not find was the boring mechanical difference I actually lived through: in steel, the drawing is the versioned artifact and the thing you build is derived from it. In software, the build is versioned and the drawing lives in a thread.

So last month I went looking for the drawing in a repo I have read access to. In an HTTP service, the closest thing to a drawing is the contract.

The service, and the count that lied to me

Node and TypeScript, order service, roughly 14k lines, 7 months old, three consumers: an iOS app, one partner integration over HTTP, and an internal dashboard. One api/openapi.yaml.

My first attempt was the obvious one. Count how much the surface moved by counting commits on the handlers.

$ git log --oneline -- src/http/handlers | wc -l
317
Enter fullscreen mode Exit fullscreen mode

That number is worthless and I threw it away. It counts a rename, a Prettier run, a logging tweak and a field deletion as the same event. It measures typing, not revisions.

Second attempt, much narrower. Only the contract file, and only the kind of change that can break somebody who is already calling you: a field that was required in a response schema and stopped being there. That definition misses plenty, which I will get to, but it has one property I wanted. It produces no false positives. If a required field is gone, somebody's parser is going to find out.

Replaying the contract commit by commit

$ git rev-list --reverse HEAD -- api/openapi.yaml | wc -l
48
Enter fullscreen mode Exit fullscreen mode

Forty-eight revisions of the drawing in 7 months. Now walk them and diff the required sets.

# contract_revisions.py  (pip install pyyaml)
import subprocess, sys, yaml

PATH = "api/openapi.yaml"
RANGE = sys.argv[1:] or ["HEAD"]

def revs():
    out = subprocess.run(["git", "rev-list", "--reverse", *RANGE, "--", PATH],
                         capture_output=True, text=True).stdout
    return out.split()

def required(rev):
    blob = subprocess.run(["git", "show", f"{rev}:{PATH}"],
                          capture_output=True, text=True).stdout
    spec = yaml.safe_load(blob) or {}
    schemas = spec.get("components", {}).get("schemas", {})
    return {n: set(s.get("required", [])) for n, s in schemas.items()}

history = revs()
prev, breaks = required(history[0]), 0
for rev in history[1:]:
    cur = required(rev)
    for name, fields in prev.items():
        gone = fields - cur.get(name, set())
        if gone:
            breaks += 1
            print(f"{rev[:8]} {name} lost required: {', '.join(sorted(gone))}")
    prev = cur
print(f"{breaks} breaking revisions across {len(history)} commits")
Enter fullscreen mode Exit fullscreen mode
$ python3 contract_revisions.py
3f9a21c7 Order lost required: deliveryWindow
7c04bb1e Order lost required: customerTaxId
a1d7e8b2 Shipment lost required: carrierCode, trackingCode
b6c13f09 Order lost required: deliveryWindow
...
11 breaking revisions across 48 commits
Enter fullscreen mode Exit fullscreen mode

Eleven. Sorted by what they actually were:

  • Four renames. customerTaxId became taxId, shipped under a commit message that said chore: naming.
  • Five fields that moved into a nested object, which reads like a tidy-up in the diff and reads like a missing field to a client.
  • Two product decisions in the same week pointing opposite ways. One meeting dropped deliveryWindow from the response. The meeting four days later asked for it back, split into deliveryWindowStart and deliveryWindowEnd.

Two of the eleven had anything written down outside the diff. The partner integration learned about one of them from a support ticket: their sync threw on the missing field and 41 jobs failed over a weekend before anyone connected the two events.

None of the eleven was negligence. Every one of them was somebody doing a sensible thing to the code without the drawing having a revision index.

The revision index, in about twenty lines

What Interlagos had was not discipline, it was friction placed at the right moment. Nobody was forbidden from moving a beam. Moving it cost a revision, a signature and a new date, and that cost alone made a good half of the requests evaporate before they reached me.

So I put the index on the file.

# api/openapi.yaml
info:
  title: orders
  version: 1.7.0
  x-contract-revision: 12
Enter fullscreen mode Exit fullscreen mode

And a check in CI that fails when a required field leaves without the number going up in the same pull request.

#!/usr/bin/env bash
set -euo pipefail
base=$(git merge-base origin/main HEAD)

if python3 contract_revisions.py "$base~1..HEAD" | grep -q 'lost required'; then
  if ! git diff "$base" HEAD -- api/openapi.yaml | grep -q '^+.*x-contract-revision'; then
    echo "breaking contract change without a revision bump" >&2
    exit 1
  fi
fi
Enter fullscreen mode Exit fullscreen mode

The ~1 is there because the script compares each commit to the one before it, so the first commit in the range needs a predecessor to be compared against. Took me two red builds to notice.

The rule that goes with the bump is the part that matters, and it is not code: the pull request body carries the date and the name of whoever on the consuming side said yes. Same as the drawing.

What this does not catch

  • Semantic drift, which is the expensive one. A status field that keeps its name and starts carrying a fourth value breaks consumers with zero movement in the required set. Narrowing an enum, widening a type and changing a unit all slip through for the same reason.
  • Anything about behaviour. Ordering, pagination defaults, response time, retry semantics. The drawing describes shape, not load.

I also have no idea whether this is worth doing in a repo with 30 consumers. If you are at that size you probably run a schema registry with compatibility rules already, and this is a poor man's version of it. It is written for the service that has three consumers and a YAML file nobody treats as an artifact.

And it does not freeze scope. I do not want it to. Interlagos never froze scope either. It just made the change arrive with a price attached, at the moment someone asked for it, instead of three weeks later as 41 failed syncs.

Five weeks in

Three contract changes since the check went in. All three bumped the revision. One of them got pulled back inside the pull request, because the person who owns the partner integration asked for two weeks of notice, and two weeks of waiting was visibly cheaper than another weekend of nobody knowing why the sync died.

That is the whole answer I would have posted on that Stack Exchange thread, if it were not already thirteen years old: software can absolutely be delivered like a building, for the part of it that is a drawing. The problem is that we version the thing we type and not the thing we promised.

So, genuinely: how do you record a contract change so the consuming side finds out before its parser does? Schema registry with enforced compatibility, a deprecation header with a sunset date, a changelog somebody reads, a consumer-driven contract test suite? I want the one that survived contact with a partner who does not read your release notes.

· · ·

Originally published on the Revin blog.

Top comments (0)