DEV Community

Yusuke Shiki
Yusuke Shiki

Posted on

A Real Deployment Exposed Two Contract Defects Before I Froze the Contract

A green CI run does not prove that the artifact built from reviewed source matches what production later exposes.

The build job may have produced one directory while the deploy job selected another. A file may have been added after review. The live manifest and representative file bytes may match the digests sealed at build time—or they may not.

I built release-evidence and its v0 contract to connect those claims. It records source and build provenance, seals artifact digests, appends deployment events, and checks a bounded part of the live production target.

But I did not freeze the first reviewed version of the contract.

Deployment contracts are unusually good at lying while every synthetic fixture stays green. A
schema can be internally consistent and still describe a topology that does not exist. Two digest
fields can both look reasonable until an adapter has to decide which one to compute first.

Keeping the contract in draft until a real adapter exercised it exposed two defects.

The goal was not another “build succeeded” record

The first target was a static dashboard deployed through GitHub Pages.

The evidence chain needed to support a narrower statement:

reviewed source
  -> one built artifact
  -> that artifact deployed to production
  -> live manifest read back
     + representative file bytes spot-checked
Enter fullscreen mode Exit fullscreen mode

The draft contract used a sealed release bundle and an append-only event ledger. The bundle held
the source commit and tree, toolchain evidence, and artifact digests. The ledger appended events
such as prepared, deployed, and verified; current state was derived by folding those events,
not by editing a mutable status row.

There were schemas, semantic checks, and accepted and rejected fixtures. The draft passed review.
The pull request still said that v0 would remain unfrozen until a real adapter had exercised it.

Defect 1: the contract expected a preview that did not exist

The first transition graph expected a preview environment before production.

This scheduled GitHub Pages deployment has no real preview tier. The workflow builds the static
dashboard and deploys it directly to the production Pages site.

I could have labeled a CI workspace “preview” and made the ledger satisfy the graph. That would
have been worse than an explicit gap: an evidence system would be changing the description of
reality to pass its own validator.

The contract changed instead.

A direct prepared -> deployed|production transition is now valid only when it carries
preview_skipped: true and a closed reason code. The same flag is rejected on the normal preview
path. The absence of a preview is an explicit property of the target, not a silently missing event.

GitHub Pages was not defective. The contract's assumption that every target had a preview was.

Defect 2: the manifest and bundle could not be finalized

The static site exposes a release-manifest.json containing a digest for each deployed path. A
read-back can fetch the manifest and compare a live file with the value it records.

The first design also made the release bundle carry the byte digest of that manifest, while the
manifest carried the digest of the bundle:

bundle
  contains sha256(release-manifest.json bytes)

release-manifest.json
  contains sha256(bundle)
Enter fullscreen mode Exit fullscreen mode

The bundle required the finished manifest. The manifest required the finished bundle. Neither
value could be computed first.

The cycle looks obvious when written as two arrows. It was much less obvious while reviewing the
two record shapes separately. It surfaced when the real adapter had to generate files in an actual
order.

The deployed manifest now contains only its schema version and content map:

{
  "schema_version": "release-evidence/v0",
  "content": {
    "index.html": "sha256:..."
  }
}
Enter fullscreen mode Exit fullscreen mode

It does not repeat the enclosing bundle digest. During production read-back, the adapter
canonicalizes the fetched content, computes its SHA-256 digest, compares that value with the
static-site artifact in the sealed bundle, and spot-checks live file bytes against the manifest.

Removing the mutual reference made the evidence flow one-directional.

I fixed both defects, then exercised production

On August 22, 2026, the corrected adapter ran in the real workflow.

Workflow run 32572501427
built the dashboard, deployed it to GitHub Pages, fetched the live manifest, compared its content
map, spot-checked a representative live file, and persisted the evidence ledger.

The ledger contains this sequence:

prepared
  -> deployed | production | preview_skipped=true
  -> verified | production
Enter fullscreen mode Exit fullscreen mode

Recomputing the digest from the live manifest's canonicalized content produced the same value as
the artifact sealed in the bundle:

sha256:35e4b7ee5a17ca64f0851e54d83385f2c478b4d04e3210a2e08639fecd3b6d0c
Enter fullscreen mode Exit fullscreen mode

Only after that receipt existed did the contract replace its synthetic dashboard fixture with the
real deployed bundle and freeze release-evidence/v0.

The order mattered:

review the draft
  -> let a real adapter break it
  -> repair the contract
  -> verify a production read-back
  -> freeze
Enter fullscreen mode Exit fullscreen mode

Freezing first would have forced the adapter either to invent a preview or immediately violate the
contract I had just declared stable.

The second success verified the repeat-run link

After the first production exercise succeeded, I found another issue in the adapter. A later
release should derive previous_release_id from the last verified production event. The first
implementation still needed that repeat-run chain logic.

I keep this separate from the two contract defects. It did not change the transition model or the
digest definition. It was an adapter defect revealed by asking what the second release must do.

After that fix reached main, scheduled
run 32615972524
created a second production release from commit 6f0b5c8, which includes
PR #7.

The new bundle records the link explicitly:

release_id:
  agent-metrics-dashboard@32615972524-1

previous_release_id:
  agent-metrics-dashboard@32572501427-1
Enter fullscreen mode Exit fullscreen mode

The first release was the preceding verified production release; its sealed bundle digest was
sha256:63392d1a635e68f670f07dcb717ad7e1437f8282be82a1dd8c0ab2c6a664c9e1.
The second release recorded prepared at 2026-08-23T03:41:48.031Z, deployed | production
at 2026-08-23T03:42:20.467Z, and verified | production at
2026-08-23T03:42:20.659Z.

Its ledger persisted the same three-event sequence under one sealed bundle digest:

prepared
  -> deployed | production | preview_skipped=true
  -> verified | production
Enter fullscreen mode Exit fullscreen mode

The sealed bundle digest is
sha256:7dabd0a36be4308cb1c70387d1097dcb37c7fde3be3aee991f8287371bc661df.
Recomputing the static-site digest from the live manifest content produced
sha256:ac69214db9909b1f1d97bd1f806214034172e0dac780e1c1470434f158b70d49,
and the live index.html bytes matched the manifest entry
sha256:ce99db76efd55bace6b50bd9ba88caa76e1dbef841c76e87dcb07b7d88380c9e.

Auditing the persisted collection against the frozen v0 contract reported two bundles, six
events, and no problems. The repeat-run link is now observed in production. The order still
matters: the first success did not prove that future linkage until a second release exercised it.

Two later runs preserved the verified chain

Two subsequent scheduled runs created production releases from the same PR #7 merge commit.

Run Release ID Previous verified release Sealed bundle digest
32687508604 agent-metrics-dashboard@32687508604-1 agent-metrics-dashboard@32615972524-1 sha256:fd2ee5b9ef6d531b1c377257a60ced576a949964be186fd9b1c6337278b2a8a4
32806015116 agent-metrics-dashboard@32806015116-1 agent-metrics-dashboard@32687508604-1 sha256:94d9f3ee15e2238eb84dcae6dbe26bb4f69afd434ce8a1491be636e24d40b8e0

The first follow-up chain recorded prepared at 2026-08-24T03:45:20.910Z, deployed |
production
at 2026-08-24T03:45:54.808Z, and verified | production at
2026-08-24T03:45:55.135Z. The second recorded the same sequence at
2026-08-25T03:40:22.482Z, 2026-08-25T03:40:53.785Z, and
2026-08-25T03:40:53.929Z. Within each chain, all three events carry the same sealed bundle
digest shown in the table. The Actions logs also record read-back verified against the
corresponding static-site artifact digest.

These runs add evidence that the previous_release_id chain and read-back continued to operate.
They do not prove that every future deployment will succeed, that every deployment is safe, or
that an external user adopted the tool.

Matching digests do not prove everything

The integrity level of release-evidence/v0 is digest_only.

There is no signature, SLSA, or SBOM claim. GitHub Actions credentials, pinned actions, the tool
checkout, and the production host remain inside the trust boundary.

A successful read-back shows that the live manifest's content map matched the static-site
artifact digest sealed in the bundle, and that a representative live file matched its manifest
entry at verification time.

It does not show that every production file was fetched and compared byte for byte, or that the
host cannot change afterward.

content integrity
!= semantic correctness

digest match
!= complete supply-chain security

successful read-back
!= future correctness
Enter fullscreen mode Exit fullscreen mode

The v0 ledger assumes one writer. The first adapter is specific to a GitHub Pages/static-site
deployment. It is not a universal release database or an automatic promotion system.

Stating those limits does not weaken the receipt. It prevents a content-digest proof from silently
turning into a broader supply-chain claim.

Make reality answer before the contract becomes permanent

Schema review and synthetic fixtures were necessary. They remain valuable regression guards after
the fixes.

They were not enough to justify freezing a deployment contract.

Real targets have topology that diagrams omit. Artifacts have generation order that isolated
schemas hide. Production read-back crosses boundaries that a local build never touches.

The point was not to delay stability. It was to keep the contract breakable until the last cheap
moment to discover that it described something impossible.

The real adapter found a preview environment that did not exist and a digest cycle that could not
be computed. Both were contract problems, not cosmetic implementation details.

Before freezing a deployment-evidence schema, I now want one answer from the production path:
can an honest adapter actually satisfy this contract without inventing state?

For release-evidence/v0, asking that question before the freeze is what found the two defects.


AI-assistance disclosure

I used ChatGPT and Codex to help structure, edit, and translate this article. I reviewed the release IDs, digests, timestamps, implementation details, and claim boundaries against the recorded evidence before publication.

Top comments (0)