Most Architecture Decision Records are written at the moment a team knows the least.
The decision has just been made. The production rollout has not finished. Failure modes are still hypothetical. Yet the document is often written in permanent language:
We use this design because it is secure, observable, and reliable.
Six months later, the code has moved, the test was renamed, one platform behaves differently, and nobody knows which sentence is still evidence-backed.
We did not need more ADRs. We needed ADRs that could admit when they were wrong.
This is Part 3 of Auditability Under Pressure.
The missing sections
The ADR community and AWS guidance both emphasize preserving context, the decision, alternatives, and consequences. Those are essential. In our production work, two more sections became equally important:
- Verification
- Invalidation conditions
Our standard record ended up with nine sections:
Status
Context and threat
Decision
Consequences
Rejected alternatives
Residual risks
Verification
Invalidation conditions
Sources
“Residual risks” prevented a partial fix from being remembered as complete.
“Verification” forced us to separate code existence, automated tests, runbooks, and dated production observation.
“Invalidation conditions” turned the decision from a commandment into a conditional statement.
Verification is a ladder, not a checkbox
We used four practical evidence levels:
| Level | What it proves |
|---|---|
| Code reference | An implementation exists at a specific path |
| Named automated test | A defined behaviour was exercised in a controlled environment |
| Runbook | An operator has a reproducible verification procedure |
| Dated production observation | The real deployment path produced the expected evidence |
A test can be strong without being production proof. A production observation can be narrow without proving every branch.
For example, the time-bounded emergency bypass had extensive tests. We observed that the bypass was closed in production because unsigned requests were rejected. We did not open it merely to strengthen the ADR's status table.
The record therefore said “not production-triggered — intentionally,” and explained why.
That sentence is more useful than a green checkmark.
The sentence that changed reviews
The most productive heading was:
This decision becomes invalid when…
For the signed-tunnel decision, invalidation conditions included a change to the gateway trust boundary or removal of the tunnel path itself.
For the local Windows evidence sink, central collection with durable offline buffering would change the local sink's role.
For the emergency bypass, a fully compatible fleet plus an independent recovery channel could remove its primary justification.
This question changed design reviews from “do we still like this architecture?” to “have the conditions that made it rational changed?”
The second question produces evidence. The first often produces taste.
We made citations executable
The ADRs cited named Go tests, TypeScript test titles, source paths, and links to other ADRs. Those references were useful until they became stale.
We added a verifier that checked:
- every cited Go test function existed;
- every cited TypeScript test title existed;
- every cited source or document path existed;
- internal ADR links resolved.
Then the verifier produced its own assumption failure.
The first version used git grep. It worked in the repository. The deployed source tree excluded .git, so the same command returned no matches and made every citation look stale.
The fix was not “ignore the error in production.” The verifier moved to filesystem traversal and introduced separate exit states:
- citation failure;
- environment failure;
- success.
A tool that verifies evidence must distinguish “the evidence is wrong” from “I could not inspect the evidence.” Otherwise it manufactures the same false certainty it was built to prevent.
What we believed
We believed documentation drift was mainly a writing problem.
It was actually a dependency problem. A sentence depended on a test name, a path, a runtime environment, a deployment shape, and a production observation. The prose stayed still while those dependencies changed.
We also believed ADRs described decisions.
The useful ones described decision state: accepted, partially verified, production-proven, superseded, or awaiting a specific event.
What changed our mind
Three production findings forced the change:
- an ADR promised that rejected requests were visible, while Windows preserved no evidence;
- an update decision claimed downgrade protection, while the compared “versions” could be unordered commit hashes;
- the citation verifier failed because its own runtime assumption was undocumented.
The ADRs were not merely behind the code. Some were confidently describing controls that only existed under unstated conditions.
Once those conditions became explicit, the documents started finding engineering work.
A small claim ledger beside the ADRs
For public communication, we added one more internal artefact: a claim ledger.
It records the strongest permitted wording for each material claim:
| Evidence | Safe wording |
|---|---|
| hash chain + anchor | tamper-evident |
| signed work order | authorized order is signed |
| unit tests only | verified in tests |
| one production agent | observed on one production agent |
| rollout incomplete | control exists where the new binary is deployed |
This prevents a technically correct mechanism from growing into a broader marketing claim during editing.
It also gives AI-assisted writing a hard boundary. A language model can improve structure, but it cannot upgrade “partially verified” to “production-proven.”
This decision becomes invalid when…
The living-ADR format can be simplified when:
- decisions and evidence live in a queryable system with equivalent history;
- test, deployment, and production evidence are linked automatically;
- invalidation conditions trigger ownership and review;
- the team can still recover rejected alternatives and residual risks.
Until then, nine explicit Markdown sections are cheaper than reconstructing a security decision during an incident.
The document found a platform gap
The biggest proof that the format worked was uncomfortable: the Verification section made us mark four decisions as not directly observable on Windows.
That red status was not documentation debt. It was a production engineering finding.
Part 4 follows the finding to the machine: The Windows Evidence Gap — when enforcement works but proof disappears.
References
- Architecture Decision Records
- AWS Prescriptive Guidance — ADR process
- NIST SP 800-218 — Secure Software Development Framework
Editorial note: This article is based on a real production engineering record. AI tools assisted with structure and language review; the author verified the technical claims, source links, and final wording.
Top comments (0)