Version AI Outputs Before People Rely on Them

A generated analysis can change after somebody has already acted on it. If your application overwrites the output in place, a later reviewer cannot reliably reconstruct which recommendation the person saw.

OpenAI’s October 6 mathematics release offers a useful engineering prompt. The announcement describes manuscripts, Lean formalizations of many proofs, reasoning summaries and information about attempted problems. The repository says corrections become new versions while earlier released versions remain accessible.

It also says results are at different verification stages, not all have accompanying formalizations, and some unformalized results could have issues. This article does not validate those mathematical results. The design lesson I am taking from the release is narrower: preserve the specific artifact that a review or decision refers to.

Separate a result from its revisions

For an AI-assisted report, keep a stable result ID and a separate revision ID. The result ID identifies the continuing task; the revision ID identifies one frozen output and its associated evidence.

A minimal record could contain the artifact digest, creation time, prior revision, reason for the change, source references and review status. Record model and prompt versions when available and relevant to reproducing the process. Do not claim that these fields make a nondeterministic generation reproducible by themselves.

Keep the output file and its receipt together. If somebody changes an assumption or conclusion after handover, create a new revision. The old revision remains the object of any earlier review. “Latest” can be a convenience pointer, but it should not be the identifier saved with a consequential decision.

Bind a receipt to exact bytes

Here is a small Python example using only the standard library. Run it in a temporary working directory. It creates a new directory for each revision and refuses to reuse an existing revision name.

from hashlib import sha256
from pathlib import Path
import json


def save_revision(root, revision, text, *, previous=None, reason):
    if not revision.isalnum():
        raise ValueError("Use an alphanumeric revision ID")
    if previous is not None:
        read_revision(root, previous)
    folder = root / revision
    folder.mkdir()  # Existing revision names are never overwritten.
    payload = text.encode("utf-8")
    (folder / "output.txt").write_bytes(payload)
    receipt = {
        "revision": revision,
        "previous": previous,
        "reason": reason,
        "artifact_sha256": sha256(payload).hexdigest(),
        "review_status": "not_reviewed",
    }
    (folder / "receipt.json").write_text(
        json.dumps(receipt, indent=2), encoding="utf-8"
    )
    return receipt


def read_revision(root, revision):
    if not revision.isalnum():
        raise ValueError("Use an alphanumeric revision ID")
    folder = root / revision
    receipt = json.loads((folder / "receipt.json").read_text("utf-8"))
    payload = (folder / "output.txt").read_bytes()
    if receipt["revision"] != revision:
        raise ValueError("Revision identity mismatch")
    if sha256(payload).hexdigest() != receipt["artifact_sha256"]:
        raise ValueError("Artifact differs from receipt")
    return payload.decode("utf-8"), receipt


root = Path("revisions")
root.mkdir(exist_ok=True)
save_revision(root, "v1", "Supplier covers region A.",
              reason="Initial unreviewed analysis")
save_revision(root, "v2", "Supplier excludes region A.",
              previous="v1", reason="Corrected coverage assumption")
assert read_revision(root, "v1")[0] == "Supplier covers region A."
assert read_revision(root, "v2")[1]["previous"] == "v1"

The example demonstrates an application convention and a byte-integrity check. It is not an immutable storage system. A person with filesystem write access can change both files, and a digest does not authenticate the author or establish that a claim is true.

There is also no transaction across the two file writes. An interrupted save can leave an incomplete directory; the reader fails instead of treating it as a valid revision. A service implementation needs a transaction or an explicit completed state, appropriate permissions and a recovery process. Keep those requirements separate from what this short example proves.

Store review decisions separately

A reviewer should reference the revision ID and artifact digest they actually inspected. Keep the reviewer identity, time, intended use and scope of checks in a separate review record.

For example, “arithmetic checked against supplied spreadsheet” is narrower than “recommendation approved.” If the spreadsheet itself has not been verified, that uncertainty should remain visible. A changed output should get a new review decision; copying an earlier approval onto the new revision would erase the very boundary the receipt is meant to preserve.

Do the same for inputs that can change. A source URL alone may later show different content. Where you are permitted to retain it, capture the relevant source version or snapshot and its timestamp. For private material, use restricted references and your existing retention rules. Do not turn a review feature into an uncontrolled copy of confidential documents.

Track the consequences of a correction

A revision chain says what came before. It does not tell you who acted on it.

If a report informs a consequential action, store that action’s reference to the specific revision. When an assumption changes, those references help identify decisions requiring another look. The correction event should describe the changed assumption and its possible consequence, rather than silently regenerating every downstream artifact.

Avoid auto-invalidating every decision merely because wording changed. Define which changes require review for your use case: a different source, an altered calculation, a changed eligibility assumption or a reversed recommendation may matter more than an editorial fix.

Test the handover, not just the happy path

Useful checks include retaining access to the earlier output after a new revision, refusing accidental reuse of a revision name and detecting an output that no longer matches its receipt. Test an interrupted write too: incomplete data should be visible as an error, not accepted as a completed record.

If you also evaluate generation quality, keep task IDs distinct from attempt IDs. OpenAI’s repository reports attempted problems and groups outputs into manuscripts and families; those different units do not support a simple success-rate division. In your own pilot, attach retries to the original task and define what counts as a usable result before comparing runs.

Start with one report or generated analysis that already crosses a team boundary. Save the revision people use, attach the scope of review, and make the next correction explicit. You can improve the storage architecture later; first make it possible to answer which version somebody trusted.