Freeze User-Visible Strings Before an OSS Patch Hits Review

A mid-size command-line library accepted a one-line error-message change on an otherwise quiet Thursday afternoon. The contributor had reproduced a confusing parser failure, corrected the branch, and then rewrote the printed message for newcomers. Local unit tests stayed green because they asserted exception classes rather than the exact printed text. Two days later a packaging script that grepped the old phrase failed inside a downstream nightly job.

Maintainers meet this pattern often enough that it no longer reads as a harmless wording nit. Scripts, runbooks, support macros, and tutorial snippets treat error text, log lines, and CLI help as shipped contracts. A coding model asked to improve clarity will rewrite those strings unless the contribution process freezes them first. The rest of this article records a ledger workflow that keeps that freeze explicit.

Why string drift survives green tests

Most OSS suites protect control flow rather than the exact bytes that leave the process. Assertions check status codes, exception types, and structured fields, while the human-readable remainder stays unpinned. Formatters and AI review passes then treat that remainder as commentary instead of interface. That is how a correct parser fix still breaks a downstream nightly job.

Treat three classes of text as user-visible until a maintainer writes otherwise in review notes.

• Error messages and panic text that operators may grep during an incident
• Log lines that on-call documents quote inside runbooks and playbooks
• CLI help, flag names, and usage examples rendered by --help output

Documentation headings can move without a migration window in most projects. Previously shipped messages and wire-format names should not move without an explicit note.

Reproduce, inventory, patch, then review

The sequence below assumes a clean local checkout of the upstream default branch. It does not require one language, though the sample helper is Python. Each stage produces a file that later stages can check. The model pass, if used at all, only compares those files.

Reproduce on a throwaway worktree

Create an isolated worktree so the inventory never mixes with unrelated dirty files. Fetch the default branch before copying so the ledger compares against current upstream text. Run the original failing command and keep stderr outside the repository tree.

git fetch origin
git worktree add /tmp/oss-string-freeze origin/main
cd /tmp/oss-string-freeze
python -m pytest tests/test_parser.py -k bad_timestamp -vv
python -m app.cli parse broken.json 2> /tmp/before.err || true

The worktree is disposable after the patch lands. The stderr capture is evidence, not product code, and should stay uncommitted. If the project uses fixtures rather than a CLI, dump the raised message with the same redirect pattern.

Build a string ledger from the intended diff

After a candidate patch exists on a branch, extract added and removed quoted strings instead of reading the diff as prose. Exclude vendored trees and snapshot files so generated noise does not flood the ledger. The helper below is a starting point, not a parser for every language.

git diff origin/main...HEAD -- ':!vendor' ':!*.snap' ':!po/*.po' > /tmp/patch.diff
python3 extract_strings.py origin/main > STRING_LEDGER.md

#!/usr/bin/env python3
"""Extract added and removed quoted strings from git diff."""
import re
import subprocess
import sys

STRING_RE = re.compile(r'["\']([^"\']{8,})["\']')

def diff_lines(base):
    cmd = ["git", "diff", "-U0", f"{base}...HEAD", "--", ":!vendor", ":!*.snap"]
    out = subprocess.check_output(cmd, text=True)
    return out.splitlines()

def main():
    base = sys.argv[1] if len(sys.argv) > 1 else "origin/main"
    added, removed = [], []
    for line in diff_lines(base):
        if line.startswith("+++") or line.startswith("---"):
            continue
        if line.startswith("+"):
            added.extend(STRING_RE.findall(line[1:]))
        elif line.startswith("-"):
            removed.extend(STRING_RE.findall(line[1:]))
    print("# String ledger")
    print("\n## Removed")
    for item in dict.fromkeys(removed):
        print(f"- {item}")
    print("\n## Added")
    for item in dict.fromkeys(added):
        print(f"- {item}")

if __name__ == "__main__":
    main()

Classify every removed line as freeze, migrate, or accident in the same file. Search the default branch for each removed string so the class is based on callers, not taste.

git grep -n -F "invalid timestamp in input" origin/main -- ':!tests' || true

# String ledger

| String | Direction | Class | Notes |
| --- | --- | --- | --- |
| invalid timestamp in input | removed | migrate | keep old text for one minor release |
| cannot parse timestamp: expected RFC3339 | added | new | document in CHANGELOG |
| debug dump written to /tmp | added | accident | drop before review |

A freeze class means the patch must restore the old bytes before review. A migrate class requires a changelog bullet and, when possible, a dual-message window. An accident class is text the author or an assistant introduced without a product reason.

Patch against the ledger, not against taste

Apply the functional fix first and only then decide whether any user-visible string must change. If a message is false because the behavior was false, keep the old wording when downstream parsers depend on it. Ship both sentences for one release when the project already has a deprecation policy.

# Proposal: dual-message window, not a silent rewrite.
# Leave unexecuted until the project agrees on deprecation policy.
raise ParserError(
    "invalid timestamp in input; expected RFC3339"
)

Label dual-message examples as proposals when the repository has no published deprecation policy. Do not let a model pick the friendlier sentence as a default. Restore any accident lines by deleting them from the working tree before the next test run.

Test the ledger, then the original behavior

Add a cheap presence check so frozen strings still appear in source. This is not a localization framework and does not parse format strings. It only stops an accidental rewrite from riding along with a logic fix.

# tests/test_string_ledger.py
from pathlib import Path

FROZEN = [
    "invalid timestamp in input",
]

def test_frozen_strings_still_present():
    paths = Path("src").rglob("*.py")
    joined = "\n".join(p.read_text(encoding="utf-8") for p in paths)
    missing = [item for item in FROZEN if item not in joined]
    assert missing == [], missing

Run the ledger test before the original reproducer so a wording regression fails first. Capture stderr again and compare it with the file from the worktree setup. If the wording diff includes a freeze class string, restore those bytes before any review pass.

python -m pytest tests/test_string_ledger.py tests/test_parser.py -vv
python -m app.cli parse broken.json 2> /tmp/after.err || true
diff -u /tmp/before.err /tmp/after.err || true

Compare the ledger with the diff on a free model

This is the only stage where a coding model should enter the loop. The task is comparison, not rewrite, and the prompt should forbid wording suggestions. Paste git diff origin/main...HEAD together with STRING_LEDGER.md and ask for omissions only.

Disclosure: This article was prepared as part of MonkeyCode's product outreach. Contributors who already build this inventory locally can send the ledger and the diff through MonkeyCode's free model access, and can use the free server option when that comparison should run off the laptop. The model exists to catch table omissions, not to invent clearer error text.

You are comparing STRING_LEDGER.md with the unified diff.
List: (1) added or removed quoted strings in the diff that lack a ledger row,
(2) ledger rows whose text does not appear in the diff,
(3) any freeze row whose old bytes are still missing from the tree.
Do not suggest wording improvements. Do not rewrite the patch.

If the model reports extra strings, update the ledger or revert the extra edits. If it reports none, the patch can go to a human maintainer, who still owns tone and deprecation policy.

Decision table for string classes

Class	When to use it	Required artifact	Ready for review
freeze	Downstream scripts or docs quote the exact bytes	ledger row plus presence test	old string still in the tree
migrate	Behavior change makes the old sentence false	ledger row plus changelog bullet	old and new text both documented
new	Entirely new failure mode or help flag	ledger row only	no removed twin hiding nearby
accident	Debug dumps, speculative help, assistant asides	none; delete the text	gone from git diff

Use the table as a gate, not as a style guide. A patch that still contains an accident row is not ready, even when tests are green. A migrate row without a changelog bullet is also not ready, because downstream authors will not learn the new bytes.

Limitations

Quoted-string extraction misses template literals, concatenated fragments, and gettext catalogs. Dynamic messages built from several pieces will under-count, so the ledger is a net rather than a proof. Generated code, vendored files, and snapshot fixtures can flood the report unless those paths are excluded from git diff.

Projects that localize every user-facing sentence should run this workflow against message catalogs rather than source literals. Security advisories sometimes must change an error to avoid leaking internals, and those patches should skip the freeze class. A hosted model will not know the deprecation policy unless that policy is pasted beside the ledger.

The free model pass does not replace the local worktree, the presence test, or maintainer judgment. No duration, quota, hardware size, or model name is implied by using a free server for the comparison. Teams that need a guarantee against string drift still have to keep the presence test in CI.

Who should skip this approach

Drive-by typo fixes in comments do not need a ledger or a model pass. Embargoed security patches should not paste diffs into any hosted model, even for a string comparison. Repositories without a shipped CLI, log contract, or scriptable error text will spend more time classifying strings than they will save in review.

Authors who cannot run the original reproducer locally should not certify that user-visible output is stable. The extra file exists to keep taste out of the first review pass, not to freeze language forever. Maintainers can still improve wording after the functional fix lands, and they should do that in a dedicated docs change.

Contributors who already keep a worktree and a ledger can run the comparison on MonkeyCode's free models when they want a second pass on the table.