Turn a Patch Session Into a PR Body: A JSONL Contribution Journal

A patch is only as reviewable as its evidence trail. Most pull request bodies are written from memory, hours after the last command. The distance between what ran and what the description claims is where reviewer time disappears.

This article defines a small journal format for open-source patches. Each reproduction, patch inspection, and test run becomes one JSON line. A standard-library Python script renders the PR body from those lines.

The artifact below is complete and runnable. It fits any patch session where a model, free or paid, sits somewhere in the loop.

The bookkeeping failure

Three problems repeat in model-assisted contribution work.

1. The failing command is never captured.
2. A model answer gets promoted to a test result.
3. The PR body describes intent, not observed exits.

None of these are model problems. They are record-keeping problems. One append-only file solves all three.

Two record kinds

A run record stores a command, exit code, duration, and the last output lines. An ask record stores a question, its source, the answer, and a verified flag.

The verified flag carries the weight. It stays false until a human checks the answer against code or output. An unverified answer is a to-do item, not evidence.

Save the next script as journal.py.

#!/usr/bin/env python3
"""journal.py - append contribution events to JSONL and render a PR body."""
import argparse
import json
import pathlib
import shlex
import subprocess
import sys
import time

JOURNAL = pathlib.Path("contribution.jsonl")


def append(record):
    with JOURNAL.open("a", encoding="utf-8") as fh:
        fh.write(json.dumps(record, ensure_ascii=False) + "\n")


def run(step, argv, note=""):
    start = time.monotonic()
    proc = subprocess.run(argv, capture_output=True, text=True)
    append({
        "kind": "run",
        "step": step,
        "cmd": " ".join(shlex.quote(a) for a in argv),
        "exit": proc.returncode,
        "seconds": round(time.monotonic() - start, 2),
        "note": note,
        "tail": (proc.stdout + proc.stderr).strip().splitlines()[-6:],
    })
    return proc.returncode


def ask(question, source, answered="", verified=False):
    append({
        "kind": "ask",
        "step": "ask",
        "cmd": "",
        "exit": 0,
        "seconds": 0,
        "note": question,
        "source": source,
        "answered": answered,
        "verified": verified,
        "tail": [],
    })


def report():
    text = JOURNAL.read_text(encoding="utf-8") if JOURNAL.exists() else ""
    rows = [json.loads(line) for line in text.splitlines() if line.strip()]
    runs = [r for r in rows if r["kind"] == "run"]
    asks = [r for r in rows if r["kind"] == "ask"]
    latest = {a["note"]: a for a in asks}
    asks = list(latest.values())

    print("## What was run\n")
    print("| step | command | exit | seconds |")
    print("| --- | --- | --- | --- |")
    for r in runs:
        print(f"| {r['step']} | `{r['cmd']}` | {r['exit']} | {r['seconds']} |")

    failed = [r for r in runs if r["exit"] != 0]
    print("\n## Non-zero exits\n")
    print("\n".join(f"- `{r['step']}`: exit {r['exit']}" for r in failed) or "- none recorded")

    print("\n## Questions asked of a model\n")
    print("| question | source | answered | verified |")
    print("| --- | --- | --- | --- |")
    for a in asks:
        mark = "yes" if a["verified"] else "no"
        print(f"| {a['note']} | {a['source']} | {a['answered'] or '-'} | {mark} |")

    open_items = [a for a in asks if not a["verified"]]
    print("\n## Unverified claims\n")
    print("\n".join(f"- {a['note']}" for a in open_items) or "- none")


def main():
    parser = argparse.ArgumentParser()
    subs = parser.add_subparsers(dest="command", required=True)
    run_p = subs.add_parser("run")
    run_p.add_argument("step", choices=["reproduce", "patch", "test"])
    run_p.add_argument("--note", default="")
    run_p.add_argument("argv", nargs=argparse.REMAINDER)
    ask_p = subs.add_parser("ask")
    ask_p.add_argument("question")
    ask_p.add_argument("--source", default="model")
    ask_p.add_argument("--answered", default="")
    ask_p.add_argument("--verified", action="store_true")
    subs.add_parser("report")
    ns = parser.parse_args()

    if ns.command == "report":
        report()
    elif ns.command == "ask":
        ask(ns.question, ns.source, ns.answered, ns.verified)
    else:
        argv = ns.argv[1:] if ns.argv[:1] == ["--"] else ns.argv
        sys.exit(run(ns.step, argv, ns.note))


if __name__ == "__main__":
    main()

The script has no third-party dependencies. It writes to contribution.jsonl in the current directory.

Step 1: Record the reproduction

Run the failing case through the journal. Do not paste it into notes afterward.

python journal.py run reproduce --note "deadline case" \
  -- python -m pytest tests/test_retry.py::test_deadline -x || true

The trailing || true matters. The script exits with the test code, which stops shells running under set -e. A non-zero exit is recorded either way.

Step 2: Record the patch shape

Cheap evidence still counts as evidence. Capture the diff summary before the test rerun.

python journal.py run patch --note "one file, retry.go" -- git diff --stat
python journal.py run patch --note "full diff" -- git diff -- src/retry.go

Two records beat one. The summary shows blast radius, the full diff shows intent.

Step 3: Record the same test again

Use the identical command from step 1. Same node id, same flags, same environment.

python journal.py run test --note "after patch" \
  -- python -m pytest tests/test_retry.py::test_deadline

The journal now holds a before and an after with exit codes. That pair is the strongest sentence in any PR body.

Step 4: Record model questions

Free model access lowers the cost of asking. It does not lower the cost of being wrong. Record the question, then record the verification separately.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

python journal.py ask "Does the retry loop reset the deadline?" \
  --source "free model review" \
  --answered "no, it reuses the first deadline"

The answer sits in the report as unverified. Read src/retry.go, find the assignment, then re-record the same question with --verified. The renderer keeps the latest entry per question, so the flag flips cleanly.

python journal.py ask "Does the retry loop reset the deadline?" \
  --source "free model review" \
  --answered "no, it reuses the first deadline, see retry.go:88" --verified

Teams already testing free model access, or the free server option, can run these commands unchanged. The journal format stays provider-agnostic, which matters when the tool of the month changes.

Step 5: Render the body

python journal.py report > pr_body.md

The output contains four sections. Reviewers get the command table, the non-zero exits, the questions with sources, and the still-unverified claims.

Any entry in the last section is a blocker. Ship the PR only when that list is empty or explained.

When a model question earns its place

Situation	Ask a model?	Check first
Assertion message names the mismatch	No	Read the assertion
Behavior spans three or more files	Yes, ask for call sites	grep -rn the symbol
Suspected race or timing bug	Rarely	Build a flaky reproduction loop
Error string is unparseable	Yes, ask what raises it	Search the string in the repo
"Should this be a config flag?"	Weak yes	Read existing config conventions
Roadmap or maintainer intent	No	Read CONTRIBUTING.md and open issues

The left column is a filter. Most patch questions fail it. That is the point.

Limits worth stating

The journal proves what ran. It does not prove the patch is correct. A green exit code can still hide a wrong fix.

The tail field captures raw output. Tokens, internal hostnames, and customer paths can land in the file. Scrub it before it leaves the machine.

Free model answers vary between calls. That variability is why the verified flag exists. The record must not inherit the model's confidence.

The file grows with every session. One journal per branch keeps it readable. Do not commit it to the repo by default.

Shared capacity on any free tier means long test suites may need local runs. Cheap review and cheap execution are separate bets.

Who should skip this

Repositories that require signed audit logs should use CI artifacts instead. A local JSONL file is not an audit trail.

One-line typo fixes do not need a journal. The bookkeeping costs more than the patch.

Teams looking for a model to approve merges should stop here. This workflow keeps the human as the verification step, permanently.

Anyone unable to store the journal outside version control should not start. Output tails leak context.

Pre-publish checklist

1. contribution.jsonl contains a reproduce run with a non-zero exit.
2. The test run repeats the same command after the patch.
3. Every ask entry has a source and a verification state.
4. pr_body.md shows no unexplained unverified claims.
5. The journal file is excluded via .gitignore.

The last check is the one people forget. Add contribution.jsonl to .gitignore before the first run, not after the first leak.