You should review the generation path like a service. Treat prompts, models, and scratch hosts as moving parts. Do not wait for a bad merge to teach the shape.
A design review is not a casual vibe check. You name constraints, trace data, and split failure domains. Then you pick one next change and write a date.
Most teams skip that ritual for generated patches. They paste a stack trace and accept the first diff. That is how a scratch lane becomes unofficial architecture.
Treat this path like a tiny distributed system today. Your laptop holds intent and the canonical git history. The remote model only proposes text you might apply.
A throwaway host may smoke the candidate later. Those three nodes do not share fate or disks. You review them as colleagues, not as one blob.
Constraints come before taste
Start the review with rules you refuse to violate. Write those rules as tests, not hallway slogans. If a constraint cannot fail a build, it is a wish.
Keep the source of truth on your own machine. Generated files stay candidates until local CI agrees. Secrets never ride along inside a prompt dump.
Context windows are not archives for the whole monorepo. You send a contract, a failing test, and a tight diff budget. Leave the rest of the tree out of the pack.
Time is a constraint you should write down too. A review that needs a week will never happen. Keep the whole ritual inside one focused hour.
Commit a constraint file before the next generate step. The listing below is a proposal, not production evidence. Load it in the packer and fail closed on disagreement.
# proposed: .ai/constraints.yaml
canonical_git: local
merge_authority: local-ci
prompt_may_include:
- failing_test
- interface_file
- error_log_redacted
prompt_must_exclude:
- .env
- private_keys
- customer_dumps
scratch_server:
role: ephemeral_smoke
may_hold: generated_artifact
may_not_hold: canonical_state
Run a checker before any generate step starts. The command should look boring on purpose. Boring tools are the ones you actually run.
# proposed: tools/check_constraints.py
from pathlib import Path
import sys, yaml
c = yaml.safe_load(Path(".ai/constraints.yaml").read_text())
blocked = {p.lower() for p in c["prompt_must_exclude"]}
pack = Path(".ai/last-prompt-pack.txt").read_text().splitlines()
hits = [line for line in pack if any(b in line.lower() for b in blocked)]
if c["canonical_git"] != "local" or c["merge_authority"] != "local-ci":
print("canonical git and merge rights must stay local")
sys.exit(1)
if hits:
print("constraint miss:")
print("\n".join(hits))
sys.exit(1)
print("constraints hold for this pack")
python tools/check_constraints.py && echo "pack is reviewable"
Walk a billing handler through those rules as a rehearsal. You own a POST route that must reject unknown fields. A generated helper that silently drops extras violates the file, not your taste.
Trace the data, not the story
A review without a data flow is only a speech. Follow bytes from intent all the way to merge. Name every hop that can rewrite those bytes.
Intent starts as a failing test sitting on disk. The packer reads that test plus one interface file. The model returns a unified diff you have not trusted yet.
Your apply step writes a worktree and never writes main. Optional smoke may hit a free scratch server after that. Local CI remains the only gate that can merge.
Picture a queue with slightly hostile consumers at each hop. Any hop may drop fields or invent a type. Any hop may replay stale context from last Tuesday.
Record a payload hash at every hop during review. Then you can replay a fight without guessing. Memories lie about what the model actually saw.
# proposed: tools/flow_trace.py
import hashlib, json, time
from pathlib import Path
def digest(path):
data = Path(path).read_bytes()
return hashlib.sha256(data).hexdigest()[:12]
event = {
"ts": round(time.time()),
"intent": digest("tests/test_billing.py"),
"pack": digest(".ai/last-prompt-pack.txt"),
"candidate": digest(".ai/candidate.diff"),
"worktree": "review/gen-billing",
"scratch_smoke": "optional",
"merge_gate": "local-ci",
}
Path(".ai").mkdir(exist_ok=True)
with Path(".ai/flow-log.jsonl").open("a") as fh:
fh.write(json.dumps(event) + "\n")
print(json.dumps(event, indent=2))
Keep the log beside the written review notes. When a bad patch lands, you replay hashes. You do not replay a standup story.
git worktree add ../review-gen-billing -b review/gen-billing
python tools/flow_trace.py
git -C ../review-gen-billing apply "$PWD/.ai/candidate.diff"
Do not apply the candidate onto the branch you ship. The worktree is a quarantine ward for text. If the diff is violent, you delete that ward.
You can send the packed contract through MonkeyCode's free model access. You can park ephemeral smoke on its free server option. Disclosure: This article was prepared as part of MonkeyCode's product outreach.
Canonical git and merge rights stay on your laptop either way. The product is a scratch lane, not a source of truth. If that server vanishes, hashes still explain the candidate.
That is the whole point of tracing data. A missing host is an inconvenience. A missing hash is amnesia.
Split failure domains on the diagram
A generation path fails in independent pieces every week. The model can invent an import that never existed. The packer can omit the constraint file you wrote.
The scratch host can reboot in the middle of smoke. Local tests can pass on mocks and still lie. Those events should not share one pager channel.
You do not need a glossy poster for this. You need four boxes that cannot take each other down. Intent lives in git with the tests you trust.
Packing lives in a local tool you can read. Proposal lives in a remote model you do not own. Smoke lives on a throwaway host you can lose.
If the model is wrong, you discard the diff. If the server is gone, you rerun smoke later. If git is wrong, you have a real incident.
Those three outcomes are not the same emergency. Write the split into the review record in plain text. A future you should not answer every miss with retry.
Retry is how separate domains collapse into one blob. Collapsed domains look fast until the wrong box pages you. Keep the seams ugly and visible.
# proposed: .ai/review-2026-09-06.txt
constraints: local git is canonical; prompts carry tests not secrets
data_flow: test -> pack -> model diff -> worktree -> optional smoke -> local CI
failure_domains: git-intent | local-packer | remote-proposal | ephemeral-smoke
next_change: pin OpenAPI for billing before another generate
Notice the last line of that record. A review that names no next change is theater. You pick one edit to the path, not twelve slogans.
Pick the next change, then stop
After you trace the flow you will want a rewrite. Do not start that rewrite during the review hour. Architecture reviews die when they become secret migrations.
Choose the smallest change that shrinks the worst domain. For many pipelines that change is a contract pin. Generated handlers drift because the prompt saw a comment.
They did not see a schema with required fields. Put the schema in the pack before you generate. Assert the same schema after the worktree apply.
# proposed: tests/test_contract_pin.py
import json
from pathlib import Path
def test_billing_handler_matches_pinned_schema():
schema = json.loads(Path("contracts/billing.json").read_text())
src = Path("src/billing/handler.py").read_text()
for key in schema["required"]:
assert key in src, f"missing {key} after generate"
Run that test before you read the rest of the diff. If the contract is broken, the patch prose does not matter. You just saved an hour of charitable interpretation.
Another honest next change is a replay fixture on disk. Save the pack, the diff, and the test result together. Next week you can ask why the path accepted a lie.
mkdir -p .ai/replays/20260906
cp .ai/last-prompt-pack.txt .ai/candidate.diff .ai/flow-log.jsonl .ai/replays/20260906/
tar -czf .ai/replays/20260906.tgz -C .ai/replays 20260906
The change you should make next is the packer. Leave the remote model alone until the packer tells truth. A sharper model on a lying pack still ships a lying patch.
For the billing rehearsal, the packer should attach contracts/billing.json. It should also attach the failing test that names unknown fields. Without those two files, the model is guessing under fluorescent lights.
# proposed packer slice, unexecuted until you wire it
{
printf 'CONTRACT\n'
cat contracts/billing.json
printf '\nFAILING TEST\n'
cat tests/test_billing.py
} > .ai/last-prompt-pack.txt
python tools/check_constraints.py
Who should skip this ritual
Skip the review record for a throwaway kata session. The overhead is for paths that can touch billing. It is also for identity flows and retention rules.
Do not wear a flight checklist to ride a bicycle. Ceremony without blast radius is costume jewelry. Save the hour for code that can leak or charge.
If your platform already packs context and gates merges, stop. Do not bolt a second tracer onto a working path. Duplicate logs become a fresh failure domain of their own.
Join the review that exists, or replace it on purpose. Two maps of the same river will fight. Pick one shoreline and write it down.
This approach will not make a weak contract strong. It will not turn a scratch host into durable infrastructure. It will not prove that a model output is correct.
It only makes the path visible enough to argue. You still read the diff with your own eyes. You still run the tests your team actually trusts.
The review is a map, not a substitute for taste. Maps do not drive. Drivers still miss exits without them.
Finish the review in one hour
Block fifty minutes plus a ten minute writeup. Use the first ten minutes on constraints.yaml against the repo. Use the next fifteen to run the tracer on one failing test.
Spend the next fifteen on the worktree against the contract pin. Spend the last ten writing the next change in one sentence. That sentence belongs in the review file, not in chat.
Chat evaporates before the next incident review starts. The file sits next to the hashes you recorded. Future you will bless the dullness.
python tools/check_constraints.py
python tools/flow_trace.py
pytest tests/test_contract_pin.py -q
printf '\nnext_change: pin billing schema in the pack\n' >> .ai/review-2026-09-06.txt
If a step fails, you stop the generate attempt. A failed constraint means the review did its job. You found the hole before the model papered it over.
Keep the scratch lane optional and the review record mandatory. The map is what you ship to your future self. Everything else is weather on the wire.
Top comments (0)