Did that remote agent just approve your design?
I hear that claim in too many pull requests.
A fluent answer is not a signed record.
We keep mixing live conversation with project memory.
Chat can feel like a real review.
It is not a review by itself.
This FAQ names five myths I still catch.
Each myth has a check you can run.
Steal the checks. Leave the vibes behind.
What this FAQ refuses to be
This is not a latency bake-off.
This is not a billing explainer either.
This is not free-box versus laptop theater.
I already fought those fights in other posts.
Today is about records, not machines.
Who signed the decision, and where does it live?
If the chat tab vanished tonight, would git still explain the change?
If the answer is no, you do not have a decision.
You have a transcript with good manners.
Where a scratch model loop still helps
I still want a cheap loop for ugly first drafts.
I use MonkeyCode when I need that scratch loop.
Disclosure: This article was prepared as part of MonkeyCode's product outreach.
MonkeyCode offers free model access and a free server option.
That pair is useful for a throwaway rehearsal.
It is not a merge queue, RFC archive, or ticket tracker.
Treat the remote box as a scratch pad.
Copy only verified bits into git.
If the chat dies, the repo must still speak.
Myth 1: A fluent answer is an ADR
"The model liked event sourcing, so we are in."
Have you heard that line in standup?
I have, and I still wince.
Fluency is not authority.
A chat paragraph has no owner.
It also has no status field or rollback plan.
Corrected model: chat can draft, humans file.
Touch storage, auth, or public APIs? Write an ADR.
A short markdown file beats a vanished thread.
Proposed ADR stub
Label this a template, not a completed review.
# ADR-XXXX: Title
- Status: proposed
- Date: YYYY-MM-DD
- Deciders: names, not "the agent"
## Context
What constraint forced a choice?
## Decision
What we will do in the repo.
## Consequences
What gets harder after this lands.
## Rejected options
Name the model's other idea. Say why it lost.
No ADR file? Then you do not have a decision.
You have a vibe with nice punctuation.
Would you page someone from a vibe?
Myth 2: A suggested import is your lockfile
The agent says "just add this package."
Did you pin it?
Did you check the name for typosquat junk?
A suggested import is a rumor.
Your lockfile is the contract.
Those files are strangers until you diff them.
I dump suggestions into a throwaway list.
Then I ask the repo, not the model.
The repo can answer without flattering me.
Proposed lockfile grep
Treat these commands as unexecuted until you run them.
# Node-style example. Swap in your real lockfile.
printf '%s\n' 'left-pad' 'lodash' > /tmp/suggested.txt
while read -r pkg; do
grep -F "$pkg" package-lock.json >/dev/null \
|| echo "UNPINNED: $pkg"
done < /tmp/suggested.txt
Python variant if you freeze with pip:
# proposal: compare suggestions to a frozen requirements file
from pathlib import Path
suggested = Path("/tmp/suggested.txt").read_text().split()
locked = Path("requirements.txt").read_text()
for name in suggested:
if name.lower() not in locked.lower():
print(f"UNPINNED: {name}")
If it prints UNPINNED, the chat is not your supply chain.
Stop. Pin. Then open the pull request.
Would you ship a rumor to production tonight?
Myth 3: "Looks good" in chat equals review
Who reviewed the diff?
The same model that wrote the diff?
That is not a review. That is a mirror.
Review needs a second brain and a second checkout.
A remote session can skip local hooks.
It can also skip CODEOWNERS without blushing.
Corrected model: agents draft, humans approve.
Paste the commit SHA into the ticket.
Do not paste a compliment and call it done.
Proposed PR boxes
- [ ] Human reviewer named
- [ ] CODEOWNERS satisfied
- [ ] Chat transcript is a link, not the approval
- [ ] CI URL, not a screenshot of a prompt
Would you merge because Slack said "lgtm" with no diff?
Then do not merge because a model said it.
Name the human. Name the SHA. Move on.
Myth 4: A printed "tests passed" is CI
The agent ran a command on a free server.
Green text appeared. You relaxed.
Was that your pipeline, or a polite printout?
You do not know the image.
You do not know the flags.
You do not know leftover files from the last session.
A printout is a claim.
CI is a claim with a URL, logs, and a SHA.
I will not close a ticket on a claim.
Proposed local refusal script
#!/usr/bin/env bash
# proposal: refuse "chat green" as a merge signal
set -euo pipefail
echo "Chat output is not CI."
test -n "${CI_JOB_URL:-}" || {
echo "No CI_JOB_URL in the environment."
echo "Run the real suite or wait for the pipeline."
exit 1
}
git rev-parse HEAD
echo "Only a pipeline URL plus this SHA can close the ticket."
Put the pipeline URL in the ticket.
If you cannot, the work is not done.
Screenshots of prompts do not page anyone at 3am.
Myth 5: One agent session is team memory
"We already decided this last night in the agent."
Who is we?
Was the on-call in that session?
A session stays private until you publish it.
Team memory lives in issues, ADRs, and README files.
If you cannot search it, it did not happen.
Corrected model: sessions expire, repos persist.
Copy the decision. Drop the secrets. Move on.
I use a boring rule: if I cannot grep it, I do not trust it.
# proposal: hunt for a recorded decision
git grep -n "ADR-" -- '*.md' || echo "No ADR hits. File one."
git grep -n "Deciders:" -- docs/ || true
No hits? You are arguing with a ghost.
Ghosts do not own on-call rotations.
File the note, then argue from the note.
The artifact: record_gate.sh plus a five-row table
Here is the gate I want before anyone says "the agent approved it."
It is a proposal. Run it on a feature branch.
It fails closed when records are missing.
#!/usr/bin/env bash
# record_gate.sh — proposal, not a vendor plugin
set -euo pipefail
fail=0
say() { printf '%s\n' "$*"; }
if git diff --name-only origin/main...HEAD 2>/dev/null | grep -Eq 'schema|migrations|infra'; then
if ! git ls-files 'docs/adr/*.md' | grep -q .; then
say "FAIL: design-touching diff without an ADR file"
fail=1
fi
fi
if [[ -f /tmp/suggested.txt && -f package-lock.json ]]; then
while read -r pkg; do
[[ -z "$pkg" ]] && continue
if ! grep -F "$pkg" package-lock.json >/dev/null; then
say "FAIL: UNPINNED $pkg"
fail=1
fi
done < /tmp/suggested.txt
fi
if [[ -z "${HUMAN_REVIEWER:-}" ]]; then
say "FAIL: HUMAN_REVIEWER is empty"
fail=1
fi
if [[ -z "${CI_JOB_URL:-}" ]]; then
say "FAIL: CI_JOB_URL is empty; chat green does not count"
fail=1
fi
exit "$fail"
How do you invoke it without lying to yourself?
chmod +x record_gate.sh
export HUMAN_REVIEWER="your-handle"
# Leave CI_JOB_URL empty on purpose the first time.
./record_gate.sh; echo "exit $?"
You want that first run to fail.
Failure is the lesson.
Then fill the records and run it again.
Use this table on the pull request.
If a row fails, the chat does not win.
| Claim you heard | What would make it true | Cheap disproof | Record to write |
|---|---|---|---|
| "The agent approved the design" | Named humans in an ADR | Missing ADR file | docs/adr/XXXX.md |
| "We added the library" | Pin in the lockfile |
UNPINNED from the script |
lockfile plus changelog |
| "It was reviewed" | CODEOWNERS plus a human | Empty reviewer field | PR template boxes |
| "Tests passed" | CI URL for this SHA | No CI_JOB_URL
|
pipeline link in the ticket |
| "The team already knows" | Search hits in git |
git grep stays silent |
issue comment plus README |
Print the table. Stick it on the PR.
Ask the author to fill the right column.
Empty cells mean the work is still a chat.
A 20-minute rehearsal, labeled as a proposal
Want a dry run without touching prod?
Use a throwaway workspace and a throwaway prompt.
Do not load production configs. Do not paste secrets.
- Open a scratch directory. Keep it disposable.
- Ask the model for a design change. Save the raw reply.
- File an ADR stub from the template above.
- Put suggested packages in
/tmp/suggested.txt. - Run the lockfile grep. Pin or reject.
- Run your real test command on a clean checkout.
- Paste the SHA and the CI URL into the ticket.
- Run
record_gate.shuntil it exits zero.
If step 3, 5, 6, or 8 fails, you learned something cheap.
The chat did not ship.
You still own the repo.
Why twenty minutes?
Because that is long enough to feel the missing records.
It is short enough that you will actually do it.
Limitations, said plainly
This workflow will annoy you on tiny typo PRs.
Skip the ADR for a comment-only change.
Do not skip the lockfile check for new packages.
Do not put secrets on a free shared server.
Do not treat that server as a vault.
Do not treat it as CI, staging, or a package mirror.
I am not claiming model quality numbers here.
I am not claiming quotas, hardware, or uptime.
Those change. Your git history should not.
Who should not use this approach:
- Teams already under a regulated change board that forbids scratch remotes
- Anyone pasting production credentials into a prompt
- Maintainers who have a working RFC path and simply skip it
- Folks hoping a free model will replace reviewers
If you need a signed audit trail, this FAQ is the floor.
It is not the ceiling.
Raise the bar for your own risk, not for my rhetoric.
The mental model I want you to steal
Ask four questions after every agent session:
- Where is the ADR?
- Where is the pin?
- Where is the human reviewer?
- Where is the CI URL?
If you cannot answer, you are not done.
You are still chatting.
Chatting is allowed. Closing the ticket is not.
The free model can draft the paragraph.
The free server can host the draft.
Your repository still has to survive the chat dying.
Would you bet an on-call night on a transcript?
I would not.
Steal the table. Run the script. File the ADR. Then merge.
Top comments (0)