DEV Community

Sam Chen
Sam Chen

Posted on

You Closed the Chat. Five Anti-Patterns Still Merged.

The chat went quiet, so the work felt finished. That feeling is the bug, not the model.

I keep a five-row anti-pattern catalog for this moment. Each row lists symptoms, root cause, and a local replacement.

A free session can draft code without any shame. Your repo still owns the merge gate, always.

Why this catalog exists

AI coding sessions now end in minutes, not days. Speed is not the same thing as a merge contract.

Have you ever pasted a green snippet as review? I have done that, and the next outage was local.

Trending posts ask if models outgrew our tests. Wrong question for this catalog on your machine. Those tests never ran against your actual checkout.

How to read each row

Use the catalog during the last ten minutes. Do not use it as a vibe check.

For every row I want three things only:

  • Symptoms you can spot in the diff or logs
  • Root cause stated in one blunt sentence
  • A replacement pattern with a real command

No score. No leaderboard. Just fail-closed checks you can rerun.

Anti-pattern 1: Clipboard as CI

You copy the patch into the tree. You never run a command that can fail.

Symptoms

  • The PR quotes the chat, not a failing command
  • Generator output never touches a scripts/ gate
  • Review says "looks good" with zero pasted output

Root cause

The session produced text. You treated that text as an execution receipt.

Replacement: a local fail-closed gate

Write one script the model cannot mark green by talking.

#!/usr/bin/env bash
# scripts/local-gate.sh
set -euo pipefail
cd "$(dirname "$0")/.."

test -f reproduce.sh
test -f decisions.md

if command -v rg >/dev/null; then
  if rg -n "AKIA|BEGIN PRIVATE|api_key\s*=" --glob '!decisions.md' .; then
    echo "secret-shaped text in the tree" >&2
    exit 1
  fi
fi

./reproduce.sh
python3 scripts/check_defaults.py
pytest -q tests/gate
Enter fullscreen mode Exit fullscreen mode

Did the chat approve the patch in prose? Fine. Did local-gate.sh exit zero on your laptop?

Anti-pattern 2: Anonymous runtime

You ran the demo on a borrowed box. Nobody on the team can replay it.

Symptoms

  • The README says it worked on "the server"
  • No commit SHA, seed, or interpreter version is recorded
  • A teammate cannot hit the same path on a laptop

Root cause

A shared runtime became the source of truth. Your laptop never signed the result.

Replacement: a reproduce file that pins the command

#!/usr/bin/env bash
# reproduce.sh
set -euo pipefail
echo "git=$(git rev-parse HEAD)"
echo "python=$(python3 --version)"
python3 -m venv .venv
# shellcheck disable=SC1091
. .venv/bin/activate
pip install -r requirements-gate.txt
python3 -m app.probe --fixture tests/fixtures/min.json
Enter fullscreen mode Exit fullscreen mode

The box can be disposable and even free. The script cannot vanish with the session.

Anti-pattern 3: Model-chosen defaults

The generator picked timeouts, pool sizes, and log levels. You shipped those numbers unchanged.

Symptoms

  • timeout=30, retries=3, workers=4 with no comment
  • Config keys appear only inside generated files
  • No test asserts a deadline or a fail-closed limit

Root cause

Defaults look like expertise in a hurry. They are the model's prior, frozen into your repo.

Replacement: an explicit defaults review

# scripts/check_defaults.py
from pathlib import Path
import re
import sys

ALLOWED = {
    "timeout_s": {5, 10, 15},
    "retries": {0, 1},
    "workers": {1, 2},
}

text = Path("app/config.py").read_text(encoding="utf-8")
fail = False
for key, allowed in ALLOWED.items():
    match = re.search(rf"{key}\s*=\s*(\d+)", text)
    if not match or int(match.group(1)) not in allowed:
        print(f"unreviewed default: {key}")
        fail = True
if fail:
    sys.exit(1)
print("defaults reviewed")
Enter fullscreen mode Exit fullscreen mode

Would you defend retries=3 in an incident channel? If not, do not merge that line.

Anti-pattern 4: Secrets in the prompt

You pasted .env so the model could "see the real shape." That thread now outlives the key rotation.

Symptoms

  • Chat history contains hostnames, tokens, or customer ids
  • Generated fixtures still point at production URLs
  • .gitignore looks clean, but the session log is not

Root cause

Context dumping felt faster than making a fake fixture. The leak is now immortal in the thread.

Replacement: a fixture that cannot authenticate

{
  "base_url": "http://127.0.0.1:9",
  "token": "test-token-not-secret",
  "user_id": "fixture-user"
}
Enter fullscreen mode Exit fullscreen mode

If the model needs shape, give it a dead socket. Never give it a live key.

Scan the tree before the gate returns green:

# last line of local-gate.sh before pytest
python3 - <<'PY'
from pathlib import Path
bad = ("AKIA", "BEGIN PRIVATE", "sk-live", "xoxb-")
root = Path(".")
for path in root.rglob("*"):
    if path.suffix in {".png", ".pyc"} or ".venv" in path.parts:
        continue
    try:
        text = path.read_text(encoding="utf-8", errors="ignore")
    except OSError:
        continue
    if any(token in text for token in bad):
        raise SystemExit(f"secret-shaped text: {path}")
print("no secret-shaped text")
PY
Enter fullscreen mode Exit fullscreen mode

Did the fixture still need a routable host? Then it is not a fixture.

Anti-pattern 5: Forty-turn design doc

The architecture lives in a scrolling chat. The repo only has a shrug.

Symptoms

  • The PR links a transcript instead of a table
  • Two engineers recount different "final" decisions
  • Re-asking the model quietly changes the story

Root cause

Conversation is cheap and feels like design. A twelve-line table is the actual design.

Replacement: a decision table in the repo

Keep decisions.md tiny. Blank cells mean the chat is not done.

Decision Options Chosen Why Revisit when
Auth session cookie / static token / none static token in tests local gate must not hit IdP we add a real user
Runtime laptop / borrowed server / CI laptop owns merge borrowed boxes lie about paths CI is green twice
Timeouts 5s / 30s / none 5s fail closed on hang p99 is measured
Retries 0 / 3 / infinite 0 retries hide probe bugs idempotency is proven
Fixtures prod dump / synthetic synthetic secrets stay out of prompts schema changes

Print this table in the PR body. If a cell is blank, stop generating more code.

The artifact: one gate, one table

Here is the workflow I want on the last commit. Treat it as a proposal, not a published study.

  1. Fill decisions.md before asking for more code.
  2. Generate against fixtures, never against live secrets.
  3. Run ./scripts/local-gate.sh on your laptop.
  4. Treat any borrowed server as a scratch lane only.
  5. Merge only when the gate exits 0 and the table is complete.

I am not attaching latency charts on purpose. Charts without a pinned reproduce.sh are theater.

A tiny pytest file keeps the happy-path lie from shipping:

# tests/gate/test_fail_closed.py
from app.probe import probe
from app import config

def test_dead_socket_fails_closed():
    result = probe("tests/fixtures/min.json")
    assert result.ok is False
    assert result.error in {"timeout", "connection_refused"}

def test_retries_are_zero():
    assert config.retries == 0

def test_timeout_is_reviewed():
    assert config.timeout_s in {5, 10, 15}
Enter fullscreen mode Exit fullscreen mode

Does a dead socket fail closed on the first try? If that probe "succeeds," your gate is theater.

Wire a stub module so the example is runnable:

# app/config.py
timeout_s = 5
retries = 0
workers = 1

# app/probe.py
from dataclasses import dataclass
from pathlib import Path
import json
import socket

@dataclass
class Result:
    ok: bool
    error: str | None

def probe(fixture_path: str) -> Result:
    data = json.loads(Path(fixture_path).read_text(encoding="utf-8"))
    host, port = "127.0.0.1", 9
    if ":" in data["base_url"].split("//", 1)[-1]:
        hostport = data["base_url"].split("//", 1)[-1]
        host, port_s = hostport.split(":")[0], hostport.split(":")[1].split("/")[0]
        port = int(port_s)
    try:
        socket.create_connection((host, port), timeout=0.2).close()
        return Result(ok=True, error=None)
    except OSError as exc:
        name = "timeout" if isinstance(exc, socket.timeout) else "connection_refused"
        return Result(ok=False, error=name)
Enter fullscreen mode Exit fullscreen mode

Run it. Watch the dead port refuse you. That refusal is the point.

Where a free model and free server fit

I still use a throwaway lane for drafts. Scratch generation is fine. Merge authority is not.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

MonkeyCode offers free model access and a free server option. I treat that pair as the scratch lane: draft the probe, try the fixture, throw the box away. The local gate above still decides the merge. I am not claiming model names, quotas, hardware, or benchmarks. Those claims go stale. These five anti-patterns do not.

If you want a disposable place to practice the gate, that scratch lane is enough. Do not promote it into CI.

Limitations

This catalog will annoy you on tiny scripts. Good. Friction on a one-line typo is a smell, not a virtue.

It will not catch semantic bugs the tests never named. It will not replace a threat model. It will not make a weak fixture honest.

Skip this approach when:

  • You are pairing live and already running the suite
  • The change is a comment, rename, or typo
  • You cannot run anything locally at all
  • You need a signed production deploy, not a laptop gate

Cannot run locally? Then you have a bigger problem than chat quality. Borrowed servers drift. Free sessions forget. Your reproduce.sh should not.

What I will not do

I will not call the chat a review. I will not paste live keys into a prompt. I will not ship retries=3 because it looked standard.

Would you bet an incident channel on a transcript? I would not.

Close the tab after the gate is green. Not before.

Top comments (0)