DEV Community

Alex Zhu
Alex Zhu

Posted on

Freeze Agent Topology Guesses With a One-Page Wiki SOP

The Monday queue that never existed

You ship a Friday agent patch that only needed a retry helper, then staging dies on Monday. The diff added a worker process, a queue name, and three environment keys nobody provisioned. Your teammate trusted the model because the code compiled and the unit tests used mocks. This playbook treats those invented topology guesses as review items, not as clever extras.

Cheap iteration makes the problem worse, because another agent pass can invent a cache, a sidecar, and a hostname before lunch. You do not need a new architecture committee for every helper function that lands in a brownfield service. You need a freeze rule, named roles, and a checker that fails the build when guesses leak into compose files. The rest of this article is a paste-ready SOP plus a small Python artifact you can drop into CI.

Why agent guesses survive code review

Most pull request templates ask for tests, screenshots, and a rollback note, then stay silent about new runtime dependencies. They rarely ask whether Redis, Kafka, or a second Postgres showed up without an owner. Reviewers skim Python and miss YAML, especially when the model formatted the YAML cleanly and the unit tests never boot the stack. A remote drafting loop makes that cycle faster, which stays useful until the guesses compound across two or three small patches.

You should assume the agent will fill every gap in your brownfield repo when the prompt says the tests must pass. Missing helm values become reasonable looking defaults that nobody in ops has ever seen. Missing base URLs become localhost plus a lucky port that collides with a laptop sidecar. Missing secrets become placeholder names that later get copied into a real shell profile during a late incident. The SOP below forces those fills into a ledger before merge, so the guess has a human owner.

Roles you can name in the wiki

Keep four roles on a single page so the handoff does not depend on who happens to be online. Write the names beside the freeze steps, not in a private doc that only the original author can find. If a role is empty during a holiday week, the merge waits rather than inventing a substitute in chat. That rule sounds harsh until you remember the queue that never existed.

  1. Change Author ran the agent, owns the diff, and must list every new runtime dependency the model introduced.
  2. Assumption Reviewer is not the author and only signs the ledger, without redesigning the feature under review.
  3. Runtime Owner confirms staging and production actually provide those hosts, queues, buckets, and credentials.
  4. Wiki Steward rejects template drift and restores the one-page run when people start pasting novels into the freeze page.

If your team is three people, the reviewer and runtime owner can be the same human, but they cannot be the author. Write that constraint in the wiki so Friday shortcuts do not quietly rewrite the rule when chat is empty. Rotate the reviewer weekly if the same pair always ships together, because familiarity is how invented hostnames survive.

The one-page run (paste this)

Copy the following numbered run into your team wiki, and keep the whole contract on one screen so people actually follow it during review. Do not attach architecture essays, vendor comparisons, or a history of past outages on this page. Those belong one click away, linked from the steward section, so the freeze stays usable under time pressure.

  1. Freeze the session. Stop the agent before a second "just fix CI" turn, then export the last prompt and the file list the model touched.
  2. Diff for topology, not only for logic. Search the branch for compose services, Helm charts, Terraform, GitHub Actions service containers, and new environment keys.
  3. Write one ledger row per guess. A guess is any host, port, queue, bucket, model endpoint, or secret name that the repo did not already document.
  4. Run the checker locally. The script below must exit non-zero when an undocumented service or environment key appears in the branch.
  5. Handoff to the Assumption Reviewer. They only confirm the ledger is complete for this diff, and they do not redesign the feature.
  6. Runtime Owner marks each row as exists, will-provision, or rejected, using the decision table later in this page.
  7. Merge only with zero unreviewed rows. If CI is red because the checker failed, treat that redness as a process success rather than a flake.

That seven-step contract is the entire operational freeze, and it should stay short enough to read during a standup. Longer essays belong in architecture docs, not in the freeze page that reviewers open while the agent is still warm. If someone needs a novel to merge a retry helper, the page has already failed its job.

Artifact: an assumption ledger and a failing checker

Store a known-good allowlist next to the SOP, and keep the file boring on purpose so you will actually maintain it after the first week. Proposed example only: this YAML is a starting template, not a description of your production estate. Replace the names with services you already run, then commit the file before you turn the checker on.

# config/topology_allowlist.yaml
services:
  - api
  - worker
  - postgres
env_keys:
  - DATABASE_URL
  - API_TOKEN
  - LOG_LEVEL
hostnames:
  - api.internal
  - postgres.internal
Enter fullscreen mode Exit fullscreen mode

Treat the next snippet as a proposed checker you must adapt, not as a battle-tested product that understands every manifest. You should extend the filename globs to match Helm, Terraform, and whatever else your agents are allowed to touch. The heuristics are intentionally narrow so a first run fails on invented queues instead of on every constant in the repo.

# tools/check_agent_assumptions.py
"""Fail CI when agent diffs invent services, env keys, or hostnames."""

from __future__ import annotations

import re
import sys
from pathlib import Path

import yaml

ROOT = Path(__file__).resolve().parents[1]
ALLOW = yaml.safe_load((ROOT / "config" / "topology_allowlist.yaml").read_text())
SERVICE_RE = re.compile(r"^\s{2}([a-zA-Z0-9_-]+):\s*$")
ENV_RE = re.compile(r"\b([A-Z][A-Z0-9_]{2,})\b")
HOST_RE = re.compile(r"\b([a-z0-9-]+(?:\.[a-z0-9-]+)+)\b")

SCAN_GLOBS = [
    "docker-compose*.yml",
    "compose*.yaml",
    ".env.example",
    "**/*.py",
    ".github/workflows/*.yml",
]


def iter_files() -> list[Path]:
    files: list[Path] = []
    for glob in SCAN_GLOBS:
        files.extend(ROOT.glob(glob))
    return [p for p in files if p.is_file()]


def services_from_compose(text: str) -> set[str]:
    names: set[str] = set()
    in_services = False
    for line in text.splitlines():
        if line.startswith("services:"):
            in_services = True
            continue
        if in_services and line and not line.startswith(" "):
            in_services = False
        if in_services:
            match = SERVICE_RE.match(line)
            if match:
                names.add(match.group(1))
    return names


def main() -> int:
    allowed_services = set(ALLOW["services"])
    allowed_env = set(ALLOW["env_keys"])
    allowed_hosts = set(ALLOW["hostnames"])
    invented: list[str] = []

    for path in iter_files():
        text = path.read_text(encoding="utf-8", errors="ignore")
        rel = path.relative_to(ROOT)
        if path.name.startswith("docker-compose") or path.name.startswith("compose"):
            for name in services_from_compose(text) - allowed_services:
                invented.append(f"{rel}: service `{name}` is not in the allowlist")
        for key in set(ENV_RE.findall(text)) - allowed_env:
            if key.endswith("_URL") or key.endswith("_TOKEN") or key.endswith("_HOST"):
                invented.append(f"{rel}: env key `{key}` is not in the allowlist")
        if path.suffix == ".py" or "workflows" in str(rel):
            for host in set(HOST_RE.findall(text)) - allowed_hosts:
                if host.endswith(".internal") or host.endswith(".local"):
                    invented.append(f"{rel}: hostname `{host}` is not in the allowlist")

    if invented:
        print("Assumption freeze failed. File ledger rows before merge:")
        for item in sorted(set(invented)):
            print(f"- {item}")
        return 1
    print("Assumption freeze passed.")
    return 0


if __name__ == "__main__":
    sys.exit(main())
Enter fullscreen mode Exit fullscreen mode

Run the checker after every agent session, including personal branches, rather than waiting for the main branch build to surprise you. Install the YAML parser in the same environment your CI already uses for lint, so you do not create a second Python toolchain. If the command fails, freeze the agent before you ask it to silence the output.

python -m pip install pyyaml
python tools/check_agent_assumptions.py
Enter fullscreen mode Exit fullscreen mode

Pair the checker with a ledger the reviewer actually signs, and keep that table in the pull request body or in docs/assumption-ledger.md so the handoff is visible. Proposed example only: copy the header, then add one row per guess instead of writing a paragraph. Empty decisions mean the branch is not mergeable, even when the feature tests are green.

| ID | Guess | File | Author claim | Runtime owner | Decision |
| --- | --- | --- | --- | --- | --- |
| A-14 | `jobs` Redis service | docker-compose.yml | Needed for retries | Unreviewed | unreviewed |
Enter fullscreen mode Exit fullscreen mode

Where a free remote drafting host belongs

You can run the agent that drafts the helper, and the checker that polices it, on separate machines without mixing their jobs. The checker should stay local or in CI so it does not depend on a model remaining available during an outage. The agent can live on a remote box if your laptop is already full of language servers, browsers, and half-finished containers. Keep that split explicit in the wiki so people do not skip the checker when the remote session is convenient.

Disclosure: This article was prepared as part of MonkeyCode's product outreach. If your team already needs a remote place to run those agent sessions, consider a dedicated drafting host. MonkeyCode's free model access and free server option can host that loop while CI keeps the freeze checker. That split matters because generation can be cheap and remote, but topology truth stays in your allowlist and in the Runtime Owner's marks. Skip any vendor if you cannot point the agent at a private clone without copying production secrets into the prompt.

Do not paste credentials into the remote session, even when the box is convenient and the models are free to call. Point the agent at redacted compose files and .env.example only, then let humans fill real values in the secret store. If the model proposes a secret name, add a ledger row instead of trying that name against staging just to see. A freeze that leaks credentials is worse than a missing retry helper.

Decision table for the Runtime Owner

Use this table when a ledger row is not an obvious reject, and paste it under the one-page run so Slack threads do not become the real process. The author fills the guess type, and the Runtime Owner fills the two existence columns from actual staging and production, not from the agent's story. If either column is unknown, the decision is unreviewed until someone logs into the environment and looks.

Guess type Exists in staging? Exists in prod? Decision Handoff
New compose service No No rejected unless a provision ticket exists Author deletes the service from the diff
New env key Yes Yes exists Reviewer signs the ledger
New env key Yes No will-provision Runtime owner files an infra ticket
New hostname No No rejected Author must use an allowlisted name
New queue or bucket Partial No will-provision Do not merge until the ticket has an owner

Keep rejected rows in the ledger for a week so the next agent session does not rediscover the same queue. The Wiki Steward can archive them after the allowlist and the docs match. Do not delete evidence of a guess that almost shipped, because that is how the freeze teaches new authors.

Limitations, and who should not use this

The checker is a heuristic, not an architecture oracle, and it will miss a Kubernetes manifest you forgot to glob. It will also nag you about test fixtures that mention EXAMPLE_TOKEN, which is noisy but cheaper than a Monday outage. A human still has to read the ledger, because a hostname regex cannot tell a documentation example from a new dependency. This SOP does not replace threat modeling, capacity planning, or a change-advisory board in a regulated environment.

You should not use this approach if you are a solo hobbyist with no shared wiki, because the roles collapse into one person who will skip the freeze. You should also skip it for throwaway spikes that never leave a personal branch and never touch shared staging. Do not point a remote agent at production kubeconfigs, customer dumps, or live tokens, even when the server is free to use. If your topology lives only in someone's head, write the allowlist first; otherwise the checker has nothing honest to compare.

Teams that already generate large refactors with agents will feel friction for a week, and that friction is the point of the freeze. After the allowlist matches reality, most helper patches pass the checker in seconds, and only the invented queue still fails. If every patch fails, your allowlist is incomplete, not your engineers. Fix the file with the Runtime Owner before you disable the job.

Keep the wiki page short on purpose

When the SOP grows past one screen, people will ignore it and ask the agent to just make CI green. Put examples in a second page, and keep the freeze page limited to roles, the seven steps, the decision table, and a link to the checker. Review the allowlist when you actually provision a service, not on a calendar ritual that nobody attends. The steward's job is deletion as much as it is addition.

If you adapt this playbook, change the globs before you congratulate yourselves on a green pipeline. A checker that never sees Helm charts will happily bless an agent that invented three new releases. Treat that miss as a process bug, file it against the Wiki Steward, and keep the Monday queue in the story so new hires understand why the freeze exists.

Top comments (0)