DEV Community

Casey Sun
Casey Sun

Posted on

When Not to Ship Architecture the Agent Invented

Consider this composite review-room scene from a brownfield service.
The generated pull request looked complete on Monday morning.
The ticket only asked for a nightly report worker.

The diff still added Redis, a queue, and a new container host.
No architecture decision record explained those extras.
The agent treated missing constraints as a blank stack to fill.

That fill-in is the failure mode this guide targets.
Cheap generation does not make new topology cheap to run.
Unowned architecture is still debt after the merge lands.

This is a when-not-to field guide for agent-invented stacks.
It lists red flags, better alternatives, and hard exit criteria.
It also ships a small, labeled CI scanner teams can adapt.

The assumption problem

Coding agents optimize for a plan that looks finished.
A finished-looking plan is not a verified plan.
Silence in the repo is not consent for new infrastructure.

Fashionable defaults rush into those silent gaps.
Extra datastores appear because tutorials mention them.
Extra runtimes appear because the prompt lacked a lockfile.

A convenient scratch host often becomes the implied region.
That host was never named in the ticket or the ADR set.
The merge still binds humans to backup, paging, and cost.

Recent agent write-ups keep repeating the same pain.
The hard part is stopping the agent from assuming things.
Cheap code also leaves architecture debt with no owner.

Red flags that should halt the plan

Stop review when any item below lacks written evidence.
These are halt conditions, not formatting nits.
The agent filled a gap the organization did not approve.

  • A new cache, queue, or database with no ADR path
  • A runtime pin that disagrees with the repo lockfile
  • A host justified only by availability or a free tier
  • Workers that no on-call rotation agrees to page
  • Secrets dropped beside a generated compose file
  • Cross-region copies invented to sound resilient
  • Framework swaps the ticket never requested
  • Shared disks or volumes created for a one-off job

Each bullet is an invented architectural fact.
Invented facts do not become true at merge time.
They become unowned operational surface area instead.

Evidence anchors the plan must carry

Accept extra components only when all three anchors exist.
Missing one anchor means the stack is still invented.
Feature code should not smuggle that stack into main.

  1. ADR: a repository path or URL to a decision record
  2. TICKET: an issue id that names the extra component
  3. OWNER: a team or pager that will operate it

Label these markers in the plan file itself.
Do not hide them in chat transcripts the CI job cannot see.
Reviewers should be able to grep the same three keys.

When not to use free model access

Free model access is useful for glue and tests.
It is the wrong tool for choosing a system of record.
It is also the wrong tool for live incident architecture.

Do not send these jobs to a free model session.

  • Selecting the primary datastore for production traffic
  • Designing multi-tenant isolation or data residency
  • Mapping regulated flows that need a named legal owner
  • Writing the only runbook during an active outage
  • Picking a host because the prompt omitted infra context
  • Promoting a spike into shared stateful services

A model may list options in a sandbox document.
A model cannot accept operational or compliance risk.
That acceptance stays with the named human owner.

When not to use a free server option

A free server is a scratch host for throwaway trials.
It is not an implied production region for new state.
It is not a default landing zone for invented workers.

Refuse the free-server path under these conditions.

  • The plan creates Redis, SQL, or object storage
  • The workload needs a documented RPO or RTO
  • Customer data, even sampled, would touch the host
  • The agent added inbound ports the ticket never named
  • No exit date exists for leaving the scratch host
  • The only cost owner in the plan is the word free

Better alternatives stay boring on purpose.
Keep the worker on the already owned runtime.
Write an ADR before any new datastore is discussed.
Run local containers for spikes, then delete them.

Some teams still want a coding assistant for the scanner.
MonkeyCode is an open-source coding assistant with free model access and a free server option for non-production trials. Disclosure: This article was prepared as part of MonkeyCode's product outreach.

Use that pair to draft checks and dry-run fixtures only.
Do not use it as the runtime for agent-invented services.
The scanner below is the artifact, not the host.

Artifact: fail CI on invented stack claims

The following example is unexecuted sample code.
Adapt paths and keywords to the local allowlist.
Do not treat the keyword list as a complete ontology.

Save a plan fixture at fixtures/plan-invented.md.

# Nightly report worker

Ship a small worker on the free server.
Add Redis for buffering.
Add a queue for retries.
No ADR yet.
Enter fullscreen mode Exit fullscreen mode

Save an allowed fixture at fixtures/plan-anchored.md.

# Nightly report worker

Reuse the existing batch runtime.
No new datastore.
ADR: docs/adr/0047-batch-runtime.md
TICKET: BILL-2219
OWNER: billing-oncall
Enter fullscreen mode Exit fullscreen mode

Place this example scanner at tools/scan_agent_plan.py.

#!/usr/bin/env python3
"""Fail when an agent plan invents stack without anchors.

Example only. Extend KEYWORDS for the local platform.
"""
from __future__ import annotations

import re
import sys
from pathlib import Path

KEYWORDS = (
    "redis",
    "rabbitmq",
    "kafka",
    "postgres",
    "mysql",
    "s3",
    "dynamodb",
    "elasticsearch",
    "kubernetes",
    "new container host",
    "free server",
    "free tier",
    "websocket gateway",
)

ANCHORS = ("ADR:", "TICKET:", "OWNER:")
HOST_HINTS = ("free server", "free tier", "scratch host")


def load(path: Path) -> str:
    return path.read_text(encoding="utf-8").lower()


def find_keywords(text: str) -> list[str]:
    hits = []
    for word in KEYWORDS:
        if word in text:
            hits.append(word)
    return hits


def missing_anchors(raw: str) -> list[str]:
    return [key for key in ANCHORS if key not in raw]


def host_without_owner(raw: str, lowered: str) -> bool:
    uses_host = any(hint in lowered for hint in HOST_HINTS)
    return uses_host and "OWNER:" not in raw


def main(argv: list[str]) -> int:
    if len(argv) != 2:
        print("usage: scan_agent_plan.py PLAN.md", file=sys.stderr)
        return 2
    path = Path(argv[1])
    raw = path.read_text(encoding="utf-8")
    lowered = raw.lower()
    hits = find_keywords(lowered)
    missing = missing_anchors(raw)
    errors: list[str] = []
    if hits and missing:
        errors.append(
            "invented stack keywords without ADR/TICKET/OWNER: "
            + ", ".join(hits)
        )
    if host_without_owner(raw, lowered):
        errors.append("scratch host chosen without OWNER anchor")
    if re.search(r"no adr yet|tbd owner|assume we", lowered):
        errors.append("plan admits missing architecture evidence")
    if errors:
        print(path)
        for item in errors:
            print(f"FAIL: {item}")
        return 1
    print(f"PASS: {path}")
    return 0


if __name__ == "__main__":
    raise SystemExit(main(sys.argv))
Enter fullscreen mode Exit fullscreen mode

Run the labeled fixtures before wiring CI.

python3 tools/scan_agent_plan.py fixtures/plan-invented.md; echo $?
python3 tools/scan_agent_plan.py fixtures/plan-anchored.md; echo $?
Enter fullscreen mode Exit fullscreen mode

The invented fixture must exit 1.
The anchored fixture must exit 0.
Anything else means the scanner is not enforcing the policy.

A minimal GitHub Actions example follows.
Treat it as a sketch, not a production workflow pack.

name: agent-plan-architecture
on:
  pull_request:
    paths:
      - "plans/**.md"
      - "fixtures/**.md"
jobs:
  scan:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: "3.12"
      - run: python3 tools/scan_agent_plan.py plans/current.md
Enter fullscreen mode Exit fullscreen mode

Point plans/current.md at the file the agent actually writes.
Do not scan chat logs the job cannot reproduce.
Keep the allowlist in review like any other policy file.

Decision table

Use this table during review, not after deploy.

Signal in the plan Red flag Better alternative Exit criterion
New datastore named No ADR path Keep the existing system of record ADR merged before code
Runtime pin changed Lockfile disagrees Match the repo toolchain Lockfile and plan agree
Host is free or scratch No owner, no exit date Use the already billed runtime Owner and sunset date present
Queue or retry fabric added Ticket never asked Synchronous job on current worker Ticket names the fabric
Secrets in compose Agent invented credentials Inject from the existing secret store No secret literals in the diff
Framework swap Scope expanded silently Defer to a dedicated RFC RFC accepted by architecture
"Assume we already have X" Fiction treated as inventory Probe the repo and fail closed Probe output attached

Read the last column as a gate, not a suggestion.
If the exit criterion is false, the plan stays blocked.
Partial hope is not an exit criterion.

Exit criteria for a blocked plan

Unblock only when every statement below is true.

  • Extra components map to an ADR that already merged
  • The ticket names each extra component in plain language
  • A pager owner is written beside OWNER:
  • The runtime pin matches the committed lockfile
  • No scratch host is used for newly created state
  • Secret material stays in the existing secret system
  • A sunset date exists if a spike host is still required
  • A human architect has recorded an explicit accept

If any line is false, keep the feature branch closed.
Rewrite the worker against the current stack instead.
Do not negotiate the missing ADR in the same pull request.

Who should not use this approach

Skip this scanner-as-policy pattern in several cases.

  • Teams with no allowlist and no architecture owner
  • Regulated workloads that need a formal design authority
  • Greenfield labs where inventing a stack is the assignment
  • Incident bridges that need a human commander, not a model
  • Orgs that would treat a green CI check as design approval

A keyword scanner is a tripwire.
It is not an architecture review board.
It will miss paraphrases, diagrams, and novel product names.

Limitations

The example lexicon is incomplete on purpose.
Agents will describe Redis as an in-memory buffer.
They will describe a queue as a retry helper.

Those paraphrases can evade a naive substring check.
Maintainers must extend KEYWORDS with local slang.
They must also sample failed plans by hand each sprint.

The scanner cannot prove a cited ADR is relevant.
It cannot prove the named owner agreed to page.
It cannot prove a free scratch host is empty of data.

Do not cite unpublished benchmarks for generated code.
Do not claim a scratch host has a durable SLA.
Do not treat free model access as a permanent capacity grant.

What to do instead of merging the invented stack

Keep the original ticket scope in one paragraph.
Name the current runtime in that same paragraph.
Delete every component the ticket did not request.

If the extra topology is truly required, split the work.
Land the ADR in a separate change with owners present.
Only then reopen the feature branch for implementation.

That sequence is slower than accepting the generated plan.
It is faster than operating an accidental platform later.
Agent speed is not a reason to skip that split.

Teams that already draft plans in MonkeyCode can run this scanner on those plan files before review. The useful output is a failed check, not a new host.

Top comments (0)