48-Hour Field Notes: Phantom Env Keys in Agent Infra Patches

The pull request looked finished. A thicker .env.example, a rewritten docker-compose.yml, and a GitHub Action that "only needed" PROD_DATABASE_URL plus CACHE_PASSWORD. The Node process still called process.env.DATABASE_URL and nothing else. The agent had drawn a second climate for a binary that only knows one season.

That gap is not a naming quibble. It is a boot-time crash that green CI will miss if the workflow injects the fictional keys. The app never asked for them. The patch still taught the pipeline to believe they were load-bearing.

This note is a 48-hour protocol, not a war story with trophy metrics. Treat the numbers in the fixtures as labeled samples. Run the commands against your own tree before you quote them.

The failure mode in one sitting

Agents do not only invent defaults inside function signatures. They invent habitats. A habitat is every name that must exist outside the process for the process to start: compose services, Action secrets, dotenv templates, Helm values, systemd Environment= lines. The model has read a thousand tutorials. Tutorials always have a REDIS_URL. Your service talks to SQLite.

The analogy is a hotel keycard encoded for floor 19 when the building stops at 12. The card prints without error. The elevator is the runtime.

A two-day pass is enough to see the pattern if you meter patches, not vibes. Day one: freeze a baseline of names the repo already reads. Day two: every agent diff is scored against that baseline. The interesting count is not total environment keys. It is keys the patch introduces that no source file loads.

Freeze the names the process actually reads

Keep the scanner boring. Boring survives copy-paste into a CI job. The script below is a proposal you can run locally; it is not a claim that any production fleet already ships it.

#!/usr/bin/env python3
"""phantom_env.py — inventory env keys vs keys a patch introduces.

Labeled example. Extend the regexes for your stack before trusting it.
"""
from __future__ import annotations

import re
import sys
from pathlib import Path

READ_PATTERNS = [
    re.compile(r"process\.env\.([A-Z][A-Z0-9_]+)"),
    re.compile(r"os\.environ(?:\.get)?\([\"']([A-Z][A-Z0-9_]+)"),
    re.compile(r"os\.getenv\([\"']([A-Z][A-Z0-9_]+)"),
    re.compile(r"ENV\[[\"']([A-Z][A-Z0-9_]+)[\"']\]"),  # Ruby
]

DECLARE_PATTERNS = [
    re.compile(r"^\s*([A-Z][A-Z0-9_]+)=", re.M),          # dotenv / compose env
    re.compile(r"secrets\.\s*([A-Z][A-Z0-9_]+)"),         # GHA
    re.compile(r"\$\{([A-Z][A-Z0-9_]+)\}"),               # compose interpolation
]

CODE_SUFFIX = {'.py', '.js', '.ts', '.tsx', '.go', '.rb', '.java'}
DECL_NAMES = {
    '.env', '.env.example', '.env.sample',
    'docker-compose.yml', 'docker-compose.yaml',
    'compose.yml', 'compose.yaml',
}

SKIP_DIRS = {'.git', 'node_modules', 'dist', 'build', '.venv', 'vendor'}


def files(root: Path):
    for p in root.rglob('*'):
        if not p.is_file():
            continue
        if any(part in SKIP_DIRS for part in p.parts):
            continue
        yield p


def harvest(text: str, patterns) -> set[str]:
    found: set[str] = set()
    for rx in patterns:
        found.update(rx.findall(text))
    return found


def inventory(root: Path) -> tuple[set[str], set[str]]:
    reads, decls = set(), set()
    for p in files(root):
        text = p.read_text(encoding='utf-8', errors='ignore')
        if p.suffix in CODE_SUFFIX:
            reads |= harvest(text, READ_PATTERNS)
        if p.name in DECL_NAMES or p.suffix in {'.yml', '.yaml'} and 'compose' in p.name:
            decls |= harvest(text, DECLARE_PATTERNS)
        if '.github/workflows' in str(p).replace('\\', '/'):
            decls |= harvest(text, DECLARE_PATTERNS)
    return reads, decls


def keys_in_diff(diff_text: str) -> set[str]:
    added = []
    for line in diff_text.splitlines():
        if line.startswith('+') and not line.startswith('+++'):
            added.append(line[1:])
    return harvest('\n'.join(added), DECLARE_PATTERNS + READ_PATTERNS)


def main() -> int:
    root = Path(sys.argv[1] if len(sys.argv) > 1 else '.')
    diff_path = Path(sys.argv[2]) if len(sys.argv) > 2 else None
    reads, decls = inventory(root)
    phantom_existing = decls - reads
    print(f'read_keys={len(reads)} declared_keys={len(decls)} '
          f'already_phantom={len(phantom_existing)}')
    for k in sorted(phantom_existing):
        print(f'  existing_phantom {k}')
    if diff_path:
        introduced = keys_in_diff(diff_path.read_text(encoding='utf-8'))
        new_phantom = (introduced - reads)
        print(f'patch_declared={len(introduced)} patch_phantom={len(new_phantom)}')
        for k in sorted(new_phantom):
            print(f'  patch_phantom {k}')
        return 1 if new_phantom else 0
    return 0


if __name__ == '__main__':
    raise SystemExit(main())

Save a tiny fixture so the protocol is falsifiable. The next block is synthetic. Do not treat it as a customer repo.

mkdir -p demo/src demo/.github/workflows
cat > demo/src/server.js <<'EOF'
const url = process.env.DATABASE_URL
const port = process.env.PORT
console.log('boot', Boolean(url), port)
EOF
cat > demo/.env.example <<'EOF'
DATABASE_URL=postgres://localhost/app
PORT=3000
EOF

Now invent the agent patch the way models often do: extra cache, extra stripe, extra "production" prefix.

cat > /tmp/agent.patch <<'EOF'
diff --git a/.env.example b/.env.example
--- a/.env.example
+++ b/.env.example
@@ -1,2 +1,5 @@
 DATABASE_URL=postgres://localhost/app
 PORT=3000
+PROD_DATABASE_URL=postgres://prod/app
+CACHE_PASSWORD=changeme
+STRIPE_WEBHOOK_SECRET=whsec_example
diff --git a/docker-compose.yml b/docker-compose.yml
--- /dev/null
+++ b/docker-compose.yml
@@ -0,0 +1,8 @@
+services:
+  api:
+    environment:
+      PROD_DATABASE_URL: ${PROD_DATABASE_URL}
+      CACHE_PASSWORD: ${CACHE_PASSWORD}
+      REDIS_URL: ${REDIS_URL}
EOF
python3 phantom_env.py demo /tmp/agent.patch

Expected shape of the output, still a fixture:

read_keys=2 declared_keys=2 already_phantom=0
patch_declared=4 patch_phantom=4
  patch_phantom CACHE_PASSWORD
  patch_phantom PROD_DATABASE_URL
  patch_phantom REDIS_URL
  patch_phantom STRIPE_WEBHOOK_SECRET

Four new names, zero readers. The compose file will look "complete" in review. The Node file will ignore three of them and die on the fourth only if someone deletes DATABASE_URL because the agent renamed it in prose.

What broke when the protocol met real diffs

Dynamic lookup is the first crack. process.env[name] and os.environ[key] do not yield literals. The scanner under-counts reads and over-counts phantoms. Mark those hits as suspects, not verdicts.

Generated clients are the second crack. An OpenAPI generator may read PETSTORE_BASE in a file you never open. If that file is in SKIP_DIRS by accident, the protocol lies.

The third crack is values versus names. This tool never prints secrets. If a reviewer pastes a real .env into a chat to "help the model debug," the protocol cannot save them. Names are the meter. Values stay on the laptop.

A fourth crack showed up in YAML that is not compose: Kubernetes manifests with valueFrom.secretKeyRef. The regex will either miss the key or treat every ConfigMap as an application read. Scope the path list on day one. Do not scan the whole monorepo because a model likes context windows.

A two-day loop that stays cheap

Hour 0: commit phantom_env.py and a fixture like demo/. Hour 1: run it on main and store read_keys as a comment in the PR template. You are not collecting a dashboard. You are collecting a denominator.

Hours 2–24: every agent-produced infra file goes through the same command. Reject the patch when patch_phantom is non-zero unless a human adds a matching read in application code. That rule is the whole policy. The script is just a flashlight.

Hours 24–48: watch false positives. If a key is read in bash ("$DATABASE_URL") the Python patterns will miss it. Add one regex, not ten. The protocol fails when the scanner becomes a second product.

Wire it as a local pre-check, then as CI if the noise is low:

git diff origin/main...HEAD > /tmp/pr.diff
python3 phantom_env.py . /tmp/pr.diff
echo "exit $?"  # 1 means the patch declared names the code does not read

A GitHub Action can call the same command. It should not receive secret values. Feed it the diff and the tree. If the model needs extra context, give it the scanner's name list, which is already non-secret.

That last split is where a hosted coding assistant earns a sentence. Disclosure: This article was prepared as part of MonkeyCode's product outreach. I used MonkeyCode here only as a place with free model access and a free server option, so the agent could propose compose and workflow edits on a throwaway box while the phantom-key script stayed on the laptop that holds real dotenv files. No model names, quotas, or hardware claims belong in this protocol; if those details matter to you, read the product's current docs rather than this paragraph.

The useful part survives if you delete the product name. Baseline the reads. Diff the declarations. Keep values off the prompt.

Decision table for the review comment

When patch_phantom is greater than zero, the review is not "please rename for style." It is one of four mechanical choices.

If the key is a true new dependency, the application code must grow a reader in the same patch. Compose cannot be the only witness.

If the key is a rename (DATABASE_URL to PROD_DATABASE_URL), the old reader must move. Leaving both is how you get staging to boot on a leftover local default.

If the key is tutorial residue (REDIS_URL in a SQLite app), delete it. Residue becomes a future secret scanner finding when someone fills it "just in case."

If the key is only for CI (DOCKER_HUB_TOKEN), it should live in the workflow secrets: block and nowhere in .env.example. Example files leak into forks. Workflow secrets do not, if you never echo them.

Write the choice in the PR, not in a wiki. Wikis do not fail the job.

Limitations, and who should skip this

The scanner is a name linter. It is not gitleaks, not a SAST tool, not an architecture review. It will not notice that DATABASE_URL points at production from a laptop. It will not notice an agent that hardcodes a password in a string that is not an env key.

Skip it if your runtime builds env names at execution time from prefixes (FOR_each_tenant). Skip it if you cannot keep real values out of prompts; a free server does not make a leaked dump smaller. Skip it if no human reads infra diffs. A red patch_phantom count with a rubber-stamp merge is theater.

Teams in regulated secret-rotation programs need their existing vault workflow, not a regex. This protocol is for small services where an agent is allowed to touch compose and Actions, and where a two-day window is the difference between a silent rename and a Monday outage.

What I would repeat

Repeat the baseline. Repeat the rule that a new declaration needs a new read in the same diff. Repeat the ban on pasting dotenv values into any model, free or not.

Do not repeat expanding the scanner into a framework. Do not repeat sending CI logs or live secrets to "give the agent more context." Names are enough. The process already told you which climate it can survive. The patch should not invent another one.

If you try the loop, keep the fixture in-repo so the next agent patch has something to fail against besides optimism.