The Agentic Code That Compiles but Crashes in Production

You see a pull request where the agent generated a function that imports requests and calls an external API. The CI passes, but the service crashes in staging because the container lacks the library. This mismatch between sandbox and production is why good developers stay wary of agentic development.

You will learn how to spot hallucinated imports, add safety wrappers around external calls, and integrate agentic output into a CI pipeline so it fails fast instead of at runtime.

Spot the hidden import bug

# Generated by an agent – looks fine locally
def fetch_data(url: str) -> dict:
    import requests
    resp = requests.get(url, timeout=5)
    return resp.json()

The snippet imports requests without checking if the package is installed. In a production container that only has requests in a dev-only extra, the import fails at runtime.

You can guard against this by adding a try-except around the import and falling back to a safe stub:

def fetch_data(url: str) -> dict:
    try:
        import requests
    except ImportError:
        class MissingPackage:
            def get(self, *args, **kwargs):
                raise RuntimeError("The 'requests' package is missing in production.")
        requests = MissingPackage()
    resp = requests.get(url, timeout=5)
    return resp.json()

The guard makes the failure explicit early, so you can add the missing dependency to your production image before the service starts.

Add a safety wrapper for external calls

Generated code often omits rate limiting or circuit breaking. An agent may produce a function that blindly retries on failure, exhausting a third-party quota.

# Agent output – no throttling
def query_service(payload: dict) -> str:
    return httpx.post("https://api.example.com/endpoint", json=payload).text

Wrap it with a simple retry with exponential backoff and a limit:

import httpx
import time

def query_service(payload: dict, max_retries: int = 3, base_delay: float = 0.5) -> str:
    for attempt in range(max_retries + 1):
        try:
            resp = httpx.post("https://api.example.com/endpoint", json=payload, timeout=10)
            resp.raise_for_status()
            return resp.text
        except (httpx.HTTPStatusError, httpx.NetworkError) as exc:
            if attempt == max_retries:
                raise RuntimeError(f"Service call failed after {max_retries} attempts") from exc
            delay = base_delay * (2 ** attempt)
            time.sleep(delay)

The wrapper ensures you do not hammer the external API and you get a clear error after the final attempt.

Watch for environment variable hallucinations

Agents frequently reference environment variables without verifying they exist. A generated function might read a secret from os.environ that was never set in the deployment environment, causing an immediate crash on startup.

# Generated code – assumes the env var exists
def get_api_key() -> str:
    return os.environ["API_KEY"]

If API_KEY is absent, this raises a KeyError with no context about whether the variable is misconfigured or genuinely missing. Replace the bare lookup with a helper that surfaces configuration errors clearly:

def get_api_key() -> str:
    key = os.environ.get("API_KEY")
    if key is None:
        raise RuntimeError("Configuration error: 'API_KEY' is not set. Check your environment variables.")
    return key

This pattern catches missing configuration at startup rather than failing later when the feature is actually used.

Integrate generated code into CI

Even well-checked generated code should be validated by the same pipeline as hand-written code. A minimal GitHub Actions workflow looks like this:

name: Validate Agentic Code
on: [push, pull_request]
jobs:
  validate:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: "3.11"
      - name: Install dependencies
        run: |
          python -m pip install --upgrade pip
          pip install -r requirements.txt
      - name: Run unit tests
        run: pytest
      - name: Run static type check
        run: mypy src/

The workflow installs the exact dependencies you declare, runs unit tests, and runs a static type checker. This catches missing imports and type mismatches before the code reaches production.

When agentic code still goes wrong

Even with guards, agentic output can introduce subtle bugs:

• Stale context – the model may reference variables that no longer exist in the current scope.
• Over-reliance – developers stop reviewing generated code, letting technical debt accumulate.
• Debugging difficulty – generated comments and variable names often lack clarity, making stack traces harder to interpret.

Mitigate these by treating generated code as a draft. Run a linter, enforce code-review checklists, and keep a log of which files were auto-generated.

Key takeaways

• Generated code can import packages that are missing in production; wrap imports in try-except blocks to surface the problem early.
• External API calls need rate limiting and circuit breaking; add a retry wrapper with exponential backoff.
• CI pipelines must validate agentic output with the same checks as hand-written code.
• Treat agentic code as a draft, not a final product, to avoid stale context and technical debt.
• Keep a record of auto-generated files and review them regularly to maintain code clarity.

Source

Why are good developers so wary of agentic development? – I added concrete code examples for import guards, retry wrappers, and environment-variable handling, plus a full CI workflow, to show how you can safely incorporate agentic output into production-ready systems.

Support this work

These write-ups are researched and published with no paywall, sponsor, or tracking. If one saved you an afternoon, a small tip keeps them coming.

USDT, USDC or USDD · TRC-20 (Tron)

TFTNsfyomKrnUutRjBTGVULp19ByW29KbY