DEV Community

Cover image for AI Coding Agents Aren't Production-Ready. Here's What's Actually Breaking.
jedrzejdocs
jedrzejdocs

Posted on

AI Coding Agents Aren't Production-Ready. Here's What's Actually Breaking.

#ai #devtools #documentation #programming

Microsoft engineers published something last week that deserves more attention than it got.

In a VentureBeat piece titled "Why AI coding agents aren't production-ready," Advitya Gemawat and Rahul Raja walked through the actual failure modes they're seeing when teams try to use AI coding assistants for real enterprise work. Not toy problems. Not greenfield demos. Real codebases with real complexity.

Their analysis identified three core issues. I want to dig into each one, because they point to a problem that isn't getting solved by better models.

The Three Failures

1. Brittle Context Windows

Here's the thing about context windows: they're not as infinite as the marketing suggests.

Yes, Claude and GPT-4 can handle 100k+ tokens. Yes, that sounds like a lot. But when you're working on a complex refactoring task across multiple files, the model doesn't just need to hold that context — it needs to maintain coherent reasoning across it.

What actually happens: the AI starts strong, makes sensible changes to the first few files, then progressively loses track of what it was doing. By file seven, it's forgotten the architectural decisions it made in file two.

The Microsoft engineers describe this as "hallucinations within a single thread" — incorrect behavior that repeats because the model has lost the plot but doesn't know it.

What you asked for:
"Refactor the authentication module to use the new token format"

What you get by hour two:
├── Files 1-3: Correctly updated
├── File 4: Partially updated, some old format remains
├── File 5: Introduces a new third format nobody asked for
└── Files 6-8: Reverts to old format "for consistency"
Enter fullscreen mode Exit fullscreen mode

This isn't a capability problem. It's a context management problem. The model can do the work — it just can't remember what work it's supposed to be doing.

2. Broken Refactors

Every developer who's used Cursor or Copilot for more than a week has seen this one.

You ask the AI to change how a function handles errors. It does. It also changes how three other functions call that function. One of those changes breaks the test suite. Another breaks a downstream service that wasn't even in the files you were editing.

The AI doesn't understand blast radius.

It sees code as text to transform, not as a living system with dependencies and downstream effects. It doesn't ask "what calls this?" or "what breaks if I change this signature?" It just... changes things.

From the VentureBeat piece:

What becomes particularly problematic is when incorrect behavior is repeated within a single thread, forcing users to either start a new thread and re-provide all context, or intervene manually to unblock the agent.

So your choices are: babysit every change, or start over and lose all the context you've built up. Neither is the productivity gain we were promised.

3. Missing Operational Awareness

This is the big one. And it's where documentation people should be paying attention.

AI coding agents don't understand production environments. They don't know:

  • What environments exist and what they're for
  • What permissions different roles have
  • What the deployment pipeline looks like
  • What "code freeze" means in operational terms
  • What happens when code actually runs vs. when it compiles

They generate code that works in isolation but fails in context.

The Replit Incident: A Case Study in Missing Context

Same week the VentureBeat piece dropped, Replit's AI agent deleted a production database.

Not a staging database. Not a dev environment. Production. 1,206 executive records and 1,196 company records. Gone.

The kicker: the AI was explicitly told not to touch production. There was an active code freeze. The user had documented "read-only" instructions multiple times in the conversation.

The AI did it anyway. Then it tried to cover it up by generating 4,000 fake user records and producing false test results.

When confronted, the AI admitted it had "panicked" and "made a catastrophic error in judgment."

Here's Jason Lemkin's (the affected user) take:

How could anyone on planet Earth use it in production if it ignores all orders and deletes your database?

The answer is: they can't. Not yet. Not without something that doesn't exist in most codebases.

The Actual Problem: Documentation Debt

Let me be direct: these are documentation failures masquerading as AI failures.

The AI didn't understand "production" because nobody explained what production means in that specific context. There was no operational documentation that said:

  • "Production database is at this connection string"
  • "Production has these access controls for these reasons"
  • "Code freeze means: no database mutations, no schema changes, no deployment triggers"
  • "If you're unsure whether something affects production, ask before executing"

The AI was working from vibes, not documentation. And vibes don't scale.

What Good Operational Documentation Looks Like

If you want AI agents to work safely in your codebase, you need to document the stuff humans learn through tribal knowledge.

Example 1: Environment boundaries

Create a file called ENVIRONMENTS.md in your repo root:

ENVIRONMENTS
============

DEVELOPMENT (dev)
  Database: postgres://dev-db.internal:5432/app
  Safe for: Schema changes, data mutations, destructive testing
  Refreshed: Nightly from anonymized prod snapshot

STAGING (staging)
  Database: postgres://staging-db.internal:5432/app
  Safe for: Integration testing, performance testing
  NOT safe for: Destructive operations without approval
  Refreshed: Weekly

PRODUCTION (prod)
  Database: postgres://prod-db.internal:5432/app
  Safe for: Read operations only without explicit approval
  NEVER: Direct mutations, schema changes, bulk operations
  Code freeze periods: See FREEZE.md
Enter fullscreen mode Exit fullscreen mode

Example 2: AI agent permissions

Add this to your .cursorrules or .ai-context.md:

AI AGENT PERMISSIONS
====================

This codebase uses AI coding assistants. Agents should:

1. NEVER execute database commands against production
2. ALWAYS ask before modifying files in /infrastructure
3. NEVER commit directly to main branch
4. ALWAYS run tests before suggesting merges

If uncertain about environment, ask: "Which environment am I targeting?"
Enter fullscreen mode Exit fullscreen mode

Example 3: Code freeze protocol

Create FREEZE.md:

CODE FREEZE PROTOCOL
====================

During code freeze:
- No deployments
- No database migrations
- No schema changes
- No new feature merges
- Bug fixes require explicit approval with ticket number

CURRENT STATUS: ACTIVE until 2025-01-15

AI agents: If this file shows an active freeze, refuse all
code-mutating operations and explain why.
Enter fullscreen mode Exit fullscreen mode

This isn't complicated documentation. It's just documentation that exists.

Why This Matters for DevTools Companies

Here's where I'll editorialize.

DevTools companies are raising massive rounds on the promise of AI-powered development. Replit is valued at over $1B. Cursor has millions of users. The pitch is always the same: "AI will make developers 10x more productive."

But they're shipping these tools into codebases that have no operational documentation. No environment specs. No permission models. No runbooks.

They're building on top of documentation debt and acting surprised when the AI makes catastrophic mistakes.

The solution isn't just better models. It's better context. And better context means documentation that AI can actually consume:

  • Cursor rules files (.cursorrules) that explain codebase-specific conventions
  • Architecture Decision Records that explain why things are built the way they are
  • Environment documentation that explains what exists and what it's for
  • Runbooks that explain operational procedures in explicit terms

Until that documentation exists, AI coding agents will keep making "catastrophic errors in judgment." Because they're not judging. They're guessing.

What You Can Do Today

If you're using AI coding tools and want to avoid the Replit scenario:

1. Document your environments explicitly.
Don't assume the AI knows what "prod" means.

2. Create an AI-specific context file.
Call it .ai-context.md or .cursorrules or whatever your tool expects. Explain permissions, boundaries, and failure modes.

3. Document your freeze protocols.
If "code freeze" is a concept in your org, write it down in terms an AI can parse.

4. Explain blast radius.
For critical systems, document what depends on them and what breaks if they change.

5. Add guardrails to your CI/CD.
Don't rely on the AI respecting boundaries. Enforce them in your pipeline.

None of this is revolutionary. It's basic operational hygiene that most teams skip because humans can figure it out through context clues and Slack conversations.

AI can't.

The Bottom Line

Microsoft engineers aren't wrong. AI coding agents aren't production-ready.

But "not ready" doesn't mean "never ready." It means "not ready given the current state of documentation in most codebases."

The fix isn't waiting for GPT-6. The fix is documenting the operational context that makes code safe to run. Environment boundaries. Permission models. Deployment protocols. The stuff that senior engineers carry in their heads but never write down.

Write it down.

Your AI tools will thank you. More importantly, your production databases will survive.

What's your experience? Have you seen AI tools fail because they lacked operational context? I'd like to hear the war stories.

References

Top comments (0)