DEV Community

Cover image for AI doesn't write bad code. It writes plausible code — so I tried to break my own AI-built app
FavCRM
FavCRM

Posted on • Originally published at microservices.sh

AI doesn't write bad code. It writes plausible code — so I tried to break my own AI-built app

#ai #cloudflare #typescript #aiagents

Disclosure: I work on one of the tools in this post (create-microservices-app). But the experiment, commands, and outputs below are real, and the pattern at the end works no matter what stack you're on — that's the part I actually want you to take.

If you ship with Claude Code, Cursor, or Codex, you know the feeling. The agent gets you 70% of the way in minutes. It compiles. The diff looks reasonable. You merge it.

And then there's the quiet doubt: did it actually get the hard 30% right — auth boundaries, payments, tenant isolation, the booking logic that stops two people taking the same slot? Because AI doesn't usually write obviously bad code. It writes plausible code. And plausible-but-wrong is the expensive kind — it passes review and breaks in production on day three.

(The data backs the doubt: 84% of devs use AI tools, only 29% trust the output, and 45% of AI-generated apps ship an exploitable vulnerability — Veracode, 2025.)

So I ran an experiment: build a real app with an agent, then deliberately make the mistake an agent makes every day, and see what — if anything — catches it.

Build the app (one command) 

npm create microservices-app@latest booking-demo -- --template booking-sveltekit
Enter fullscreen mode Exit fullscreen mode

A full Cloudflare SvelteKit booking app — public flow, admin, D1, auth. The detail that matters for this experiment: it ships its own contract into the repo — README.agent.md, docs/api-boundary.md, and an executable spec, microservices.check.mjs. The layering rule is one line: routes are thin adapters; domain logic lives in verified modules, not in your handlers.

Baseline:

$ microservices check
Template checks: pass
Enter fullscreen mode Exit fullscreen mode

Now break it the way an agent would

The request an agent gets constantly: "simplify the bookings endpoint." So I did the eager-agent thing — inlined the write straight to the DB and dropped the module:

// src/routes/api/bookings/+server.ts — the "simplified" version
export const POST: RequestHandler = async ({ request, locals }) => {
  const body = await request.json();
  await locals.bookingRepository.insert({
    serviceId: body.serviceId,
    startsAt: body.startsAt,
    customerId: body.customerId
  });
  return json({ ok: true });
};
Enter fullscreen mode Exit fullscreen mode

It type-checks. It runs. It would pass review. And it silently drops the slot-conflict guard the verified createBooking use case enforced — a double-booking waiting to happen. Classic plausible-but-wrong.

Then I ran the check:

$ microservices check
Error: One or more generated app checks failed.

$ microservices check --json
FAIL: spec:src/routes/api/bookings/+server.ts
      — Booking API route stays a thin adapter over createBooking and injected repositories.
Enter fullscreen mode Exit fullscreen mode

It named the exact file and the exact contract I broke — not a vague lint warning, but "you bypassed the verified booking use case." Restore the delegation to the module, and:

$ microservices check
Template checks: pass
Enter fullscreen mode Exit fullscreen mode

Green. The slot-conflict protection is back where it belongs.

The pattern (this is the part that's yours, tool or not)

Forget my tool for a second — the transferable idea is this:

The fix for plausible-but-wrong isn't a smarter model. It's a boundary your agent can't cross without a named, machine-readable failure.

Three moves you can apply on any stack:

  1. Push the dangerous 30% behind a real boundary — auth, payments, domain integrity in a module/package with a tested API, not regenerated inline every time.
  2. Write a contract check that asserts the boundary held — "this route only calls the use case," "this handler imports the verified module." A few assertions beat a thousand eyeballed diffs.
  3. Put that check in the agent loop — run it after every edit. Let the agent move fast on the safe 70% (the adapter/UI layer); make the 30% fail loudly when it's touched.

You can roll this yourself with a test file and a grep. I happen to ship it as a contract + check for Cloudflare apps — but the move is the move.

What I verified vs. what you'd run

I ran the scaffold → contract → check → break → fix loop above for real. The parts that need your own machine — npm install, npm run dev, a deploy — are yours to run; I'm not going to claim outputs I didn't produce:

npm create microservices-app@latest booking-demo -- --template booking-sveltekit
cd booking-demo && npm install
npm run microservices -- check     # the gate — wire it into your agent loop
npm run dev
Enter fullscreen mode Exit fullscreen mode

(If you ship apps for clients on Cloudflare, the same gate is what lets you hand the result to a security review without the 2am call — but that's a different post.)

Repo + the rest of the modules: https://microservices.sh

Genuinely curious: how are you keeping your agent from quietly rewriting the dangerous 30%? Contract tests, review checklists, just vibes? What's caught a plausible-but-wrong change for you — and what slipped through?

Top comments (2)

Collapse
 
alexshev profile image
Alex Shev •

Plausible code is exactly the right failure mode to test against. AI-generated apps often pass the happy path because the surface looks coherent, but the real risk is permission checks, weird state transitions, and error paths nobody asked the model to imagine. Trying to break the app is the only honest review step.

Collapse
 
fastanchor_io profile image
FastAnchor_io •

Totally agree with this take. AI code looks perfectly reasonable at first glance, but it hides all kinds of edge-case flaws and logical loopholes that only surface under real-world stress testing. Breaking AI-generated apps has basically become part of my regular QA workflow now.