The AI said it was done. It wasn't. Three times.

Edward Labuschagne — Wed, 08 Jul 2026 10:51:50 +0000

Last Tuesday, every production check on my app came back green. HTTP 200 on every route. Deploys clean. Tests passing. And any real visitor who opened the site saw a blank white page.

It had been that way since the first feature shipped. Four consecutive rounds of "production verified" claims, all technically true and all completely wrong, had sailed past it. What finally caught it wasn't me, and it wasn't the AI that built the app. It was a second AI whose entire job is to distrust the first one.

Let me back up.

I'm not a developer

I'm a system designer: twenty years in tech, plenty of architecture, and close to zero shipped code. AI coding tools changed that equation for me, the way they have for a lot of people. I now plan software in a chat interface and execute it through an agentic coding tool, and real products come out the other end.

But if you've done any serious AI-assisted building, you already know the actual problem. It isn't code quality. Modern models write decent code. The problem is that an AI implementer will confidently report a milestone as complete when it isn't. And it isn't lying, exactly. Its context is contaminated with "make this work". It checked what it thought to check, the checks passed, and it declared victory. Three milestones later, the fudged detail surfaces as a real bug in cold, expensive-to-fix context.

I call this "looks done but isn't", and after it burned me enough times, I stopped treating it as bad luck and started treating it as a structural problem with a structural fix.

The fix, in one paragraph

Forge is the methodology I built around one old engineering principle: the entity that built something is the wrong entity to confirm it works. You plan in chat, execute in a coding harness, and every milestone ends at a gate. At the gate, an independent Verifier (a separate AI instance in a completely fresh context, which has never seen the builder's reasoning or its "make this work" framing) runs a fixed eight-check spec against the actual diff, the test output, and captured observable outcomes: screenshots, stdout, generated files. Its brief is blunt: find the fudge, don't bless the work. Verdicts come with file-and-line citations or they don't count. No evidence, no pass.

That's the theory. Here's what it caught, on one small real build (an attendance portal for an online course, built over a weekend), in the order it caught it.

Catch #1: the screenshot that lied

Milestone 0 is project setup: repo, skeleton, toolchain, first deploy. The builder reported it done, including "deployed and live" with a screenshot of the deployed URL as evidence.

At the gate, the Verifier compared artefacts and found the "deployed URL" screenshot was byte-identical to the localhost screenshot from the local test run. Not similar. Identical. Whatever it showed, it was not evidence of a live deploy.

The claim got corrected in the commit history; the deploy was then actually verified with a request against the real URL and an origin assertion, and the build moved on. Total cost: minutes. The same defect discovered weeks later, after building on the assumption that deploys worked, would have cost far more than minutes.

This happened at milestone zero. Before a single feature existed, the "looks done but isn't" pattern had already shown up once.

Catch #2: HTTP 200, blank page

Three milestones later, the feature work was done and the gate was running its final checks. Every previous milestone had included a "production verified" step: hitting the live API routes and the deployed shell, confirming clean status codes and correct responses. All green, every time.

This time, the Verifier went one step further than curl. It loaded the actual production URL in headless Chromium with a JavaScript error listener attached.

The root <div> was empty. The page error read:

VITE_SUPABASE_URL and VITE_SUPABASE_ANON_KEY must be set

Independently confirmed: the deployed JS bundle contained that literal error string and zero occurrences of any of the app's own UI strings. The entire application was dead code, tree-shaken away because one module threw unconditionally on load, before anything else became reachable. Every real visitor since the first feature milestone had been getting a blank white page.

The root cause is a genuinely instructive one if you deploy SPAs to an edge platform. Frontend env vars are baked in at build time (import.meta.env.*), and the build happens on the host's build infrastructure when you push. That's a completely different variable scope from the runtime vars and secrets configured for the server-side worker, all of which had been set correctly, which is why every API check passed. Nobody had ever set the build-time pair. Two scopes, one dashboard, easy to conflate.

The fix took minutes once diagnosed: set the two build-time vars, trigger a rebuild. The re-verification is the part worth stealing. Not just "the page renders now", but a real browser load of every route with zero page errors, and a hash comparison confirming the deployed bundle was byte-identical to a local build of the same source, proving the host's build environment now matched local rather than hoping.

The one-line lesson, straight from the project's own handoff notes: a blank white page and a healthy 200 are indistinguishable to curl. If your "production verified" step doesn't execute the bundle in a browser, it's verifying your server, not your product.

Catch #3: the test that was smarter than manual testing

One more, because it's a different species: not a false claim, but a real bug that no amount of clicking around locally would have found.

The product's core feature is a personal join link sent by email. The link hits a server route that validates a signed token, records a check-in, and redirects. During end-to-end testing, page.goto('/join/<token>') rendered the app's login page instead of running the join logic, while direct API-style requests to the same path worked perfectly the whole time.

The cause: the hosting platform routes navigation requests to unmatched paths to the SPA fallback by default, and only non-navigation fetches to the worker. Which is exactly right for app pages, and exactly wrong for a link that is always a navigation, because it's clicked from an email. The feature would have worked in every API test and failed for every real user.

Even better: the first fix made things worse. Scoping "worker-first" routing to just the join path silently flipped the default for every other route to the SPA fallback: the API routes started returning HTML instead of JSON in local dev. The final fix was the unambiguous one (worker-first for everything, with explicit fallthrough to static assets), and every route category was re-verified individually before the test suite was trusted again.

The honest part

Here's what I refuse to do in this article: pretend the gate is magic.

That blank production page lived through three earlier gates. The methodology's own verification spec demands captured observable outcomes for every criterion, but those captures ran against local end-to-end tests, and the production checks were status codes. The gate had a blind spot, and the bug sat in it.

Forge's answer to that is a rule I stole from ordinary engineering practice: every real-world miss becomes a new check, the same discipline as writing a regression test for every bug. This miss is now a named verification requirement (production checks must include a real browser load with error listening) and a candidate case for the methodology's eval set.

Which brings up the part of Forge I get the most raised eyebrows about: the Verifier itself gets verified. The repo ships a ten-case eval set: planted defects the Verifier must FAIL (unmet criteria, doctored acceptance ledgers, scope violations, string-built SQL, tests deleted to hide a regression) and clean-but-plain code it must PASS, because a reviewer that flags everything is drifting just as badly as one that blesses everything. When the Verifier's model changes or a verdict smells wrong, the evals run. Trust, but grade.

And one more honesty note: the Verifier isn't flawless in the small, either. Earlier gates on this build produced citation-accuracy slips (right verdicts, sloppy line references), and the fix was procedural, not magical: instruct it to double-check its own references before writing them. Independent review is a mechanism, not a miracle.

What this is, if you want it

Forge is a model-agnostic methodology, not a tool: a small set of documents you drop into your planning chat and your coding harness. Plan first. Build one milestone at a time. Gate every milestone through an independent, fresh-context Verifier. Capture observable outcomes (screenshots, stdout, artefacts), because tests prove what someone thought to assert, not that the result is right. Mark every shortcut, budget the debt, and stop trusting "done".

It's MIT-licensed and public: https://github.com/edlabuschagne/project-forge

I built it because I'm exactly the person the failure mode preys on: someone who can't personally code-review the output, and therefore can't afford to take the builder's word for anything. If that's you too, or if you're a developer who's tired of discovering three milestones later that "done" wasn't, the docs are yours. I'd genuinely value the scrutiny. The gate would expect nothing less.

DEV Community: Edward Labuschagne