DEV Community

Cover image for Before You Deploy an AI-Built Website: A Developer's Launch Review
FlawPilot
FlawPilot

Posted on Originally published at flawpilot.com

Before You Deploy an AI-Built Website: A Developer's Launch Review

AI coding tools can take a prompt to a working UI in an afternoon. The preview looks finished, the happy path works, and it is tempting to ship.

But a preview proves one thing: the intended path works in one controlled environment. It says nothing about whether user A can read user B's data, whether a key is sitting in your JavaScript bundle, or whether staging's noindex made it to production.

This is the review we run before deploying generated code. It is not a threat model, pen test, or compliance audit. It is a way to catch the common launch gaps while they are still cheap to fix.

Block the release if any of these is true:

  1. A real secret reaches the browser or the repo history.
  2. A normal user can reach another user's data or an admin action.
  3. The core journey (signup, checkout, form) fails on the production URL.
  4. Production inherited a staging restriction (noindex, wrong canonical, password wall).
  5. Nobody can roll back or knows who is watching after launch.

Everything below is how to check these quickly.

1. Test the deployed journey, not the generated screens

Run the review against the production URL or an environment that mirrors it. Preview builds often differ in caching, asset paths, env vars, and domains.

Pick the one action the site exists for (contact form, signup, checkout, upload) and complete it end to end. A success toast is not proof. Confirm the data arrived, the email was delivered, and the right system received it.

Then break it on purpose:

  • Submit empty and malformed values
  • Reuse an expired token or old confirmation link
  • Reload mid-flow, then use the back button after completion
  • Upload a blocked or oversized file
  • Cancel or fail a payment in test mode

A failure should be clear and safe. It should never leak stack traces, internal paths, database errors, or tokens.

2. Test authentication and authorization separately

Authentication answers "who are you?" Authorization answers "what may you do?" A working login only proves the first.

Create at least two users with different roles, then try to cross the boundary:

# Logged in as user A, request a record owned by user B
curl -i -H "Authorization: Bearer $TOKEN\_A" \\
  https://example.com/api/invoices/<ID\_OWNED\_BY\_B>

# Expect 403 or 404. A 200 means broken access control.
Enter fullscreen mode Exit fullscreen mode

Also try an admin endpoint as a normal user, change a role value in a request body, and attempt a write with a read-only account.

Hiding a button in the UI does nothing if the endpoint is open. Enforce ownership on the server, in the query itself:

app.get("/api/invoices/:id", requireAuth, async (req, res) => {
  const invoice = await db.invoice.findFirst({
    where: { id: req.params.id, ownerId: req.user.id },
  });
  if (!invoice) return res.status(404).end();
  res.json(invoice);
});
Enter fullscreen mode Exit fullscreen mode

Multi-tenant apps need extra care: a user from one organization must not be able to read, search, or infer another organization's data.

Broken access control sits at the top of the OWASP Top 10, and generated code is a common place for it to hide.

3. Trace every secret from source to browser

If a value reaches the browser, assume anyone can read it. Frameworks differ on which env vars are bundled into client code, and generated examples do not always say.

Search your production build output, not just your source:

# Adjust the build folder for your framework (dist, build, .next/static, out)
grep -rEn "sk\_live|AKIA\[0-9A-Z]{16}|api\[\_-]?key|secret" dist/ 2>/dev/null

# Source maps can expose original code
find dist -name "\*.map"

# Was a key ever committed?
git log --all --oneline -S"sk\_live"
Enter fullscreen mode Exit fullscreen mode

Also review network requests in DevTools, build logs, error messages, and example config files.

If a real credential was exposed, rotate or revoke it. Deleting it from the latest commit does not remove it from Git history, caches, logs, or deployment artifacts. Give each key the minimum permissions it needs and restrict it by origin, environment, or quota where the provider allows.

4. Prune dependencies and third-party scripts

Generated projects collect packages and scripts across prompts, and some outlive the feature that needed them.

npm audit --omit=dev
npx depcheck
Enter fullscreen mode Exit fullscreen mode

Remove what is unused, review what is left, and apply upgrades in a branch with tests and a rollback path. Do not bulk-upgrade right before release.

For every third-party script on the deployed page (analytics, chat, ads, widgets), know who owns it, why it is there, and what data it receives.

5. Inspect the real production response

Your hosting platform, CDN, or proxy can change what the browser actually receives. Look at the live response:

# Headers on the final response
curl -sI https://example.com

# Redirect chain: http, www, and non-www should end on one preferred URL
curl -sIL http://example.com | grep -iE "^(HTTP|location)"
curl -sIL https://www.example.com | grep -iE "^(HTTP|location)"
Enter fullscreen mode Exit fullscreen mode

Check:

  • HTTPS everywhere, no mixed content
  • One preferred host, with short redirect chains
  • Cookie flags (Secure, HttpOnly, SameSite)
  • CORS rules that match your actual clients
  • Security headers: Content-Security-Policy, Strict-Transport-Security, X-Content-Type-Options, Referrer-Policy, and framing controls

Do not paste a strict CSP from another site and deploy it. It can block your own scripts, fonts, payments, or auth provider. Start in report-only mode, watch what it flags, then enforce:

Content-Security-Policy-Report-Only: default-src 'self'; script-src 'self'; img-src 'self' data:; report-to csp-endpoint
Enter fullscreen mode Exit fullscreen mode

The MDN CSP guide covers each directive.

If you use a CDN or proxy, also confirm the origin server is not directly reachable in a way that bypasses it.

6. Check what crawlers actually receive

Staging settings leak into production more often than you would expect.

# Robots directives and canonical on the live page
curl -s https://example.com | grep -iE "<meta\[^>]+robots|<link\[^>]+canonical"

# Header-level noindex
curl -sI https://example.com | grep -i x-robots-tag

curl -s https://example.com/robots.txt
Enter fullscreen mode Exit fullscreen mode

Verify for key pages: correct status code, no accidental noindex, a canonical pointing to the preferred production URL, unique title and description, one h1, crawlable internal links, and a sitemap with live URLs only.

For client-rendered apps, inspect the rendered HTML. If essential copy, links, or metadata only appear after heavy JavaScript runs, a crawler may not see them. Google's indexing docs explain how rendering and canonicals are handled.

Also check social previews: a correct canonical can still ship with an Open Graph image from a builder preview domain.

7. Test performance and accessibility on the real build

Measure the deployed build on a mobile profile and a constrained network, not your laptop.

npx lighthouse https://example.com \\
  --only-categories=performance,accessibility,seo \\
  --view
Enter fullscreen mode Exit fullscreen mode

Common causes of slow generated sites: oversized hero images, unused JavaScript, duplicate UI libraries, many font weights, animation packages for small effects, and blocking third-party widgets. Check loading, responsiveness, and layout stability, then complete the main task by hand. A good aggregate score can hide a form that still feels laggy.

Automated accessibility checks catch missing labels and contrast issues, but not the whole experience. The W3C WAI is explicit that tools cannot cover every requirement. Manually check:

  • Keyboard-only navigation, focus visibility, and focus order
  • Persistent form labels and readable error messages
  • Alt text on meaningful images
  • Zoom and reflow at small viewports
  • Reduced-motion behavior

8. Assign owners and make a release decision

A report fixes nothing by itself. Give every release-blocking issue one owner, one deadline, and one piece of evidence:

  • Critical journeys: product or QA, with a completed success and failure run
  • Access control and app behavior: developer, with role-based test results against the production build
  • Hosting, DNS, TLS, CDN: platform owner, with verified live configuration
  • Indexing and metadata: SEO or marketing, with live URL inspection
  • Analytics and forms: marketing ops, with confirmed events and delivery

One person can own several rows. What matters is that ownership is explicit.

Then sort findings by action rather than a scanner's severity label:

  • Block the release: exposed production credentials, broken authorization, public sensitive data, unsafe payment or account behavior, a critical journey that cannot be completed
  • Fix before public traffic: broken forms, accidental indexing blocks, wrong redirects, severe mobile problems, no monitoring on a critical service
  • Schedule and track: lower-impact hardening and cosmetic issues, each with an owner and a date
  • Investigate: unclear, duplicated, or environment-specific findings. Confirm the affected asset before changing production

After every fix, retest on the live environment. A merged PR does not prove your CDN, cache, DNS provider, or pipeline delivered the change.

9. Plan the first hours after deploy

Before release, decide who watches errors and uptime, who checks form and email delivery, who can roll back, and where launch issues are logged. Know how to restore from backup. Keep the domain registrar, DNS, hosting, repo, and email accounts under team control rather than one person's login. Re-run this review after any change to the domain, hosting, framework, auth, analytics, or integrations.

Automating part of this

Most of the checks above can be scripted or run with free tools: curl, Lighthouse, npm audit, OWASP ZAP, and axe. Several hosted scanners also cover the public-surface checks (security headers, performance, SEO, infrastructure) in one pass.

Disclosure: I work on one of them, FlawPilot. Whatever you use, keep the limits in mind. A public scan cannot sign in as each role, inspect private business logic, or judge accessibility in full, and source scanning does not replace runtime testing. Treat automated results as inputs to the release decision, not permission to deploy.

Final checklist

  • [ ] Production-like URL tested, primary journey works end to end
  • [ ] Failure and cancellation paths fail safely
  • [ ] Authentication and authorization tested separately, with multiple roles
  • [ ] No secrets in the bundle, source maps, logs, or Git history
  • [ ] Dependencies and third-party scripts reviewed
  • [ ] HTTPS, redirects, cookies, CORS, and headers checked on the live response
  • [ ] Important pages indexable, with correct canonicals and metadata
  • [ ] Performance tested on the production build under mobile conditions
  • [ ] Accessibility tested automatically and manually
  • [ ] Every open issue has an owner and a release decision
  • [ ] Monitoring and rollback responsibilities are clear
  • [ ] Critical fixes verified after deployment

AI changes how fast you can build. It does not change what production has to withstand.

What is the worst thing you have found in an AI-generated codebase before launch? Curious what others check that is missing from this list.

Top comments (0)