DEV Community

aimEncoders
aimEncoders

Posted on

How We Ship 100% QA-Tested Websites: Our Playwright + CI Pipeline

#playwright #testing #qa #cicd

Every agency says their work is "tested." Most of them mean someone clicked around the homepage on their own laptop before the launch call.

We do it differently. At aimEncoders, nothing ships to a client until it passes an automated regression suite in CI — every release, every time. Here's the exact pipeline we run, why we built it this way, and the Playwright patterns that caught real bugs before our clients ever saw them.

Why we stopped trusting manual QA

Manual testing has three problems: it's slow, it's inconsistent, and it doesn't scale with the number of pages. A typical client site we ship has 20–40 pages, 5–10 forms, and a handful of API integrations. A human tester needs the better part of a day to cover that surface — and they'll still miss the regression where last Tuesday's "small CSS fix" broke the mobile checkout button.

Automated end-to-end tests flip that: the suite runs in minutes, covers the same paths identically every time, and — critically — it runs before merge, not after launch.

The pipeline: GitHub Actions + Playwright + Postman

Here's the shape of our CI workflow for a standard client website:

  1. Install & build — npm ci, then a production build. If the build breaks, nothing else runs.
  2. Unit & component tests — fast, run first, fail fast.
  3. Playwright E2E suite — against a preview deployment, covering critical user journeys.
  4. Postman API regression — every backend endpoint, run via Newman in CI.
  5. Lighthouse CI — performance, accessibility, and SEO budgets as hard gates.
  6. OWASP ZAP baseline scan — automated security pass on every release.

A release is "approved for production" only when all six stages are green. No exceptions, no "we'll fix it after launch."

The Playwright patterns that actually catch bugs

1. Test user journeys, not pages

Page-level tests ("does the homepage load?") catch almost nothing. Journey tests catch everything:

test('visitor can request a quote end-to-end', async ({ page }) => {
  await page.goto('/services');
  await page.getByRole('link', { name: /request a quote/i }).click();
  await page.getByLabel(/name/i).fill('Test User');
  await page.getByLabel(/email/i).fill('test@example.com');
  await page.getByLabel(/message/i).fill('Need a website for my restaurant.');
  await page.getByRole('button', { name: /send/i }).click();
  await expect(page.getByText(/we'll be in touch/i)).toBeVisible();
});
Enter fullscreen mode Exit fullscreen mode

This single test has caught: forms that submit into the void, success messages that never render, and redirects that drop the visitor on a 404. All bugs that "clicking around" missed.

2. Run the mobile viewport as a first-class citizen

Most client traffic is mobile, so half our suite runs at mobile viewports:

// playwright.config.ts
projects: [
  { name: 'desktop', use: { ...devices['Desktop Chrome'] } },
  { name: 'mobile', use: { ...devices['iPhone 14'] } },
],
Enter fullscreen mode Exit fullscreen mode

The number of bugs this catches — unreachable buttons, overlapping text, menus that won't open — is embarrassing for the industry and great for our clients.

3. Make flaky tests a build-breaker, not background noise

A flaky test is worse than no test: it trains the team to ignore red builds. Our rule is simple — a test that flakes twice gets quarantined and fixed within the sprint, or deleted. We use Playwright's retries only to distinguish infrastructure flakes from real failures, never to paper over them:

// playwright.config.ts
retries: process.env.CI ? 1 : 0,
Enter fullscreen mode Exit fullscreen mode

4. API contracts with Postman/Newman

Frontend tests tell you the UI works. They don't tell you the API still returns what the UI expects. We keep a Postman collection per project — auth flows, order endpoints, form submissions — and run it with Newman in the same pipeline:

newman run api-regression.json -e production-env.json --bail
Enter fullscreen mode Exit fullscreen mode

--bail matters: the first contract break stops the release. We'd rather block a deploy than debug a production incident.

What this costs vs. what it saves

Setting up this pipeline for a new project takes us about a day. It has, conservatively, saved us from shipping broken checkout flows, dead forms, and mobile layout disasters on nearly every project since we adopted it. One prevented production incident pays for the setup ten times over — in avoided emergency fixes, and in the client trust you never have to rebuild.

The takeaway

You don't need a QA department to ship tested software. You need a pipeline that refuses to ship untested software. Start with journey tests on your three most important user flows, run them in CI on every push, and make green builds a merge requirement. Everything else is refinement.

Your clients will never see the pipeline. They'll just notice that your launches are boring — and boring launches are the best kind.

Author bio:
Muhammad Asad Ullah is the founder of aimEncoders, a website development agency where every release passes an automated QA pipeline before it reaches the client. He writes about web development, testing, and shipping software that doesn't break at 2am.

Top comments (0)