DEV Community: Dmitry

Flaky Tests You Can't Fix With Better Selectors

Dmitry — Tue, 02 Jun 2026 16:47:04 +0000

You've fixed your locators. You've switched to web-first assertions. Your tests still fail intermittently — but now the failures look different. Duplicate records in the database. Tests that pass alone but fail in parallel. Mocks that say everything is fine while production is broken.

This is the next layer of flakiness. It lives in your API calls, your test doubles, and your database. Better selectors won't help here.

Code examples are simplified for clarity — focus on the idea, not the boilerplate.

TL;DR

Use idempotency keys on POST requests — one network glitch shouldn't create two orders
page.route mocks the browser, not your server — know the difference
Use WireMock for server-to-server integrations you can't control
Contract tests catch API drift before it reaches production
Never rely on afterEach for database cleanup — use TTL or a cleanup queue instead

The Problem: Flakiness That Looks Like Application Bugs

When a selector fails, the error is obvious. When a test creates a duplicate order because a network request was retried, the error looks like a business logic bug. You spend an hour investigating something that has nothing to do with your application code.

Three categories cause this:

API flakiness — a request succeeds on the server but the response never arrives. Playwright retries. Now you have two orders.

Lying mocks — your mocks say the API returns { order_id: "123" }. The backend deployed last week and now returns { orderId: "123" }. Tests are green. Production is broken.

Data pollution — tests create users, orders, and transactions but don't clean up reliably. After a week, the test database is a graveyard that slows down queries and causes unique constraint violations.

Rule #1: Idempotency Keys — One Request, One Result

Networks are unreliable. A POST request can reach the server, create a record, and then the response gets lost in transit. Playwright sees a timeout and retries. The server sees a new request and creates another record.

The fix is an idempotency key — a unique header that tells the server "if you've seen this request before, return the same result instead of processing it again."

import { createHash } from 'crypto';

export function generateIdempotencyKey(
  method: string,
  url: string,
  data: unknown,
  testId: string,
): string {
  // Mixing testId ensures that parallel workers using identical factory data won't collide
  const payload = `${testId}:${method}:${url}:${JSON.stringify(data)}`;
  return createHash('sha256').update(payload).digest('hex').slice(0, 16);
}

// api/clients/BaseApiClient.ts
import { test } from '@playwright/test';

export abstract class BaseApiClient {
  protected async post(url: string, data?: unknown) {
    // Fetch the unique identifier of the currently running test
    const testId = test.info().testId;
    const key = generateIdempotencyKey('POST', url, data, testId);

    return await this.request.post(url, {
      data,
      headers: { 'X-Idempotency-Key': key },
    });
  }
}

The key is generated from the request method, URL, and body — so two identical requests get the same key, but two different requests (create user, then create order) get different keys. One network glitch no longer creates two records.

Important nuance: if your test legitimately creates two identical orders (same body, same URL), they'll get the same key — and the server will return the first result for both. This is intentional behaviour for retries, but it means this approach assumes each unique operation has unique data. If you need two genuinely identical records, add a unique field (like requestId) to the body.

Note: This only works if your backend handles the X-Idempotency-Key header. Check with your backend team — many order APIs support this out of the box. If your payment provider doesn't support it — that's their problem to solve, not yours. Look for their sandbox or test mode, or use WireMock to mock them entirely. If it's your own backend that's missing support — that's a tech-debt conversation with your backend team, not something to work around in tests.

Rule #2: Know What Your Mocks Actually Cover

page.route is Playwright's built-in way to intercept requests. It's great for testing UI behavior in isolation — how does the page look when the API returns an error?

// Good use of page.route — testing UI error state
await page.route('**/api/orders', (route) => {
  route.fulfill({
    status: 500,
    body: JSON.stringify({ error: 'Internal Server Error' }),
  });
});

await page.goto('/orders');
await expect(page.getByText('Something went wrong')).toBeVisible();

The catch: page.route only intercepts network traffic generated inside the browser instance (like clicks or fetch/XHR requests from the page script). If your test bypasses the UI and makes direct API calls via Node.js—for instance, using Playwright’s backend request fixture to seed data before the test—page.route will completely ignore them.

Why route.fulfill() instead of route.abort()? abort() causes the request to fail with a network error. Some apps handle this gracefully, but others enter an infinite retry loop waiting for a response that never comes. fulfill() returns a proper HTTP response (even a fake one) so the app moves on cleanly.

For direct API calls in tests, you need mocks at a different level — either a wrapper around request, or an infrastructure mock like WireMock.

Rule #3: WireMock for Integrations You Don't Control

Your backend calls Stripe for payments. It calls Twilio for SMS. It calls a shipping provider to get rates. In tests, you don't want any of that to actually happen.

page.route can't help here — these are server-to-server calls that never touch the browser. The solution is WireMock: a mock server that runs alongside your test environment and intercepts HTTP calls at the network level.

# docker-compose.yml
services:
  wiremock:
    image: wiremock/wiremock:3.3.1
    ports:
      - '8080:8080'
    volumes:
      - ./wiremock/mappings:/home/wiremock/mappings

// wiremock/mappings/stripe-payment.json
{
  "request": {
    "method": "POST",
    "url": "/v1/payment_intents"
  },
  "response": {
    "status": 200,
    "jsonBody": {
      "id": "pi_test_123",
      "status": "succeeded"
    }
  }
}

Now your backend hits localhost:8080 in tests instead of the real Stripe API. Tests are fast, isolated, and don't depend on external uptime.

Point your backend's base URLs to WireMock via environment variables in your test environment:

STRIPE_BASE_URL=http://localhost:8080
TWILIO_BASE_URL=http://localhost:8080

One thing to be aware of: this requires your backend to use configurable base URLs for external services. In most well-structured backends this is already the case. If it's not — that's a conversation with the backend team, not a reason to skip WireMock.

Rule #4: Contract Tests — Stop Trusting Your Mocks

Here's the problem with all mocks: they can lie. Your WireMock returns { payment_id: "pay_123" }. The backend team renames the field to paymentId. Your tests stay green. Production breaks.

This is called a Lying Mock — a test double that no longer matches reality.

Contract testing fixes this. Instead of just mocking the response, you write a contract: "I expect this request to return this response." The backend then verifies that contract against its actual code.

// tests/contracts/payment.pact.spec.ts
import { PactV3, MatchersV3 } from '@pact-foundation/pact';

const provider = new PactV3({
  consumer: 'frontend-tests',
  provider: 'payment-service',
  dir: './pacts',
});

describe('Payment API contract', () => {
  it('returns payment confirmation', async () => {
    await provider
      .given('a valid payment intent exists')
      .uponReceiving('POST /v1/payment_intents')
      .withRequest({ method: 'POST', path: '/v1/payment_intents' })
      .willRespondWith({
        status: 200,
        body: {
          id: MatchersV3.string('pi_test_123'),
          status: MatchersV3.string('succeeded'),
        },
      })
      .executeTest(async (mockServer) => {
        const result = await createPayment(mockServer.url);
        expect(result.status).toBe('succeeded');
      });
  });
});

After this test runs, it generates a JSON contract file in ./pacts. The backend team runs that contract against their actual API. If they rename id to paymentId — the contract verification fails in their pipeline, before the change is merged.

Where to start: You don't need to contract-test everything. Start with the APIs that change most often or have caused the most incidents. One contract on your payment flow is worth more than ten contracts on stable read-only endpoints.

One important caveat: contract testing requires the backend team to actually run the verification in their pipeline. This is an organizational commitment, not just a technical one. For small teams, storing contract JSON files in the backend repo and running verification manually is a simpler starting point than a full Pact Broker setup. Also note that contracts verify response structure — they don't catch business logic bugs or side effects.

Rule #5: Stop Relying on `afterEach` for Cleanup

The classic approach to test data cleanup:

// Unreliable
afterEach(async () => {
  await api.deleteUser(userId);
});

This fails silently when a test crashes before setting userId. It doesn't run when the test runner itself crashes. After a CI failure mid-run, your database has orphaned records that affect the next run.

Three approaches that actually work:

TTL — let the database clean up automatically

Add an expires_at field to your test entities and set it when creating them:

// When creating test data
await api.createUser({
  email: `test_${Date.now()}@example.com`,
  expires_at: new Date(Date.now() + 24 * 60 * 60 * 1000).toISOString(),
});

In PostgreSQL, a scheduled job handles cleanup:

-- Runs every hour via pg_cron
SELECT cron.schedule('cleanup-test-data', '0 * * * *', $$
  DELETE FROM users WHERE expires_at < NOW() AND is_test = true;
  DELETE FROM orders WHERE expires_at < NOW() AND is_test = true;
$$);

In MongoDB and Redis, TTL indexes handle this natively — no cron job needed.

Cleanup queue — collect IDs, delete in bulk

Track everything your tests create, then clean it all up in a global teardown:

// In your base API client
protected async post(url: string, data?: unknown) {
  const response = await this.request.post(url, { data });
  const body = await response.json();

  if (body.id) {
    cleanupQueue.push({ url, id: body.id });
  }
  return response;
}

// global-teardown.ts
export default async function globalTeardown() {
  for (const item of cleanupQueue) {
    await api.delete(`${item.url}/${item.id}`);
  }
}

Even if individual tests fail, the global teardown runs and cleans up the queue.

Which approach to use: TTL is the more reliable default — it works even if the test runner is killed with SIGKILL, because cleanup happens at the database level independently of your test process. Use TTL as your first line of defence. The cleanup queue is a good complement when you need guaranteed cleanup order or when your database doesn't support scheduled jobs — but it won't run if the process is hard-killed.

Putting It Together: The Data Reliability Cheat Sheet

Symptom	Root cause	Fix
Duplicate records after CI failure	No idempotency on POST requests	Add `X-Idempotency-Key` header
Tests green, production broken	Mocks don't match real API	Add contract tests for critical endpoints
`page.route` mock not working	Request bypasses browser	Use WireMock or request wrapper
Database full of test garbage	`afterEach` cleanup unreliable	TTL field + pg_cron or cleanup queue
External API causing flakiness	Real network calls in tests	WireMock for server-to-server calls

What's Next?

You now have three layers covered: test infrastructure, object lifecycle, and data reliability. The next layer is observability — how do you measure test health, identify patterns in flakiness, and prove to your manager that stability work has business value?

Want to go deeper on any of these topics? Check out the advanced version: Why Your Test Suite Lies to You at Scale

All patterns in this article are implemented in the Playwright BDR Template on GitHub.

Your Playwright Tests Will Need Refactoring. Here's How to Make It Painless

Dmitry — Tue, 26 May 2026 15:48:07 +0000

You write 50 tests. Everything works. Six months later the team grows, tests become 300, and someone changes a constructor — and you spend two days updating imports across the entire project. Sound familiar?

This isn't a discipline problem. It's an architecture problem. And it's fixable before it happens.

Code examples are simplified for clarity — focus on the idea, not the boilerplate.

TL;DR

Never instantiate Page Objects with new inside tests — use fixtures
Use getters instead of constructor assignments in Page Objects
Seed your test data with a combination of testId + RUN_ID + repeatEachIndex for reproducibility
Split fixtures by domain when the file gets large — use mergeTests
Use Namespacing to avoid silent fixture name collisions

What Is a Flow? (Quick Explainer)

Before we dive in — this article uses the term Flow, which might be unfamiliar.

In a well-structured Playwright project, tests are built in three layers:

Page Object (POM) — knows how to interact with elements on a specific page: find a button, fill a field, click a link
Flow — knows how to complete a business scenario: "checkout", "register a user", "reset a password". It orchestrates Page Objects in the right sequence so tests don't have to
Test — just calls the Flow and checks the result

So when you see checkoutFlow.submitOrder() in a test, that one line is hiding a sequence of page navigations, form fills, and button clicks — all managed by the Flow. The test doesn't need to know the details.

The Problem: Architecture That Fights You at Scale

At 50 tests, messy architecture is invisible. At 300 tests, it becomes expensive. Two separate problems compound each other:

Data isolation breaks in parallel runs. Two workers create a user named "Ivan", one test reads the other's data, both fail. You spend an hour debugging something that has nothing to do with your application. This is a data seeding problem — solved in Rule #3.

Refactoring takes days instead of hours. Someone changes a constructor signature. Now you have 150 files to update. With modern tools this is still risky — you might miss one. This is a dependency management problem — solved in Rule #1.

Tests are impossible to read. Ten lines of setup before the actual test logic. New team members can't tell what's being tested and what's just noise. This too is a dependency management problem — when setup lives in fixtures, tests read like specifications.

Rule #1: Stop Using `new` Inside Tests

This is the most common pattern that makes refactoring painful:

// Every test manages its own dependencies
test('checkout', async ({ page }) => {
  const cartPage = new CartPage(page);
  const checkoutPage = new CheckoutPage(page);
  const checkoutFlow = new CheckoutFlow(cartPage, checkoutPage);

  await checkoutFlow.submitOrder();
});

If CartPage needs a new dependency tomorrow — a logger, a config object, an API client — you update every single test that creates it. That's your two days of refactoring.

The fix: fixtures as a DI container

// fixtures.ts — one place to manage all object creation
export const test = base.extend({
  cartPage: async ({ page }, use) => {
    await use(new CartPage(page));
  },

  checkoutFlow: async ({ cartPage, checkoutPage }, use) => {
    await use(new CheckoutFlow(cartPage, checkoutPage));
  },
});

// The test reads like a specification
test('checkout', async ({ checkoutFlow }) => {
  await checkoutFlow.submitOrder();
});

When CartPage constructor changes — you update fixtures.ts. One file. Done.

Why fixtures even when Flow seems stateless today:

Your CheckoutFlow might be pure today — no state, no side effects. But requirements change. Tomorrow it needs to track an order ID. Next month it opens a WebSocket connection that needs to be closed after the test.

If Flow is created via new in every test, adding teardown means updating hundreds of files. If it's in a fixture, you add after use cleanup in one place:

checkoutFlow: async ({ cartPage, checkoutPage }, use) => {
  const flow = new CheckoutFlow(cartPage, checkoutPage);
  await use(flow);
  await flow.cleanup(); // added in one place, applies everywhere
};

The upfront investment is real — a few hours to set up fixtures properly. The cost of refactoring later: days, proportional to how many tests you have.

A note on pragmatism: Fixtures are for managing state and lifecycle. If you have a stateless utility function — like formatDate or a math helper — don't wrap it in a fixture. A simple ES6 import is faster and less complex. Use fixtures for things that hold a page context or require setup/teardown. Everything else is just a function.

Rule #2: Use Getters in Page Objects, Not Constructor Assignments

This is subtle but important. Most tutorials show this:

// Locator computed once at construction time
class CartPage {
  private submitButton: Locator;

  constructor(page: Page) {
    this.submitButton = page.locator('button#submit');
  }
}

This looks fine. Playwright locators are lazy — they don't query the DOM at construction time, they query it when you interact with them. So assigning a locator in the constructor is technically safe.

The real danger of constructor assignments is that they bloat your files with boilerplate. You have to declare a private property at the top, and then tie it to a locator inside the constructor.

The fix: lazy getters
We use getters purely for code cleanliness and boilerplate reduction. A getter combines property declaration and initialization into a single line, keeping your Page Objects strictly declarative.

// Fresh locator on every access, clean and concise
class CartPage {
  constructor(private page: Page) {}

  get submitButton() {
    return this.page.getByRole('button', { name: 'Place order' });
  }

  get cartItems() {
    return this.page.locator('.cart-item');
  }

  // For computed state, keep it as an explicit async method
  async getItemCount(): Promise<number> {
    return this.cartItems.count();
  }
}

This forces the Page Object to focus on mapping human-readable elements to fresh locators, leaving no room for accidental state caching during initialization.

Rule #3: Isolate Test Data for Parallel Runs

When you run 1000 tests in parallel across multiple CI shards, data collisions are inevitable — unless you design against them.

The common mistake is using workerIndex as a seed for test data. It seems logical: each worker gets a unique number, so data should be unique. The problem is that workerIndex resets per shard. On 10 parallel CI agents, each has its own "Worker 0". Collisions are guaranteed.

The fix: combine test identity with CI build ID — not worker index

// utils/faker.utils.ts
import { TestInfo } from '@playwright/test';
import { faker } from '@faker-js/faker';

function hashCode(str: string): number {
  const hash = str.split('').reduce((acc, char) => {
    return (Math.imul(31, acc) + char.charCodeAt(0)) | 0;
  }, 0);
  return Math.abs(hash);
}

export function seedFaker(testInfo: TestInfo) {
  const RUN_ID = process.env.RUN_ID || 'local';
  const seed = hashCode(`${testInfo.testId}-${RUN_ID}-${testInfo.repeatEachIndex}`);
  faker.seed(seed);
  return faker;
}

// fixtures.ts
export const test = base.extend({
  faker: async ({}, use, testInfo) => {
    await use(seedFaker(testInfo));
  },
});

Three components in the seed:

testId — unique hash of the test file path and test name
RUN_ID — the CI build ID (e.g. GITHUB_RUN_ID), so different builds get different data
repeatEachIndex — handles retries correctly

Note: RUN_ID is an environment variable provided by your CI system — for example, GITHUB_RUN_ID in GitHub Actions. If it's missing, the code falls back to 'local', so everything works on your machine without any extra setup.

The payoff: when a test fails in CI, grab the RUN_ID from the pipeline logs, run the test locally with the same ID, and you get the exact same names, emails, and UUIDs that were generated in CI. Reproducible failures instead of "I can't reproduce this locally."

Rule #4: Structure Test Data With Factories and Overrides

Random data everywhere creates noise. If a field doesn't affect the test outcome, it shouldn't be visible in the test.

// user.factory.ts — sensible defaults
export function createUser(overrides?: Partial<User>, f = faker): User {
  return {
    id: f.string.uuid(),
    email: f.internet.email(),
    name: f.person.fullName(),
    role: 'customer',
    ...overrides,
  };
}

// In the test — only what matters
test('VIP discount applies at checkout', async ({ checkoutFlow, faker }) => {
  const user = createUser({ role: 'vip', discount: 0.15 }, faker);
  await checkoutFlow.asUser(user).applyPromo();
});

The test declares intent, not implementation. When you read it, you know exactly what's being tested: VIP role and discount. Everything else — name, email, UUID — is noise that the factory handles.

For data that represents specific business cases and appears repeatedly, extract it as a named dataset:

// data/datasets/users.ts
export const VIP_USER = { role: 'vip', discount: 0.15 } as const;

// In tests
const user = createUser({ ...VIP_USER }, faker);

Pro tip: Use the satisfies operator (TypeScript 4.9+) instead of as const for datasets. It ensures your data matches the User type without losing the specific literal values — catching type errors before you even run the test:
export const VIP_USER = {
  role: 'vip',
  discount: 0.15,
} satisfies Partial<User>;
If someone adds a required field to User and forgets to update the dataset, TypeScript will tell you at compile time, not at runtime.

Rule #5: Scale Fixtures With `mergeTests` and Namespacing

One fixtures.ts file is fine at the start. At 20+ fixtures it becomes a 400-line file that multiple people edit simultaneously.

Split by domain:

// auth.fixtures.ts
import { test as base } from '@playwright/test';
import { LoginPage } from '../pages/LoginPage';

export const authTest = base.extend<{ loginPage: LoginPage }>({
  loginPage: async ({ page }, use) => {
    await use(new LoginPage(page));
  },
});

// cart.fixtures.ts
import { test as base } from '@playwright/test';
import { CartPage } from '../pages/CartPage';

export const cartTest = base.extend<{ cartPage: CartPage }>({
  cartPage: async ({ page }, use) => {
    await use(new CartPage(page));
  },
});

// fixtures.ts — merge everything
import { mergeTests } from '@playwright/test';
import { authTest } from './auth.fixtures';
import { cartTest } from './cart.fixtures';

export const test = mergeTests(authTest, cartTest);

Tests don't change at all — they still import from fixtures.ts. The split is purely organizational.

Watch out for name collisions:

If auth.fixtures.ts and cart.fixtures.ts both define a fixture called user, Playwright won't warn you. The last one wins silently. This creates subtle bugs that are very hard to track down.

The fix is namespacing — group fixtures by domain:

// No collision possible
import { test as base } from '@playwright/test';
import { Admin } from '../pages/Admin';
import { User } from '../pages/User';

export const test = base.extend<{ auth: { admin: Admin; user: User } }>({
  auth: async ({ page }, use) => {
    await use({
      admin: new Admin(page),
      user: new User(page),
    });
  },
});

// In tests
test('admin can manage users', async ({ auth }) => {
  await auth.admin.login();
  await auth.user.register();
});

Rule #6: Write Business Steps, Not Technical Logs

If you use Allure or any step-based reporter, the quality of your step descriptions determines how useful the report is.

The native Playwright way is test.step():

// Technical log — describes implementation
async login() {
  await test.step('Click the login button', async () => {
    await this.page.getByRole('button', { name: 'Login' }).click();
  });
}

// Business intent — describes what happened
async loginAs(user: User) {
  await test.step(`Authenticate as "${user.username}"`, async () => {
    await this.loginPage.login(user.username, user.password);
  });
}

The first version breaks when you rename the button. The second version remains valid even if the entire login mechanism changes from a form to SSO. The report reads like a scenario, not a DOM manipulation log.

In BDR methodology we use a @Step decorator instead of wrapping every method manually — same result, cleaner syntax. If you're interested in that approach, check it out.

ESLint: Enforce the Architecture Automatically

The best rule is one that doesn't require a code review comment:

// .eslintrc.js
module.exports = {
  overrides: [
    {
      // Only applies inside test files — won't flag Page Object factories or helpers
      files: ['tests/**/*.ts', '**/*.spec.ts'],
      rules: {
        'no-restricted-syntax': [
          'error',
          {
            selector: 'NewExpression[callee.name$=Page]',
            message: 'Use fixtures instead of new for Page Objects. See fixtures.ts.',
          },
          {
            selector: 'NewExpression[callee.name$=Flow]',
            message: 'Use fixtures instead of new for Flow objects. See fixtures.ts.',
          },
        ],
      },
    },
  ],
};

Scoping to tests/** prevents false positives — new Pagination() in your app code won't trigger this. Only new LoginPage() inside test files will.

Architecture Cheat Sheet

Symptom	Root cause	Fix
Refactoring takes days	`new PageObject()` in every test	Move to fixtures
Parallel tests corrupt each other's data	`workerIndex` as seed	Seed with `testId` + `RUN_ID`
Can't reproduce CI failures locally	Non-deterministic test data	Seeded faker fixture
fixtures.ts is 400 lines	No domain separation	`mergeTests` + domain files
Fixture collision, wrong object used	Flat fixture namespace	Namespace by domain
Report is unreadable	Technical step descriptions	`test.step()` with business intent (or `@Step` in BDR)

What's Next?

This architecture handles the object lifecycle and data isolation. The next layer is async reliability — expect.poll, idempotency keys for parallel API calls, and cleaning up test data without relying on afterEach.

Want to go deeper? Check out the advanced version: Playwright Architecture at Scale: What Senior Engineers Do Differently

All patterns in this article are implemented in the Playwright BDR Template on GitHub.

Why Your Playwright Tests Fail in CI (And Never Locally)?

Dmitry — Wed, 20 May 2026 17:28:20 +0000

You run your tests locally — everything is green. You push to CI — three tests fail. You run CI again — different three tests fail. Sound familiar?

This isn't bad luck. It's a set of fixable architectural mistakes. In this guide I'll walk you through the six rules that eliminated flakiness in our test suite. No magic, no "just increase the timeout" advice.

All code examples are simplified for clarity — focus on the idea, not the boilerplate.

TL;DR

Use Dependency Projects instead of globalSetup — if the environment is down, stop immediately instead of running 1000 failing tests
Locator priority: getByRole > getByLabel > getByTestId. CSS selectors — last resort only
Never use isVisible() in assertions — it's a snapshot. Use Web-first assertions that wait
Block analytics and tracking scripts with page.route — they cause networkidle to hang
Trace Viewer is your debugging tool. Screenshots show you what, traces show you why
Always authenticate via API, not UI — 50ms vs 5 seconds, per test

Why CI breaks tests that pass locally

Your local machine is fast. CI is not. Less CPU, higher latency between services, multiple parallel processes all competing for resources. Asynchronous problems exist locally too — a powerful machine and fast network just hide them. When conditions get slightly worse, timings fall apart.

This is why "works on my machine" is such a common story in test automation.

Rule #1: Stop Running Tests in a Vacuum

When your staging environment goes down at night, do you want to run 1000 tests just to get 1000 failures? Of course not. But that's exactly what happens without a proper dependency chain.

The solution: Dependency Projects

Instead of one big globalSetup file, build a dependency graph in your Playwright config:

// playwright.config.ts
export default defineConfig({
  projects: [
    // Step 1: Authenticate and save session
    {
      name: 'auth-setup',
      testMatch: /.*\.auth\.setup\.ts/,
    },
    // Step 2: Check if the environment is actually alive
    {
      name: 'healthcheck',
      testMatch: /.*\.health\.setup\.ts/,
      dependencies: ['auth-setup'],
    },
    // Step 3: Only run real tests if steps 1 and 2 passed
    {
      name: 'chromium',
      use: { ...devices['Desktop Chrome'] },
      dependencies: ['healthcheck'],
    },
  ],
});

If auth fails or the environment is down — Playwright stops immediately. No wasted CI minutes, no flood of useless alerts.

Why not globalSetup?

globalSetup gives you dry logs when something fails. Dependency Projects give you full Trace Viewer support — you can see exactly what happened during setup: network requests, screenshots, console errors. And you can run just one project in isolation: npx playwright test --project=auth-setup.

Rule #2: Authenticate via API, Not UI

UI login is slow. A full page load with all assets and rendering takes 2–5 seconds. An API login call takes 50–100ms. At CI scale, this difference adds up fast.

More importantly: you shouldn't be testing your login form 500 times. Test it once, in a dedicated test. For everything else, just reuse the session.

// auth.setup.ts
test('authenticate', async ({ request }) => {
  // Direct API call — no browser rendering needed
  await request.post('/api/login', {
    data: { username: 'user@example.com', password: 'secret' },
  });

  // Save cookies and storage state for all other tests
  await request.storageState({ path: '.auth/user.json' });
});

Then in your config:

use: {
  storageState: '.auth/user.json',
}

Every test now starts already authenticated. Zero UI login overhead.

Rule #3: Use the Right Locators — and Know Why

A locator isn't just a way to find an element. It's a statement about what your test actually cares about. The wrong locator makes tests brittle. The right locator makes failures meaningful.

Why getByRole is the default choice

getByRole finds elements by their semantic role in the accessibility tree — button, heading, link, dialog. This matters because role is tied to behavior, not implementation. A CSS class can be renamed, a DOM structure can be refactored — but if the element is still a button, getByRole still finds it.

One important nuance: getByRole often takes a { name: '...' } parameter to narrow down which element you mean. That name comes from the button's text or aria-label. If you rely on visible text and the app is multilingual — that name changes per locale, and your locator breaks. The role survives translation. The name doesn’t. In multilingual apps, visible text and aria-labels change per locale, making getByRole tricky unless you pass a localized string variable into the name parameter. If managing localized strings inside tests is too painful, getByTestId becomes your fallback.

There’s a bonus: if getByRole can’t find your element, it often means the element has no semantic role — which is an accessibility bug. Your test is catching a real problem.

// Finds the button regardless of CSS class or DOM structure
await page.getByRole('button', { name: 'Place order' }).click();

Why getByLabel for form fields

getByLabel finds inputs by their associated label text. The label is a contract between the UI and the user — if it changes, that's a UX change worth knowing about. This locator also catches cases where a field exists but has no label — another real bug.

await page.getByLabel('Email address').fill('user@example.com');

When getByTestId is the right answer

getByTestId is stable but semantically blind — it finds the element regardless of its role, text, or visual state. That's a feature in specific situations:

Ant Design, Material UI, or other component libraries — these generate DOM structures where a single Select or Combobox contains multiple elements with the same role: a hidden native input, a trigger button, a text field. getByRole('combobox') picks the first one in DOM order, which is often not the one you need to interact with — and it can change between library versions
Multi-language apps — button text changes per locale; getByTestId doesn't care
A/B tests or personalization — the label varies per user variant
Icon buttons without text — SVG icons with no aria-label

// Stable regardless of language or variant
await page.getByTestId('checkout-button').click();

The tradeoff: getByTestId passes even if the button is visually broken, hidden by styles, or inaccessible to screen readers. You're trading semantic coverage for stability. That's a conscious choice, not a default.

The decision algorithm

Try getByRole first — if the element has a semantic role, this is always better
If text is dynamic (translations, A/B) — pass a localized string variable into the { name } option.
If managing localized strings in tests is too painful or the element has no stable role — use getByTestId without guilt.

// Both of these use getByRole — role is stable
await page.getByRole('button', { name: 'Place order' }).click();
await expect(page.getByRole('heading')).toHaveText('Order confirmed');

// Both of these use getByTestId — text is dynamic
await page.getByTestId('checkout-button').click();
await expect(page.getByTestId('order-status')).toHaveText('Confirmed');

Rule #4: Stop Using `isVisible()` in Assertions

This is one of the most common sources of flakiness. Here's why:

// This checks visibility at this exact millisecond
const isVisible = await page.getByRole('button').isVisible();
expect(isVisible).toBeTruthy();

If the page is still loading at that millisecond — the test fails. Not because something is broken, but because you asked too early.

Web-first assertions wait for you:

// This polls the DOM until the condition is true (or timeout)
await expect(page.getByRole('button')).toBeVisible();

The difference: expect(locator).toBeVisible() keeps checking every ~100ms until the element appears or the timeout is reached. It's a built-in retry loop.

Quick reference:

Instead of this	Use this
`await loc.isVisible()`	`await expect(loc).toBeVisible()`
`await loc.textContent() === '...'`	`await expect(loc).toHaveText('...')`
`await loc.count()`	`await expect(loc).toHaveCount(3)`
`await loc.isChecked()`	`await expect(loc).toBeChecked()`
`await loc.isEnabled()`	`await expect(loc).toBeEnabled()`

One exception: isVisible() is fine inside conditional logic — for example, to decide whether to close a cookie banner before continuing. Just don't use it as a final assertion.

Rule #5: `waitForTimeout` is not a solution — here's what to use instead

If you feel the urge to add waitForTimeout — stop. In 95% of cases there's a better tool. The question is which one.

Use web-first assertions (toBeVisible, toHaveText, toHaveURL, etc.) when:

An element appears or disappears after a click
The URL changes after navigation
Text updates after data loads
A form shows a validation error
Anything that is visible in the UI

This covers the vast majority of cases. Web-first assertions have built-in retry — you don't need anything else.

// Built-in retry — no polling needed
await expect(page.getByText('Order confirmed')).toBeVisible();
await expect(page).toHaveURL('/dashboard');

Use expect.poll when:

A background job updated order status in the DB, and the UI only shows a spinner
A payment webhook arrived from Stripe or PayPal and updated the payment status
A message was processed from a queue (Kafka, RabbitMQ) by another service

The common pattern: you clicked something, the UI shows nothing useful (or just a spinner), but something should have happened behind the scenes. You can only verify it via a direct API call.

// Background job updated order status — not visible in UI
await expect
  .poll(
    async () => {
      const response = await request.get(`/api/orders/${orderId}`);
      const order = await response.json();
      return order.status;
    },
    {
      message: 'Waiting for order status to become PAID',
      timeout: 30_000,
    },
  )
  .toBe('PAID');

Use expect.toPass when:

You need to click a button repeatedly until the UI shows the expected result
An action needs to be repeated until a condition is met

// Click Refresh until status appears in UI
await expect(async () => {
  await page.getByRole('button', { name: 'Refresh' }).click();
  await expect(page.getByText('Status: Ready')).toBeVisible();
}).toPass({
  intervals: [1_000, 2_000, 5_000],
  timeout: 15_000,
});

Warning: If you find yourself writing expect.poll more than once or twice per test file — stop and reconsider. Either the UI is missing proper loading indicators, or the architecture needs rethinking. expect.poll is a last resort, not a default tool.

Rule #6: Block Analytics and Tracking Scripts

Your app loads Google Analytics, a support chat widget, maybe a heatmap tool. These services are slow, sometimes unreliable, and completely irrelevant to what you're testing. They also interfere with networkidle waits.

Block them:

// In your fixture or beforeEach
await page.route(/google-analytics\.com|intercom\.io|hotjar\.com/, (route) => {
  // Use fulfill instead of abort so the app doesn't hang waiting for a response
  route.fulfill({ status: 200, body: 'ok' });
});

Watch out for fonts: Blocking external fonts can cause layout shifts, which may trigger Playwright's stability checks and slow things down. Either allow fonts through or make sure your app handles missing fonts gracefully.

Rule #7: Use Trace Viewer, Not Screenshots

When a test fails in CI, a screenshot shows you what the page looked like. Trace Viewer shows you why it failed.

A screenshot: a frozen image of a page that looks fine.

Trace Viewer: every action, every network request, every console error, the DOM state before and after each step — all in a timeline you can scrub through.

Enable it in your config:

// playwright.config.ts
use: {
  // Only save traces when tests fail — keeps your artifacts small
  trace: 'retain-on-failure',
  screenshot: 'only-on-failure',
}

What to look for in Trace Viewer:

Actionability tab: If a click didn't work, this tells you exactly which element was blocking it (a loading skeleton, an overlay, a tooltip)
Network tab: See which API calls were slow or failed
Console tab: See JavaScript errors that don't show up in your test output
Snapshots: The actual DOM state at each step — you can open DevTools on a past moment in time

When a test fails because a button was "covered by another element" — Trace Viewer shows you the exact element, with a red dot on the snapshot. No guessing required.

Hydration: Why Clicks Sometimes Do Nothing

If you work with React, Next.js, Vue, or Nuxt — you've probably seen this: Playwright clicks a button, no error is thrown, but nothing happens.

This is hydration. The server sends HTML that looks like a working page, but the JavaScript hasn't loaded yet. The button exists in the DOM but has no event listeners. Playwright clicks it, the click lands, and nothing responds.

The fix: Wait for a signal that the app is ready before interacting:

// Wait for a loading indicator to disappear
await expect(page.locator('#global-loader')).toBeHidden();

// Or wait for a class that your app adds when hydration is complete
await page.waitForSelector('.app-ready', { state: 'attached' });

About force: true:

You might be tempted to use force: true to bypass Playwright's checks. Before you do, understand what you're skipping. Playwright's actionability checks verify that an element is:

Visible — not hidden by CSS or outside the viewport
Stable — not moving (animations, transitions)
Enabled — not disabled or read-only
Receiving events — not covered by another element like a modal or overlay

When you add force: true, all four checks are disabled. You're no longer testing what a real user experiences — you're manipulating the DOM directly. The test passes, the user is still stuck.

There is one legitimate exception: hidden file inputs (<input type="file">). Browsers render this element as a native, hard-to-style button. Developers often intentionally hide it (make it invisible) and draw a custom button on top, consistent with the rest of the design. In such cases, Playwright cannot interact with the hidden element without force: true.

// force: true required — file input is visually hidden by design,
// replaced by a styled button that triggers it
await page.locator('input[type="file"]').setInputFiles('file.pdf', { force: true });

For everything else — find the root cause. If an element is covered, wait for the overlay to disappear. If it's disabled, wait for the enabled state. force: true without a comment is a red flag in code review.

ESLint: Let the Robot Enforce the Rules

Don't explain these rules in every code review. Automate it:

// .eslintrc.js
module.exports = {
  extends: ['plugin:playwright/recommended'],
  rules: {
    'playwright/no-wait-for-timeout': 'error', // No sleeps
    'playwright/no-focused-test': 'error', // No test.only in commits
    'playwright/no-page-pause': 'error', // No page.pause() in commits
    'playwright/prefer-web-first-assertions': 'error', // Nudge toward better assertions
    'playwright/no-force-option': 'warn', // Flag force: true usage
  },
};

error for things that definitely break your tests or CI. warn for architectural debt that's worth addressing but not blocking.

One more thing: rules exist to be broken consciously. If you're working with a component library that generates dynamic selectors you can't control, // eslint-disable-next-line is sometimes the honest answer. The key word is consciously — disable the rule, write a comment explaining why, and move on. What you want to avoid is blanket disables that hide real problems.

Migration Cheat Sheet: Old Playwright vs Current

If you're coming from Selenium or older Playwright patterns, here's the direct translation:

What you used to do	What to do now	Why
`page.$()`, `page.$$()`	`getByRole()`, `getByLabel()`, `getByTestId()`	Lazy evaluation + automatic retry on assertions
`waitForSelector()`	Not needed — built into actions	Playwright waits for actionability before every click/fill
`waitForTimeout(3000)`	`expect(loc).toBeVisible()`	Polls until ready instead of guessing
`waitForNavigation()`	`await expect(page).toHaveURL('/dashboard')`	`toHaveURL` has built-in polling, no race condition
`isVisible()` in assertions	`expect(loc).toBeVisible()`	One is a snapshot, the other waits
`console.log('HERE')`	Trace Viewer	Full timeline with network, DOM, console — in CI

Flakiness Cheat Sheet

Symptom	Likely cause	Fix
Click lands, nothing happens	Hydration	Wait for app-ready signal
Timeout in CI, passes locally	Slow network / analytics	Block third-party scripts
Selector not found after deploy	Fragile CSS / text changed	Use `data-testid` or `getByRole`
Random failures, no pattern	Race condition in assertions	Switch to Web-first assertions
All tests fail at once	Environment down	Add healthcheck dependency

What's Next?

These six rules cover the most common sources of flakiness. Once you have them in place, the next level is async handling at scale — expect.poll, idempotency keys, contract testing, and data hygiene.

Want to go deeper into the architecture? Check out the advanced version of this guide: Playwright CI: What Senior Engineers Do Differently

All patterns in this article are implemented in the Playwright BDR Template on GitHub — clone it and see how everything fits together.

Nobody reads your test reports. Here's how I re-engineered them with a 3-layer architecture

Dmitry — Thu, 07 May 2026 17:50:56 +0000

Note: BDR (Behavior-Driven Living Requirements) is my own architectural approach to organizing Playwright tests — a Cucumber-free alternative to BDD that I designed and documented at bdr-methodology.dev.

Monday morning. Coffee. You open GitLab — and CI is red. Classic.

You open the report. There's a wall of text, five screens long. Somewhere in there: TimeoutError on a click. The selector looks fine — data-testid="checkout-submit". But why did it fail? Was the database down? Did the frontend not render the button? Did some API return an unexpected response?

To find out, you have to dive into the test code and debug it line by line. Mentally reconstruct what the app state was. Read through fifty lines of setup just to understand what was being tested.

This is the real cost of unreadable test reports. Not the failure itself — but the hour you spend just figuring out what failed and why.

The classic POM: looks clean, reports terribly

Most teams start here. You write a clean Page Object:

import { Page } from '@playwright/test';

export class CartPage {
  constructor(private readonly page: Page) {}

  async clickCheckout() {
    await this.page.getByTestId('checkout-submit').click();
  }
}

The code looks great. Clean, atomic, no logic in the wrong place.

But the report? It looks like this:

✓ Test: User can complete purchase
  - clickCheckout
  - fillDetails
  - submit

How do you understand the context from that in five seconds? You can't. The developer opens the test code, reads through it, swears, mentally reconstructs what was happening. Time gone.

"Just use test.step everywhere" — don't do this

Someone will suggest: "Just wrap everything in test.step, what's the problem?"

Don't. It works for three tests. At a hundred, it kills the project.

Copy-paste will destroy you. The login → cart → checkout chain ends up in most test files. Login logic changes? Congratulations, you're editing fifty files by hand.

Maintenance becomes a nightmare. Checkout now requires a "agree to terms" checkbox? Go insert await page.click(...) in a hundred places.

Tests lose their meaning. A ten-line test balloons to fifty lines of await test.step(...) noise. The actual business intent disappears behind the boilerplate.

The fix: a Flow layer between POM and tests

The solution is a layer between "dumb" pages and tests. But here's the key insight most teams miss: a Flow is not just a reusable helper. It's a business entity.

Think of an e-commerce app. You have three distinct business actions:

Adding a product to the cart — a self-contained business event
Placing an order — another self-contained business event
Processing payment — yet another

Each of these deserves its own Flow class. Not because of DRY (though that's a nice side effect), but because each one represents a real business concept with its own rules and responsibilities.

Then your Spec just assembles them like Lego:

// Scenario 1: full happy path
await cart.addProduct(laptop);
await checkout.placeOrder(address);
await payment.pay(card);

// Scenario 2: just verify cart behaviour
await cart.addProduct(laptop);
await cart.verifyTotal(1200);

Same building blocks, different scenarios. The Spec doesn't care how "add product" works internally — it just uses the business entity.

This distinction has a real consequence. If the business process for checkout changes from one screen to three, your test remains the same:

await checkoutFlow.completePurchase(orderData);

You change the implementation inside the Flow, but the test — the business intent — stays untouched. That's the difference between a brittle script and a resilient test framework.

A Flow is a conductor — it knows nothing about selectors or clicks. It only knows about the business process.

export class CheckoutFlow {
  constructor(
    private cartPage: CartPage,
    private paymentPage: PaymentPage,
  ) {}

  async completePurchase(orderData: OrderData) {
    await test.step('WHEN: User proceeds to checkout', async () => {
      await this.cartPage.clickCheckout();
      // High-level POM check, keeping the Flow clean from raw elements
      await this.paymentPage.verifyPaymentFormIsVisible();
    });

    await test.step('WHEN: User fills payment details', async () => {
      await this.paymentPage.fillDetails(orderData.card);
      await this.paymentPage.submit();
    });
  }
}

Now the report looks like this:

✓ Test: User can complete purchase
  ✓ WHEN: User proceeds to checkout
  ✓ WHEN: User fills payment details
  ✓ THEN: Order confirmation is displayed

Test failed? The developer opens the report. Thirty seconds — and they know exactly which business step broke. No code diving required.

Why three layers — and what breaks if you skip one

This is the part most teams skip. They add a Flow layer but let the boundaries blur. A month later, everything is tangled again.

Here's why each layer exists and what happens when you violate it:

POM knows about selectors. Nothing else.
If your POM starts containing multi-page business logic — “click checkout, wait for navigation, and pre-fill the shipping form” — you’ve coupled UI mapping with business rules. A single Page Object should never orchestrate user journeys across multiple entities.

Flow knows about business processes. Nothing about selectors.
If your Flow starts calling page.getByTestId(...) directly, you've lost the separation that makes refactoring safe. Now a selector change requires touching both the POM and the Flow.

Spec knows about intent. Nothing about implementation.
Your test should read like a user story. If it's full of .fill() and .click() calls, a non-engineer can't read it — and you've lost the "living documentation" value entirely.

The rule: each layer talks only to the layer directly below it. Spec → Flow → POM. Never skip a level.

What the report becomes

With this architecture, your Allure report stops being a log of browser actions and becomes a record of business events.

When a test fails, the report answers three questions immediately:

What was being tested (the test name)
Where it broke (the step name)
What the state was (attached tables with data)

That's the difference between a report that developers ignore and one they actually use.

Try it

This architecture is the foundation of BDR — Behavior-Driven Living Requirements.

BDR Methodology — full architecture docs
Playwright BDR Template — working implementation to clone

I'm open to QA Automation roles — remote, contract, or full-time.
dmitryAQA@outlook.com | @DmitryMeAQA

DEV Community: Dmitry

Flaky Tests You Can't Fix With Better Selectors

TL;DR

The Problem: Flakiness That Looks Like Application Bugs

Rule #1: Idempotency Keys — One Request, One Result

Rule #2: Know What Your Mocks Actually Cover

Rule #3: WireMock for Integrations You Don't Control

Rule #4: Contract Tests — Stop Trusting Your Mocks

Rule #5: Stop Relying on afterEach for Cleanup

Putting It Together: The Data Reliability Cheat Sheet

What's Next?

Your Playwright Tests Will Need Refactoring. Here's How to Make It Painless

TL;DR

What Is a Flow? (Quick Explainer)

The Problem: Architecture That Fights You at Scale

Rule #1: Stop Using new Inside Tests

Rule #2: Use Getters in Page Objects, Not Constructor Assignments

Rule #3: Isolate Test Data for Parallel Runs

Rule #4: Structure Test Data With Factories and Overrides

Rule #5: Scale Fixtures With mergeTests and Namespacing

Rule #6: Write Business Steps, Not Technical Logs

ESLint: Enforce the Architecture Automatically

Architecture Cheat Sheet

What's Next?

Why Your Playwright Tests Fail in CI (And Never Locally)?

TL;DR

Why CI breaks tests that pass locally

Rule #1: Stop Running Tests in a Vacuum

Rule #2: Authenticate via API, Not UI

Rule #3: Use the Right Locators — and Know Why

Rule #4: Stop Using isVisible() in Assertions

Rule #5: waitForTimeout is not a solution — here's what to use instead

Rule #6: Block Analytics and Tracking Scripts

Rule #7: Use Trace Viewer, Not Screenshots

Hydration: Why Clicks Sometimes Do Nothing

ESLint: Let the Robot Enforce the Rules

Migration Cheat Sheet: Old Playwright vs Current

Flakiness Cheat Sheet

What's Next?

Nobody reads your test reports. Here's how I re-engineered them with a 3-layer architecture

The classic POM: looks clean, reports terribly

"Just use test.step everywhere" — don't do this

The fix: a Flow layer between POM and tests

Why three layers — and what breaks if you skip one

What the report becomes

Try it

Rule #5: Stop Relying on `afterEach` for Cleanup

Rule #1: Stop Using `new` Inside Tests

Rule #5: Scale Fixtures With `mergeTests` and Namespacing

Rule #4: Stop Using `isVisible()` in Assertions

Rule #5: `waitForTimeout` is not a solution — here's what to use instead