DEV Community: Anton Gulin

Let's give an AI agent its first testing task

Anton Gulin — Sat, 10 Oct 2026 15:01:20 +0000

Originally published on anton.qa.

You can learn a lot from one small testing task. You need a rule, some code, and a clear stopping point.

Here is a practice task for Claude Code. It checks PDF filenames. You will ask Claude to write tests and run them. Then you will inspect one failure. You will stop before asking for a code fix.

That last step matters. A failing test can be the result you asked for. It shows whether the test found the intended mistake.

What you need

Use a scratch folder that contains only this exercise. You need a terminal and Node.js 24. Check your version with node --version.

You also need Claude Code and a supported account. Follow the official Claude Code quickstart for installation and sign-in. Then check claude --version. Installation and sign-in were not tested in this exercise.

The complete files are below. This example needs no additional packages.

You can also copy the complete public example.

Make the helper

Create a new folder and enter it:

mkdir first-agent-task
cd first-agent-task

Save this as pdf-filename.mjs:

export function isPdfFilename(name) {
  return typeof name === 'string' && name.endsWith('.pdf');
}

The intended rule is simple. Accept filenames ending in .pdf, regardless of letter case. Reject other endings. The helper contains one deliberate mistake: endsWith('.pdf') rejects report.PDF.

An extension is a file ending, such as .pdf. The helper checks the name alone. It does not open a PDF or validate its contents.

Give Claude one bounded task

Start Claude Code inside the scratch folder with claude. Its quickstart shows that command. Paste this prompt into the session:

Read pdf-filename.mjs. The requirement is: accept a filename whose final
extension is .pdf, ignoring letter case. Reject other endings.

Write pdf-filename.test.mjs using Node's built-in node:test runner.
Test invoice.pdf, report.PDF, notes.txt, report.pdf.exe, and an empty name.
Do not change pdf-filename.mjs or install packages.

Run node --test pdf-filename.test.mjs. Report the command, pass and fail counts, and failing case.
Include exact expected and actual values. Then stop.
Do not fix the implementation in this task.

The prompt names the file, rule, test cases, command, and stopping point. It asks for an observable result. Anthropic recommends giving the agent a check it can run. Ask it to show the output.

Read Claude's proposed file change before accepting it. Keep the work inside the scratch folder. If it asks to change the helper, restate the stopping point. After the run, check both filenames. The helper should still match the original. The new test file should hold five cases.

What the test file could look like

Claude may write the tests differently. This complete file gives you a comparison. Save it as pdf-filename.test.mjs if you want to run the example yourself:

import test from 'node:test';
import assert from 'node:assert/strict';
import { isPdfFilename } from './pdf-filename.mjs';

test('accepts a lowercase PDF extension', () => {
  assert.equal(isPdfFilename('invoice.pdf'), true);
});

test('accepts an uppercase PDF extension', () => {
  assert.equal(isPdfFilename('report.PDF'), true);
});

test('rejects a text file', () => {
  assert.equal(isPdfFilename('notes.txt'), false);
});

test('rejects a suffix after .pdf', () => {
  assert.equal(isPdfFilename('report.pdf.exe'), false);
});

test('rejects an empty name', () => {
  assert.equal(isPdfFilename(''), false);
});

Run node --test pdf-filename.test.mjs from that folder. An assertion compares the actual value with the expected value. The runner marks a mismatch as a failure.

In the isolated Node 24.18.0 sample run, four tests passed and one failed. The report.PDF case expected true and received false. Node exited with code 1. The lowercase and rejection cases passed. That is the intended result for this stage.

This example proves the sample test detects the sample mistake. No live Claude Code session was run for this example. Claude's output may differ. Compare its test file and command output with the rule above.

Read the failure before fixing anything

Look for the failing case name and the values false !== true. Check that the failed test points to report.PDF. Then open pdf-filename.mjs and find the case-sensitive ending check.

Also read the passing tests. invoice.pdf should pass. notes.txt, report.pdf.exe, and the empty name should return false. If another case fails, inspect that result before treating the exercise as complete. If all five pass, check whether the helper changed. This task asked for tests only.

Save the command output before starting another task. It records which case failed and what the helper returned. You can compare a later fix against that result.

Now stop. You have a small test that explains a real gap. A separate task can fix the helper and rerun the same tests. Keeping those steps separate makes the first result easier to judge.

I'm Anton. I build AI tools and measure how fast they really run. Former Apple engineer. I make ollamatps.com and write about building and measuring AI at anton.qa.

Should your Playwright page object use a getter?

Anton Gulin — Sat, 03 Oct 2026 15:01:14 +0000

Originally published on anton.qa.

A page object groups reusable actions for part of your app. For example, a checkout page object might expose its Save button.

Should it store that button's locator in a field? Or should a getter create the locator when you need it?

A locator describes how to find an element. That distinction makes the choice easier.

A stored locator can find a new button

Imagine a page that redraws its Save button after an edit. The old button disappears, and a new button takes its place.

A stored locator can still find the new matching button. Playwright finds the current element each time you use the locator for an action. Its locator documentation explains this behavior.

You aren't storing the original button itself. You're storing the description Playwright uses to find it.

For a fixed button, a field can be enough. A field stores a value on your page object.

This illustrative JavaScript fragment assigns a locator when the page object is created:

constructor(page) {
  this.save = page.getByRole('button', {
    name: 'Save', exact: true
  });
}

The getByRole call describes the button by its role and name. Later, this.save.click() finds the current matching button and clicks it.

This fragment belongs inside a class. It isn't a complete test or an executed demonstration.

Playwright's page object example uses stored locator fields too.

A getter changes when you build the locator

A getter runs a function when you read a property. It can return a locator each time you ask for it.

Here is the equivalent illustrative fragment:

constructor(page) {
  this.page = page;
}

get save() {
  return this.page.getByRole('button', {
    name: 'Save', exact: true
  });
}

Reading this.save runs the getter. Calling this.save.click() then uses the returned locator.

Both fragments describe the same button on the same page. The getter rebuilds that description on each access. The field retains the description created earlier.

Neither requires the button to exist when you create the locator. The page redraw alone doesn't make the getter necessary.

MDN's getter reference explains the underlying language behavior.

Changing search values need a separate decision

Now imagine a page object that stores the currently selected product name. You build a locator using that name, then change the name later.

The stored locator still contains the value used when you created it. Finding the current element doesn't mean rereading your page object's other properties.

A getter can build a fresh locator from the current name. That can be useful when the page object's state drives the search.

I'd make the input visible in the method call. For example, buyButtonFor('Notebook') tells the reader which product the test means.

That's a design preference, not a Playwright requirement. A getter can be clear when the changing state is already obvious.

Keep the choice small

For a fixed Save button, I'd start with a stored locator. When the search depends on an input, I'd consider a method accepting that input.

If reading current page-object state is the goal, a getter may fit. Keep the chosen pattern consistent enough for another person to follow.

There are limits to this explanation. A locator won't repair a changed button name or an ambiguous match. Both versions still need a valid page and a correct description of the target.

Before changing an existing page object, identify the behavior you want to improve. A redraw by itself doesn't require rewriting stored locators as getters.

I'm Anton. I build AI tools and measure how fast they really run. Former Apple engineer. I make ollamatps.com and write about building and measuring AI at anton.qa.

Let's test dark mode and reduced motion

Anton Gulin — Sat, 26 Sep 2026 15:01:01 +0000

Originally published on anton.qa.

A browser can report dark mode while the page stays white.
Checking the setting alone would miss that broken page.

Let's build a check that catches it, then deliberately break the page.
We will do the same for reduced motion, a request for less movement.

This walkthrough uses Playwright, a browser testing tool, and a small practice page.
The complete project includes the correct page, two broken versions and three tests.

The result we want

The practice page has a card and an animated notice.
Its styling rules should respond to two browser preferences:

Dark mode: use the expected dark background, light text and lighter border.
Reduced motion: make the notice's visual change immediate.

These are requirements for this example. Your page may need different colours or movement.

We also keep a test for the normal light appearance.
Otherwise, a page that always looks dark could pass the dark check.

Here is the expected result across the three versions:

Page version	Light check	Dark check	Motion check
Correct page	Pass	Pass	Pass
Missing dark styling rule	Pass	Fail	Pass
Missing reduced-motion rule	Pass	Pass	Fail

The two failures are intentional. Each shows that a test detects its target problem.

Run the example first

Use Node.js 24, Git and a terminal.
Node runs JavaScript. Git downloads the project. npm installs its packages.
The project pins Playwright to version 1.63.0.

git clone https://github.com/antongulin/anton-qa-resources.git
cd anton-qa-resources/posts/test-dark-mode-reduced-motion
npm ci
npx playwright install chromium
npm run verify

Chromium is the browser used for these checks.
On Linux, browser installation may need system libraries too:

npx playwright install --with-deps chromium

The verifier starts the local practice server and checks all three versions.
Its final output should include:

Corrected app: baseline, dark colors, and reduced motion all pass.
Broken colors: dark check failed with the light-rgb reason; motion check still passed.
Broken motion: reduced-motion check failed with the 2s reason; dark check still passed.
Light baseline stays green under both deliberately broken variants.
Verification complete: 3 checks, 2 deliberate defects with correct independent failure reasons.

Success here includes the expected failures on the broken pages.
A browser startup error does not count as detecting a styling problem.
The verifier checks the failure messages and the unaffected tests too.

For a quick run against only the correct page, use npm test.

Ask for dark mode, then check the card

Setting a browser preference and checking a page's response are separate steps.

page.emulateMedia makes the browser report a preference to the page.
matchMedia lets us confirm that the browser reports it.
Neither tells us which colours the page applied.

For that, getComputedStyle reads the style values calculated by the browser.
The following test comes from tests/theme.spec.ts:

import { test, expect } from '@playwright/test';
import { assertDarkTheme, variantPath } from './helpers';

test('rendered card colors follow the dark color scheme preference', async ({ page }) => {
  await page.emulateMedia({ colorScheme: 'dark', media: 'screen' });
  await page.goto(variantPath());

  // Confirm the emulation flag itself is active so a failure means the CSS did
  // not apply, not that the emulation was dropped.
  expect(await page.evaluate(() => matchMedia('(prefers-color-scheme: dark)').matches)).toBe(true);

  const card = page.getByTestId('card');
  const surface = await card.evaluate(node => getComputedStyle(node).backgroundColor);
  const text = await card.evaluate(node => getComputedStyle(node).color);
  const accent = await card.evaluate(node => getComputedStyle(node).borderTopColor);

  // The rendered values must be the dark tokens, not just "some dark color".
  // Assert the reasoned color contract first so a failure names the defect and
  // the light value actually rendered.
  assertDarkTheme(surface, text, accent);
  expect({ surface, text, accent }).toEqual({
    surface: 'rgb(16, 20, 24)',
    text: 'rgb(244, 246, 248)',
    accent: 'rgb(138, 180, 248)',
  });
});

The test checks three colours because the practice card has three relevant requirements.
The background should be dark, the text light, and the border lighter than the background.

The custom helper gives a readable failure message.
The final comparison checks the exact expected colours.
Both live in the linked project. You do not need to recreate the helper.

These are browser style values, not sampled screenshot pixels.

Check reduced motion separately

The notice normally changes over two seconds.
For this page, the reduced-motion rule removes that transition.

transitionDuration tells us how long the visual change takes.
The expected value here is zero seconds, plus the page's “reduced” marker.

Here is tests/motion.spec.ts:

import { test, expect } from '@playwright/test';
import { assertReducedMotion, variantPath } from './helpers';

test('rendered transition honors the reduced-motion preference', async ({ page }) => {
  await page.emulateMedia({ reducedMotion: 'reduce', media: 'screen' });
  await page.goto(variantPath());

  expect(await page.evaluate(() => matchMedia('(prefers-reduced-motion: reduce)').matches)).toBe(true);

  const banner = page.getByTestId('banner');
  const duration = await banner.evaluate(node => parseFloat(getComputedStyle(node).transitionDuration));
  const flag = await page.getByTestId('banner-flag').evaluate(node => getComputedStyle(node, '::after').content);

  assertReducedMotion(duration, flag.replace(/"/g, ''));
});

The helper checks the duration and marker together.
It reports the actual duration when the page keeps moving.

Zero seconds is this example's requirement, not a universal reduced-motion rule.
For your application, agree on the expected behaviour before writing the assertion.
An assertion is the comparison that decides whether the test passes.

Remove one rule and watch the result

CSS is the page styling language.
The server can leave out one CSS rule at a time.

The css-stuck version omits the dark-mode rule.
The browser still reports dark mode, but the card stays white.
The dark check fails with rgb(255, 255, 255) in its message.

The motion-stuck version omits the reduced-motion rule.
The browser still reports reduced motion, but the transition lasts two seconds.
The motion check reports received 2s.

Run either broken version yourself on macOS or Linux:

APP_VARIANT=css-stuck npm test
APP_VARIANT=motion-stuck npm test

Each command should fail one test and exit unsuccessfully.
The remaining checks should pass.

On PowerShell, use this form and clear the variable afterward:

$env:APP_VARIANT="css-stuck"
npm test
Remove-Item Env:APP_VARIANT

Use motion-stuck as the value to try the other defect.
Then run npm test without a variant to return to the correct page.

Adapt one check to your application

Start with a page whose expected appearance you understand.
Choose one element, such as a settings card, and write down its expected dark colours.

Set the preference before opening the page.
Read the element's applied styles and compare them with those expected values.
Keep a light-mode check beside it.

For movement, choose the exact animation or transition your requirement covers.
Check its expected behaviour under reduced motion and under the normal setting.

Finally, remove the relevant styling rule in a safe local copy.
Confirm that the intended check fails for the expected reason.
Restore the rule and confirm that it passes again.

You now have evidence that the check catches that specific defect.

What this example does not establish

The project checks one synthetic page in Chromium.
It does not establish coverage across every browser or component.
It does not compare screenshot pixels or measure accessible colour contrast.

Browser emulation reports a preference. It does not change your operating system's setting.
Screen-reader behaviour and other accessibility requirements need their own checks.

The practice server runs locally, binds to 127.0.0.1 and writes no files.
Its expected colours belong to this example, not your design system.

About the author

Anton Gulin is the AI QA Architect, the first person to claim this title on LinkedIn. He builds AI-powered test automation systems where AI agents and human engineers collaborate on quality. Former Apple SDET (Apple.com / Apple Card pre-release testing). Find him at anton.qa or on LinkedIn.

Sources

Playwright media emulation.
Complete practice project, including helpers and the verifier.
Exact tested revision.

How to Migrate from Selenium to Playwright in 2026: A Complete Guide

Anton Gulin — Tue, 22 Sep 2026 15:01:34 +0000

Why Migrate from Selenium to Playwright?

I've been using Selenium since 2015. It served me well at Apple and plenty of other companies. But after migrating a large suite to Playwright, I won't go back.

Here's how they compare:

Setup time:

Selenium: 30+ minutes
Playwright: 5 minutes
Auto-waiting:
Selenium: Manual (causes flaky tests)
Playwright: Built-in
Cross-browser:
Selenium: Needs separate drivers
Playwright: Bundled
Parallel testing:
Selenium: Requires Grid setup
Playwright: Built-in
Test execution:
Selenium: Slower
Playwright: 40-60% faster
Debugging:
Selenium: Stack traces only
Playwright: Trace viewer with recordings
If your Selenium tests are slow, flaky, or a pain to maintain, migration is worth it.

The Migration Strategy

Don't rewrite everything at once. Follow this incremental approach:

Phase 1: New tests in Playwright

Stop writing new Selenium tests. All new tests go in Playwright.

Phase 2: Migrate critical paths first

Identify your highest-value tests (login, checkout, core features) and migrate those.

Phase 3: Gradual Selenium deprecation

As you touch Selenium tests for maintenance, migrate them. Eventually, the old suite goes to zero.

This approach minimizes risk and lets your team learn Playwright gradually.

Step 1: Set Up Playwright Alongside Selenium

You don't need to remove Selenium to start. Create a parallel Playwright project:

npm init playwright@latest -- --quiet

Your project structure becomes:

tests/
├── selenium/  # Existing Selenium tests
└── playwright/  # New Playwright tests

Both can run in CI simultaneously during transition.

Step 2: Translate Common Patterns

Here's a side-by-side comparison of common operations:

Navigation:

Selenium (Java):

driver.get("https://example.com");

Playwright (TypeScript):

await page.goto('https://example.com');

Finding elements:

Selenium:

driver.findElement(By.id("username"));

driver.findElement(By.cssSelector(".submit-btn"));

driver.findElement(By.xpath("//button[text()='Submit']"));

Playwright:

page.locator('#username');

page.locator('.submit-btn');

page.getByRole('button', { name: 'Submit' });

Clicking:

Selenium:

driver.findElement(By.id("submit")).click();

Playwright:

await page.locator('#submit').click();

Or better:

await page.getByRole('button', { name: 'Submit' }).click();

Typing:

Selenium:

driver.findElement(By.id("email")).sendKeys("test@example.com");

Playwright:

await page.locator('#email').fill('test@example.com');

Or:

await page.getByLabel('Email').fill('test@example.com');

Waiting:

Selenium (the source of most flakiness):

WebDriverWait wait = new WebDriverWait(driver, Duration.ofSeconds(10));

wait.until(ExpectedConditions.visibilityOfElementLocated(By.id("result")));

Playwright (auto-waits by default):

Just interact. Playwright waits automatically

await page.locator('#result').click();

Or explicitly assert visibility:

await expect(page.locator('#result')).toBeVisible();

Assertions:

Selenium (with JUnit):

String title = driver.getTitle();

assertEquals("Expected Title", title);

Playwright:

await expect(page).toHaveTitle('Expected Title');

Screenshots:

Selenium:

File screenshot = ((TakesScreenshot) driver).getScreenshotAs(OutputType.FILE);

FileUtils.copyFile(screenshot, new File("screenshot.png"));

Playwright:

await page.screenshot({ path: 'screenshot.png' });

Step 3: Migrate Page Objects

If you're using Page Object Model in Selenium, the pattern translates directly.

Selenium (Java):

public class LoginPage {
    private WebDriver driver;
    public LoginPage(WebDriver driver) {
        this.driver = driver;
    }
    public void login(String email, String password) {
        driver.findElement(By.id("email")).sendKeys(email);
        driver.findElement(By.id("password")).sendKeys(password);
        driver.findElement(By.id("submit")).click();
    }
}

Playwright (TypeScript):

export class LoginPage {
  constructor(private page: Page) {}
  async login(email: string, password: string) {
    await this.page.getByLabel('Email').fill(email);
    await this.page.getByLabel('Password').fill(password);
    await this.page.getByRole('button', { name: 'Sign in' }).click();
  }
}

The structure is almost identical. You're just swapping APIs.

Step 4: Replace WebDriver Waits with Auto-Wait

This is where you get the biggest reliability win.

Remove all explicit waits:

In Selenium, you probably have code like:

WebDriverWait wait = new WebDriverWait(driver, 10);

wait.until(ExpectedConditions.elementToBeClickable(By.id("button")));

driver.findElement(By.id("button")).click();

In Playwright, just:

await page.locator('#button').click();

Playwright automatically waits for:

Element to be attached to DOM
Element to be visible
Element to be enabled
Element to be stable (not animating) This single change eliminates most flaky tests.

Step 5: Set Up CI/CD

Playwright's CI setup is simpler than Selenium Grid.

GitHub Actions example:

name: Playwright Tests
on: [push, pull_request]
jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: 20
      - run: npm ci
      - run: npx playwright install --with-deps
      - run: npx playwright test
      - uses: actions/upload-artifact@v4
        if: failure()
        with:
          name: playwright-report
          path: playwright-report/

No Selenium Grid. No browser driver management. Just install and run.

Common Migration Pitfalls

1. Trying to rewrite everything at once

This is the number one cause of failed migrations. Use the incremental approach.

2. Copying Selenium's explicit waits

Don't translate WebDriverWait to waitForSelector. Trust Playwright's auto-waiting.

3. Keeping fragile locators

Migration is a chance to improve. Replace XPath and positional CSS with role-based locators.

4. Not training the team

Playwright has different patterns. Invest in a training session or pair programming during the first weeks.

Migration Checklist

Before declaring migration complete:

All new tests written in Playwright
Critical user journeys migrated
CI/CD running Playwright tests
Team trained on Playwright patterns
Old Selenium tests deleted (don't leave dead code)
Trace viewer configured for failure debugging

How Long Does Migration Take?

Based on my experience:

Small suite (under 50 tests): 1-2 weeks

Medium suite (50-200 tests): 1-2 months

Large suite (200+ tests): 3-6 months

The biggest factor isn't the number of tests. It's how tangled your Selenium architecture is. Clean Page Objects migrate fast. Spaghetti code requires refactoring.

The Results Are Worth It

On that project, after completing migration:

Test execution time: 40% faster
Flaky test rate: Reduced by 80%
CI pipeline reliability: 98%+ pass rate
Developer experience: "I actually like writing tests now" That last point matters more than any metric. When tests are pleasant to write, people write more tests.

Need Help with Your Migration?

If you're planning a Selenium to Playwright migration and want guidance from someone who's done it multiple times, I can help. I offer consulting packages that include:

Migration strategy and roadmap
Team training on Playwright patterns
Architecture review of existing tests
Hands-on migration for critical paths Let's talk

Let's save the clues from a failed Playwright test

Anton Gulin — Sat, 19 Sep 2026 15:00:56 +0000

Originally published on anton.qa.

Let's run a checkout test that fails once, then passes.
We'll use Playwright, a browser testing tool.
You'll compare two settings and open the recordings they keep.

A trace is a detailed test recording.
You can inspect actions, saved page views, and requests from that attempt.
A retry is another attempt after a failure.

The useful difference in this example:

on-first-retry keeps the successful retry's recording.
retain-on-failure keeps the original failure's recording.

The test creates both responses so you can repeat this comparison.
You can see the supplied error in the original failure's saved response.
It runs on your computer and doesn't contact a payment provider.

The complete runnable project includes all files.
An agent verified a fresh public download while preparing the article.

Set up the practice project

Use Node.js 24, which runs JavaScript programs.
Its included npm tool installs project packages.
You'll also need Git, which downloads and tracks source code.

Run these commands in your terminal:

git clone https://github.com/antongulin/anton-qa-resources.git
cd anton-qa-resources/posts/save-failed-playwright-test
npm ci
npx playwright install chromium

The project fixes Playwright at version 1.63.0 for this example.
The last command installs Chromium, the browser used for testing.

On Linux, missing browser libraries may require this command:

npx playwright install --with-deps chromium

You don't need to start the practice website yourself.
The tests start and stop their own local server at 127.0.0.1:4191.
If you've started npm run serve yourself, stop it with Ctrl+C first.

Run the example and check its recordings

npm run verify

This command runs the tests and checks the saved recordings.
It replaces the project's generated results from any previous run.

Expect the checkout test to fail first, then pass on retry.
Playwright labels that result flaky: it needed another attempt to pass.
Here, we created that behavior so you can repeat the comparison.

The verifier checks both attempt numbers and the recorded responses.
It also confirms that the failed recording contains the original failed check.
You should see successful verification despite the deliberately failed first attempts.

The project includes three additional checks for related browser-testing examples.
Those checks should pass too.

See how the test creates its failure

Open tests/core-trace.spec.ts in your editor.
The test uses testInfo.retry to read the attempt number.
Zero means the original attempt. One means the first retry.

It fills an order note and presses Save checkout.
A route handler, code that intercepts requests, supplies the checkout response.
On attempt zero, it supplies an error. On retry one, it supplies success.

A fixture means a controlled test setup here.
The example uses that word in its exact response messages:

Attempt	Response code	Message supplied by the test
Original, 0	503, service unavailable	Injected first-attempt save failure
Retry, 1	200, success	Saved on retry by synthetic fixture

Both attempts run the same assertion, a check of the expected result.
This exact excerpt comes from the test:

await expect(page.getByRole('status')).toHaveText('Saved on retry by synthetic fixture');

The first attempt displays the error message, so this check fails.
The retry displays the expected success message, so it passes.
No timing guess or random failure decides the outcome.

Compare the two recording settings

The project runs that test under two sets of settings.
Playwright calls these projects, named groups of test settings.

These two project entries come from playwright.config.ts:

[
  {
    name: 'on-first-retry',
    testMatch: /core-trace\.spec\.ts/,
    retries: 1,
    use: { trace: 'on-first-retry' },
  },
  {
    name: 'retain-on-failure',
    testMatch: /core-trace\.spec\.ts/,
    retries: 1,
    use: { trace: 'retain-on-failure' },
  },
]

These entries belong inside the existing configuration, followed by the companion project.
Keep the downloaded file intact. You don't need to edit it for this walkthrough.

on-first-retry records the first retry and keeps that recording.
In our example, you get the successful second attempt.

retain-on-failure records each attempt and keeps recordings of failed attempts.
In our example, you get the original failed attempt.

Playwright's recording settings describe additional modes and combinations.
The two here let you compare a successful retry with its original failure.

Open the successful retry

Run this command after verification:

npx playwright show-trace test-results/*on-first-retry-retry1/trace.zip

Trace Viewer opens the recording for inspection.
Select the Save checkout action, then look at the saved page view.
Open Network, the list of recorded requests.
Find the checkout request ending in /api/checkout.

The request asks the app to save the order note.
Its response carries the result.
You should find response code 200 and the supplied success message.

This recording lets you inspect what happened during the retry.
To inspect the first failure, open its own recording next.

Open the original failed attempt

npx playwright show-trace test-results/*retain-on-failure/trace.zip

Find the failed text check in the action list.
Compare the expected message with the received message.
The page received Injected first-attempt save failure.
The test expected Saved on retry by synthetic fixture.

Next, find /api/checkout under Network.
Inspect its response code and response body, the returned data.
The recording contains code 503 and the injected error message.

You can now connect the received response with the page's message and failed check.
In this example, the test supplied the error on purpose.
This demonstrates saved evidence, rather than diagnosing a real service outage.

The Trace Viewer guide explains its inspection panels.

Keep a record of what you found

Open the complete test report with:

npm run show-report

For the verifier's findings, open artifacts/verification-report.json in your editor.
JSON is a structured data format.
This file lists attempt numbers, saved response bodies, and recording file locations.

The verifier reads those recordings to confirm the comparison.
A filename alone doesn't establish which response the browser received.
The tested project revision preserves the version used for this article.

For your own investigation, record a few concrete details:

The test name and attempt number.
The expected result and what appeared instead.
The relevant response and a link to the saved recording.

That gives the next person somewhere useful to start.

Choose recording settings for your project

If you need the original failure, choose a mode that keeps that attempt.
If you're investigating the retry, record the retry.

Recording more attempts uses processing time and storage.
Saved pages and responses can also contain private data.
Decide who can open those files and when to delete them.

Try this comparison on the practice project before changing your team's settings.
You'll know which recording to look for when a real test fails.

How to Fix Flaky Tests in Playwright: 10 Battle-Tested Strategies

Anton Gulin — Tue, 15 Sep 2026 15:01:17 +0000

The Cost of Flaky Tests

A flaky test is one that sometimes passes and sometimes fails without any code changes. They seem harmless at first. Just re-run the pipeline, right?

Wrong. Flaky tests are silent killers:

Eroded trust: Teams stop believing test results
Wasted time: Engineers debug tests instead of building features
Slower releases: "Just re-run it" becomes the norm
Hidden bugs: Real failures get dismissed as "flaky" At one company, I inherited a test suite with a 68% pass rate. Not because the application was broken, but because the tests were. Here's how I fixed it.

Strategy 1: Use Role-Based Locators

The problem: CSS selectors and XPath break when developers change class names or restructure HTML.

The fix: Use Playwright's role-based locators:

Bad:

page.locator('#submit-btn')

page.locator('.form-container > button:nth-child(2)')

Good:

page.getByRole('button', { name: 'Submit' })

page.getByLabel('Email address')

page.getByText('Welcome back')

Role-based locators are more resilient because they target what users see, not implementation details.

Strategy 2: Never Use Hard-Coded Waits

*The problem: *waitForTimeout() is the number one cause of flaky tests.

await page.waitForTimeout(5000); // ❌ Please don't do this

Why it fails: 5 seconds might be enough on your machine but not in CI. Or it might be way too long, slowing tests unnecessarily.

The fix: Wait for specific conditions:

await page.waitForLoadState('networkidle');

await expect(page.getByRole('button')).toBeEnabled();

await expect(page.getByText('Success')).toBeVisible();

Playwright's auto-waiting handles most cases automatically. Trust it.

Strategy 3: Isolate Test State

The problem: Tests depend on state from previous tests.

test('login', ...); // Creates session

test('add to cart', ...); // Expects logged-in state

If the login test fails, the cart test also fails, but not because of a cart bug.

The fix: Each test should set up its own state:

test.beforeEach(async ({ page }) => {
  await loginAsUser(page, 'testuser');
});

Or use Playwright's storage state to share authentication without dependencies:

await page.context().storageState({ path: 'auth.json' });

Strategy 4: Handle Loading States Explicitly

The problem: Clicking a button that's still loading, or reading text before it's rendered.

The fix: Wait for loading indicators to disappear:

// Wait for spinner to go away
await expect(page.getByTestId('loading-spinner')).toBeHidden();
// Then interact with the element
await page.getByRole('button', { name: 'Submit' }).click();

Or wait for the element to be in a specific state:

await expect(page.getByRole('button')).toBeEnabled();
await page.getByRole('button').click();

Strategy 5: Use data-testid for Dynamic Content

The problem: Elements generated dynamically have unpredictable locators.

The fix: Add data-testid attributes for testing:

In your application code:

<button data-testid="checkout-button">Checkout</button>

In your test:

await page.getByTestId('checkout-button').click();

This creates a contract between frontend and tests that survives refactoring.

Strategy 6: Retry Failed Assertions (Not Whole Tests)

The problem: A test fails once and you re-run the entire suite.

The fix: Use Playwright's built-in expect retries:

// playwright.config.ts
export default defineConfig({
  expect: {
    timeout: 10000, // Wait up to 10 seconds for assertions
  },
});

Assertions like toBeVisible() and toHaveText() will automatically retry until timeout. No manual retries needed.

Strategy 7: Handle Network Variability

The problem: API calls take longer in CI than locally.

The fix: Wait for network responses explicitly:

// Wait for specific API call to complete
await page.waitForResponse(resp =>
  resp.url().includes('/api/products') && resp.status() === 200
);

Or use networkidle for simpler cases:

await page.goto('/dashboard', { waitUntil: 'networkidle' });

Strategy 8: Run Tests in Parallel Correctly

The problem: Tests interfere with each other when running in parallel.

Test A creates user "testuser@example.com"

Test B also creates user "testuser@example.com"

One fails due to duplicate email.

The fix: Use unique data per test:

import { faker } from '@faker-js/faker';
const email = faker.internet.email();
await page.getByLabel('Email').fill(email);

Or isolate tests in separate browser contexts (Playwright does this by default).

Strategy 9: Use Trace Viewer for Debugging

The problem: You can't see what happened when a test failed in CI.

The fix: Turn traces on for failures:

// playwright.config.ts
export default defineConfig({
  use: {
    trace: 'on-first-retry',
  },
});

Now, when a test fails and retries, Playwright captures:

Screenshots at every step
DOM snapshots
Network requests
Console logs Open traces with:

npx playwright show-trace trace.zip

This is the single best debugging tool for flaky tests.

Strategy 10: Set Realistic Timeouts

The problem: Default timeouts are too short for slow environments.

The fix: Configure appropriate timeouts based on your CI environment:

// playwright.config.ts
export default defineConfig({
  timeout: 60000,        // Test timeout: 60 seconds
  expect: {
    timeout: 10000,      // Assertion timeout: 10 seconds
  },
  use: {
    actionTimeout: 15000, // Click/fill timeout: 15 seconds
    navigationTimeout: 30000, // Page load timeout: 30 seconds
  },
});

Don't make them too long. Slow failures are frustrating. Find the right balance for your infrastructure.

Bonus: The Flaky Test Triage Process

When you encounter a flaky test, follow this process:

1. Reproduce locally

Run the test 10 times:

npx playwright test tests/checkout.spec.ts --repeat-each=10

If it passes every time locally but fails in CI, it's likely a timing or environment issue.

2. Check the trace

Open the trace file from CI and look for:

Slow network requests
Elements not visible when clicked
Unexpected modal or popup 3. Identify the root cause

Common causes:

Hard-coded wait times
Race conditions
Shared test state
Unstable locators 4. Fix or delete

If you can't fix it after 30 minutes, delete it. A flaky test is worse than no test. You can always rewrite it properly later.

My Flaky Test Scorecard

After applying these strategies on one project, we went from a 68% pass rate to 98%+ in three months.

Pass rate:

Before: 68%
After: 98.5%
Average test time:
Before: 4.2 minutes
After: 1.8 minutes
"Re-run pipeline" requests:
Before: Daily
After: Rare
Team trust in automation:
Before: Low
After: High
The biggest win wasn't technical. It was cultural. When tests are reliable, developers actually care about failures.

Need Help with Your Flaky Tests?

If your test suite is unreliable and slowing down your team, I can help. I've fixed test suites at Fortune 500 companies and can usually identify the main issues in just a few hours.

Book a consultation

Let's turn a browser task into a Playwright test

Anton Gulin — Sat, 12 Sep 2026 15:01:06 +0000

Originally published on anton.qa.

This walkthrough teaches one thing: turn recorded actions into a test that checks the result.

Our example is a small practice website with a display-name field.
The name starts as Alex. We change it to Sam and press Save.
We are editing this website, not a Chrome profile or Google account.

Playwright is a browser testing tool. It can record those actions as code.
We add a check: reload the page and confirm the name is still Sam.
The finished test repeats the actions and checks the result automatically.

Here is the problem that check can catch.
Our demo includes a deliberately broken Save button.
It says Saved, but reloading brings back Alex.
The test expects Sam, finds Alex, and fails.
With saving fixed, the same test passes.

The lesson: record the steps, then check that the intended change actually happened.

The sections below show how to build and run that example.
Playwright's command-line tool accepts typed terminal commands.
Version 0.1.19 added commands for recording browser actions.
The recorder provides the actions. We choose and add the result checks.

An agent ran this example while preparing the article.
It uses invented names and browser storage, not a customer account.

1. Create a small practice app

Use Node.js 24, which runs JavaScript programs.
The files use TypeScript, JavaScript with types.

Create a folder, then run these commands inside it:

npm init -y
npm pkg set type=module
npm install --save-dev @playwright/test@1.63.0 @playwright/cli@0.1.19
npx playwright install chromium
mkdir tests

Create server.ts with this app:

import { createServer } from 'node:http';

createServer((request, response) => {
  const url = new URL(request.url ?? '/', 'http://127.0.0.1');
  const broken = url.searchParams.get('broken') === '1';
  response.setHeader('Content-Type', 'text/html; charset=utf-8');
  response.end(`<!doctype html>
<html lang="en"><meta charset="utf-8"><title>Profile practice</title>
<body><main><h1>Profile settings</h1>
<form><label for="name">Display name</label>
<input id="name"><button>Save</button></form>
<p role="status"></p>
<p>Current name: <strong data-testid="current-name"></strong></p>
<script>
const field = document.querySelector('#name');
const current = document.querySelector('[data-testid="current-name"]');
field.value = localStorage.getItem('displayName') || 'Alex';
current.textContent = field.value;
document.querySelector('form').addEventListener('submit', event => {
  event.preventDefault();
  if (!${broken}) localStorage.setItem('displayName', field.value);
  current.textContent = localStorage.getItem('displayName') || 'Alex';
  document.querySelector('[role="status"]').textContent = 'Saved';
});
</script></main></body></html>`);
}).listen(4187, '127.0.0.1');

Browser storage keeps the name between page reloads.
This small app has a deliberate broken mode.
Adding ?broken=1 makes Save display success without storing the name.
That gives us a known problem to test.

Start the app in one terminal:

node server.ts

2. Record one task

In a second terminal, open a visible browser:

npx playwright-cli -s=profile open http://127.0.0.1:4187 --headed
npx playwright-cli -s=profile recording-start

Change Display name from Alex to Sam. Click Save.
Then stop recording:

npx playwright-cli -s=profile recording-stop
npx playwright-cli -s=profile close

The checked example returned these actions:

await page.getByRole('textbox', { name: 'Display name' }).click();
await page.getByRole('textbox', { name: 'Display name' }).press('ControlOrMeta+A');
await page.getByRole('textbox', { name: 'Display name' }).fill('Sam');
await page.getByRole('button', { name: 'Save' }).click();

Your output can differ with your clicks, browser, and operating system.
The local proof used simulated pointer and keyboard input.
These actions came from the recorder. The saved-result checks below were added separately.

We can simplify these steps. fill replaces the field's current text.
The first two actions are unnecessary for this particular test.

The recording describes what happened in the browser. It does not decide whether saving worked.

A field label also makes the action easier to read. Here, the test finds the textbox named Display name. It finds the Save button. Those descriptions connect the code to controls the reader can see.

For another page, review the recorded descriptions before keeping them. A changing number or unrelated click may not belong in the finished test.

3. Check the result after reload

Stop the app in the first terminal with Control-C.
The test runner will start its own copy.

Create playwright.config.ts:

import { defineConfig } from '@playwright/test';

export default defineConfig({
  testDir: './tests',
  retries: 0,
  use: { baseURL: 'http://127.0.0.1:4187' },
  webServer: {
    command: 'node server.ts',
    url: 'http://127.0.0.1:4187',
    reuseExistingServer: false,
  },
});

Create tests/save.spec.ts:

import { test, expect } from '@playwright/test';

test('Save keeps the edited name after reload', async ({ page }) => {
  const path = process.env.APP_BROKEN === '1' ? '/?broken=1' : '/';
  await page.goto(path);

  await page.evaluate(() => localStorage.removeItem('displayName'));
  await page.reload();
  await expect(page.getByLabel('Display name')).toHaveValue('Alex');

  await page.getByRole('textbox', { name: 'Display name' }).fill('Sam');
  await page.getByRole('button', { name: 'Save', exact: true }).click();
  await expect(page.getByRole('status')).toHaveText('Saved');

  await page.reload();
  await expect(page.getByLabel('Display name')).toHaveValue('Sam');
  await expect(page.getByTestId('current-name')).toHaveText('Sam');
});

expect checks that the app matches an expected result.
Here, Sam must remain in both places after reloading.

The status message is useful feedback. It cannot prove that the name was saved.
Our final checks cover that missing part.

We also restore Alex before editing. Each run starts from a known value.
Playwright gives each test a fresh browser context, an isolated browser session.
That does not reset a real shared server or database.
This example uses only browser storage.

The reset and the final check answer different questions. The reset establishes where the test begins. The final check establishes what the action must preserve. Keeping both visible makes the example easier to change later.

4. Confirm the test catches the problem

Run the broken app mode first:

# macOS or Linux
APP_BROKEN=1 npx playwright test tests/save.spec.ts

PowerShell users can set the same value separately:

$env:APP_BROKEN='1'
npx playwright test tests/save.spec.ts

The test should fail after reloading. It expects Sam and finds Alex.
The Save message still said Saved, which explains why clicking alone was insufficient.

Now run the working mode:

# macOS or Linux
APP_BROKEN=0 npx playwright test tests/save.spec.ts

$env:APP_BROKEN='0'
npx playwright test tests/save.spec.ts

The same test should pass.

If your result differs, read the failed check before changing the test. Does it show Alex where Sam was expected? That is the intended failure in broken mode. A browser launch error or an occupied port is a setup problem instead.

First confirm the example opened the page. Then compare the expected name with the actual name. Keeping that distinction helps you fix the right part. A failed setup does not demonstrate that the saving check caught a defect.

During preparation, the broken run failed and the corrected run passed. Both used the same test, with no retries.
This proves the check caught our deliberately introduced problem.
It does not prove coverage for every possible saving problem.

5. Choose the next small flow

Try this with one task you already understand.

Write the expected result before adding more recorded steps.
For a saved setting, reload and check the value.
For a submitted form, check the resulting record where your test can inspect it.
Use test accounts and practice data when recording.

Keep unrelated clicks out of the finished test.
Give each test its own data where possible.
Keep checks that explain the user's expected outcome.

The useful question is simple: what should still be true after this action?

The recorder helps with the steps. Your understanding supplies the reason to check them.

Sources

Playwright CLI 0.1.19 release: recording commands and their scope.
Playwright best practices: test isolation and user-visible checks.
Playwright assertions: checks that wait for expected results.

About the author

Playwright Tutorial for Beginners: Your First Test in 10 Minutes

Anton Gulin — Tue, 08 Sep 2026 15:01:15 +0000

What You'll Learn

By the end of this tutorial, you'll have:

Playwright installed on your machine
Written and run your first automated test
Understood the basic Playwright concepts
A foundation to build real-world test suites Total time: about 10 minutes. Let's go.

Prerequisites

You need Node.js 18+ installed. Check your version:

node --version

If you need to install Node.js, download it from nodejs.org.

That's it. No other dependencies required.

Step 1: Create Your Project

Open your terminal and create a new folder:

mkdir playwright-tutorial

cd playwright-tutorial

Step 2: Install Playwright

Run the Playwright installer:

npm init playwright@latest

You'll see some prompts. Accept the defaults:

TypeScript or JavaScript? → TypeScript (recommended)
Where to put tests? → tests
Add GitHub Actions? → Yes (optional, but useful later)
Install browsers? → Yes This installs Playwright, downloads browser binaries, and creates a basic project structure.

Step 3: Explore the Project Structure

After installation, you'll see:

tests/: Your test files go here
playwright.config.ts: Configuration file
tests/example.spec.ts: A sample test (we'll replace this)

Step 4: Write Your First Test

Delete the example test and create a new file called tests/first.spec.ts:

import { test, expect } from '@playwright/test';
test('homepage has correct title', async ({ page }) => {
  await page.goto('https://playwright.dev');
  await expect(page).toHaveTitle(/Playwright/);
});

What this does:

test(): Defines a test with a name
page.goto(): Navigates to a URL
expect().toHaveTitle(): Asserts the page title contains "Playwright"

Step 5: Run Your Test

In your terminal:

npx playwright test

You should see:

Running 1 test using 1 worker
✓ first.spec.ts:3:1 › homepage has correct title (1.2s)
1 passed (2.1s)

Congratulations! You just ran your first Playwright test.

Step 6: See the Test Report

Playwright generates beautiful HTML reports. View it:

npx playwright show-report

This opens a browser with your test results, including timing, screenshots, and traces.

Step 7: Run Tests in UI Mode (Game Changer)

Playwright's UI mode lets you watch tests run in real-time:

npx playwright test --ui

This opens a visual interface where you can:

Watch tests execute step-by-step
Inspect DOM at any point
Debug failures visually
Re-run individual tests This is my favorite Playwright feature for development.

Step 8: Add More Assertions

Let's expand our test to click a link and verify navigation:

import { test, expect } from '@playwright/test';
test('can navigate to getting started page', async ({ page }) => {
  await page.goto('https://playwright.dev');
  // Click the Getting Started link
  await page.getByRole('link', { name: 'Get started' }).click();
  // Verify we're on the right page
  await expect(page).toHaveURL(/.*intro/);
  // Check the heading exists
  await expect(page.getByRole('heading', { name: 'Installation' })).toBeVisible();
});

New concepts:

getByRole(): Finds elements by their accessibility role (best practice)
click(): Clicks an element
toHaveURL(): Asserts the current URL matches a pattern
toBeVisible(): Asserts an element is visible on page

Step 9: Use the Code Generator

Playwright can generate test code by recording your actions:

npx playwright codegen playwright.dev

This opens a browser. Click around, and Playwright writes code for you in real-time. Copy the generated code into your test file.

This is extremely useful for learning locators and speeding up test creation.

Understanding Key Concepts

Locators:

Playwright finds elements using locators. Best practices:

getByRole(): By accessibility role (button, link, heading)
getByText(): By visible text
getByLabel(): By form label
getByTestId(): By data-testid attribute Avoid CSS selectors and XPath when possible. Role-based locators are more resilient.

Auto-waiting:

Playwright automatically waits for elements to be ready before interacting. No more:

await page.waitForSelector('.button'); // Not needed!

Just write:

await page.click('.button'); // Playwright handles waiting

This dramatically reduces flaky tests.

Next Steps

Now that you have the basics:

Add more tests: Start testing your own application
Learn Page Object Model: Organize tests for maintainability
Set up CI/CD: Run tests on every pull request
Explore API testing: Playwright also tests REST APIs

Common Beginner Mistakes to Avoid

1. Using sleep/delay:

Bad:

await page.waitForTimeout(5000);

Good: Let Playwright's auto-waiting handle it, or wait for specific conditions.

2. Fragile locators:

Bad:

page.locator('#app > div:nth-child(3) > button')

Good:

page.getByRole('button', { name: 'Submit' })

3. Not using test isolation:

Each test should be independent. Don't rely on state from previous tests.

Resources

Official docs: playwright.dev/docs/intro
Test generator:
npx playwright codegen [url]
UI mode:
npx playwright test --ui
Report viewer:
npx playwright show-report

Need Hands-On Help?

If you're building a test automation framework for your team and want guidance from someone who's done it at Apple and Fortune 500 companies, I offer consulting and training.

Learn more about working with me

How to tell whether an AI feature actually works

Anton Gulin — Sat, 05 Sep 2026 20:23:16 +0000

Your AI feature worked when you demoed it. That is the only evidence most teams collect before shipping.

In August 2026 Microsoft published a benchmark that measures the thing that demo misses. They ran 507 real business workflows through AI agents, across retail, hospitality, auto insurance, banking IT and consulting support. Every workflow was attempted 20 times, and each attempt was graded on what actually changed in the backend database, not on what the agent said it did.

The strongest model completed 65.36% of workflows on the first try. It completed all twenty attempts on only 25.25% of them.

Their paper is titled "One Success Isn't Reliability."

Testers have a shorter word for the gap between those two numbers. We call it a flaky test: one that passes and fails on the same code, with nothing changed but time.

This post is the method for measuring it in your own product.

Why one good run tells you almost nothing

A traditional test is deterministic. Given the same input and the same code, it returns the same answer every time. When it fails, something is broken.

An AI feature is not. The same prompt against the same model can return a different answer on Tuesday than it did on Monday. Temperature, model updates, retrieval order, and load all move the output.

So "it worked" is not a property of the feature. It is a property of one attempt. The question worth answering is: how often, and how far does it drift when it misses?

The ten-run check

Here is the smallest honest version. Drop it in a spec file, replace score() with whatever "good" means for your feature, and run it against the AI path you trust least.

import { test, expect } from '@playwright/test'

const RUNS = 10
const MIN_PASS_RATE = 0.9   // 9 of 10 runs must be good enough
const MAX_SPREAD = 0.34     // best run minus worst run

test('the summary feature holds up over ten runs', async ({ page }) => {
  const scores: number[] = []

  for (let i = 0; i < RUNS; i++) {
    await page.goto('/summarize')
    await page.getByRole('button', { name: 'Summarize' }).click()
    const text = await page.getByTestId('summary').innerText()
    scores.push(score(text))
  }

  const passRate = scores.filter(s => s >= 0.67).length / RUNS
  const spread = Math.max(...scores) - Math.min(...scores)
  console.log(`pass rate ${passRate}  spread ${spread.toFixed(2)}  [${scores}]`)

  expect(passRate).toBeGreaterThanOrEqual(MIN_PASS_RATE)
  expect(spread).toBeLessThanOrEqual(MAX_SPREAD)
})

// Yours to write. Start with the cheapest checks that are not "did it crash".
function score(text: string) {
  let s = 0
  if (text.length > 40) s += 0.34                  // it said something
  if (/\d{4}-\d{2}-\d{2}/.test(text)) s += 0.33    // it kept the date
  if (!/i cannot|as an ai/i.test(text)) s += 0.33  // it did not refuse
  return s
}

Two things make this useful instead of noisy.

Log all ten scores every run, and keep them. One bad day means nothing. The same feature drifting down across a week is the real signal.

Fail on the spread, not only on the average. A feature scoring 1.0, 1.0, 0.33, 1.0 has a worse problem than one scoring 0.8 every time. The average hides it. The spread does not.

The flag you already have

If you would rather not write a loop, Playwright ships this:

npx playwright test --repeat-each=10

It runs the same test ten times, and every run has to pass.

That tells you whether something failed. It does not tell you how far the answers moved, which is the part that warns you early. Use it as the free first step, then graduate to scoring when the feature matters.

The setting that hides all of this

Most projects carry this in playwright.config.ts:

export default defineConfig({
  retries: 2,
})

A failed test gets two more chances. If any of them passes, the run reports the test as flaky and still exits 0. The build goes green and nobody opens it.

Retries exist for a good reason: they keep a shared pipeline usable. But they are the opposite tool from the one above. Retries let one pass overwrite two failures. Repetition refuses to let that happen.

You need both, and most teams only have the first.

What to do this week

Pick the three AI-backed paths you trust least.
Run each ten times and write down the pass rate and the spread.
Decide, before you see the numbers, what pass rate would stop a release.
Put that number in the test, not in a document.

Step 3 is the one teams skip, and it is the one that makes the rest mean anything. A threshold chosen after seeing the result is not a threshold.

The part worth keeping

Microsoft needed 507 workflows, twenty runs each, and a graded backend state to say something testers have said for twenty years: one success is not reliability.

The measurement is not new. What is new is that the software now behaves differently on Tuesday than it did on Monday, so everyone needs the measurement, not just the people with "quality" in their title.

Page Object Model in Playwright with TypeScript: The Complete Guide

Anton Gulin — Tue, 01 Sep 2026 15:00:46 +0000

What is the Page Object Model?

The Page Object Model (POM) is a design pattern that creates an abstraction layer between your tests and your web pages. Instead of writing locators directly in tests, you encapsulate page elements and actions inside reusable classes.

Without POM (hard to maintain):

test('user can login', async ({ page }) => {
  await page.goto('/login');
  await page.locator('#email').fill('user@example.com');
  await page.locator('#password').fill('password123');
  await page.locator('button[type="submit"]').click();
  await expect(page.locator('.dashboard-welcome')).toBeVisible();
});

With POM (scalable):

test('user can login', async ({ page }) => {
  const loginPage = new LoginPage(page);
  await loginPage.goto();
  await loginPage.login('user@example.com', 'password123');
  await expect(loginPage.welcomeMessage).toBeVisible();
});

When the login form changes, you update one file, not 50 tests.

Why Use Page Objects?

1. Maintainability

Locators live in one place. When UI changes, update once.

2. Readability

Tests read like user stories: loginPage.login(), dashboard.navigateToSettings().

3. Reusability

4. Separation of concerns

Tests focus on behavior. Page objects handle implementation.

5. Team scalability

New team members understand tests faster.

Project Structure

Here's the structure I use for all my Playwright projects:

tests/
├── pages/
│   ├── BasePage.ts
│   ├── LoginPage.ts
│   ├── DashboardPage.ts
│   └── CheckoutPage.ts
├── components/
│   ├── Header.ts
│   ├── Modal.ts
│   └── Sidebar.ts
├── fixtures/
│   └── fixtures.ts
├── specs/
│   ├── login.spec.ts
│   ├── dashboard.spec.ts
│   └── checkout.spec.ts
└── playwright.config.ts

Creating a Base Page Class

Start with a base class that all pages extend:

// tests/pages/BasePage.ts
import { Page, Locator } from '@playwright/test';
export abstract class BasePage {
  protected page: Page;
  constructor(page: Page) {
    this.page = page;
  }
  async goto(): Promise<void> {
    await this.page.goto(this.url);
  }
  abstract get url(): string;
  // Common elements across all pages
  get header(): Locator {
    return this.page.getByRole('banner');
  }
  get footer(): Locator {
    return this.page.getByRole('contentinfo');
  }
  // Common actions
  async waitForPageLoad(): Promise<void> {
    await this.page.waitForLoadState('networkidle');
  }
}

Creating a Login Page Object

Now create a specific page class:

// tests/pages/LoginPage.ts
import { Page, Locator, expect } from '@playwright/test';
import { BasePage } from './BasePage';
export class LoginPage extends BasePage {
  constructor(page: Page) {
    super(page);
  }
  get url(): string {
    return '/login';
  }
  // Locators (use getters for lazy evaluation)
  get emailInput(): Locator {
    return this.page.getByLabel('Email address');
  }
  get passwordInput(): Locator {
    return this.page.getByLabel('Password');
  }
  get submitButton(): Locator {
    return this.page.getByRole('button', { name: 'Sign in' });
  }
  get errorMessage(): Locator {
    return this.page.getByRole('alert');
  }
  get forgotPasswordLink(): Locator {
    return this.page.getByRole('link', { name: 'Forgot password?' });
  }
  // Actions
  async login(email: string, password: string): Promise<void> {
    await this.emailInput.fill(email);
    await this.passwordInput.fill(password);
    await this.submitButton.click();
  }
  async loginAndExpectDashboard(email: string, password: string): Promise<void> {
    await this.login(email, password);
    await expect(this.page).toHaveURL(/.*dashboard/);
  }
  async loginWithInvalidCredentials(email: string, password: string): Promise<void> {
    await this.login(email, password);
    await expect(this.errorMessage).toBeVisible();
  }
}

Creating a Dashboard Page Object

// tests/pages/DashboardPage.ts
import { Page, Locator } from '@playwright/test';
import { BasePage } from './BasePage';
export class DashboardPage extends BasePage {
  constructor(page: Page) {
    super(page);
  }
  get url(): string {
    return '/dashboard';
  }
  // Locators
  get welcomeMessage(): Locator {
    return this.page.getByRole('heading', { name: /Welcome/ });
  }
  get userMenu(): Locator {
    return this.page.getByRole('button', { name: 'User menu' });
  }
  get logoutButton(): Locator {
    return this.page.getByRole('menuitem', { name: 'Logout' });
  }
  get settingsLink(): Locator {
    return this.page.getByRole('link', { name: 'Settings' });
  }
  // Actions
  async logout(): Promise<void> {
    await this.userMenu.click();
    await this.logoutButton.click();
  }
  async navigateToSettings(): Promise<void> {
    await this.settingsLink.click();
  }
}

Using Page Objects in Tests

Now your tests are clean and readable:

// tests/specs/login.spec.ts
import { test, expect } from '@playwright/test';
import { LoginPage } from '../pages/LoginPage';
import { DashboardPage } from '../pages/DashboardPage';
test.describe('Login functionality', () => {
  test('user can login with valid credentials', async ({ page }) => {
    const loginPage = new LoginPage(page);
    const dashboardPage = new DashboardPage(page);
    await loginPage.goto();
    await loginPage.login('valid@example.com', 'password123');
    await expect(dashboardPage.welcomeMessage).toBeVisible();
  });
  test('user sees error with invalid credentials', async ({ page }) => {
    const loginPage = new LoginPage(page);
    await loginPage.goto();
    await loginPage.loginWithInvalidCredentials('invalid@example.com', 'wrong');
    await expect(loginPage.errorMessage).toContainText('Invalid credentials');
  });
  test('user can navigate to forgot password', async ({ page }) => {
    const loginPage = new LoginPage(page);
    await loginPage.goto();
    await loginPage.forgotPasswordLink.click();
    await expect(page).toHaveURL(/.*forgot-password/);
  });
});

Advanced: Component Objects

For reusable UI components (modals, headers, sidebars), create component classes:

// tests/components/Modal.ts
import { Page, Locator } from '@playwright/test';
export class Modal {
  private page: Page;
  private container: Locator;
  constructor(page: Page) {
    this.page = page;
    this.container = page.getByRole('dialog');
  }
  get title(): Locator {
    return this.container.getByRole('heading');
  }
  get closeButton(): Locator {
    return this.container.getByRole('button', { name: 'Close' });
  }
  get confirmButton(): Locator {
    return this.container.getByRole('button', { name: 'Confirm' });
  }
  get cancelButton(): Locator {
    return this.container.getByRole('button', { name: 'Cancel' });
  }
  async close(): Promise<void> {
    await this.closeButton.click();
  }
  async confirm(): Promise<void> {
    await this.confirmButton.click();
  }
}

Use it in page objects:

// In DashboardPage.ts
async openDeleteModal(): Promise<Modal> {
  await this.deleteButton.click();
  return new Modal(this.page);
}

Advanced: Using Playwright Fixtures

For even cleaner tests, integrate page objects with Playwright fixtures:

// tests/fixtures/fixtures.ts
import { test as base } from '@playwright/test';
import { LoginPage } from '../pages/LoginPage';
import { DashboardPage } from '../pages/DashboardPage';
type MyFixtures = {
  loginPage: LoginPage;
  dashboardPage: DashboardPage;
};
export const test = base.extend<MyFixtures>({
  loginPage: async ({ page }, use) => {
    await use(new LoginPage(page));
  },
  dashboardPage: async ({ page }, use) => {
    await use(new DashboardPage(page));
  },
});
export { expect } from '@playwright/test';

Now tests are even cleaner:

// tests/specs/login.spec.ts
import { test, expect } from '../fixtures/fixtures';
test('user can login', async ({ loginPage, dashboardPage }) => {
  await loginPage.goto();
  await loginPage.login('user@example.com', 'password123');
  await expect(dashboardPage.welcomeMessage).toBeVisible();
});

Best Practices

1. Use getters for locators

Getters are lazily evaluated. The locator is resolved when accessed, not when the class is instantiated.

// Good
get submitButton(): Locator {
  return this.page.getByRole('button', { name: 'Submit' });
}
// Avoid (evaluated too early)
submitButton = this.page.getByRole('button', { name: 'Submit' });

2. Keep actions atomic

Each method should do one logical thing:

// Good
async login(email: string, password: string) { ... }
async fillEmail(email: string) { ... }
// Avoid combining unrelated actions
async loginAndCheckSettings() { ... }

3. Don't add assertions in page objects (usually)

Assertions belong in tests. Page objects encapsulate interactions.

Exception: Verification methods like loginAndExpectDashboard() can include assertions for convenience.

4. Use role-based locators

More resilient than CSS selectors:

// Good
page.getByRole('button', { name: 'Submit' })
// Fragile
page.locator('#submit-btn-primary')

Common Mistakes to Avoid

1. God page objects

If your page object has 50+ methods, break it into smaller components.

2. Exposing raw locators everywhere

Encapsulate interactions: loginPage.login() instead of loginPage.submitButton.click().

3. Not waiting for navigation

After clicks that navigate, wait for the URL or element:

await this.submitButton.click();

await this.page.waitForURL(/.*dashboard/);

4. Sharing state between tests

Each test should be independent. Don't rely on previous test state.

Folder Structure for Large Projects

For 100+ tests, organize by feature:

tests/
├── pages/
├── components/
├── fixtures/
├── auth/
│   ├── login.spec.ts
│   ├── register.spec.ts
│   └── password-reset.spec.ts
├── checkout/
│   ├── cart.spec.ts
│   ├── payment.spec.ts
│   └── confirmation.spec.ts
└── admin/
    ├── users.spec.ts
    └── settings.spec.ts

Need Architecture Help?

If you're building a test automation framework and want it done right from the start, I can help design the architecture, set up page objects, and train your team on best practices.

Get in touch

You Were Handed Testing and No One Trained You: A Developer's Field Guide

Anton Gulin — Sat, 29 Aug 2026 15:00:41 +0000

Your team moved testing to the left. That is the industry's word for it: shift-left. Testing that used to happen at the end, done by a separate QA team, now happens earlier, done by the people writing the code.

On my most recent project, that meant the developers. Most had never opened Playwright. None of them thought like a tester. And they still had to ship tested code every sprint.

The same five questions came in every week. This is the field guide I wish I could have handed each of them on day one. If testing just landed on your desk, start here.

1. "How do I wait for the page to be ready?"

The instinct is to sleep for a few seconds and hope. Do not. A sleep is a guess. Guess too short and the test is flaky. Guess too long and your suite crawls.

Playwright already waits for you. When you act on an element, it waits until that element is ready to use. Visible, stable, clickable. Then it acts.

// Fragile: a blind guess
await page.waitForTimeout(3000);
await page.locator('#save').click();

// Correct: Playwright waits for the button to be ready, then clicks
await page.getByRole('button', { name: 'Save' }).click();

Wait for the thing, never for the clock.

2. "Why does my test pass, then fail with no change?"

That is a flaky test. It is not bad luck. It means two things raced and your test did not say which one had to finish first.

The fix is not to add a sleep. The fix is to wait for the real signal, the row appears, the spinner is gone, the network call returned.

// Flaky: asserts before the row has rendered
await page.getByRole('button', { name: 'Add' }).click();
await expect(page.getByRole('row')).toHaveCount(1);

// Stable: web-first assertion retries until the state is true
await page.getByRole('button', { name: 'Add' }).click();
await expect(page.getByRole('row', { name: 'New item' })).toBeVisible();

A flaky test is not a small annoyance. It teaches the team to ignore red, and an ignored red is where real bugs ship.

3. "What am I even supposed to check?"

Check what the user gets. Not what the console logs, not that the page loaded, the actual outcome on the screen.

// Weak: proves the page rendered, nothing more
await expect(page).toHaveURL(/checkout/);

// Strong: proves the user sees the right total
await expect(page.getByTestId('order-total')).toHaveText('$41.00');

If your assertion would still pass when the feature is broken, it is decoration, not proof.

4. "My tests are so slow."

Most slow suites log in through the UI before every single test. If you have 200 tests, that is 200 logins. Do it once.

// One-time setup: save the signed-in state
await page.context().storageState({ path: 'auth.json' });

// Every test reuses it, no login step
test.use({ storageState: 'auth.json' });

Testing the login belongs in one test. The other 199 should assume it works.

5. "It works on my machine."

Then it is not tested. It is remembered. A test that only passes with your leftover data and your logged-in browser is not proof anyone else can trust.

Run it clean and isolated, every time: fresh data, no shared login, no state left behind from the last run. If it passes on a machine that knows nothing about you, it is real.

The one habit under all five

A test is a claim that the software works. Your job is to make that claim honest. Wait for real signals, check what the user sees, keep each test independent, and when a test fails, read the evidence before you touch it. Playwright's trace viewer records exactly what the browser did, so you can tell a real bug from a broken test in about two minutes.

Shift-left handed you testing. Nobody handed you the manual. Now you have a start.

Your Regression Suite Is a Museum: 5 Questions That Decide Delete vs Keep

Anton Gulin — Sat, 22 Aug 2026 15:00:46 +0000

Every sprint adds tests. Almost no sprint deletes one.

That one-way flow has a predictable end. The suite gets slower. Flaky tests
(tests that fail randomly) pile up. Engineers stop trusting red builds.
Eventually someone suggests rewriting everything, and the cycle restarts.

There is a cheaper fix. Audit the suite with five questions.
On the last suite I inherited, they retired a third of the tests.
Nothing those tests "guarded" ever broke.

Question 1: When did this test last fail for a real reason?

Your CI history (the record of server test runs) already knows.

A test that has been green for a year has two possible explanations.
Either the code it guards never changed, or the test cannot detect change.
Both are worth knowing. Only one deserves compute on every commit.

Practical rule: pull the last 90 days of runs. Tag every test that failed
only for environment reasons, or never failed at all. Those are audit candidates.

Question 2: What user risk does it guard?

Write the risk in one sentence, naming a user.

"It tests the profile page" is not a risk.
"A user loses saved work when the session expires" is a risk.

If nobody on the team can produce that sentence, the test is guarding
an implementation detail, not a user. Details change on purpose all the time.
Tests guarding them fail on purpose all the time. That is where flakiness lives.

Question 3: Would anyone notice if it vanished?

Run the experiment instead of debating it:

# in a branch: skip the suspect test, run everything else
npx playwright test --grep-invert "@museum-candidate"

If coverage of the named risk survives through other tests, and no gap
appears in the risk list from Question 2, the test was furniture.

Delete it in the branch. Keep the branch open a week. Merge with confidence.

Question 4: Does it check results or steps?

A test that clicks through checkout and asserts the button was clickable
is a tour, not a test.

// a tour: asserts the step happened
await page.getByRole('button', { name: 'Pay' }).click();
await expect(page).toHaveURL(/confirmation/);

// a test: asserts the outcome is real
await expect(page.getByTestId('invoice-total')).toHaveText('$34.20');
await expect(page.getByTestId('invoice-number')).not.toBeEmpty();

The URL can change while the invoice ships blank. Assert on the thing
the user came for.

Question 5: Can it fail at all?

Flip the condition and run it.

// original
await expect(status).toBe('paid');
// flipped: this MUST fail. If it passes, the test is dead.
await expect(status).not.toBe('paid');

A test that passes both ways asserts nothing. AI-written tests fail this
check more than any other kind, because generators optimize for green.

Sixty seconds per suspicious test. The flip never lies.

What you get back

Five questions, one afternoon, on the oldest third of your suite.

The suite you keep is faster, and every test in it can answer
"what breaks if I go red?" That is what makes a red build mean something.

And when an AI agent starts writing tests into your suite, this audit
is the contract you hold its work against. A generated test enters
only if it survives the same five questions.