Let's save the clues from a failed Playwright test

Originally published on anton.qa.

Let's run a checkout test that fails once, then passes.
We'll use Playwright, a browser testing tool.
You'll compare two settings and open the recordings they keep.

A trace is a detailed test recording.
You can inspect actions, saved page views, and requests from that attempt.
A retry is another attempt after a failure.

The useful difference in this example:

• on-first-retry keeps the successful retry's recording.
• retain-on-failure keeps the original failure's recording.

The test creates both responses so you can repeat this comparison.
You can see the supplied error in the original failure's saved response.
It runs on your computer and doesn't contact a payment provider.

The complete runnable project includes all files.
An agent verified a fresh public download while preparing the article.

Set up the practice project

Use Node.js 24, which runs JavaScript programs.
Its included npm tool installs project packages.
You'll also need Git, which downloads and tracks source code.

Run these commands in your terminal:

git clone https://github.com/antongulin/anton-qa-resources.git
cd anton-qa-resources/posts/save-failed-playwright-test
npm ci
npx playwright install chromium

The project fixes Playwright at version 1.63.0 for this example.
The last command installs Chromium, the browser used for testing.

On Linux, missing browser libraries may require this command:

npx playwright install --with-deps chromium

You don't need to start the practice website yourself.
The tests start and stop their own local server at 127.0.0.1:4191.
If you've started npm run serve yourself, stop it with Ctrl+C first.

Run the example and check its recordings

npm run verify

This command runs the tests and checks the saved recordings.
It replaces the project's generated results from any previous run.

Expect the checkout test to fail first, then pass on retry.
Playwright labels that result flaky: it needed another attempt to pass.
Here, we created that behavior so you can repeat the comparison.

The verifier checks both attempt numbers and the recorded responses.
It also confirms that the failed recording contains the original failed check.
You should see successful verification despite the deliberately failed first attempts.

The project includes three additional checks for related browser-testing examples.
Those checks should pass too.

See how the test creates its failure

Open tests/core-trace.spec.ts in your editor.
The test uses testInfo.retry to read the attempt number.
Zero means the original attempt. One means the first retry.

It fills an order note and presses Save checkout.
A route handler, code that intercepts requests, supplies the checkout response.
On attempt zero, it supplies an error. On retry one, it supplies success.

A fixture means a controlled test setup here.
The example uses that word in its exact response messages:

Attempt	Response code	Message supplied by the test
Original, 0	503, service unavailable	Injected first-attempt save failure
Retry, 1	200, success	Saved on retry by synthetic fixture

Both attempts run the same assertion, a check of the expected result.
This exact excerpt comes from the test:

await expect(page.getByRole('status')).toHaveText('Saved on retry by synthetic fixture');

The first attempt displays the error message, so this check fails.
The retry displays the expected success message, so it passes.
No timing guess or random failure decides the outcome.

Compare the two recording settings

The project runs that test under two sets of settings.
Playwright calls these projects, named groups of test settings.

These two project entries come from playwright.config.ts:

[
  {
    name: 'on-first-retry',
    testMatch: /core-trace\.spec\.ts/,
    retries: 1,
    use: { trace: 'on-first-retry' },
  },
  {
    name: 'retain-on-failure',
    testMatch: /core-trace\.spec\.ts/,
    retries: 1,
    use: { trace: 'retain-on-failure' },
  },
]

These entries belong inside the existing configuration, followed by the companion project.
Keep the downloaded file intact. You don't need to edit it for this walkthrough.

on-first-retry records the first retry and keeps that recording.
In our example, you get the successful second attempt.

retain-on-failure records each attempt and keeps recordings of failed attempts.
In our example, you get the original failed attempt.

Playwright's recording settings describe additional modes and combinations.
The two here let you compare a successful retry with its original failure.

Open the successful retry

Run this command after verification:

npx playwright show-trace test-results/*on-first-retry-retry1/trace.zip

Trace Viewer opens the recording for inspection.
Select the Save checkout action, then look at the saved page view.
Open Network, the list of recorded requests.
Find the checkout request ending in /api/checkout.

The request asks the app to save the order note.
Its response carries the result.
You should find response code 200 and the supplied success message.

This recording lets you inspect what happened during the retry.
To inspect the first failure, open its own recording next.

Open the original failed attempt

npx playwright show-trace test-results/*retain-on-failure/trace.zip

Find the failed text check in the action list.
Compare the expected message with the received message.
The page received Injected first-attempt save failure.
The test expected Saved on retry by synthetic fixture.

Next, find /api/checkout under Network.
Inspect its response code and response body, the returned data.
The recording contains code 503 and the injected error message.

You can now connect the received response with the page's message and failed check.
In this example, the test supplied the error on purpose.
This demonstrates saved evidence, rather than diagnosing a real service outage.

The Trace Viewer guide explains its inspection panels.

Keep a record of what you found

Open the complete test report with:

npm run show-report

For the verifier's findings, open artifacts/verification-report.json in your editor.
JSON is a structured data format.
This file lists attempt numbers, saved response bodies, and recording file locations.

The verifier reads those recordings to confirm the comparison.
A filename alone doesn't establish which response the browser received.
The tested project revision preserves the version used for this article.

For your own investigation, record a few concrete details:

• The test name and attempt number.
• The expected result and what appeared instead.
• The relevant response and a link to the saved recording.

That gives the next person somewhere useful to start.

Choose recording settings for your project

If you need the original failure, choose a mode that keeps that attempt.
If you're investigating the retry, record the retry.

Recording more attempts uses processing time and storage.
Saved pages and responses can also contain private data.
Decide who can open those files and when to delete them.

Try this comparison on the practice project before changing your team's settings.
You'll know which recording to look for when a real test fails.

Anton Gulin is the AI QA Architect, the first person to claim this title on LinkedIn. He builds AI-powered test automation systems where AI agents and human engineers collaborate on quality. Former Apple SDET (Apple.com / Apple Card pre-release testing). Find him at anton.qa or on LinkedIn.