DEV Community

Cover image for Why Your Playwright Tests Pass But Images Are Broken
Tanmay Gupta
Tanmay Gupta

Posted on

Why Your Playwright Tests Pass But Images Are Broken

#playwright #testing #qa #webdev

Your tests are green. CI passed. You deploy.

And then someone Slacks you: "Half the images on the product page are broken."

You check the test run. Everything passed. No failures. No warnings. The suite did exactly what it was told to do.

That's the gap nobody talks about.

Playwright doesn't catch what it's never asked to check

Broken images don't show up as passing tests. They show up as passing tests. Playwright isn't wrong here. It checked what your assertions told it to check. Nothing more.

Four things cause this:

1. A 404 The CDN path changed, a file got deleted, the src attribute points at something that no longer exists. The browser renders a broken icon. The test moves on.
2. An invisible image CSS hides the element. The container collapses to zero height. Something overlays it. isVisible() comes back true because the element is technically in the DOM. Users see nothing.

3. A 1x1 placeholder The image loads but it's a pixel-sized fallback from a bad API response or a failed upload. naturalWidth is 1. The browser is satisfied. You're not.

4. A lazy-loaded image your test never reached If the test didn't scroll far enough, the image never entered the viewport. Playwright only sees what's on screen.

Same result in your Playwright report for all four: a green checkmark.

What you can catch with Playwright today

Network interception

The most direct approach. Listen to every response, filter for image resource types, throw on anything returning 4xx.

// global-setup.ts or a shared fixture
page.on('response', response => {
  if (
    response.request().resourceType() === 'image' &&
    response.status() >= 400
  ) {
    console.error(`Broken image: ${response.url()} — ${response.status()}`);
  }
});
Enter fullscreen mode Exit fullscreen mode

Wire it into a global fixture and every test picks it up automatically. The catch: fixtures break quietly. Someone refactors the setup file, the listener disappears, and you won't know until users report broken images.

naturalWidth check

Catches images that failed to load entirely. Pair this with solid Playwright assertions to keep the checks reliable.

const brokenImages = await page.$$eval('img', imgs =>
imgs
.filter(img => img.naturalWidth === 0)
.map(img => img.src)

);

if (brokenImages.length > 0) {
  throw new Error(
    `${brokenImages.length} broken image(s) found:\n${brokenImages.join('\n')}`
  );
}
Enter fullscreen mode Exit fullscreen mode

Catches hard 404s. Misses the 1x1 placeholder case. Add a dimension check to cover those:

const suspiciousImages = await page.$$eval('img', imgs =>
  imgs
    .filter(img => img.naturalWidth <= 1 || img.naturalHeight <= 1)
    .map(img => ({ src: img.src, w: img.naturalWidth, h: img.naturalHeight }))
);
Enter fullscreen mode Exit fullscreen mode

Allure Playwright integration

If you're using Allure as your Playwright reporting tool, you can attach image check findings directly into the report as step annotations or attachments.

import { allure } from 'allure-playwright';

test('check for broken images', async ({ page }) => {
  await page.goto('/products');

  const brokenImages = await page.$$eval('img', imgs =>
    imgs.filter(img => img.naturalWidth === 0).map(img => img.src)
  );

  if (brokenImages.length > 0) {
    await allure.attachment(
      'Broken Images',
      Buffer.from(brokenImages.join('\n')),
      'text/plain'
    );
    throw new Error(`Found ${brokenImages.length} broken image(s)`);
  }
});
Enter fullscreen mode Exit fullscreen mode

Allure renders the attachment inline when someone views the failed test. The reviewer sees the list of broken image URLs without opening a separate trace file. That's a genuine improvement in report readability.

What Allure doesn't do is detect broken images on its own. The detection logic is yours. Allure surfaces what you push into it. If you don't write the check, nothing shows up regardless of which reporter you're using.

Visual regression snapshot testing

The most complete approach. Playwright visual testing uses toHaveScreenshot() to compare the current render against a saved baseline.

test('product page images render correctly', async ({ page }) => {
  await page.goto('/products');
  await page.waitForLoadState('networkidle');
  await expect(page).toHaveScreenshot('product-page.png');
});
Enter fullscreen mode Exit fullscreen mode

A pixel diff catches any image that changed, disappeared, or loaded incorrectly. The tradeoff is maintenance. Baselines drift between environments, font rendering differs between Linux CI and local macOS, and sensitivity thresholds need tuning. On a large team with frequent UI changes, managing baselines becomes a significant ongoing task.

Where all of this breaks down

Setting it up once is not the hard part. Living with it at scale is.

One CDN image breaks. 15 tests fail. Your Playwright HTML report shows 15 separate entries with no indication they're connected. Someone on your team opens each Playwright trace one by one, checks the network tab each time, and pieces together that it's all the same 404. That's 45 minutes on a good day.

Traces aren't fast even for a single failure. Download the zip, open the viewer locally, find the network panel, cross-reference with the screenshot. Useful, yes. Fast, no. At 40 failures you've just taken up someone's entire morning.

The part that frustrates teams most: you can't tell if this is new. Did the image break today or has it been flaky for two weeks? Did it start after last Tuesday's deploy? Both the Playwright HTML reporter and Allure focus on single-run visibility. Neither carries memory across runs, so even with solid detection logic, the reporting layer can't tell you whether this is a pattern or a one-off. Tracking tests across runs is where single-run reporters hit their ceiling.

What a reporting layer built for Playwright actually changes

TestDino is a Playwright-specific reporting and test management platform. Once your image assertions catch a failure, here's what's different.

In-platform traces, no downloading.Screenshot, network logs, console output, and trace data are all in one panel per test. What used to take 10 minutes to piece together takes about 30 seconds.

Error grouping.Those 15 tests failing from the same broken image show up as one issue in the Error Analysis tab, not 15. You look at it once, understand what broke, fix it once. See how Playwright debugging changes when failures are grouped by root cause.

AI failure classification.TestDino labels each failure as Bug, UI Change, Flaky, or Misc. For broken images, this routes the fix correctly without a triage meeting:

Test history. TestDino shows you the exact CI run where a test started failing. If your landing page hero image broke on Tuesday at 3pm, that's one click. No correlating GitHub Actions timestamps with deployment logs.

Tag analytics.Tag image tests with @visual or @images and TestDino gives you pass/fail trend data for that group across every run. You stop finding out images are broken when users report them. See how Playwright reporting metrics look when tracked over time.

GitHub PR comments. If an image assertion fails on a pull request, TestDino posts a test summary comment on that PR automatically. The developer who introduced the broken path sees it before the code merges. Production never sees it. The Playwright automation checklist covers how to wire this up end to end.

Quickstart fixture

Drop this into your Playwright project and every test gets automatic image coverage:

// fixtures/imageCheck.ts
import { test as base } from '@playwright/test';

export const test = base.extend({
  page: async ({ page }, use) => {
    const brokenImages: string[] = [];

    page.on('response', response => {
      if (
        response.request().resourceType() === 'image' &&
        response.status() >= 400
      ) {
        brokenImages.push(`${response.status()} — ${response.url()}`);
      }
    });

    await use(page);

    if (brokenImages.length > 0) {
      throw new Error(
        `Broken images detected:\n${brokenImages.join('\n')}`
      );
    }
  }
});
Enter fullscreen mode Exit fullscreen mode

Use this test instead of Playwright's default in your spec files:

// product-images.spec.ts
import { test } from '../fixtures/imageCheck';
import { expect } from '@playwright/test';

test('product page has no broken images', async ({ page }) => {
  await page.goto('/products');
  await page.waitForLoadState('networkidle');
  // fixture handles the image check automatically
  // add your other assertions here
});
Enter fullscreen mode Exit fullscreen mode

You can also build a custom Playwright reporter that outputs broken image data directly into your CI pipeline logs if you want the findings surfaced at the runner level rather than test level.

Where to start

If your tests aren't catching broken images at all, start with the fixture above. 20 minutes of setup, every test in your suite gets coverage without touching individual spec files.

If you're using Allure and want to enrich your reports with image check data, the allure-playwright attachment approach earlier works well alongside the fixture. The fixture throws on failures. The attachment gives reviewers context without opening traces.

If you're catching failures but triage is eating time, try TestDino at Sandbox. Nothing changes in how you run tests. Connect it, run a suite, see the grouping and classification on your own data.

The detection side of this is solvable with what Playwright gives you today. The part that actually costs teams time is everything that happens after a test fails. That's the part worth getting right.

Top comments (0)