Why the same page gives different Playwright screenshots: five sources, each measured

A visual test that fails once in ten runs is usually blamed on "the browser being flaky". I wanted to know which parts of a page actually change the pixels, so I took the same page over and over and counted differing pixels while switching one suspect at a time.

Short version, from one machine:

• A live clock, a running CSS animation and a screenshot taken mid-transition each produced different PNGs from the same page.
• A blinking text caret changed 5 pixels, depending only on timing.
• Re-launching the browser, using new headless, --disable-gpu and --font-render-hinting=none changed nothing.
• --disable-lcd-text changed 1085 pixels, because the default launch draws coloured sub-pixel text.

Setup

Linux 6.8, AMD EPYC VM, Node v24.22.2, Playwright 1.63.0 with Chromium 153.0.8010.12 (headless shell). Pages are served from 127.0.0.1. Differences are counted with pixelmatch at threshold: 0, so any change in any channel counts.

npm i playwright@1.63.0 pngjs@7 pixelmatch@7

The fixture has a timestamp, an infinite pulse animation, a 400 ms transition that starts after load, and a focused input.

<!-- file: index.html -->
<!doctype html><meta charset="utf-8">
<style>
body{font:16px/1.4 system-ui,sans-serif;margin:24px;width:600px}
.pulse{width:120px;height:40px;background:#36c;animation:p 1s ease-in-out infinite alternate}
@keyframes p{from{opacity:.2}to{opacity:1}}
.slide{width:60px;height:20px;background:#c63;transition:transform 400ms}
.slide.go{transform:translateX(200px)}
input{font:16px sans-serif;width:200px}
</style>
<h1>Order #1042</h1>
<p>Rendered at <span id="t"></span></p>
<div class="pulse"></div>
<div class="slide" id="s"></div>
<p><input id="i" value="hello"></p>
<script>
document.getElementById('t').textContent = new Date().toISOString().slice(0, 19) + 'Z';
requestAnimationFrame(() => requestAnimationFrame(() => document.getElementById('s').classList.add('go')));
document.getElementById('i').focus();
</script>

A small helper serves it and compares PNGs.

// file: lib.js
const http = require('http'), fs = require('fs'), crypto = require('crypto');
const { PNG } = require('pngjs');
const pixelmatch = require('pixelmatch').default || require('pixelmatch');

exports.serve = () => new Promise(resolve => {
  const html = fs.readFileSync(__dirname + '/index.html');
  const s = http.createServer((req, res) => { res.setHeader('content-type', 'text/html'); res.end(html); });
  s.listen(0, '127.0.0.1', () => resolve({ url: `http://127.0.0.1:${s.address().port}/`, close: () => s.close() }));
});
// number of pixels that differ at all (threshold 0) between two PNG buffers
exports.diff = (a, b) => {
  const A = PNG.sync.read(a), B = PNG.sync.read(b);
  return pixelmatch(A.data, B.data, null, A.width, A.height, { threshold: 0 });
};
exports.sha = buf => crypto.createHash('sha1').update(buf).digest('hex').slice(0, 8);

The ablation

The baseline controls everything: a frozen clock (page.clock.install), animations: 'disabled', caret: 'hide' and a 700 ms wait. Each other row undoes one control. Every screenshot is a fresh page, 12 per row, and the numbers are compared with the first screenshot of the row.

// file: ablate.js
const { chromium } = require('playwright');
const { serve, diff, sha } = require('./lib');
const N = 12;
const BASE = { clock: true, animations: 'disabled', caret: 'hide', settleMs: 700, pulse: true };
const rows = [
  ['everything controlled', {}],
  ['live clock', { clock: false }],
  ['CSS animation running', { animations: 'allow' }],
  ['transition mid-flight', { animations: 'allow', pulse: false, settleMs: 0 }],
  ['transition finished', { animations: 'allow', pulse: false, settleMs: 700 }],
  ['mid-flight, disabled', { pulse: false, settleMs: 0 }],
];

(async () => {
  const srv = await serve();
  const browser = await chromium.launch();
  console.log('Chromium', browser.version(), `| ${N} fresh pages per row`);
  console.log('row'.padEnd(26), 'distinct PNGs', 'max px differing from first');
  for (const [name, change] of rows) {
    const c = { ...BASE, ...change };
    const shots = [];
    for (let i = 0; i < N; i++) {
      const page = await browser.newPage({ viewport: { width: 700, height: 360 } });
      if (c.clock) await page.clock.install({ time: new Date('2026-01-01T12:00:00Z') });
      await page.goto(srv.url);
      if (!c.pulse) await page.addStyleTag({ content: '.pulse{animation:none!important}' });
      if (c.settleMs) await page.waitForTimeout(c.settleMs);
      shots.push(await page.screenshot({ animations: c.animations, caret: c.caret }));
      await page.close();
    }
    const worst = Math.max(...shots.map(s => diff(shots[0], s)));
    console.log(name.padEnd(26), `${new Set(shots.map(sha)).size}/${N}`.padStart(8), String(worst).padStart(14));
  }

  // the caret: one settled page, six screenshots 260 ms apart, caret left as it is
  const page = await browser.newPage({ viewport: { width: 700, height: 360 } });
  await page.clock.install({ time: new Date('2026-01-01T12:00:00Z') });
  await page.goto(srv.url);
  await page.waitForTimeout(700);
  const shots = [];
  for (let i = 0; i < 6; i++) {
    shots.push(await page.screenshot({ animations: 'disabled', caret: 'initial' }));
    await page.waitForTimeout(260);
  }
  console.log('caret left visible, px differing from shot 0:', shots.map(s => diff(shots[0], s)).join(' '));
  await browser.close();
  srv.close();
})();

Output:

Chromium 153.0.8010.12 | 12 fresh pages per row
row                        distinct PNGs max px differing from first
everything controlled          1/12              0
live clock                    12/12            151
CSS animation running          7/12           4800
transition mid-flight          5/12           2360
transition finished            1/12              0
mid-flight, disabled           1/12              0
caret left visible, px differing from shot 0: 0 5 5 0 0 5

The exact counts change from run to run. I ran this script three times: the live clock gave 11 to 12 distinct images in 12, the running animation 5 to 7, and mid-flight transitions 3 to 6. Every row that is controlled gave 1 image and 0 differing pixels each time. Read the zeros as the result and the other counts as "not stable".

What this says:

• Clock. Anything that prints a time or date differs from run to run: 11 to 12 different PNGs in 12. The difference is only about 100 to 150 pixels, which is small, so a loose tolerance would hide it, but it would also hide real regressions of that size. Freeze the clock or mask the element.
• Infinite animation. 5 to 7 distinct images in 12, up to 4800 pixels. Not all 12 differ, presumably because some screenshots landed on the same animation phase. Equal results here are luck, not stability.
• Transition. Mid-flight, 3 to 6 distinct images in 12. Once finished, identical. Waiting works when you know the duration. animations: 'disabled' also worked: the last row takes the screenshot with no wait and got one image. Playwright's source documentation says it fast-forwards finite animations and transitions to completion and cancels infinite ones to their initial state (screenshot options).
• Caret. It is 5 pixels. With six screenshots 260 ms apart on one page, each was either 0 or 5 pixels from the first, in a pattern that timing decides. Another run of the same loop printed 0 5 5 0 5 5 instead of the 0 5 5 0 0 5 shown above, so do not rely on a lucky phase. page.screenshot() hides the caret by default according to the same document, which is why I had to set caret: 'initial' to see this.

Things that did not matter

Next I changed the browser instead of the page. The fixture is the same, with everything above controlled.

// file: launches.js
const { chromium } = require('playwright');
const { serve, diff, sha } = require('./lib');

(async () => {
  const srv = await serve();
  const shoot = async launchOptions => {
    const browser = await chromium.launch(launchOptions);
    const page = await browser.newPage({ viewport: { width: 700, height: 360 } });
    await page.clock.install({ time: new Date('2026-01-01T12:00:00Z') });
    await page.goto(srv.url);
    await page.waitForTimeout(700);
    const png = await page.screenshot({ animations: 'disabled', caret: 'hide' });
    await browser.close();
    return png;
  };
  const ref = await shoot({});
  const again = [ref];
  for (let i = 0; i < 4; i++) again.push(await shoot({}));
  console.log('5 separate launches, distinct PNGs:', new Set(again.map(sha)).size);
  const variants = {
    'channel: chromium (new headless)': { channel: 'chromium' },
    '--disable-gpu': { args: ['--disable-gpu'] },
    '--font-render-hinting=none': { args: ['--font-render-hinting=none'] },
    '--disable-lcd-text': { args: ['--disable-lcd-text'] },
  };
  for (const [name, opts] of Object.entries(variants)) {
    console.log(name.padEnd(34), 'px differing from default:', diff(ref, await shoot(opts)));
  }
  srv.close();
})();

5 separate launches, distinct PNGs: 1
channel: chromium (new headless)   px differing from default: 0
--disable-gpu                      px differing from default: 0
--font-render-hinting=none         px differing from default: 0
--disable-lcd-text                 px differing from default: 1085

Five launches gave byte-identical files, so a new process is not a source of noise by itself. The new headless mode and --disable-gpu gave the same pixels on this page, so I would not spend time on them first. --font-render-hinting=none also changed nothing here, but on this host and this fixture only.

--disable-lcd-text is the odd one. It changed 1085 pixels. A check on the <h1> explains why.

// file: lcd.js
const { chromium } = require('playwright');
const { serve } = require('./lib');
const { PNG } = require('pngjs');

(async () => {
  const srv = await serve();
  for (const [label, args] of [['default launch', []], ['--disable-lcd-text', ['--disable-lcd-text']]]) {
    const browser = await chromium.launch({ args });
    const page = await browser.newPage({ viewport: { width: 700, height: 360 } });
    await page.clock.install({ time: new Date('2026-01-01T12:00:00Z') });
    await page.goto(srv.url);
    await page.waitForTimeout(700);
    const box = await page.locator('h1').boundingBox();
    const png = PNG.sync.read(await page.screenshot({ animations: 'disabled', clip: box }));
    // black text on white: any pixel whose R, G and B differ by more than 8 is coloured, so subpixel-antialiased
    let coloured = 0;
    for (let i = 0; i < png.data.length; i += 4) {
      const [r, g, b] = [png.data[i], png.data[i + 1], png.data[i + 2]];
      if (Math.max(r, g, b) - Math.min(r, g, b) > 8) coloured++;
    }
    console.log(label.padEnd(20), 'coloured pixels in the <h1>:', coloured);
    await browser.close();
  }
  srv.close();
})();

default launch       coloured pixels in the <h1>: 1242
--disable-lcd-text   coloured pixels in the <h1>: 0

Black text on white should contain only greys. The default launch has 1242 coloured pixels in the heading, which is sub-pixel (LCD) antialiasing. With the flag the heading is pure grey. Whether the setting is on depends on the build and platform, so the same code can produce coloured text on one machine and grey on another. I could only test one machine, so I cannot say how often that happens. In my run, a plain paragraph lower on the page had no coloured pixels in the default launch either; I did not find out why, so check your own page rather than assume.

What I would do with this

1. Freeze time with page.clock.install() or mask the element with the mask option.
2. Use animations: 'disabled', or wait for known transitions to end. Leave caret at its default of hide.
3. Generate baselines in the same environment that compares them. The Playwright test docs say the same: rendering varies with OS, settings, hardware and headless mode, so run tests where the baselines were made (snapshots).
4. If you see coloured fringes on text, try --disable-lcd-text on both sides before loosening a pixel tolerance. I have only shown that it changes the output, not that it makes two different machines agree.

Limits: one fixture, one OS, Chromium only, 12 samples per row. I did not test web fonts that load late, images, video, canvas, scrollbars, or toHaveScreenshot() and its own retry logic. A real page with ads or carousels will have more noise than this one.

Written with AI assistance. Every code sample was run on 2026-11-09 with Playwright 1.63.0 (Chromium 153.0.8010.12), Node v24.22.2, pngjs 7.0.0 and pixelmatch 7.2.0; the output above is from that run.