DEV Community

Libme
Libme

Posted on

Playwright vs Cypress When Your E2E Tests Pass Locally but Fail in CI

#testing #cicd #javascript #webdev

Tests that pass on your laptop and fail in CI are almost never a tool bug; they are timing assumptions that a slower, colder CI runner exposes. Both Playwright and Cypress can be made stable, but they get there differently: Playwright gives you built-in retries, per-test traces, and free sharding across runners; Cypress gives you retry-able assertions inside the browser and a very good interactive debugger, with parallelization living behind its paid cloud service. If you are choosing today, Playwright is the lower-friction path for a CI-first team, and Cypress is the stronger pick when the people writing tests are mostly debugging them by watching the browser.

What does "flaky in CI only" actually look like?

The two failure texts I see most often are worth recognizing on sight. In Playwright:

Error: locator.click: Test timeout of 30000ms exceeded.
Call log:
  - waiting for getByRole('button', { name: 'Save' })
  -   locator resolved to <button disabled …>Save</button>
  - attempting click action
  -   waiting for element to be visible, enabled and stable
  -   element is not enabled
Enter fullscreen mode Exit fullscreen mode

In Cypress:

CypressError: Timed out retrying after 4000ms: Expected to find element: `[data-test=save-button]`, but never found it.
Enter fullscreen mode Exit fullscreen mode

Locally these never fire because your dev server responds in tens of milliseconds and the button becomes enabled before the test even looks for it. In CI, the app boots cold, the runner has two vCPUs shared with the browser, and the same request takes long enough that the test's assumption ("by the time I click, the form is ready") stops holding. The tool is telling the truth: the element genuinely was not ready inside its default budget.

The dead ends people try first are the same in both tools: adding waitForTimeout(2000) or cy.wait(2000), then bumping the number when it fails again. That trades a flaky test for a slow one and still fails on the one run where CI is slower than your padding. The fix is always to wait for the condition the click depends on, and both tools have a proper primitive for that.

Takeaway: a CI-only failure is a hidden sleep in your test logic, and the fix is naming the condition you were implicitly assuming.

How do Playwright and Cypress wait differently?

Playwright auto-waits before every action: click() will not fire until the element is attached, visible, stable, enabled, and receiving pointer events. Assertions like expect(locator).toBeEnabled() retry until the expect timeout. What it does not do is wait for arbitrary application state, so if your button is enabled before its data is loaded, you have to say what you mean:

import { test, expect } from '@playwright/test';

test('saves the profile', async ({ page }) => {
  const profileLoaded = page.waitForResponse(
    (r) => r.url().includes('/api/profile') && r.status() === 200
  );
  await page.goto('/settings');
  await profileLoaded;

  await page.getByLabel('Display name').fill('Jay');
  await page.getByRole('button', { name: 'Save' }).click();
  await expect(page.getByText('Saved')).toBeVisible();
});
Enter fullscreen mode Exit fullscreen mode

Cypress runs inside the browser and makes commands retry-able rather than actions auto-waiting. cy.get() and the assertion chained to it keep retrying until the default command timeout, and the idiomatic wait for network is an intercept alias:

it('saves the profile', () => {
  cy.intercept('GET', '/api/profile').as('profile');
  cy.visit('/settings');
  cy.wait('@profile');

  cy.findByLabelText('Display name').type('Jay');
  cy.get('[data-test=save-button]').should('be.enabled').click();
  cy.contains('Saved').should('be.visible');
});
Enter fullscreen mode Exit fullscreen mode

Both are correct. The difference that matters in CI is what happens when they are not correct. Playwright's error above came with a call log that said "element is not enabled", which points straight at the cause. Cypress's error tells you the element was never found, and to see why you open the run's screenshot or video, or use Test Replay if you are on Cypress Cloud. In my experience the call log alone resolves a good share of CI-only failures without downloading anything.

One Cypress-specific trap: because commands are queued and run asynchronously, mixing them with plain await or if on a value you have not yet yielded produces tests that pass locally by luck. Playwright's plain async/await model has fewer of these footguns, but it has its own: forgetting an await on an action silently races the next line, and TypeScript will not always warn you.

Takeaway: Playwright waits on the action and tells you which precondition failed; Cypress waits on the query and tells you it eventually gave up.

What do retries and artifacts cost you in each tool?

Retries are a diagnostic tool, not a fix, and both tools support them in a way that keeps the first-run failure visible.

Playwright's config lets you retry only in CI and capture a full trace only when a retry happens, which keeps artifact size small on green runs:

// playwright.config.ts
import { defineConfig } from '@playwright/test';

export default defineConfig({
  retries: process.env.CI ? 2 : 0,
  workers: process.env.CI ? 2 : undefined,
  fullyParallel: true,
  reporter: process.env.CI ? 'blob' : 'html',
  use: {
    trace: 'on-first-retry',
    screenshot: 'only-on-failure',
    video: 'retain-on-failure',
  },
});
Enter fullscreen mode Exit fullscreen mode

A test that passes on retry is reported as "flaky" rather than "passed", so you can grep the report for tests that are quietly degrading. The trace file opens in the trace viewer with a DOM snapshot at every step, network log, and console output; it is the single feature that made me stop reproducing CI failures locally.

Cypress configures retries per mode, and since version 13 (as of my last check) does not record video by default, so you opt in:

// cypress.config.js
const { defineConfig } = require('cypress');

module.exports = defineConfig({
  retries: { runMode: 2, openMode: 0 },
  video: true,
  e2e: { baseUrl: 'http://localhost:3000' },
});
Enter fullscreen mode Exit fullscreen mode

The honest limitation on the Cypress side is that the richest artifact, Test Replay, requires recording to Cypress Cloud. Without it you are working from screenshots and video, which show what happened but not the network timeline. On the Playwright side, the honest limitation is that traces on a large suite with trace: 'on' get big fast and slow uploads down, which is why the on-first-retry setting is the one to keep.

Takeaway: turn on retries in CI only, and treat "flaky" in the report as a bug queue, not a pass.

When does parallelization change the decision?

This is where the tools diverge most sharply. Playwright shards a suite across CI machines with a flag and merges the reports afterward, with no service involved:

# .github/workflows/e2e.yml (excerpt)
strategy:
  fail-fast: false
  matrix:
    shard: [1, 2, 3, 4]
steps:
  - run: npx playwright test --shard=${{ matrix.shard }}/4
  - uses: actions/upload-artifact@v4
    with:
      name: blob-report-${{ matrix.shard }}
      path: blob-report
Enter fullscreen mode Exit fullscreen mode

A follow-up job downloads the blobs and runs npx playwright merge-reports --reporter html ./all-blob-reports. Test isolation comes from a fresh browser context per test, which is cheap, so fullyParallel is usually safe unless tests share a database row.

Cypress runs one spec at a time per machine. Splitting specs across machines is a Cypress Cloud feature (load-balanced by historical duration), and the open-source alternatives are third-party orchestrators you host yourself. For a small suite this does not matter. For a suite that has crossed the point where a single runner takes fifteen minutes, it is often the deciding factor, because you are choosing between a recurring bill and operating another service.

Concern Playwright Cypress
Waiting model Auto-wait on actions, retrying expect Retry-able commands and chained assertions
Default failure detail Step call log in the error Screenshot; video and Test Replay optional
Retries retries in config, per-project retries per run/open mode
Trace/time-travel Trace viewer, local files Time-travel in interactive runner; Test Replay via Cloud
Parallel across machines Built-in --shard, free Cypress Cloud or self-hosted orchestrator
Multi-tab / multi-origin Native contexts and pages cy.origin() for cross-origin; no true multi-tab
Browsers Chromium, Firefox, WebKit Chrome family, Firefox, WebKit experimental
Debugging locally UI mode, --debug, trace viewer Interactive runner with DOM snapshots per command

If your team's bottleneck is CI wall-clock time and you want sharding without a vendor, Playwright is the one that gives you free cross-machine parallelism with a single flag. If your team's bottleneck is people understanding why a test failed while watching it run, Cypress is the one whose interactive runner makes every command's before-and-after DOM state clickable without extra setup.

Takeaway: the moment your suite outgrows one runner, the parallelization model stops being a feature comparison and becomes a budget line.

FAQ

Why do my Playwright tests pass locally but fail in CI?
Because CI is slower and colder, so elements that were already ready on your machine are still loading when the test acts. Replace fixed waits with expect(locator) assertions or page.waitForResponse for the specific request the action depends on, and enable trace: 'on-first-retry' to see the exact step that stalled.

How do I make Cypress tests less flaky in CI?
Set retries: { runMode: 2, openMode: 0 }, wait on intercepted requests with cy.intercept().as() and cy.wait('@alias') instead of cy.wait(ms), and chain .should('be.enabled') before clicking. Treat any test that only passes on retry as a bug to fix, not a green result.

Is Playwright or Cypress better for CI?
Playwright has the lower CI cost because retries, traces, and cross-machine sharding are built in and free. Cypress is competitive on a single runner but its parallelization and richest failure artifacts depend on Cypress Cloud.

Bottom line

If you are starting a new suite and it will run primarily in CI, pick Playwright: auto-waiting reduces the class of timing bugs, the trace viewer answers "why did this fail" without a local repro, and sharding is a flag rather than a subscription. If your team already has a Cypress suite and mostly debugs by watching the interactive runner, stay and fix the flake with intercept aliases and CI-only retries; the tool is not the problem. Migrate only when a single runner's wall-clock time becomes the thing blocking merges, because that is the one gap the free tier of Cypress does not close.

Related reading

Top comments (0)