Why Your Playwright Tests Randomly Fail on CI and How to Fix It

Flaky end-to-end tests are one of the most frustrating experiences in CI/CD. They pass on your machine, fail on the pipeline, and the error messages rarely point you toward the actual problem. Most developers do the intuitive thing: increase the timeout. Sometimes that works. Most of the time, it delays the failure rather than preventing it.

The real culprit is usually a race condition between page navigation, asynchronous network requests, and DOM updates.

The Root Cause: Unwaited Network Activity

When you click a button that triggers an AJAX call, the browser doesn't wait for that request to complete before moving to the next line of your test. If your test then asserts on data that arrives via that AJAX call, you're gambling on network latency.

Locally, your machine is fast and close to the API. The race condition almost always resolves in your favor. On CI, especially in containerized environments, latency is variable and unpredictable. Now that same test fails intermittently, and it feels random.

It's not random. It's deterministic once you understand what's happening.

The Wrong Solution: Arbitrary Waits

// This is guessing, not waiting
await Task.Delay(3000);
await page.ClickAsync("#submit-btn");

Bumping delays to 5 seconds, 10 seconds, or higher might reduce flakiness, but it won't eliminate it. You're still guessing. And you're slowing down your test suite significantly.

The Right Solution: Event-Based Waits

Playwright provides explicit wait mechanisms that pause execution until a specific condition is met.

Wait for a Specific Response

If your test triggers a known API call, wait for that response explicitly:

await page.ClickAsync("#submit-btn");

// Wait for the specific network call to complete
var responseTask = page.WaitForResponseAsync("**/api/orders");
await responseTask;

var response = await responseTask;
Assert.Equals(200, response.Status);

// Now it's safe to check the DOM
await page.WaitForSelectorAsync(".order-confirmation");

Wait for Network Idle

If you're not sure which calls a click triggers, or if there are multiple concurrent requests:

await page.ClickAsync("#load-dashboard-btn");
await page.WaitForLoadStateAsync(LoadState.NetworkIdle);

// At this point, all network activity has settled
var dashboardContent = await page.TextContentAsync(".dashboard");
Assert.Contains("Welcome", dashboardContent);

Wait for DOM Changes

For dynamic content that doesn't necessarily involve network calls:

await page.ClickAsync("#filter-btn");
await page.WaitForSelectorAsync(".filtered-results", 
    new PageWaitForSelectorOptions { State = WaitForSelectorState.Visible });

var results = await page.Locator(".filtered-results li").Count();
Assert.True(results > 0);

Mock External APIs

Third-party APIs introduce uncontrollable latency and occasional failures. In CI, you want deterministic behavior. Use Playwright's API mocking:

// In your test setup or a beforeEach hook
await page.RouteAsync("**/api/external-rate-limit", route => 
{
    route.FulfillAsync(new RouteFulfillOptions
    {
        Status = 200,
        ContentType = "application/json",
        Body = "{\"rate\": 0.95, \"currency\": \"USD\"}"
    });
});

This ensures your tests aren't affected by external service availability or performance.

Isolate State Between Tests

Shared state between tests is a silent flakiness generator. One test sets something in localStorage, another test depends on a clean slate, and they interfere with each other.

// Use a fresh context for each test
private IBrowserContext _context;

[SetUp]
public async Task Setup()
{
    var browser = await Playwright.Chromium.LaunchAsync();
    _context = await browser.NewContextAsync();

    // Clear all storage for a clean state
    await _context.ClearCookiesAsync();
    await _context.ClearPermissionsAsync();
}

// Or if using storage state, clear explicitly
await _context.StorageStateAsync(new BrowserContextStorageStateOptions
{
    Path = null // Prevents state leakage between runs
});

Instrument, Don't Speculate

When a test fails on CI but passes locally, resist the urge to just bump the timeout. Instead, add logging to understand the timing:

var stopwatch = Stopwatch.StartNew();
await page.ClickAsync("#submit-btn");
await page.WaitForResponseAsync("**/api/submit");
stopwatch.Stop();

Console.WriteLine($"Response arrived after {stopwatch.ElapsedMilliseconds}ms");

// If this consistently takes 2.8 seconds on CI, you have data.
// Bumping to 3 seconds might be legitimate, or you might have a scaling issue.

This diagnostic approach tells you whether you need explicit waits, better mocking, or something else entirely.

Practical Takeaway

Replace arbitrary waits with explicit, event-based waits. waitForResponse and waitForLoadState are your primary tools. Mock external dependencies to eliminate variables. Clear browser context between tests to prevent state leakage.

Your test suite should be trustworthy across local dev, staging, and CI. When a test fails, it should be because something is actually broken, not because of timing luck.

---