DEV Community

Pirt
Pirt

Posted on • Originally published at poly.dev

How to Fix Flaky Tests: 10 Proven Strategies That Actually Work in 2026

#testing #ci #playwright #cypress

Every engineering team has that one test. You know the one. It fails every third run. Nobody wants to fix it because it "mostly works." But "mostly works" means "sometimes blocks your deploy" — and that's not good enough.

After analyzing thousands of flaky test patterns across CI pipelines, here are the 10 strategies that actually move the needle, ordered from quick wins to structural changes.

Strategy 1: Replace Hard Waits with Smart Assertions

The #1 cause of flakiness across all frameworks.

Hard waits (sleep, setTimeout, cy.wait(5000)) are a gamble. Sometimes 500ms is enough. Sometimes the API takes 800ms. You either wait too long (slow tests) or not long enough (flaky tests).

Before (Flaky):

await page.click('#submit');
await page.waitForTimeout(3000); // gambling
expect(await page.locator('.success').isVisible()).toBe(true);
Enter fullscreen mode Exit fullscreen mode

After (Stable):

await page.click('#submit');
await expect(page.locator('.success')).toBeVisible(); // waits until it appears
Enter fullscreen mode Exit fullscreen mode

Every modern test framework has assertion-based waiting:

  • Playwright: await expect(locator).toBeVisible()
  • Cypress: cy.get('.success').should('be.visible')
  • Jest + Testing Library: await waitFor(() => expect(screen.getByText('Success')).toBeInTheDocument())

Strategy 2: Use Stable, User-Visible Selectors

Tests that rely on CSS structure break when the UI changes. A designer wraps content in a new div, and your nth-child(3) selector finds the wrong element.

Before (Flaky):

await page.click('.card > div > button:nth-child(2)');
Enter fullscreen mode Exit fullscreen mode

After (Stable):

await page.getByRole('button', { name: 'Submit Order' });
await page.getByTestId('submit-order-button');
await page.getByLabel('Email address');
Enter fullscreen mode Exit fullscreen mode

Prioritize selectors in this order:

  1. Role + name (getByRole) — most stable, most accessible
  2. Test ID (getByTestId) — explicit, never changes with UI
  3. Label (getByLabel) — tied to form semantics
  4. Text content (getByText) — user-visible, but watch for duplicates
  5. CSS selector — last resort, most brittle

Strategy 3: Isolate Test Data Per Test

Tests sharing data is a silent flakiness bomb. Test A creates a user. Test B assumes that user exists. When Test B runs before Test A (parallel execution), it fails.

The Fix:

  • Create fresh data in beforeEach or at the start of each test
  • Use unique identifiers (UUIDs, timestamps) to prevent collisions
  • Clean up test data in afterEach
  • For databases, use transactions that rollback after each test 
beforeEach(async () => {
  const uniqueEmail = `test-${Date.now()}@example.com`;
  await createTestUser({ email: uniqueEmail });
});
Enter fullscreen mode Exit fullscreen mode

Strategy 4: Mock External Dependencies at the Network Level

Tests that call real external APIs (Stripe, SendGrid, AWS) will fail when those services are slow, rate-limited, or down.

The Fix: Intercept network requests at the browser/driver level, not by modifying application code.

Playwright:

await page.route('**/api/stripe/charge', (route) => {
  route.fulfill({
    status: 200,
    body: JSON.stringify({ id: 'ch_mock', status: 'succeeded' }),
  });
});
Enter fullscreen mode Exit fullscreen mode

Cypress:

cy.intercept('POST', '**/api/stripe/charge', {
  statusCode: 200,
  body: { id: 'ch_mock', status: 'succeeded' },
});
Enter fullscreen mode Exit fullscreen mode

Strategy 5: Eliminate Test Ordering Dependencies

If testA.js must run before testB.js, you have a hidden dependency that will break in parallel mode.

Signs of order dependency:

  • Tests pass sequentially but fail in parallel
  • A specific test always fails when the suite runs but passes individually
  • Flipping test file names changes which tests fail

Fixes:

  • Each test must be independently runnable
  • Use beforeAll/beforeEach to set up prerequisites
  • Run tests in random order in CI (--shuffle flag in Jest)
  • If you can't make a test independent, it's an integration test, not a unit test

Strategy 6: Handle Date and Time Properly

Time-dependent tests fail at boundaries: midnight, month-end, daylight saving, leap years.

Fixes:

  • Inject a clock dependency instead of using Date.now() directly
  • Use test-specific fixed dates
  • Test across timezone boundaries if your app is multi-timezone

Jest:

jest.useFakeTimers().setSystemTime(new Date('2026-07-09T12:00:00Z'));
// ... test ...
jest.useRealTimers();
Enter fullscreen mode Exit fullscreen mode

Strategy 7: Deduplicate and Consolidate Overlapping Tests

Some flakiness comes from test count. 2,000 tests running in parallel creates more resource contention than 500 well-structured tests. Many teams have redundant tests that cover the same code path 3-5 times.

Audit your test suite:

  1. Run tests with coverage reporting
  2. Identify code paths with 5+ tests covering them
  3. Keep the most specific test, delete the rest
  4. Replace 10 narrow unit tests with 1 integration test

This doesn't just reduce flakiness — it makes CI faster.

Strategy 8: Containerize Your Test Environment

"Different results on my machine vs CI" usually means environment differences. Different Node versions, different browser versions, different system libraries.

Fixes:

  • Use Docker containers for CI test runs
  • Pin exact versions in package.json (not ^ ranges)
  • Use package-lock.json and commit it
  • Run the same browser version locally and in CI 
FROM mcr.microsoft.com/playwright:v1.45.0-jammy
WORKDIR /app
COPY package*.json ./
RUN npm ci
COPY . .
RUN npx playwright install --with-deps
CMD ["npx", "playwright", "test"]
Enter fullscreen mode Exit fullscreen mode

Strategy 9: Implement the 95/30 Rule

Before investing time in fixing a flaky test, prove it's actually flaky with data.

The 95/30 Rule: Run the test 30 times. If it doesn't pass at least 95% of the time (28+ passes), quarantine it.

This prevents spending hours "fixing" a test that's actually revealing a real intermittent bug.

# Quick flakiness check
for i in {1..30}; do
  npx playwright test --grep "checkout flow" --reporter=line 2>&1 | tail -1
done | sort | uniq -c
Enter fullscreen mode Exit fullscreen mode

Strategy 10: Add Real-Time Failure Classification to CI

The most impactful structural change: instead of treating every CI failure equally, classify each failure as "likely real" or "likely flaky" in real-time.

This means:

  • Real failures → Block the merge, alert the team
  • Likely flaky failures → Auto-flag, don't block, track for fixing later

This eliminates the diagnostic time cost — engineers no longer waste 15-20 minutes investigating failures that turn out to be flaky. They see a classified failure and immediately know whether to investigate or re-run.

The Reality: You Can't Fix Them All

Here's the uncomfortable truth: in any non-trivial test suite, you will always have some level of flakiness. New code introduces new race conditions, new async patterns create new timing issues, and new dependencies add new failure modes.

The most successful teams don't try to eliminate flakiness entirely. They:

  1. Fix the top offenders (strategies 1-8 above)
  2. Quarantine the chronic ones (strategy 9)
  3. Classify failures in real-time so flaky ones don't waste team time (strategy 10)

The combination of fixing what you can and intelligently handling what you can't is what separates teams with 5-minute CI feedback loops from teams stuck in "re-run and pray" mode.

Stop wasting engineering hours on false alarms. Poly classifies every CI failure in real-time — real bugs get fixed, flaky tests get tracked, and your team gets their time back.

Top comments (0)