DEV Community: Sai Krishna Oggu

Why Do Playwright Tests Pass Locally but Fail in CI/CD?

Sai Krishna Oggu — Sat, 19 Sep 2026 08:09:42 +0000

“It works perfectly on my machine.”

Every automation engineer has said it.

Your Playwright test passes 20 times locally.
You push the code.
The CI pipeline starts.
And suddenly:

❌ Test failed.

You run it again locally.

✅ Passed.

So, what changed?

Usually, the test didn’t suddenly become bad. The environment changed.

Here are the most common reasons behind this frustrating problem — and how to debug them.

Timing Issues — The Classic Flaky Test Your local machine may give the application enough time to render.

CI may be under CPU or memory pressure, making everything slightly slower.

A fragile test might look like:

await page.click('#submit');
await page.waitForTimeout(1000);
expect(await page.locator('.success-message').isVisible()).toBeTruthy();
Instead, let Playwright wait for the expected state:

await page.getByRole('button', { name: 'Submit' }).click();
await expect(
page.getByText('Success')
).toBeVisible();
Rule:
Don’t wait for time. Wait for state.

Avoid using waitForTimeout() as a solution for flaky tests.

Your Local Environment ≠ CI Environment This is one of the first things I check. Your local environment might use:

QA1
while the pipeline is configured for:

QA2
Or your local .env may contain variables that aren't available in CI.

Check:

baseURL
API endpoints
environment variables
feature flags
tenant configuration
test credentials
database state
A test can be perfectly written and still fail because it is talking to the wrong environment.

Browser or Playwright Version Differences Your local machine may be running one browser version while CI runs another.

Check your Playwright version:

npx playwright --version
Make sure your project uses a consistent dependency version and that CI installs the expected browsers.

For CI environments, browser installation should be explicit when needed:

npx playwright install --with-deps
Consistency matters.

Same code + different browser/runtime = potentially different behavior.

Headless vs Headed Execution Most CI pipelines run Playwright in headless mode.

Locally, you may be running:

npx playwright test --headed
while CI runs:

npx playwright test
This can expose problems involving:

animations
responsive layouts
viewport assumptions
visual elements
timing
Try reproducing the pipeline locally in headless mode.

Parallel Execution Can Expose Hidden Dependencies This is a big one, you might run locally:

npx playwright test --workers=1
But your CI pipeline may execute multiple workers, now imagine two tests using the same user:

Test A → Update User
Test B → Delete User
Run independently:

PASS ✅

Run simultaneously:

FAIL ❌

Shared test data is often the real culprit, watch for dependencies involving:

users
database records
files
API data
tenants
orders
accounts
Tests should be as isolated as possible.

Test Data Exists Locally — but Not in CI Your local database may already contain:

User: testuser123
Order: 45678
Tenant: ABC
Your pipeline may not.

Download the Medium app
If your test assumes that data exists, it can fail immediately.

Instead of relying on existing state, create the required data as part of the test setup.

For example:

test.beforeEach(async ({ request }) => {
await createTestUser(request);
});
A reliable test should control the data it depends on.

Authentication & Storage State Another common issue:

Local
↓
Valid storageState
↓
Authenticated
↓
Test passes
CI:

Missing/expired storageState
↓
Login/session fails
↓
Test fails
Check:

cookies
access tokens
session state
authentication setup
CI secrets
storage state files
Don’t assume authentication works simply because it works locally.

CI Secrets Are Different You may have:

process.env.USERNAME
process.env.PASSWORD
process.env.BASE_URL
working locally because they’re defined in your .env.

But CI needs those values configured separately.

A missing environment variable can create a failure that looks like a Playwright problem — but isn’t.

Timezone Differences Here’s a subtle one.

Your machine:

IST
CI server:

UTC
Now consider a test involving:

Today's date
Expiry date
Scheduled job
Booking time
Timestamp
Your test may behave differently.

If dates matter, make timezone assumptions explicit rather than relying on the machine’s local timezone.

Windows Works. Linux Doesn’t. This catches many automation engineers; your local machine may be Windows.

Your CI runner may be Linux.

Windows filesystem handling can be different from Linux, particularly around filename casing.

For example:

import LoginPage from './pages/loginpage';
while the actual file is:

LoginPage.ts
It may work locally and fail in a Linux pipeline.

Treat filenames and imports as case-sensitive.

The Most Powerful Debugging Tool: Trace Viewer
When a test fails in CI, don’t immediately increase the timeout.

First, find out why it failed.

Configure Playwright:

use: {
trace: 'retain-on-failure',
screenshot: 'only-on-failure',
video: 'retain-on-failure'
}
Then inspect the trace:

npx playwright show-trace trace.zip
You can investigate:

the exact action that failed
DOM state
screenshots
network activity
console messages
timing
locator behavior
Instead of guessing:

“Maybe the page wasn’t loaded?”

You can actually see what happened.

My CI Debugging Checklist
When a Playwright test passes locally but fails in CI, I check these in roughly this order:

Environment / Base URL ↓
Test data ↓
Authentication ↓
Timing / synchronization ↓
Browser & Playwright versions ↓
Headless execution ↓
Parallel workers ↓
Viewport / responsive behavior ↓
Timezone ↓
CI resources / network Then I reproduce the pipeline conditions locally:

npx playwright test --workers=1
and inspect the trace if the failure persists.

The Bigger Lesson
A CI failure isn’t necessarily a Playwright problem.

It can reveal a weakness in your:

test → data → environment → application → infrastructure

chain.

That’s why good automation isn’t just about writing locators.

It’s about creating tests that are:

Reliable. Isolated. Reproducible. Observable.

The goal isn’t to make CI green by adding more waits.

The goal is to understand why it wasn’t green in the first place.

Final Thought
Local passing tells you the test can work.
CI passing tells you the test can be trusted.

And that’s the real difference between a test that runs and an automation suite you can rely on.

Integrating Playwright with CI/CD Using GitHub Actions

Sai Krishna Oggu — Fri, 18 Sep 2026 10:20:54 +0000

Automating Playwright tests locally is useful, but running them automatically in a CI/CD pipeline gives teams faster feedback on every code change.

In this guide, we'll integrate Playwright with GitHub Actions using a simple YAML workflow.

1. Project Setup

Assuming you already have a Playwright project:

playwright-project/
├── tests/
├── playwright.config.ts
├── package.json
└── package-lock.json

Make sure the tests run locally:

npx playwright test

2. Create the GitHub Actions Workflow

Create the following file:

.github/workflows/playwright.yml

Add:

name: Playwright Tests

on:
  push:
    branches: [main]

  pull_request:
    branches: [main]

jobs:
  test:
    runs-on: ubuntu-latest

    steps:
      - name: Checkout code
        uses: actions/checkout@v4

      - name: Setup Node.js
        uses: actions/setup-node@v4
        with:
          node-version: 20
          cache: npm

      - name: Install dependencies
        run: npm ci

      - name: Install Playwright browsers
        run: npx playwright install --with-deps

      - name: Run Playwright tests
        run: npx playwright test

      - name: Upload Playwright report
        if: always()
        uses: actions/upload-artifact@v4
        with:
          name: playwright-report
          path: playwright-report/

3. How It Works

Whenever code is pushed to main or a pull request is created, GitHub Actions will:

Code Push / Pull Request
          ↓
   Checkout Repository
          ↓
     Install Node.js
          ↓
    Install Dependencies
          ↓
 Install Playwright Browsers
          ↓
    Run Playwright Tests
          ↓
    Generate Test Report

If a test fails, the workflow fails and the Playwright report is still uploaded because of:

if: always()

This makes it easier to investigate failures directly from GitHub Actions.

4. Why Integrate Playwright with CI/CD?

CI/CD integration helps teams:

Run tests automatically
Catch regressions early
Validate pull requests
Execute tests consistently
Store reports and artifacts
Reduce manual testing effort

For larger frameworks, you can extend the pipeline with parallel execution, environment variables, secrets, scheduled regression runs, test sharding, and deployment gates.

Final Thought

Playwright becomes much more valuable when automation is part of the development pipeline—not just something executed manually on a QA engineer's machine.

Write the tests once. Run them continuously. Get feedback early.

🚀 1,200+ Test Cases Migrated from Tosca to Playwright – A Major QA Transformation Success! 🚀

Sai Krishna Oggu — Mon, 08 Jun 2026 03:46:01 +0000

Excited to share the successful migration of 1,200+ automated test cases from Tosca to Playwright, helping our customer modernize their automation landscape and build a faster, more scalable, and cost-effective testing ecosystem.

Why Move from Tosca to Playwright?
✅ No Licensing Costs – Open-source framework that eliminates licensing fees and reduces automation costs.
✅ Faster Test Execution – Native parallel execution enables significantly quicker regression cycles.
✅ Better CI/CD Integration – Seamlessly integrates with modern DevOps and continuous testing pipelines.
✅ Superior Web Application Support – Auto-waiting and robust locators improve test stability and reliability.
✅ Rich Debugging & Reporting – Built-in traces, screenshots, videos, and logs simplify failure analysis.
✅ Cross-Browser Testing – Supports Chromium, Firefox, and WebKit from a single automation framework.
.
Value Delivered:

🎯 Successfully migrated 1,200+ automated test cases
🎯 Improved execution speed and test stability
🎯 Reduced long-term automation costs
🎯 Enhanced maintainability and scalability
🎯 Accelerated feedback cycles for development teams
🎯 Established a modern automation foundation for future growth

Large-scale automation transformations require the right strategy, technical expertise, and a strong focus on quality. Proud to have contributed to this journey and to the measurable value delivered to the customer.

🚀 1,200+ Test Cases Migrated from Tosca to Playwright – A Major QA Transformation Success! 🚀 Excited to share the successful migration of 1,200+ automated test cases from Tosca to Playwright, helping our customer modernize their automation landscape

Sai Krishna Oggu — Mon, 08 Jun 2026 03:44:41 +0000

Testing the product as an end user.

Sai Krishna Oggu — Sun, 05 Oct 2025 02:11:05 +0000

🧪 Testing isn’t about ticking boxes. It’s about thinking like a user.

Requirements are written with assumptions; they tell you what’s expected.
Users arrive with expectations, and they need experience they really don't care about user stories in Jira.

The gap between the two? That’s where startups succeed… or stumble.

I test like your first customer, your 100th customer, and your toughest investor. 🐞🚀

Great startups don’t just launch features; they launch experiences people believe in. We identify the gap between user expectations and user stories of your product.