DEV Community: TestDino

What to Check When Your Playwright Test Passes Locally But Fails in CI

TestDino — Thu, 25 Jun 2026 10:46:05 +0000

You pushed the test. Passed locally 10 times clean. CI runs it. Fails.

You open the report. Screenshot is blank. Stack trace points to a selector you know exists.

You add a waitForTimeout(2000). Push again. Passes.
Next week, same test. Same blank screenshot. Different error message.

You are not fixing the test. You are guessing until it stops failing temporarily.

Here is what is actually happening and how to get out of it.

Why Does a Playwright Test Pass Locally But Fail in CI?

Your test is probably not broken. The CI environment is the variable nobody talks about.

On your machine:

One test runs. Full CPU. Full network. Nothing competing.
Page loads in under a second. Images render. API responds in under 200ms.
Screenshot captures a fully loaded state.

In CI with parallel tests running:

CPU is split across all parallel processes
Network bandwidth is shared across all shards
API calls queue behind dozens of others hitting the same endpoint
Page starts loading, resources get starved halfway through, rendering stops
Your selector checks for an element that never finished rendering
Screenshot captures a blank or half-loaded state

Your selector was never wrong. The environment killed the render before the element existed.

But the report says element not found. So you go fix the selector.

That is the loop.

What Does a Blank Playwright Screenshot in CI Actually Mean?

Not "test failed." One of these four specific things:

1. Page timed out before full render

Network got thin mid-load. Playwright captured the loading state. The element you checked for was never painted onto the screen.

You see: blank screenshot
What happened: resource contention during page load, not a selector problem

2. Image format changed and the CDN did not catch up

Your main branch now serves WebP or AVIF optimized images. Staging CDN does not support that format yet. Image request returns 404 silently or hangs. Placeholder stays. Your selector looks for the real rendered image. Fails.

You see: element not found: img.product-image
What happened: image format change on main branch broke the CDN request in staging

3. API timeout cascade

Parallel tests hammer the same endpoint. Your call joins the queue. Times out. Page never finishes rendering. Screenshot captures whatever loaded before timeout fired.

You see: blank or partial screenshot
What happened: API queue filled up under parallel load, your request died in it

4. Resource starvation from parallel execution

OS deprioritized your page load because 99 other processes were competing. Load that takes under a second locally takes 3 to 4 seconds in parallel CI. Playwright timeout fires before the page finishes.

You see: blank screenshot
What happened: physics, not your test code

Why Can't You Tell Which One It Is From the Report?

Standard Playwright reporter shows you:

Assertion failed
Blank screenshot
Stack trace pointing at the selector line

It does not show you:

Which network requests failed or timed out
When your selector ran relative to when the API responded
How many tests were running at the same time
Whether the page was mid-render or fully loaded when failure fired

So you have a failure with no context. You guess.

Most QAs guess timing and add a wait. But it was a CDN or environment issue. The wait sometimes helps by coincidence. The real problem stays. It comes back next week with a slightly different error and you start the loop again.

How Do You Actually Debug a Playwright CI Failure?

Here is the step-by-step process to stop guessing and start knowing.

Step 1: Check the network tab before you touch your test

Before you change a selector or add a wait, get the network data from that CI run.

First, make sure you are capturing traces. If trace: 'on-first-retry' is not in your config, you have zero network data on failures.

// playwright.config.ts
use: {
  trace: 'on-first-retry',
  screenshot: 'only-on-failure',
  video: 'retain-on-failure'
}

Playwright Trace Viewer

Once you have trace data, open Trace Viewer and go to the Network tab. Look for:

Any request with a 404 or 5xx status
Any request that shows no response at all (started, then nothing)
Image requests specifically if the failure involves a missing image element

If a network request failed before your selector ran: your test is innocent. Stop there.

Step 2: Check the timeline, not just the error message

The order of events matters more than what the error says.

You want to know:

When did the page load finish?
When did the API call complete?
When did your selector check run?

If your selector ran at 2100ms and the API responded at 2200ms, that is a timing issue. The element existed, your check ran 100ms too early.

If the API never responded at all, that is an environment issue. Not your test.

Step 3: Classify the failure type before writing any fix

There are three kinds of CI failures and each needs a different response:

Real test failure
Same error, fails consistently on every run, every environment.
What you do: fix your test code.

Environment failure
Passes when running alone, fails under parallel load. Network timeouts, resource starvation.
What you do: fix the environment config or escalate to ops.

Timing failure
Passes after you add a wait, was checking before the element existed.
What you do: replace waitForTimeout with waitForSelector or waitForResponse.

When you add waitForTimeout to everything without classifying first, you cover up environment failures instead of surfacing them. The suite fills up with arbitrary waits and inflated timeouts. It gets slower, more fragile, and the real problems stay invisible.

Step 4: Replace static waits with explicit waits

Once you confirm it is a timing issue and not a network issue, fix it properly.

Replace:

await page.waitForTimeout(2000); // this is a guess

With:

await page.waitForResponse(
  resp => resp.url().includes('/api/product') && resp.status() === 200
);

Or for a specific element:

await page.waitForSelector('img.product-image', { state: 'visible' });

Static waits hide the real problem. waitForResponse and waitForSelector make timing explicit and debuggable.

Playwright waitForResponse docs
Playwright waitForSelector docs

How Does the Image Optimization Problem Break CI in 2026?

This one trips up QA teams specifically because it looks exactly like a selector problem and the fix looks obvious but is completely wrong.

Your main branch started serving WebP and AVIF images for performance optimization. Your staging CDN config has not been updated in months. It does not support AVIF. Image requests either return 404 silently or hang until timeout.

Here is the actual sequence when your test hits CI:

0ms    -- Page starts loading
400ms  -- Image request fires (AVIF format from main branch)
2000ms -- Main page HTML content loads fine
3000ms -- Image CDN timeout threshold hit. No response from CDN.
3100ms -- Your selector checks for img.product-image
3200ms -- Selector fails. Image element never rendered

Report says: element not found: img.product-image
Screenshot shows: page looks loaded, image area is a blank placeholder
You think: wrong selector, or needs a wait
What actually happened: CDN does not support AVIF in staging, element never existed

Quick check to confirm this in your CI environment:

url -I https://your-staging-cdn.com/images/product.avif

If that returns 404: your blank screenshot is explained. Fix the CDN config. Do not touch your test.

Why Does Parallel Execution Make This Worse?

Because it is the default now and most reports do not reflect it.

GitHub Actions, GitLab CI, and CircleCI all shard and parallelize by default. Running 100 tests in parallel on a runner with 2 CPUs means you have already created resource starvation.

What this creates:

CPU contention: Your test alone runs in under a second. With 99 others competing on a 2-CPU runner, it runs in 3 to 4 seconds. Same test. Different resources. Nothing wrong with your code.

API queue buildup: Your endpoint responds in 150ms under normal load. Under 100 parallel requests, it responds in 1800ms or times out entirely. Same API. Different load.

CDN cache miss storm: Locally, your browser has the images cached. In parallel CI, 100 tests hit the CDN simultaneously with cold caches. Some requests hit the raw origin. Some time out.

A test that passes running alone but fails with 99 others is not a flaky test. It is a resource allocation problem.

Check this in your CI config:

`# In your GitHub Actions workflow

If you have a 2-CPU runner, 4 shards is reasonable. 100 is not.

run: npx playwright test --shard=1/4`

More shards than your runner can support is the problem. More waits will not fix that.

What Should a CI Failure Report Actually Show You?

Here is what good failure visibility looks like:

Visual state at failure:

Screenshot at the exact moment of failure
What was rendered and what was not
Was it a blank page, a partial load, or a fully loaded page with one missing element

Network state at failure:

Which requests completed with 200
Which returned 4xx or 5xx
Which timed out and at exactly what timestamp
Full sequence so you can see what failed first

Timing:

When did page load complete
When did each API call complete
When did your selector check run
Gap between last network event and selector check

Environment context:

How many tests were running at the same time
Whether resources were under pressure

When you have all four, you go from blind guessing to "image CDN request timed out at 3000ms, selector ran at 3100ms, element never rendered, not a test issue" in under two minutes.

Without all four, you are doing forensics manually every time.

What Is the Complete Process to Follow on Every CI Failure?

Follow these in order. Do not skip to your test code first.

1. Open trace. Go to the Network tab.

Found a 404 or 5xx? Image or API infrastructure issue. Your test is not the problem. Check CDN config or escalate to ops.
Found a request with no response at all? If it is an image request, check your CDN supports current image formats. If it is an API request, you are hitting a rate limit or queue issue under parallel load. Not your test.
All requests returned 200? Move to step 2.

2. Check the timing in the trace timeline.

Selector ran before the API completed? Use waitForResponse instead of waitForTimeout.
Selector ran after everything completed and element still not found? Now check your selector. This is the one case where your test code is actually the issue.

Check if the failure is consistent or only under parallel load.

Fails consistently on every run? Real test bug. Fix the selector or assertion.
Only fails under parallel load? Resource starvation. Check your shard count against runner CPU count. Reduce shards or increase runner size.

Network first. Timing second. Selector third. That order is everything.

How Does Getting the Right Visibility Change Everything?

The QA teams that stopped spending hours on every CI failure did not find better selectors or learn to write better waits. They got the right information in front of them when a failure happened.

Once you can see network requests, timing, and visual state together in one report:

You stop adjusting selectors that were never wrong
You stop adding waits for problems that were never timing issues
You surface CDN and infra problems to the right people instead of absorbing them as test flakiness
You spend 2 minutes on a failure instead of 45

Your tests do not need to get better first. Your visibility does.

TL;DR

Blank screenshot in CI = environment problem until the network tab proves otherwise
Turn on trace: 'on-first-retry' in playwright.config.ts or you are debugging blind
Check network requests before touching your selector
Check timing before adding a wait
WebP or AVIF image optimization on main branch causes "element not found" that looks like a selector issue but is a CDN config problem
100 shards on a 2-CPU runner is resource starvation, not flaky tests
The fix order is: network first, timing second, test code last

Your test is probably fine. Your visibility is the gap.

You do your job right. You catch things. You dig in when everyone else moves on.

The problem is that most reports stop at "it failed" and leave you doing the detective work alone.

You follow the process, you surface what broke, you fix the right thing. We are here to make sure that process takes two minutes instead of two hours. The hard work is yours.The visibility is on us.

TestDino Public API: One Call for Errors, Coverage, Specs, and Traces. Here Is What That Enables.

TestDino — Mon, 01 Jun 2026 09:18:37 +0000

If you have ever tried to get Playwright test data out of a reporting tool programmatically, you know the problem. You call the list endpoint, get statuses, then make a separate call per run for errors, another for coverage, another for spec breakdown. By the time you have enough data to post something useful to a PR or a CI pipeline export, you have chained four requests and written more glue code than the feature itself.

The TestDino Public API handles this differently. One parameter changes what you get back from a single run detail call. That difference is what makes building a Playwright reporting API integration actually worth the time.

Note: TestDino is Playwright-only. Cypress, Vitest, Jest: not supported.

The ?include= Parameter

Default call to a run detail endpoint returns run metadata: status, duration, branch, environment, test counts. Lean and fast.

Add ?include=errors,coverage,specs,artifacts and the same request returns:

errors: failures grouped by error message with stack traces
coverage: Istanbul code coverage metrics for that run
specs: spec files with flattened test cases and individual status per test
artifacts: authenticated download URLs for Playwright traces, screenshots, videos

One call. No chaining.

GET /{projectId}/test-runs/{runId} include=errors,coverage,specs,artifacts

The ?include= parameter only works on two endpoints. GET /{projectId}/test-runs/{runId} accepts errors, coverage,specs,artifacts, all. GET /{projectId}/test-cases/{caseId} accepts history, all. List endpoints, analytics, and dashboard endpoints do not support it.

Full Endpoint List

Base URL:

https://api.testdino.com/api/public/v1/{projectId}/

Auth: Bearer token with tdp_ prefix, public-api scope. Generate from Project Settings. Token is scoped to one project. Project A token returns 403 on project B.

Rate limits: 100 req/min per token, 200/min per IP. Every response includes RateLimit-Remaining and RateLimit-Reset. PDF generation is 1/min per token.

GET  /token-info                 # verify PAT, read project and org metadata
GET  /test-runs                  # list runs, filter by branch/env/developer/tag/date
GET  /test-runs/{runId}          # run detail + ?include=errors,coverage,specs,artifacts
GET  /test-cases/{caseId}        # test case detail + ?include=history
GET  /specs                      # spec file health: pass rate, flakiness score, duration
GET  /test-case-explorer         # aggregated metrics across runs
GET  /analytics/summary          # summary metrics and trend data
GET  /dashboard                  # project health overview
GET  /manual-tests               # manual suites, cases, steps
POST /reports/pdf                # generate a PDF report
GET  /filters                    # environments, branches, developers, tags in a project
GET  /usage                      # subscription usage and allocation

Full auth and error handling: docs.testdino.com/api-reference/conventions
Five-minute quickstart: docs.testdino.com/api-reference/quickstart

What You Can Actually Build

1. GitHub PR Annotation

The pain before: Reviewers merging without knowing if tests passed. The "can you check" comment in every PR thread.

After CI finishes, a workflow step calls the latest run for that branch, pulls pass rate and top failed test names with errors via ?include=errors, posts a PR comment. Reviewer sees test status without leaving GitHub.

GET /test-runs?branch=feature/my-branch&limit=1
GET /test-runs/{runId}?include=errors

2. Nightly Digest Email

The pain before: Engineering leadership has no visibility into test health trends unless someone manually checks and reports.

A scheduled script paginates through the previous day's runs, aggregates pass rate and failure counts per branch, grabs the five most flaky spec files via GET /specs, formats a plain text email, sends it. Director reads it before standup. Automated. Nobody had to remember.

3. Client-Facing Status Page

The pain before: Clients emailing "what is the current test status" weekly.

A script pulls the latest run and writes pass rate, timestamp, and branch to a Notion database or simple webpage. Clients check it themselves. No TestDino account required on their end. Those messages stop.

4. Flakiness Heat Map

The pain before: Knowing a suite is getting unstable but not knowing which files are pulling the numbers down.

GET /specs returns per-file flakiness scores across runs. GET /test-cases/{caseId}?include=history&historyStatus=flaky gives per-test flaky history. Combine them into a weekly view that surfaces exactly which files and tests are degrading. Teams that build this catch the problem before it becomes a standup conversation.

5. MCP: Querying Test Data from an AI Tool

The TestDino MCP server is a separate access path from the REST API. Two versions:

A. Local MCP Install via npx, connect Cursor, Claude Code, or Claude Desktop. The IDE tool queries live run data directly during your session.

npx @testdino/mcp

Local MCP setup

B. Remote MCP Hosted at mcp.testdino.com, connects ChatGPT, Claude on the web, and other remote-MCP-capable tools. No local install.

Remote MCP setup

The pain before: Copying stack traces into an AI prompt and hoping it reasons correctly from pasted context.

The practical shift with MCP: You ask "what error pattern appeared most in staging this week" and the assistant queries actual run data to answer. Not a guess. Not training data. Your data.

Based on the public alternatives comparison, no other Playwright reporting tool has a remote MCP server.

Access and Availability

The API and MCP are included on every plan. There is no separate API tier, no add-on, and no unlock. You can start on the free plan and make your first API call today without a credit card.

Most alternatives in this space gate their API behind paid plans or charge significantly more for equivalent access. Full breakdown on the alternatives comparison page.

Get started | Pricing

Mistake 14/14: Not using the tools Playwright already gives you

TestDino — Tue, 21 Apr 2026 13:30:37 +0000

This is the final mistake in the series.

A lot of teams still debug failed tests the hard way:

read the error
add a console log
re-run the suite
repeat

That works, but it is slow.

Playwright already ships with tools that make debugging much easier:

Trace Viewer

Replay a failed run step by step and inspect DOM state, actions, network calls, and timing.

npx playwright show-trace trace.zip

UI Mode

Run tests in an interactive mode and watch them execute with time-travel debugging.

npx playwright test --ui

Codegen

Generate locators and actions by recording browser interactions.

npx playwright codegen https://your-app.com

Last failed

Re-run only the tests that failed, not the whole suite.

npx playwright test --last-failed

Use scripts

You can also add these to package.json so the team actually uses them regularly.

{
  "scripts": {
    "test": "npx playwright test",
    "test:ui": "npx playwright test --ui",
    "test:failed": "npx playwright test --last-failed",
    "test:trace": "npx playwright show-trace trace.zip",
    "codegen": "npx playwright codegen"
  }
}

The biggest lesson from this series is simple: Most debugging pain is not caused by missing tools. It is caused by not using the tools you already have.

Playwright Mistake 13/14: You have no test cleanup strategy

TestDino — Fri, 17 Apr 2026 13:09:08 +0000

Your tests create users, orders, and comments. But they never delete them.

A few runs later, your staging database is full of ghost users, orphan orders, and search results polluted with test data.

Or worse, your tests begin failing because they rely on a “clean” state that is no longer clean. Yesterday’s leftovers become today’s flaky test.

Test automation shouldn’t leave a trace. The fix is simple: create data before the test, then clean it up after. Even if the test fails.

import { test as base, expect } from '@playwright/test';

type Product = { id: string; name: string; price: number };

export const test = base.extend<{ testProduct: Product }>({
  testProduct: async ({ request }, use) => {
    const response = await request.post('/api/products', {
      data: { name: 'Test Product', price: 29.99 },
    });

    const product = await response.json();
    await use(product);

    await request.delete(`/api/products/${product.id}`);
  },
});

test('edit product', async ({ page, testProduct }) => {
  await page.goto(`/admin/products/${testProduct.id}`);
  await expect(page.getByText('Test Product')).toBeVisible();
});

That’s the kind of cleanup strategy that keeps test suites reliable and staging data sane.

Does your test suite clean up after itself, or does it leave a mess behind?

Mistake 12/14: You're not catching visual regressions

TestDino — Thu, 16 Apr 2026 13:22:33 +0000

A simple CSS change broke the checkout button on mobile. Every functional test passed.

The scary part? Nobody knew. The button was technically in the DOM. It had the correct text. But to an actual user, it was completely invisible, overlapped by a rogue div on mobile screens only. The kind of bug that gets caught in production when a customer tries to check out and can't.

Functional tests are great for verifying behavior. They are not designed to check appearance. They can't tell you if your app actually looks right.

BEFORE : Functional test passes. Button is covered on mobile.

// ❌ BEFORE — Functional test passes, but the button is covered on mobile.
await expect(page.getByRole('button', { name: 'Checkout' })).toBeVisible();
// Passes because the button is in the DOM, even if users can't actually click it.

AFTER : Visual regression catches layout issues across devices.

// ✅ AFTER — Visual regression catches layout issues across devices.
test('cart page looks correct', async ({ page }) => {
  await page.goto('/cart');

  await expect(page).toHaveScreenshot('cart.png', {
    mask: [
      page.getByTestId('timestamp'),   // changes every run
      page.getByTestId('order-id'),    // unique per session
    ],
    maxDiffPixelRatio: 0.01,
  });
});

// 🧪 Test across viewports in playwright.config.ts
{ name: 'desktop', use: { viewport: { width: 1280, height: 720 } } },
{ name: 'mobile',  use: { viewport: { width: 375, height: 667 } } },

Playwright's built-in visual comparisons (toHaveScreenshot) catch these layout breaks instantly. The quick pro-tip that saves every team time: always mask dynamic elements like timestamps or order IDs to avoid flaky false positives.

The distinction most teams miss is this: functional tests verify that something works. Visual tests verify that users can actually see and interact with it. Both questions matter. Most suites only answer one of them.

When you add visual coverage at the viewport level, you stop shipping layout bugs that your test suite is structurally incapable of catching.

Have visual tests ever caught a bug your functional tests completely missed? Drop it in the comments.

Mistake 11/14: Your E2E tests are checking email format

TestDino — Wed, 15 Apr 2026 13:46:00 +0000

if you are running 400 E2E tests and half of them validate form inputs like email format, you are burning browser time on logic that never needed a browser.

The inconsistency isn't obvious at first. These tests pass. They feel useful. But every one of them navigates to a login page, authenticates, clicks through menus, waits for redirects. All to verify that a regex rejects "not-an-email". That is 30 seconds of browser overhead for a check that has nothing to do with the browser.

BEFORE - E2E for email validation. 30 seconds of overhead.

// Every spec file repeats this flow
test('rejects invalid email', async ({ page }) => {
  await page.goto('/login');                          // navigate
  await page.getByLabel('Email').fill('not-an-email');// fill
  await page.getByRole('button', { name: 'Sign in' }).click();
  await expect(page.getByText('Invalid email')).toBeVisible(); // 30 sec for this
});

AFTER - Component test. Same assertion. 200ms

import { test } from '@playwright/experimental-ct-react';
import { LoginForm } from '../src/components/LoginForm';

test('rejects invalid email', async ({ mount }) => {
  const form = await mount(<LoginForm />);            // no navigation. no login.
  await form.getByLabel('Email').fill('not-an-email');
  await form.getByRole('button', { name: 'Sign in' }).click();
  await expect(form.getByText('Invalid email')).toBeVisible();
});

The distinction most teams miss is this: E2E tests exist to cover what only E2E can cover. Multi-page flows, payment processing, cross-service coordination. That is the territory.

Form validation, input constraints, UI state changes. Those belong at the component level.

When you separate them, everything gets faster and each layer actually earns its place. ⚡

What percentage of your E2E tests are actually testing component behavior? Drop it in the comments.

Playwright Mistake 10/14: Flaky Tests Are Not Random

TestDino — Tue, 14 Apr 2026 13:50:10 +0000

You re-run. It passes. You move on.

Until your whole team is trained to ignore failures. That's when real bugs slip through.

Here are the 3 root causes behind 90% of flaky Playwright tests.

Race Condition

// bad
const text = await page.locator('.welcome').textContent();

// good
await expect(page.getByText('Welcome back')).toBeVisible();
// auto-retries until element exists

Shared Data

// bad
await request.delete('/api/users/test-user-1');
// 5 workers fight over this

// good
const user = await createUniqueUser(request);
await request.delete(`/api/users/${user.id}`);
// each test gets its own

Network Timing

// bad
await page.goto('/search?q=laptop');
await page.locator('.result').first().click();
// results not loaded yet

// good
await page.goto('/search?q=laptop');
await page.waitForResponse('**/api/search');
await page.getByTestId('result').first().click();

Which of these three has burned you the most on your current project?

Mistake 9/14: Stop Relying on Staging Data for Edge Case Testing

TestDino — Mon, 13 Apr 2026 13:04:11 +0000

You finish the happy path tests. Coverage looks solid. Then a user reports the error screen is broken. You never caught it because your suite never actually triggered a server error.

Most teams skip edge cases entirely , or spend hours crafting corrupted data in staging just to trigger one scenario. Then the data gets cleaned up, and the test breaks again.

This is the exact pattern that makes test suites unreliable as they grow.

Playwright's page.route() removes the backend dependency completely. You return whatever response you want : empty arrays, server errors, malformed JSON, slow responses; directly from the test. Your staging environment stays untouched.

The Clean Pattern

Instead of relying on staging state, intercept the network call and return exactly what you need to test:

// Test 1: Empty state
test('shows empty state when no users found', async ({ page }) => {
  await page.route('/api/users', route =>
    route.fulfill({
      status: 200,
      contentType: 'application/json',
      body: JSON.stringify([]),
    })
  );
  await page.goto('/users');
  await expect(page.getByText('No users found')).toBeVisible();
});

Then handle the failure and timeout cases the same way:

// Test 2: Server error
test('shows error message on server failure', async ({ page }) => {
  await page.route('/api/users', route =>
    route.fulfill({
      status: 500,
      body: JSON.stringify({ error: 'Internal Server Error' }),
    })
  );
  await page.goto('/users');
  await expect(page.getByText('Something went wrong')).toBeVisible();
  await expect(page.getByRole('button', { name: 'Retry' })).toBeVisible();
});

// Test 3: Slow response
test('shows loading spinner and timeout message', async ({ page }) => {
  await page.route('/api/users', async route => {
    await new Promise(resolve => setTimeout(resolve, 10000));
    await route.fulfill({ status: 200, body: '[]' });
  });
  await page.goto('/users');
  await expect(page.getByTestId('loading-spinner')).toBeVisible();
  await expect(page.getByText('Request timed out')).toBeVisible({ timeout: 15000 });
});

Zero backend involvement: No staging data setup or cleanup needed
Fully deterministic: The test controls exactly what the API returns
Three edge cases covered: Empty state, server error, and timeout , all in one go

Empty states, error screens, and timeout messages are exactly what users see when things go wrong. Those paths deserve real test coverage, not a "we'll check it manually" comment in the PR.

Do you enforce API mocking for edge cases on your team, or is it still a free-for-all?

Drop it in the comments.

Playwright Mistake 8/14: You Only Check Status Codes on API Responses

TestDino — Sat, 11 Apr 2026 12:36:05 +0000

The docs said the field was a string. Production returned null. Your test passed because you only asserted status === 200.

That's the problem.

What actually happened

The frontend called .toUpperCase() on null. Page crashed. Your test had no idea.

Checking the status code and calling it tested is like confirming the door is closed without checking if it's locked.

These two error messages tell the whole story

"Expected string, received null at 'name'" → 2-minute fix
"Cannot read properties of undefined"      → 2-hour debug

One catches the issue at test time. The other catches it in production.

The fix: validate the response shape

// BEFORE
test('get user profile', async ({ request }) => {
  const response = await request.get('/api/users/123');
  await response.json();
  expect(response.status()).toBe(200);
  // Backend returns { name: null }. Test passes. Frontend breaks.
});

// AFTER
import { z } from 'zod';

const UserSchema = z.object({
  id:        z.string().uuid(),
  name:      z.string(),
  email:     z.string().email(),
  role:      z.enum(['admin', 'member']),
  createdAt: z.string().datetime(),
});

test('get user profile - validate contract', async ({ request }) => {
  const response = await request.get('/api/users/123');
  expect(response.status()).toBe(200);

  const body   = await response.json();
  const result = UserSchema.safeParse(body);

  // name null?       Caught here.
  // field removed?   Caught here.
  // type changed?    Caught here.
  expect(result.success).toBe(true);
});

Why it matters at scale

One method: safeParse covers every field in one call
Visible failure: tells you exactly which field broke and why
No prod surprises: field renames, null returns, type changes all fail at test time

When you have 200+ API tests only checking status codes, you have 200+ tests that can pass while the app is broken.

Tag someone who's still only asserting status codes.

Mistake 7/14: You are still using beforeEach for everything

TestDino — Fri, 10 Apr 2026 13:36:25 +0000

If you are still copy-pasting beforeEachblocks across fifty spec files to initialize your Page Objects, you are building a maintenance trap.

The inconsistency isn't laziness. It is what happens when your setup has no enforced contract across the codebase.

BEFORE - Brittle boilerplate

// Every spec file repeats this logic
let settingsPage;

test.beforeEach(async ({ page }) => {
  settingsPage = new SettingsPage(page);
});

AFTER - Scalable dependency injection

// fixtures.ts
export const test = base.extend({
  settingsPage: async ({ page }, use) => {
    await use(new SettingsPage(page));
  },
});

Playwright built Fixtures to eliminate the "which pattern did we agree on?" conversation.

-Encapsulation: Setup and teardown live together in the fixture.

-Lazy Loading: If a test doesn't ask for the fixture, it never runs.

-Consistency: One source of truth for every Page Object in your framework.

Stop managing state manually. Let the runner do the heavy lifting. ⚡

Do you enforce a fixture strategy on your team, or is it a free-for-all? Drop it in the comments.

Mistake 6/14: Stop Putting Assertions in Page Objects

TestDino — Thu, 09 Apr 2026 13:29:08 +0000

A developer wants to keep their tests "dry," so they put the expect statement inside the Page Object method. It makes sense at first. login() handles everything: fill the form, click the button, confirm you’re on the dashboard. One method. Clean. ✅

Until you need to test a failed login. ✖️

Now your login() method asserts success but your negative test is looking for a failure. To fix it, you add loginExpectingFailure(). Then loginAndCheckError(). Six login methods later, your POM is a graveyard of duplicated logic.

This is the exact trap teams fall into when a suite grows.

Page Objects should be simple. They store locators and perform actions. That is it. Assertions belong in the test file, where the intent of the test is immediately visible.

The Clean Pattern

Instead of baking the assertion into the class, keep the class "pure":

class LoginPage {
  async login(user, pass) {
    await this.emailInput.fill(user);
    await this.passInput.fill(pass);
    await this.submitBtn.click();
  }
}

Then, handle the different outcomes in your actual test files:

// Happy path
test('success', async ({ page, loginPage }) => {
  await loginPage.login('user@test.com', 'pass123');
  await expect(page).toHaveURL('/dashboard');
});

// Negative path
test('failure', async ({ page, loginPage }) => {
  await loginPage.login('user@test.com', 'wrong');
  await expect(loginPage.error).toBeVisible();
});

One method: A single login() method with zero assertions.
Visible Intent: Anyone reading the test knows exactly what is being verified.
No Bloat: No duplicated methods. No boolean flags. No loginVersion4().

Do you enforce a "no assertions in POM" rule on your team, or is it a free-for-all? Drop it in the comments.

playwright #testing #automation #sdet

Mistake 5/14: Your Tests Are Unreadable in the Report

TestDino — Wed, 08 Apr 2026 13:29:19 +0000

A test fails in CI. You open the Playwright HTML report and see 60 lines of code in one giant block. It says "line 47 timed out." You have no idea what the test was actually trying to do. You are left staring at a stack trace guessing which part of the flow broke.

The damage isn't just time. It's clarity. When test results are a pain to decipher, developers stop reading them. Failures get dismissed.

test.step() fixes this.

BEFORE: One giant block. The report just says "X test failed."

test('submit contact form', async ({ page }) => {
  await page.goto('/contact');
  await page.getByLabel('Name').fill('Jane');
  await page.getByLabel('Email').fill('jane@test.com');
  await page.getByRole('button', { name: 'Submit' }).click();
  await expect(page.getByText('Thank you')).toBeVisible();
});

AFTER: Named steps. The report reads like a story.

test('submit contact form', async ({ page }) => {
  await test.step('GIVEN user is on contact page', async () => {
    await page.goto('/contact');
  });

  await test.step('WHEN user fills and submits form', async () => {
    await page.getByLabel('Name').fill('Jane');
    await page.getByLabel('Email').fill('jane@test.com');
    await page.getByRole('button', { name: 'Submit' }).click();
  });

  await test.step('THEN success message appears', async () => {
    await expect(page.getByText('Thank you')).toBeVisible();
  });
});

The rule: Wrap your code in named steps using GIVEN, WHEN, and THEN.

Now your report says "WHEN user fills and submits form failed" instead of "line 47 timed out".

You know exactly what broke instantly. Debugging goes from minutes to seconds.

Do you use test.step() to structure your tests, or is your code one giant block? Drop it in the comments.