DEV Community

MADAN DHOUNDIYAL
MADAN DHOUNDIYAL

Posted on • Originally published at resumeorbitz.com

I Had 160 Playwright Tests. I Deleted All of Them. Here's Why.

#webdev #javascript #playwright #testing

I Had 160 Playwright Tests. I Deleted All of Them. Here's Why.

Last week someone asked me a simple question about my E2E test suite:

"How can you assure that these test cases are actually checking?"

I opened the files to answer confidently. Instead, I spent the next hour realizing I couldn't.

The Setup

I'd built ResumeOrbitz — a resume builder with an editor, live preview, template selector, PDF download, cover letters, and public sharing. The kind of app where a lot of things need to work together.

So I did the responsible thing: I wrote tests. 160 of them. Spread across 20 spec files. GitHub Actions ran them on every push. Green checkmarks everywhere.

✓ 160 passed (4.2m)
Enter fullscreen mode Exit fullscreen mode

I felt good about it.

The Question That Broke Everything

When I looked closely at what the tests were actually doing, I saw things like this:

test('dashboard loads and shows new resume button', async ({ page }) => {
  await page.goto('/dashboard');
  await expect(page.getByRole('button', { name: /new resume/i })).toBeVisible();
});
Enter fullscreen mode Exit fullscreen mode 
test('login page renders', async ({ page }) => {
  await page.goto('/login');
  await expect(page.getByPlaceholder('you@example.com')).toBeVisible();
  await expect(page.getByPlaceholder('Your password')).toBeVisible();
  await expect(page.getByRole('button', { name: /sign in/i })).toBeVisible();
});
Enter fullscreen mode Exit fullscreen mode 
test('forgot password page renders and accepts email', async ({ page }) => {
  await page.goto('/forgot-password');
  await page.getByPlaceholder('you@example.com').fill('test@example.com');
  await page.getByRole('button', { name: /send reset/i }).click();
  await expect(page.locator('body')).toBeVisible(); // 🚩
});
Enter fullscreen mode Exit fullscreen mode

That last one expects body to be visible. The body of an HTML page is always visible. That test will pass even if the page is a blank white screen, a 500 error, or a "this site has been taken down" message.

These aren't tests. They're theater.

What Theater Tests Look Like

A theater test checks that something exists, not that it works.

Theater (presence check) Real (behavioral check)
Button is visible Clicking button navigates to /editor
Input field is on page Typing in field updates the live preview
Page has a title Invalid login shows an error message
Element count > 0 Deleting a resume removes it from the list

The critical difference: a theater test passes even when the feature is broken.

If I remove the click handler from the "New Resume" button, the first test still passes — the button is still visible. The second one doesn't catch that the sign-in form does nothing on submit. The forgot-password one would pass if I swapped the page for a cat photo.

The Sabotage Test

There's a clean way to find out if a test is actually checking something:

Deliberately break the feature and run the test. If it still passes, the test is worthless.

For example, to verify TC-LP-02 ("first name typed in form appears in live preview"):

  1. In EditorPage.tsx, comment out the line that syncs the first name field to the preview state
  2. Run the test

If the test goes red → it's real. It caught the regression.

If the test stays green → it's theater. It was never checking what it claimed.

I applied this thinking to every one of my 160 tests. The ones that checked actions and state changes were real. The ones that checked visibility and presence were not.

The Tests That Were Actually Good

Out of 160, the ones worth keeping were the behavioral ones:

// ✅ Real: action → URL change (tests the route guard)
test('unauthenticated user redirected from dashboard to login', async ({ page }) => {
  await page.goto('/dashboard');
  await expect(page).toHaveURL(/login/);
});

// ✅ Real: wrong input → specific error (tests validation logic)
test('register with mismatched passwords shows error', async ({ page }) => {
  await page.goto('/register');
  await page.getByPlaceholder('Min. 8 characters').fill('Password123!');
  await page.getByPlaceholder('Repeat your password').fill('Different123!');
  await page.getByRole('button', { name: /create free account/i }).click();
  await expect(page.getByText(/passwords do not match/i)).toBeVisible();
});

// ✅ Real: creates data → navigates away → data still exists (tests persistence round-trip)
test('new resume then back to dashboard shows the resume', async ({ page }) => {
  await page.getByRole('button', { name: /new resume/i }).click();
  await page.waitForURL('**/editor/**');
  await page.goto('/dashboard');
  await expect(page.locator('[href*="/editor/"]').first()).toBeVisible();
});

// ✅ Real: type → check specific value in different part of UI (tests live sync)
test('first name typed in form appears in live preview', async ({ page }) => {
  await page.getByLabel(/first name/i).fill('Alexandra');
  await expect(page.locator('.fixed.right-0').getByText('Alexandra')).toBeVisible();
});
Enter fullscreen mode Exit fullscreen mode

These tests verify state transitions, not static presence. They can actually go red.

What I Did About It

I deleted all 160 tests. Then I generated a proper manual test sheet — 160 rows in a spreadsheet with the module, preconditions, steps, expected result, and a Pass/Fail column I fill in myself before every release.

Yes, manual. Because a manual test you actually run beats an automated test that never fails by design.

The plan going forward: write new E2E tests only for flows that are:

  1. High-risk — auth guards, data persistence, PDF generation
  2. Behavioral — they verify an action produced a specific outcome
  3. Sabotage-verified — they go red when the feature is broken

Ten real tests are worth more than 160 green lies.

The Rule I Wish I'd Had Earlier

Before committing any test, ask:

"If I remove the feature this test claims to cover, will this test fail?"

If the answer is "no" or "I'm not sure" — rewrite the test or delete it. A test that can't go red is not protecting you. It's just giving you false confidence while your bugs ship undetected.

Quick Reference: Theater vs Real

Theater patterns (delete or rewrite):

  • expect(element).toBeVisible() with no preceding action that could hide it
  • expect(page.locator('body')).toBeVisible()
  • expect(page).toHaveURL(currentPage) — checking you didn't navigate away
  • Any test that checks an element is present but never interacts with it

Real patterns (keep and add more):

  • action → expect(page).toHaveURL(/new-route/)
  • fill wrong value → expect(error message).toBeVisible()
  • create → reload → expect(data still exists)
  • type in field A → expect(value in panel B).toBeVisible()
  • delete item → expect(item).not.toBeVisible()

Building something? Check out ResumeOrbitz — a free resume builder with live preview, 100+ templates, and AI-powered suggestions.

Originally published at https://resumeorbitz.com/blog/playwright-theater-tests

Top comments (0)