DEV Community

AutomationDataCamp
AutomationDataCamp

Posted on

3 Playwright tests that pass but prove nothing (with a practice repo to fix them)

#playwright #ai #testing #qa

A green test is not a proof. Here are three Playwright tests that pass, and still let a bug through.

I put them in a small public repo so you can run them, break the app on purpose, and see which test notices.

The setup

The repo has a tiny shop app with no dependencies: two products (keyboard 50, mouse 25), a promo code ADC10 that gives 10% off, a confirmation message, and an empty-cart message.

Two switches help you experiment:

  • LATENCE=1200 slows the products API, to expose timing mistakes.
  • BUG=remise disables the ADC10 discount, to expose tests that never check the total.

Exercise 1: the test that checks the message, not the result

The original test applies ADC10 and checks that "Code appliqué." appears. With BUG=remise, the message still appears but the total stays at 50,00 €. The test passes.

The fix: assert the total (45,00 €), and check that an invalid code leaves it at 50,00 €.

Exercise 2: the fixed sleep followed by a one-shot count

The original waits 500 ms, then reads count() once. With LATENCE=1200, the list is not loaded yet and the test fails, but only on a slow run. A flaky test is a test you stop trusting.

The fix: await expect(page.locator('#produits li')).toHaveCount(2). The assertion retries until it is true, or times out.

Exercise 3: the AI-style test

It swallows errors in a try/catch and checks count() === 1. It looks thorough, and it can never fail for the right reason.

The fix: role-based locators, the total, the confirmation message, and the empty-cart case.

Run it 

git clone https://github.com/automationdatacamp/adc-playwright-practice-lab
cd adc-playwright-practice-lab
npm ci
npx playwright install chromium
npm test
Enter fullscreen mode Exit fullscreen mode

Each exercise has an avant.spec.js (before) and a solutions/NN/apres.spec.js (after) with a short explanation. The CI runs all 8 tests on GitHub Actions. MIT license.

The statements and explanations in the repo are in French. If you want an English version, tell me in the comments.

Repo: https://github.com/automationdatacamp/adc-playwright-practice-lab

Top comments (0)