How I Use AI to Write Regression Tests in Plain English (Playwright + Passmark)

Traditional E2E tests are a maintenance nightmare. I switched to AI-powered testing and here's my workflow.

The Problem

Every time a designer changes a class name, 20 tests break. None of them found a real bug.

The Solution: Passmark

Passmark lets you write tests in English:

import { test, expect } from "@playwright/test";
import { runSteps } from "passmark";

test("User can add item to cart", async ({ page }) => {
  await runSteps({
    page,
    userFlow: "Add product to shopping cart",
    steps: [
      { description: "Navigate to the store homepage" },
      { description: "Click on the first product" },
      { description: "Click Add to Cart",
        waitUntil: "Cart icon shows 1 item" },
    ],
    assertions: [
      { assertion: "Cart contains the selected product" },
      { assertion: "Cart total price is greater than $0" },
    ],
    test,
    expect,
  });
});

How It Works Under the Hood

1. Gemini reads the page DOM and executes each step
2. Smart caching — repeated steps skip AI calls (Redis-based)
3. Auto-healing — when cached actions fail, AI re-evaluates
4. Multi-model assertions — Claude AND Gemini both verify, with an arbiter for disagreements

Setup in 2 Minutes

npm init playwright@latest my-tests
cd my-tests
npm install passmark dotenv

// playwright.config.ts
import { configure } from "passmark";
configure({ ai: { gateway: "openrouter" } });

Real-World Results

I tested 5 production apps with 30 tests:

• Zero CSS selectors used
• Zero flaky tests from timing
• Tests survive UI redesigns

Try it: github.com/bug0inc/passmark

---

Have questions about AI testing? Ask in the comments!