AI-Driven Playwright E2E Tests From One Reference Doc

Muhammad Usman — Mon, 20 Jul 2026 11:11:17 +0000

Writing E2E Tests With One Doc Instead of New Tooling

We had a growing monorepo with two frontend apps and zero end-to-end coverage. The blocker was never Playwright itself. It was that every new feature meant babysitting an AI agent through the tests, click by click, and the output still needed rewriting by hand.

Here is what actually fixed it, and the part I got wrong on the first try.

Why a dedicated package, not a shared test util

The first instinct was to drop Playwright into the existing shared testing package. That does not work, for boring but real reasons:

The unit-test runner and Playwright's runner cannot cohabit in one package.
E2E runs must not be build-cached, while unit tests are cacheable. One package can't be both.
Browser binaries and report output have no business leaking into a shared utility package.

So the e2e suite is its own package, built from scratch. That separation is what let the rest of the decisions stay clean.

The structure that made tests cheap to write

Three rules, enforced by the setup rather than by review comments:

Selectors live only in Page Object Models. Spec files never contain a raw locator.
Page objects are injected as fixtures. No manual instantiation in a spec, ever.
Auth runs once. A setup project logs in, saves the session state to disk, and every test loads it. No login flow inside a test.

The result is that a spec reads like the user story and nothing else:

test('user can open the create form', async ({ createItemPage }) => {
  await createItemPage.goto();
  await expect(createItemPage.heading).toBeVisible();
});

Already authenticated, page object already wired in, no selector in sight.

The part I got wrong first

With the structure in place, I expected the AI-assisted authoring to be the easy win. It was the opposite. Pointing Codegen and the agent at a page produced tests that looked plausible and were quietly wrong. The agent guessed expected values it had no way to know, and every generated spec came back to my desk for a manual rewrite. It was slower than just writing them myself.

The missing piece was context. The agent knew how to click a button. It did not know what the feature was supposed to do — the state machine, the validation limits, which behaviors were correct and which were known bugs.

So I wrote a skill for one feature that spelled all of that out: the flows, the rules, the routes, the confirmed bugs. The next tests it wrote were good. Not "good for AI" good. Better than what I was writing by hand, because it now had the right direction and never lost the thread halfway through a complex flow.

Then I generalized it.

One doc per feature, not a skill per feature

A skill per feature does not scale. The insight was that the skill was generic; only the facts were feature-specific. So I split them.

The authoring engine became one reusable skill. It reads a per-feature reference doc, discovers the routes, reuses or extends the page objects, drives the real app, captures the actual outcomes as assertions, then runs and self-heals the spec. On top of that sits a thin layer: the official Playwright browser-driving skill for the low-level interaction, and one dedicated skill for the single most complex domain that earned extra shortcuts.

What a new feature needs now is one document: entities, state machine, validation rules, and a numbered list of known bugs. That is it. The reference doc is the contract. If it does not exist, the agent is told to stop and ask for it instead of inventing rules from the UI, which is exactly the failure mode that cost me the first week.

Before: walk the agent through every flow, then rewrite the output.
After: write one ground-truth doc, point the generic skill at it, review real tests.

The reusable idea

The lesson had nothing to do with Playwright. Agents do not fail e2e work because they can't drive a browser. They fail because they guess at behavior no one wrote down. Encode the business truth in a document the agent is forced to read, keep the authoring engine generic, and adding coverage for the next feature costs one doc instead of a new pile of tooling.