idavidov13

Posted on Aug 18 Originally published at idavidov.eu

Agentic Playwright Is Now Open Source

#ai #playwright #testing #agents

You ask your AI assistant to write a Playwright test. It delivers instantly. The code looks clean, the review passes, you merge it.

Two days later it fails in CI. You open the file and there it is: an XPath selector, a waitForTimeout(5000), and an any type hiding a broken assertion. Sound familiar?

The problem was never the AI. The problem is that nobody gave it the rules. Today I'm fixing that for everyone: Agentic Playwright is now open source.

🎉 What I'm Announcing

Agentic Playwright is a production-grade Playwright + TypeScript scaffold built for AI-assisted test automation. It ships a complete, running test framework - page objects, API contracts, data factories, CI - plus the AI rules your assistant picks up automatically, from the very first prompt.

One command gets you there:

npm create agentic-playwright . -- --demo

Zero questions. It scaffolds the framework, installs dependencies and a browser, and ends by running a smoke test against a live demo application. You see green before writing a single line. Under 5 minutes on normal broadband.

I spent the last year building, curating, maintaining, and extending this framework as a paid product. On August 13 I open sourced the main part of it under an MIT license, because I want every QA engineer working with AI to start from this baseline instead of assembling one from scratch.

I also recorded a short announcement on my channel where you can watch the scaffold go from empty folder to green smoke test:

And this is only the beginning. I'm recording a full video series on @ArchQA that walks through every part of the framework, from page objects and API contracts to the concepts behind agentic testing itself: the harness, the outer loop, progressive disclosure, and fighting context rot.

🤔 What "Agentic" Actually Means Here

Imagine a new engineer joins your team. Smart, fast, eager. On day one, would you let them push straight to main with no onboarding, no coding standards, no review checklist? Of course not. You hand them the employee handbook first.

Your AI assistant is that new engineer, except it joins your team fresh every single session. It never remembers yesterday's corrections. Re-explaining your conventions in every prompt is onboarding the same hire hundreds of times.

Agentic Playwright inverts that. The rulebook lives in the repository itself, and the assistant loads it automatically. Three layers make it stick:

The Constitution - MUST/SHOULD/WON'T tables in CLAUDE.md that define the safety floor: no XPath, no hard waits, no any, strict Zod schemas, one tag per test.
17 skills - focused rule files that load per area. The AI reads page object rules only when touching pages/**, API testing rules only in API specs. Small context, specific rules.
An enforcement hook - a PreToolUse script that mechanically blocks forbidden patterns before the file is ever written. Prompt rules can be ignored under context pressure. This can't.

The same system is mirrored for Claude Code, Cursor, and GitHub Copilot, so your team isn't maintaining three diverging rule sets.

If you want the deeper story behind this approach, I've written about what agentic QA means and the scaffold's architecture in the Agentic QA series.

⚖️ The Same Prompt, Two Different Tests

Here is the difference the rulebook makes. The same prompt - "write a test for the product search" - to the same AI assistant.

Without the rules:

// ❌ Wrong: XPath, hard waits, `any`, magic timeouts, no structure
test('search', async ({ page }) => {
    await page.goto('https://practicesoftwaretesting.com');
    await page.locator('//input[@id="search-query"]').fill('pliers');
    await page.locator('//button[@type="submit"]').click();
    await page.waitForTimeout(5000);
    const cards: any = await page.$$('.card');
    expect(cards.length > 0).toBe(true);
});

With Agentic Playwright:

// ✅ Correct: fixtures, steps, web-first assertions
import { expect, test } from '../../../fixtures/pom/test-options';

test(
    'should show only matching products when searching',
    { tag: '@regression' },
    async ({ homePage }) => {
        await test.step('GIVEN the user is on the home page', async () => {
            await homePage.open();
        });

        await test.step('WHEN the user searches for "pliers"', async () => {
            await homePage.searchFor('pliers');
        });

        await test.step('THEN every result matches the search term', async () => {
            await expect(homePage.searchCaption).toContainText('pliers');
            await expect(homePage.productNames.first()).toContainText('Pliers');
        });
    }
);

Dependency-injected page objects, Given/When/Then steps, web-first assertions, no hard waits. Nobody reviewed this into shape. The assistant produced it on the first try because the rules were already in place.

🧱 What's in the Box

The scaffold is not a folder of empty configs. It is a running framework with every architectural decision already made:

Area	What you get
Test architecture	Page Object Model with dependency-injected fixtures, reusable UI components
API testing	`apiRequest` fixture with Zod 4 strict schema validation
Test data	Faker factories for dynamic data, `as const` TypeScript files for static boundary data
Authentication	Pre-configured storage state for authenticated test projects
Code quality	ESLint, Prettier, and Husky pre-commit hooks, all wired
AI workflow	A confidence-gated workflow where the agent must state its confidence and stop to ask when inputs are missing, instead of building on guesses
Exploration	`playwright-cli` as the default path for the AI to explore the live DOM before writing locators, so selectors are real instead of hallucinated
Environment	A Dev Container with pre-warmed caches, plus setup scripts for local installs

Every piece follows the same Constitution. The example files show the patterns, and the README walks you through replacing them with your real application step by step.

🚀 Try It in the Next Five Minutes

From an empty directory:

npm create agentic-playwright . -- --demo

Then open the project in Claude Code, Cursor, or VS Code with Copilot and ask it to write a test. Watch what comes back.

Prefer your own application instead of the demo? Run the bare variant and point the environment file at your URLs:

npm create agentic-playwright . -- --bare

If something bites you, open an issue on the repository. Stars help other engineers find it, and issues help me make it better.

🔓 What's Free and What's Pro

The open-source repository is the complete, working scaffold. No crippled features, no trial period. You get the full framework, the Constitution with all 17 skills for Claude Code, Cursor, and Copilot, the PR-review skill, the basic write-time enforcement hook, and the lint gates that keep the rule system from rotting.

So what does Agentic Playwright Pro add? One sentence from the README says it best: rules are advice, enforcement is a guarantee.

The free scaffold tells your AI what good tests look like. Pro makes it unable to ship anything else:

The full enforcement hook suite, with write guards plus refactor and verification reminders
Custom AST lint rules for every single WON'T in the Constitution
Generated, always-in-sync rule trees across all three AI tools, no manual mirroring
Drift, parity, and canary gates that fail CI the moment rules and mirrors diverge
An automated skill-evaluation CI that continuously re-tests the rules themselves
Every new release first, plus direct access to me

If you're one engineer exploring agentic testing, start with the open-source version. If you're a team standardizing AI usage and you need guarantees instead of guidelines, Pro is built for you.

🙏🏻 Thank you for reading!

Scaffold a project, hand your AI the rulebook, and tell me what it writes. The video series on @ArchQA will go deep on every concept behind the framework, so subscribe there if you want the internals as they land.

DEV Community