AI-Powered Test Automation: How Playwright Agents Plan, Write, and Fix Tests for Us

There’s a point in every QA engineer’s journey when maintaining automated tests becomes harder than writing them. You fix one flaky selector, another breaks — a small refactor ripples through half the suite.

We’ve spent years making automation smarter with patterns, abstractions, and better reporting — but structure alone can’t keep up with change.

Now, with Planner, Generator, and Healer, Playwright can analyze your UI, plan test cases, fix broken steps, and explain issues in plain English. We’re entering a phase where AI acts like a test engineer that never sleeps — one that learns from every failure.

In this post, I’ll show how I used Playwright Agents on a simple Dockerized Juice Shop app — how they plan, write, and fix tests on their own, and what that means for QA.

Setting the Stage: Starting with Structure

Before generating anything, I wanted the AI to understand my folder logic — a clean structure is half the battle in Playwright automation.

To help the Agent learn that structure, I first created the folders manually inside VS Code Insider — one for each part of the framework: components, fixtures, pages, selectors, tests, and utils. Once the structure was ready, I opened the workspace and prompted the Agent to explore it. From that point, every generated file automatically followed the same naming and placement pattern I had defined.

💡 Tip: Always define your folder structure before running the Planner. This gives the AI context about your framework organization, so instead of creating random files, it will follow your layout and naming conventions.

PLAYWRIGHT-AGENT-DEMO
components/  = reusable UI logic and visual parts
fixtures/    = shared test data and setup helpers
pages/       = page objects with user flows
selectors/   = locators and test IDs
tests/       = main test specs
utils/       = helper methods and utilities

Setting Up Playwright Agents

With the structure ready, it’s time to bring the Agents to life.

1️⃣ Use VS Code Insiders

Playwright Agents need VS Code Insiders for full automation — the regular version can only suggest, not act.
👉 https://code.visualstudio.com/insiders

2️⃣ Initialize Playwright Agents

Run this inside your project:

npx playwright init-agents --loop=vscode

• Creates all necessary AI agent files inside .github/chatmodes/
• Sets up three roles:Planner,Generator, and Healer
• Re-run after major Playwright updates to refresh definitions

Enabling File Editing in Chat Mode

Before using the Agents, give them permission to edit your workspace:

Command Palette → Chat → Configure Tools

• ✅ Edit workspace files
• ✅ Run commands
• ✅ Search files

This allows the Agents to actually create and modify files, not just suggest code.

Planner: From Prompt to Login Test Plan

Prompt I used for the Planner: this is the message I sent to the Agent to generate a test plan

Plan a concise login test for Juice Shop (http://localhost:3000) — happy path + invalid creds, with ACs and selector/fixture notes. Output to /specs/login-plan.md.

• Planner explored the login screen at http://localhost:3000
• Read my workspace and recognized folder structure
• Produced /specs/login-plan.md with ACs and implementation notes

Generator: From a Simple Plan to a Complete Login Suite

Prompt I used for the Generator: the message I sent to make the Agent build my entire login flow — pages, selectors, fixtures, tests, and the config — all structured and ready to run.

Generate all login files per /specs/login-plan.md into my folders (components/fixtures/pages/selectors/tests/utils) and include/update playwright.config.ts

What the Generator did:

• Created login.components.ts for form logic
• Added users.json fixture with test data
• Generated LoginPage.ts (page object)
• Defined login.selectors.ts for stable locators
• Created four spec files:
  • login-happy-path.spec.ts
  • login-invalid-credentials.spec.ts
  • login-security.spec.ts
  • login-smoke.spec.ts
• ⚙️ Updated playwright.config.ts and package.json

Note: The output depends on how detailed your test plan is. Since mine was for the OWASP Juice Shop app, the Agent also generated some security-related tests. They weren’t needed for my scope, but I let it run just to see what it could do.

Healer: Fixing Broken Tests

To test reliability, I intentionally broke a selector in login.selectors.ts — changing #email to #mail. The Healer analyzed the DOM, found the mismatch, and fixed it automatically.

This was a small example, but it showed real potential. Beyond broken selectors, the Healer can also refactor — remove redundant code, simplify methods, and clean up repetition.

Observation: the Healer isn’t always consistent — sometimes a one-line fix becomes a long refactor. Still, it gets the job done, tirelessly, while you focus on bigger problems.

✅ Running the Complete Test Suite

After the Planner created the test plan, the Generator built all the files, and the Healer fixed a few initial issues — I hadn’t written a single line of code myself.

Everything was planned, generated, and healed automatically by the Playwright Agents.

When I finally ran the full test suite, this is how it looked in the terminal:

All login-related scenarios — valid, invalid, and security — executed successfully across multiple browsers. Seeing all tests green without touching a single spec file really shows how far automation has come.

💭 Final Thoughts

After using these Playwright Agents, I can say this is a big step forward for smarter, faster testing. They widen coverage, reduce time on repetitive tasks, and make automation feel more collaborative.

Everything depends on how clearly you describe and structure things — the better your setup, the better the results. I wanted this post to be more hands-on than theoretical — similar to what Debbie O’Brien showed in the Playwright Live session.

The Generator does a solid job but still needs guidance — sometimes repeating logic or overcomplicating simple functions. That’s where we, as QA engineers, come in: to fine-tune, guide, or rewrite when needed. For example, it often adds a .goto('/') in every test — not ideal, but easy to fix and impressive for a start.

I also noticed that sometimes even simple tests get generated in a broken state — for example, with incorrect assertions or missing waits. However, the Healer usually steps in and fixes them automatically once you run the suite. Still, it’s clear that without our QA intuition and manual oversight, these agents wouldn’t deliver tests in the exact way we expect them to behave. A quick look at a failing report is often enough to spot what went wrong and understand how the AI reasoned its way to that error — which is fascinating in itself.

The Healer can overdo things but always finds a way back. Later, you can run the Generator again to tidy up the result.

Recommendation: try these Agents, even on a demo app. You’ll quickly see how much time they save and how AI can become your QA teammate.

Thanks for reading! Feel free to share your thoughts or questions in the comments — let’s keep learning and stay in step with the future of testing. 🚀