Solving Flaky Playwright Tests: How We Automated E2E Generation with an AI Agent and MCP

Pluswithai — Wed, 04 Nov 2026 22:19:14 +0000

Every developer knows the dread of seeing a green CI pipeline turn red because of an end-to-end test failure after a harmless frontend update.

You change a Tailwind class, reorder a flexbox container, or update an icon, and suddenly three Playwright specs break. The underlying logic works perfectly, but the tests were coupled to brittle DOM structures or arbitrary class names.

Maintaining E2E test suites is one of those engineering tasks we know is critical, but teams often push aside because scaffolding and debugging selectors feels like an endless chore.

In this post, I want to break down an architectural approach to solving this: using an autonomous AI agent that explores web apps, infers user journeys, and generates standard Playwright code with MCP support.

The Three Core Bottlenecks of Modern E2E Testing

Brittle Locators: Tests relying on CSS classes (.btn-primary-v2) or deep XPath selectors break constantly. Modern frontends with utility CSS or CSS modules make this even worse.
Scaffolding Friction: Writing boilerplate for auth flows, form validations, and navigation paths takes valuable engineering hours.
The Vendor Lock-in Trap: Many modern AI testing tools hide your tests inside proprietary black-box runners or cloud dashboards. If you cancel your subscription, you lose your entire test suite.

The Agentic Approach: Exploration Over Recording

Traditional test recorders simply watch where you click and record raw coordinates or selectors. Instead, an agentic model interacts with the web application like a human QA engineer:

1. Semantic DOM Inspection

Rather than scraping raw CSS selectors, the agent analyzes the page's Accessibility Tree and semantic structure. It identifies interactive elements by their functional roles (role="button", accessible labels, and ARIA attributes), matching how actual users perceive the UI.

2. Autonomous Journey Mapping

The agent crawls the application dynamically, discovering actionable paths: signup flows, navigation states, filter updates, and checkout steps.

3. Emitting Standard Playwright Code

The most important design decision: no proprietary formats. The output must be idiomatic Playwright TypeScript code using standard practices like page.getByRole() and page.getByLabel():

import { test, expect } from '@playwright/test';

test('completes user onboarding flow', async ({ page }) => {
  await page.goto('/signup');

  // Using resilient accessible locators rather than brittle class names
  await page.getByLabel('Email address').fill('dev@example.com');
  await page.getByLabel('Password').fill('SecurePassword123!');
  await page.getByRole('button', { name: 'Create account' }).click();

  // Validate state progression
  await expect(page.getByRole('heading', { name: 'Welcome aboard' })).toBeVisible();
});

Because it outputs standard files, you can commit them directly to your Git repository, run them locally, or plug them into your existing GitHub Actions pipeline.

Integrating with Developer Workflows: CLI & MCP

Developers don't want to live inside another isolated web dashboard. Testing belongs inside the developer loop.

To bridge this gap, we built support for two developer interfaces:

A CLI tool (@pluswithai/cli): Run explorations and trigger isolated cloud test executions straight from your terminal.
Model Context Protocol (MCP) Server: By providing an MCP server, environments like Cursor, Claude Desktop, or agent-enabled IDEs can directly interface with the test engine. You can ask your agentic IDE to "explore my staging app and generate a Playwright suite for the checkout flow", and the tool will execute the exploration and return the code to your editor.

Where We're Headed (and Current Challenges)

We built this architecture into Pluswithai (https://pluswithai.com), an agent that explores your site, writes standard Playwright suites, and runs them on a schedule in isolated cloud browsers.

Building autonomous testing agents is an ongoing engineering challenge. Right now, our focus is on:

Handling multi-factor authentication (MFA) and complex single-sign-on (SSO) edge cases cleanly.
Managing realistic mock database states for destructive operations (e.g., deleting a tenant or cancelling subscriptions).
Auto-healing selectors when fundamental UI redesigns happen.

Try It Out & Give Us Your Feedback

We're currently running an open trial (no credit card required, no waitlist) so developers can test it on their own web apps and see how it handles their flows:

👉 Visit Pluswithai - https://pluswithai.com

I'd love to hear from other engineers maintaining Playwright or Cypress suites:

What is currently your most painful source of test flakiness?
What features would you like to see exposed through an MCP server for test automation?

Leave a comment below—I'll be actively answering technical questions about the architecture and how it works under the hood!

DEV Community: Pluswithai