DEV Community

Ken Fukuyama
Ken Fukuyama

Posted on

Demystifying Playwright Test Agents' seed.spec.ts: What I Learned from Reading the MCP Code

#playwright #ai #typescript #testing

Introduction

I recently had a chance to try out Playwright Test Agents, and I'm impressed—this feels like a powerful new tool that could genuinely change how we approach E2E test creation and maintenance.

In this article, I'll dig into something that tripped me up when using Playwright Test Agents: the role of seed.spec.ts. The official documentation didn't make it immediately clear to me, so I ended up reading the MCP source code to understand how it actually works. I'll walk you through what I learned.

Note: This article assumes you have some familiarity with Playwright Test Agents. If you haven't tried it yet, you can still follow along and understand what seed.spec.ts does, but I'd recommend watching the video above first for deeper context. It's mainly in Japanese, but you can get the idea from the English auto dubs.

Where the Official Docs Left Me Confused

The first thing that confused me when starting with Playwright Test Agents was seed.spec.ts.

Looking at the official documentation (as of December 4, 2025), there's just this one line:

Official documentation screenshot

Seed tests provide a ready-to-use page context to bootstrap execution.

Now that I understand how it works, this makes perfect sense. But when I first read it, I honestly had no idea what it meant.

Specifically, what wasn't clear:

  • Whose ready-to-use page context?
  • Who is doing the bootstrap execution?

The missing subjects make this sentence vague. Is this about Playwright's test execution, or something specific to the Agents feature?

Reading the MCP Code

Since staring at the documentation wasn't getting me anywhere, I decided to look at the actual MCP (Model Context Protocol—the standard protocol for AI agents to call tools) code.

Checking the Agent Prompts

The key is in the two agent configuration files generated by the initialization command:

npx playwright init-agents --loop=vscode
Enter fullscreen mode Exit fullscreen mode

Running this command generates:

  • playwright-test-planner.md
  • playwright-test-generator.md

Looking at the Planner's prompt, I found this:

1. **Navigate and Explore**
   - Invoke the `planner_setup_page` tool once to set up page before using any other tools
   - Explore the browser snapshot
   - Do not take screenshots unless absolutely necessary
   - Use `browser_*` tools to navigate and discover interface
   - Thoroughly explore the interface, identifying all interactive elements, forms, navigation paths, and functionality
Enter fullscreen mode Exit fullscreen mode

The important part: it instructs the agent to use the planner_setup_page tool.

Checking the Generator similarly:

# For each test you generate
- Obtain the test plan with all the steps and verification specification
- Run the `generator_setup_page` tool to set up page for the scenario
Enter fullscreen mode Exit fullscreen mode

This one uses generator_setup_page.

Looking at the MCP Tool Implementation

So what do these tools actually do? Let's look at the implementation:

https://github.com/microsoft/playwright/blob/f9e1797a0d6025a0cc499692d79bb10a806eadc1/packages/playwright/src/mcp/test/plannerTools.ts#L23C14-L40

export const setupPage = defineTestTool({
  schema: {
    // Tool name from the prompt
    name: 'planner_setup_page',
    title: 'Setup planner page',
    description: 'Setup the page for test planning',
    inputSchema: z.object({
      project: z.string().optional().describe('Project to use for setup. For example: "chromium", if no project is provided uses the first project in the config.'),
      // Uses seed.spec.ts as seedFile by default
      seedFile: z.string().optional().describe('A seed file contains a single test that is used to setup the page for testing, for example: "tests/seed.spec.ts". If no seed file is provided, a default seed file is created.'),
    }),
    type: 'readOnly',
  },

  handle: async (context, params) => {
    const seed = await context.getOrCreateSeedFile(params.seedFile, params.project);
    // It runs seed.spec.ts first
    const { output, status } = await context.runSeedTest(seed.file, seed.projectName);
    return { content: [{ type: 'text', text: output }], isError: status !== 'paused' };
  },
});
Enter fullscreen mode Exit fullscreen mode

The crucial part is context.runSeedTest(). Before the Planner or Generator does its job, it first executes seed.spec.ts.

Mystery Solved: What seed.spec.ts Really Is

With this understanding, the documentation quote becomes crystal clear:

Seed tests provide a ready-to-use page context to bootstrap execution.

In other words:

  • Whose ready-to-use page context? → For the Planner and Generator (AI agents)
  • Who does the bootstrap execution? → The Planner and Generator run it

So seed.spec.ts isn't a seed file for Playwright tests—it's a setup file that runs before the Planner or Generator starts navigating and exploring the browser.

What Should Go in seed.spec.ts

Given this mechanism, in most cases your seed.spec.ts should include:

  • Login procedures for the user you want the Planner/Generator to explore as
  • Data seeding or environment initialization needed for planning/generation

For example, if you want to generate tests for an admin panel, you can log in as an admin in seed.spec.ts, and the Planner will start exploring from that logged-in state.

Here's a concrete example:

import { test, expect } from '@playwright/test';

const authFile = 'playwright/.auth/user.json';
const testUsername = 'testuser';

test.describe('Test group', () => {
  // request is Playwright's APIRequestContext
  // https://playwright.dev/docs/api/class-apirequestcontext
  test('seed', async ({ page, request }) => {
    // Navigate to login page
    await page.goto('/');

    // Fill in login form
    await page.getByLabel('Username').fill(testUsername);
    await page.getByLabel('Password').fill('password123');
    await page.getByRole('button', { name: 'Login' }).click();

    // Verify successful login
    await expect(page).toHaveURL('/products');

    // Reset user data for testing
    // Note: This assumes your app has a test data reset API
    // In real projects, implement your own initialization logic as needed
    const resetResponse = await request.post('/api/test/reset', {
      data: { username: testUsername },
    });
    expect(resetResponse.ok()).toBeTruthy();

    // Save login state
    await page.context().storageState({ path: authFile });
  });
});
Enter fullscreen mode Exit fullscreen mode

This way, you put any necessary preparation—login procedures, test data resets, etc.—in seed.spec.ts before the agent starts exploring.

How Is This Different from Existing Setup/Fixtures?

At this point, I had a question:

"Wait, doesn't Playwright already have similar mechanisms?"

Global Setup / Teardown

Playwright has documented best practices for setup:

https://playwright.dev/docs/test-global-setup-teardown

Fixtures

There's also the Fixtures concept for writing robust tests:

https://playwright.dev/docs/test-fixtures

The Critical Difference

These mechanisms serve fundamentally different purposes from seed.spec.ts.

Mechanism For Whom When It Runs
Global Setup Test execution When npx playwright test runs
Fixtures Test execution When each test runs
seed.spec.ts AI Agents When Planner/Generator launches

seed.spec.ts isn't executed by Playwright's test runner—it's executed by AI agents via MCP tools. It runs on a completely separate lifecycle from test execution.

About Comments in Generated Code

Another source of confusion was the comments in test code generated by the Planner/Generator:

// spec: specs/basic-operations.md
// seed: tests/seed.spec.ts  // <-- This comment was misleading

import { test, expect } from '../fixtures';

test.describe('Adding New Todos', () => {
  test('Add Valid Todo', async ({ page }) => {
    // 1. Click in the "What needs to be done?" input field
    const todoInput = page.getByRole('textbox', { name: 'What needs to be done?' });
    // ...
  });
});
Enter fullscreen mode Exit fullscreen mode

Seeing // seed: tests/seed.spec.ts, I wondered: "Does this comment somehow trigger seed.spec.ts to run during Playwright test execution?"

Turns out, this comment is just metadata. It records which seed file was used to generate the test, but doesn't cause any special behavior during test execution.

No magical new features here—it's simply recording "this test was generated using this seed environment."

How This Might Work in Mid-to-Large Scale Projects (Hypothesis)

Warning: This section is speculative. I haven't actually verified this in practice, so take it as food for thought.

With this understanding, I can imagine how this could work in medium to large projects.

Using Multiple Seed Files

Looking at the MCP tool implementation, we saw that a seedFile parameter can be passed. Taking advantage of this, you could create multiple seed files for different actors or use cases:

tests/
├── seeds/
│   ├── admin-seed.spec.ts      # Logged in as admin
│   ├── member-seed.spec.ts     # Logged in as regular member
│   ├── guest-seed.spec.ts      # Not logged in
│   └── checkout-seed.spec.ts   # Cart has items
└── ...
Enter fullscreen mode Exit fullscreen mode

When launching the Planner/Generator, you could specify the appropriate seed file for your purpose, enabling more efficient test planning and generation.

Pros and Cons

Pros

  • Skip repetitive operations like login procedures in each exploration session
  • Start test generation from specific states (e.g., items in cart)
  • Separate test generation starting points by actor (admin/regular user/guest)

Cons

  • Need to manage seed files (must keep them in sync with app changes)
  • Maintenance cost for seed files themselves (this is unavoidable)
  • With multiple seed files, you need to decide which one to use (also unavoidable for complex apps)

It's not a silver bullet, but used appropriately, it could significantly improve test generation efficiency.

Summary

In this article, I've explored the role of seed.spec.ts in Playwright Test Agents.

Key Points

  • seed.spec.ts is for the AI agents (Planner/Generator), not for Playwright tests
  • It runs when agents launch via MCP tools, not during Playwright test execution
  • It serves a different purpose and lifecycle from existing Global Setup and Fixtures
  • The // seed: ... comment in generated tests is just metadata

The feature is still new, so documentation hasn't fully caught up yet. But by reading the MCP code, I was able to understand how it works, and now I can use it confidently.

Either way, Playwright Test Agents has the potential to make working with E2E tests more enjoyable if used well. I'm excited to see how it evolves!

Top comments (0)