A Test-First Approach with Playwright for Unstable Systems

This article introduces a method for achieving a test-first approach using E2E testing frameworks such as Playwright. This method addresses the following challenges encountered when attempting to automate tests:

• You want to verify behavior while implementing test code, but the system under test is still unstable and doesn't run properly.
• It's difficult to explain the content of the test code to non-engineers.

This approach assumes the design described in the previous article introducing the E2E test design pattern "Extended Page Object Model" here:

• A common base class is created for PageElements, separating the Testing Framework APIs from PageElement and centralizing them in the base class.

A working implementation of this approach is available on GitHub.

Overview

This approach attempts to solve the above issues by adding the following features to the test code:

• Execute test code without operating the browser.
• Output the browser operation instructions to a file.

These features are collectively referred to as Dry Run below. The execution structure of the Dry Run feature is shown below:

In normal execution (real-run), BasePageElement operates the browser via BrowserController to perform tests. In contrast, during Dry Run execution (dry-run), BasePageElement does not use BrowserController and only outputs to DryRunLog.

The following sequence diagram illustrates this behavior:

1. PageElement instructs BasePageElement to operate screen elements.
2. In execution mode:
   1. BasePageElement instructs BrowserController to operate the browser.
3. In Dry Run mode:
   1. BasePageElement instructs DryRun to output a log.
   2. DryRun outputs the screen operation details to a DryRunLog file.

The interactions and implementations of each component are described in the following sections.

PageElement -> BasePageElement

PageElement uses methods provided by BasePageElement to operate screen elements.
For example, the clickLoginButton method in LoginPagePageElement:

import BasePageElement from '@arch/BasePageElement';

export default class LoginPagePageElement extends BasePageElement {
  get pageNameKey() {
    return 'login';
  }

  async clickLoginButton() {
    await this.click('#login-form input[name="login"]');
  }
}

DryRun is not involved in the interaction between PageElement and BasePageElement. However, it is important that PageElement never directly uses the Playwright API and always does so via BasePageElement.

BasePageElement -> BrowserController

BasePageElement uses BrowserController to operate screen elements. In implementation, BrowserController is the Playwright API’s Page. BasePageElement adds control logic to skip using BrowserController during Dry Run and instead log the actions. Here's a simplified implementation of the click method with such control (see BasePageElement.ts for the full version):

import { expect, type Page } from '@playwright/test';
import { Action, DryRun } from '@arch/DryRun';

export default abstract class BasePageElement {
  private page : Page;
  abstract get pageNameKey(): string;

  protected async click(selector: string) {
    if (!DryRun.isOn) {
      await this.page.locator(selector).click();
    } else {
      const log = DryRun.log(this.pageNameKey, selector, Action.CLICK);
      await this.page.evaluate(`console.log(\`${log}\`);`);
    }
  }
}

The DryRun.log method receives information for identifying the action and outputs it to a file, while also printing it to the browser’s console.log, making it visible in Playwright’s Trace Viewer. Similar branching logic is implemented for methods like inputText and select.

DryRun

DryRun manages execution mode (normal or dry run), builds log strings, and outputs them to a file. Here's a simplified implementation (see DryRun.ts for the full version):

import fs from 'fs';

export enum Action {
  GOTO,
  CLICK,
  INPUT,
}

export class DryRun {

  constructor(public isOn: boolean, private readonly formatter: LogFormatter) {}

  log(pageNameKey: string, itemNameKey: string, action: Action, input?: string) {
    const pageName = this.resolvePageName(pageNameKey);
    const itemName = this.resolveItemName(itemNameKey);

    const logStr = this.formatter.format(pageName, itemName, action, input);

    fs.appendFileSync(this.filePath, str + '\n');
  }
}

DryRun logs are meant to show "what screen operations are performed in the test scenario." Therefore, the log string must include the following information:

• Which screen? (pageNameKey)
• Which screen element? (itemNameKey)
• What action was taken? (action)
• What value was used? (input)

Example DryRun log output:

Navigates to "/" on the top page.
Inputs "admin" in "Username" on the login screen.
Inputs "newadminpass" in "Password" on the login screen.
Clicks "Login Button" on the login screen.
Clicks "Project Menu" on the top page.
Clicks "Demo Project" on the top page.
Clicks "Work Packages Menu" on the project page.
Clicks "Create Button" on the project page.
Clicks "Task Link" on the project page.
Inputs "Test Subject onak9fp7aj8" in "Title" on the task input screen.
Inputs "High" in "Priority" on the task input screen.
Clicks "Save Button" on the task input screen.

Dry Run Log on Trace Viewer

The DryRun log output via console.log in BasePageElement appears in the Playwright Trace Viewer during normal execution like this:

You can view actual Playwright Report and Trace Viewer results of a Dry Run execution at the following URL:

Playwright Report + Trace Viewer

Conclusion

By integrating the Dry Run feature into test code, it becomes possible to execute test logic without operating the browser. In other words, test code can be run without accessing the target system. This allows implementation of test code even when the system is still unstable. Additionally, Dry Run logs provide a readable overview of test scenarios without reading the actual code, making them a valuable resource for non-engineering project members. For an implementation example and demo, refer to the following GitHub repository:

GitHub Repository