Box Test Steps in Playwright

We have a scenario where we are running a variety of tests to test that a user can add products to the shopping cart. On doing this we have realised that we have some duplicate code.

Each test has the same code to add a product to the cart. When we create a helper function for this we can use the test.step method to group several actions into one named step and then set the box option to true so that errors inside the step are not shown and we therefore hide the implementation details for this step. Let's take a look at how this works.

Our Test Scenarios

On our site there are many ways to add a product such as from the main hero banner, from the search box and from the all products page. We have created a test for each of these scenarios.

import { test, expect } from '@playwright/test';

test.describe('add to cart scenarios', () => {
  test.beforeEach(async ({ page }) => {
    await page.goto('https://cloudtesting.contosotraders.com/')
  });

  test('add to cart from carousel', async ({ page }) => {
    await page.getByRole('button', { name: 'Buy Now' }).click();
    await page.getByRole('button', { name: 'Add To Bag' }).click();
    await page.getByLabel('cart').click();
    await expect(page.getByText('Xbox Wireless Controller Lunar Shift Special Edition')).toBeVisible();
  });

  test('add to cart from search', async ({ page }) => {
    const product = 'Xbox Wireless Controller Mineral Camo Special Edition'
    const placeholder = page.getByPlaceholder('Search by product name or search by image')
    await placeholder.click();
    await placeholder.fill('xbox');
    await placeholder.press('Enter');
    await page.getByRole('img', { name: product }).click();
    await page.getByRole('button', { name: 'Add To Bag' }).click();
    await page.getByLabel('cart').click();
    await expect(page.getByText(product)).toBeVisible();
  });

  test('add to cart from all products page', async ({ page }) => {
    const product = 'Xbox Wireless Controller Lunar Shift Special Edition'
    await page.getByRole('link', { name: 'All Products' }).first().click();
    await page.getByRole('img', { name: product }).click();
    await page.getByRole('button', { name: 'Add To Bag' }).click();
    await page.getByLabel('cart').click();
    await expect(page.getByText(product)).toBeVisible();
  });
});

Helper Function

We can create a helper function called addAndViewCart which captures the common functionality of adding a product to the cart. This function finds an element on the page with the role of button and the name Add To Bag and clicks it:

async function addAndViewCart(page: Page) {
  await page.getByRole('button', { name: 'Add To Bag' }).click();
  await page.getByLabel('cart').click();
}

We can then use this helper function throughout our tests so we have less repetitive code. Also if we were to make a change to this button we would only have to modify it in one place in our code, in the helper function.

await addAndViewCart(page);

In the example below our helper function is used in all 3 of our tests.

import { test, expect, Page } from '@playwright/test';

async function addAndViewCart(page: Page){
  await page.getByRole('button', { name: 'Add To Bag' }).click();
  await page.getByLabel('cart').click();
}

test.describe('add to cart scenarios', () => {
  test.beforeEach(async ({ page }) => {
    await page.goto('https://cloudtesting.contosotraders.com/')
  });

  test('add to cart from carousel', async ({ page }) => {
    await page.getByRole('button', { name: 'Buy Now' }).click();
    await addAndViewCart(page);
    await expect(page.getByText('Xbox Wireless Controller Lunar Shift Special Edition')).toBeVisible();
  });

  test('add to cart from search', async ({ page }) => {
    const product = 'Xbox Wireless Controller Mineral Camo Special Edition'
    const placeholder = page.getByPlaceholder('Search by product name or search by image')
    await placeholder.click();
    await placeholder.fill('xbox');
    await placeholder.press('Enter');
    await page.getByRole('img', { name: product }).click();
    await addAndViewCart(page);
    await expect(page.getByText(product)).toBeVisible();
  });

  test('add to cart from all products page', async ({ page }) => {
    const product = 'Xbox Wireless Controller Lunar Shift Special Edition'
    await page.getByRole('link', { name: 'All Products' }).first().click();
    await page.getByRole('img', { name: product }).click();
    await addAndViewCart(page);
    await expect(page.getByText(product)).toBeVisible();
  });
});

Making our tests fail

What happens when we have some errors in our test? Let's fail our test in the line before where we use our helper function. We can simply comment this line out on two of our tests.

Our test will try to click the add to cart button but won't be able to because the previous step is what takes us to the product page that contains this button. Without this step our test will fail.

import { test, expect, Page } from '@playwright/test';

async function addAndViewCart(page: Page){
  await page.getByRole('button', { name: 'Add To Bag' }).click();
  await page.getByLabel('cart').click();
}

test.describe('add to cart scenarios', () => {
  test.beforeEach(async ({ page }) => {
    await page.goto('https://cloudtesting.contosotraders.com/')
  });

  test('add to cart from carousel', async ({ page }) => {
    await page.getByRole('button', { name: 'Buy Now' }).click();
    await addAndViewCart(page);
    await expect(page.getByText('Xbox Wireless Controller Lunar Shift Special Edition')).toBeVisible();
  });

  test('add to cart from search', async ({ page }) => {
    const product = 'Xbox Wireless Controller Mineral Camo Special Edition'
    const placeholder = page.getByPlaceholder('Search by product name or search by image')
    await placeholder.click();
    await placeholder.fill('xbox');
    await placeholder.press('Enter');
    // await page.getByRole('img', { name: product }).click();
    await addAndViewCart(page);
    await expect(page.getByText(product)).toBeVisible();
  });

  test('add to cart from all products page', async ({ page }) => {
    const product = 'Xbox Wireless Controller Lunar Shift Special Edition'
    await page.getByRole('link', { name: 'All Products' }).first().click();
    // await page.getByRole('img', { name: product }).click();
    await addAndViewCart(page);
    await page.getByLabel('cart').click();
    await expect(page.getByText(product)).toBeVisible();
  });
});

Now lets run our test using the terminal.

npx playwright test --project=chromium

Two of our tests have failed as expected. However, the error messages tells us that the click is failing inside the function addAndViewCart on the line that involves clicking the button with the name 'Add to Bag'. While this is true, it would be much more helpful to see what happened before we called the function / where in the actual test this function got called.

In this case, we can't click this button because we are not on a page that has this button displayed, which we could have spotted directly. So this helps us to save time while looking at the actual error message instead of getting pointed to a helper function where the click has failed.

This would then result e.g. in such an error:

Helper Function using test steps and boxed

We need a better way to show the errors and that is where boxed steps come in.

The test.step is used to group several actions into one named step, useful for better test reports. Let's add a test.step to our helper function. The test.step method takes a name followed by an async function. Inside this function we add our click event and at the end of the function we add another parameter setting box to true. Make sure to await the test.step so it will await the inner function as well.

async function addAndViewCart(page: Page){
  await test.step('add to cart', async () => {
    await page.getByRole('button', { name: 'Add To Bag' }).click();
    await page.getByLabel('cart').click();
  }, { box: true });
}

We don't need to make any changes to our test code as we have only updated the helper function. Let's go ahead and run our test again to see what the difference is and how box steps can help us.

npx playwright test --project=chromium

You can see from the screenshot of the terminal results that even though the error is the same, as in the test timed out while waiting for the Add to Bag button to appear the actual error code and line numbers are different.

Instead of showing us the helper function code it is in fact showing us the test code of where that helper function is being used meaning we can easily see what was happening before the helper function so we can much easier debug our tests.

Check out our release video to see a live demo of box test steps.

Conclusion

Boxed steps allows us to hide the implementation details of our helper functions and instead show the test code where the helper function is being used. This makes it much easier to debug our tests when they fail either through error messages in the terminal or in the reporters (e.g. HTML).

Useful Links

• Check out the latest release notes explaining box test steps.
• Check out the docs for more info on test.step.
• Check out the sample code on GitHub
• Join our Discord server