Iterate quickly using the new --only-changed option

#playwright #e2e #testing

Playwright v1.46 ships with a nifty new feature for local development.
By specifying the --only-changed option, Playwright looks at uncommited changes and runs all affected test files.

If you've used Jest before, this is pretty similar to Jest's --onlyChanged flag.

Let's see it in action. If you prefer video, watch the Demo in the release video. In this example, we have a repository without uncommitted changes:

// utils.ts
export const inputPlaceholder = 'What needs to be done?'
export const todoItemTestID = 'todo-item'
export const todo = 'buy milk'

// test.spec.ts
import { test, expect } from '@playwright/test'
import { inputPlaceholder, todo, todoItemTestID } from './utils.ts'

test('adding a todo', async ({ page }) => {
    await page.goto('https://demo.playwright.dev/todomvc')

    const inputField = page.getByPlaceholder(inputPlaceholder)
    await inputField.fill(todo);
    await inputField.press('Enter');

    await expect(inputField).toBeEmpty();
    await expect(page.getByTestId(todoItemTestID)).toHaveText(todo);
})

If we run a test with --only-changed, we see that no tests are run:

$ npx playwright test --only-changed
Error: No tests found

Now let's make a change to one of the files, but not commit it yet:

// utils.ts
export const inputPlaceholder = 'What needs to be done?'
export const todoItemTestID = 'todo-item'
- export const todo = 'buy milk'
+ export const todo = 'make cappucino'

❯ npx playwright test

Running 1 test using 1 worker

  ✓  1 [chromium] › test.spec.ts:4:5 › adding a todo (426ms)

  1 passed (837ms)

Playwright found changes in utils.ts, so it executed all test files that depend on utils.ts. Wonderful!

By default, Playwright will compare against HEAD to determine changed files.
But you can also specify a different Git revision to compare against: Using --only-changed=main on a feature branch executes all tests that are affected by changes on the feature branch. And --only-changed=<commit sha> executes all changes between HEAD and the specified commit.

How do I use this?

--only-changed is another tool in the toolbelt to help you iterate quickly. Here's five ideas for how you can integrate it into your workflow:

1. Local development: Use --only-changed=main to quickly run affected tests as you make changes. This is especially useful when you're working on a feature and want to see the impact of your changes.

2. pre-commit hook: Running with --only-changed in a pre-commit hook prevents failing tests from being commited. Here's an example hook you can use:

npx playwright test --only-changed

3. pre-push hook: As the name says, a pre-push hook runs before pushing. This prevents failing tests from invoking a costly CI run. Here's an example implementation:

while read local_ref local_sha remote_ref remote_sha
do
    # deleting branch, nothing to check
    if [ "$local_sha" = '0000000000000000000000000000000000000000' ]; then continue; fi

    echo "Running Playwright for HEAD...$remote_sha"
    npx playwright test --only-changed="$remote_sha"
done

4. Faster feedback in CI: Prior to a full run, use --only-changed to get faster feedback on new failures. This is especially useful in large test suites, where running all tests can take a long time. Here's how that'd look in GitHub Actions:

...
    - name: Install Playwright Browsers
      run: npx playwright install --with-deps
+   - name: Run changed Playwright tests
+     run: npx playwright test --only-changed=${{ github.base_ref }}
    - name: Run Playwright tests
      run: npx playwright test
...

5. Bonus idea: If you're interested in --only-changed, you might also be interested in Playwright's Watch mode that's available in the UI and in the VS Code Extension.

That's it! Let us know what you think and how you're using --only-changed. Either in the comments here, in the Community Discord, or on LinkedIn! And have a wonderful day.

Top comments (2)

Pramod Kumar Yadav • Sep 4

Trying to run "changed playwright tests" in github using the solution mentioned on step 4, i,e.

+   - name: Run changed Playwright tests
+     run: npx playwright test --only-changed=${{ github.base_ref }}

Results in below error.

Error: Cannot detect changed files for --only-changed mode:
git ls-files --others --exclude-standard

fatal: detected dubious ownership in repository at '/__w/playwright-sandbox/playwright-sandbox'
To add an exception for this directory, call:

    git config --global --add safe.directory /__w/playwright-sandbox/playwright-sandbox

Adding the above step after checkout (such as below):

 - name: Checkout Repository
        uses: actions/checkout@v4

      - name: Add safe directory exception
        run: git config --global --add safe.directory /__w/playwright-sandbox/playwright-sandbox

Also results in the same error. Any clue what I maybe doing wrong here and/or need to fix this issue?