DEV Community

Cover image for Iterate quickly using the new --only-changed option
Simon Knott for Playwright end to end Testing

Posted on

49 1 1 1 1

Iterate quickly using the new --only-changed option

Playwright v1.46 ships with a nifty new feature for local development.
By specifying the --only-changed option, Playwright looks at uncommited changes and runs all affected test files.

If you've used Jest before, this is pretty similar to Jest's --onlyChanged flag.

Let's see it in action. If you prefer video, watch the Demo in the release video. In this example, we have a repository without uncommitted changes:

// utils.ts
export const inputPlaceholder = 'What needs to be done?'
export const todoItemTestID = 'todo-item'
export const todo = 'buy milk'
Enter fullscreen mode Exit fullscreen mode
// test.spec.ts
import { test, expect } from '@playwright/test'
import { inputPlaceholder, todo, todoItemTestID } from './utils.ts'

test('adding a todo', async ({ page }) => {
    await page.goto('https://demo.playwright.dev/todomvc')

    const inputField = page.getByPlaceholder(inputPlaceholder)
    await inputField.fill(todo);
    await inputField.press('Enter');

    await expect(inputField).toBeEmpty();
    await expect(page.getByTestId(todoItemTestID)).toHaveText(todo);
})
Enter fullscreen mode Exit fullscreen mode

If we run a test with --only-changed, we see that no tests are run:

$ npx playwright test --only-changed
Error: No tests found
Enter fullscreen mode Exit fullscreen mode

Now let's make a change to one of the files, but not commit it yet:

// utils.ts
export const inputPlaceholder = 'What needs to be done?'
export const todoItemTestID = 'todo-item'
- export const todo = 'buy milk'
+ export const todo = 'make cappucino'
Enter fullscreen mode Exit fullscreen mode
❯ npx playwright test

Running 1 test using 1 worker

  βœ“  1 [chromium] β€Ί test.spec.ts:4:5 β€Ί adding a todo (426ms)

  1 passed (837ms)
Enter fullscreen mode Exit fullscreen mode

Playwright found changes in utils.ts, so it executed all test files that depend on utils.ts. Wonderful!

By default, Playwright will compare against HEAD to determine changed files.
But you can also specify a different Git revision to compare against: Using --only-changed=main on a feature branch executes all tests that are affected by changes on the feature branch. And --only-changed=<commit sha> executes all changes between HEAD and the specified commit.

How do I use this?

--only-changed is another tool in the toolbelt to help you iterate quickly. Here's five ideas for how you can integrate it into your workflow:

1. Local development: Use --only-changed=main to quickly run affected tests as you make changes. This is especially useful when you're working on a feature and want to see the impact of your changes.

2. pre-commit hook: Running with --only-changed in a pre-commit hook prevents failing tests from being commited. Here's an example hook you can use:

npx playwright test --only-changed
Enter fullscreen mode Exit fullscreen mode

3. pre-push hook: As the name says, a pre-push hook runs before pushing. This prevents failing tests from invoking a costly CI run. Here's an example implementation:

while read local_ref local_sha remote_ref remote_sha
do
    # deleting branch, nothing to check
    if [ "$local_sha" = '0000000000000000000000000000000000000000' ]; then continue; fi

    echo "Running Playwright for HEAD...$remote_sha"
    npx playwright test --only-changed="$remote_sha"
done
Enter fullscreen mode Exit fullscreen mode

4. Faster feedback in CI: Prior to a full run, use --only-changed to get faster feedback on new failures. This is especially useful in large test suites, where running all tests can take a long time. Here's how that'd look in GitHub Actions:

...
    - name: Install Playwright Browsers
      run: npx playwright install --with-deps
+   - name: Run changed Playwright tests
+     run: npx playwright test --only-changed=${{ github.base_ref }}
    - name: Run Playwright tests
      run: npx playwright test
...
Enter fullscreen mode Exit fullscreen mode

5. Bonus idea: If you're interested in --only-changed, you might also be interested in Playwright's Watch mode that's available in the UI and in the VS Code Extension.

That's it! Let us know what you think and how you're using --only-changed. Either in the comments here, in the Community Discord, or on LinkedIn! And have a wonderful day.

Image of AssemblyAI tool

Transforming Interviews into Publishable Stories with AssemblyAI

Insightview is a modern web application that streamlines the interview workflow for journalists. By leveraging AssemblyAI's LeMUR and Universal-2 technology, it transforms raw interview recordings into structured, actionable content, dramatically reducing the time from recording to publication.

Key Features:
πŸŽ₯ Audio/video file upload with real-time preview
πŸ—£οΈ Advanced transcription with speaker identification
⭐ Automatic highlight extraction of key moments
✍️ AI-powered article draft generation
πŸ“€ Export interview's subtitles in VTT format

Read full post

Top comments (2)

Collapse
 
pramodkumaryadav profile image
Pramod Kumar Yadav β€’

Trying to run "changed playwright tests" in github using the solution mentioned on step 4, i,e.

+   - name: Run changed Playwright tests
+     run: npx playwright test --only-changed=${{ github.base_ref }}
Enter fullscreen mode Exit fullscreen mode

Results in below error.

Error: Cannot detect changed files for --only-changed mode:
git ls-files --others --exclude-standard

fatal: detected dubious ownership in repository at '/__w/playwright-sandbox/playwright-sandbox'
To add an exception for this directory, call:

    git config --global --add safe.directory /__w/playwright-sandbox/playwright-sandbox
Enter fullscreen mode Exit fullscreen mode

Adding the above step after checkout (such as below):

 - name: Checkout Repository
        uses: actions/checkout@v4

      - name: Add safe directory exception
        run: git config --global --add safe.directory /__w/playwright-sandbox/playwright-sandbox

Enter fullscreen mode Exit fullscreen mode

Also results in the same error. Any clue what I maybe doing wrong here and/or need to fix this issue?

Collapse
 
yash_softeng profile image
Yashwanth β€’

Hi let me know if you were able to fix this issue?

Image of Timescale

Timescale – the developer's data platform for modern apps, built on PostgreSQL

Timescale Cloud is PostgreSQL optimized for speed, scale, and performance. Over 3 million IoT, AI, crypto, and dev tool apps are powered by Timescale. Try it free today! No credit card required.

Try free

πŸ‘‹ Kindness is contagious

Discover a treasure trove of wisdom within this insightful piece, highly respected in the nurturing DEV Community enviroment. Developers, whether novice or expert, are encouraged to participate and add to our shared knowledge basin.

A simple "thank you" can illuminate someone's day. Express your appreciation in the comments section!

On DEV, sharing ideas smoothens our journey and strengthens our community ties. Learn something useful? Offering a quick thanks to the author is deeply appreciated.

Okay