DEV Community

Cover image for How to speed up slow Cypress Component Tests by running only affected tests in parallel
Davide Cantelli
Davide Cantelli

Posted on

How to speed up slow Cypress Component Tests by running only affected tests in parallel

#testing #javascript #react #ci

In TrustLayer our Cypress component test suite had grown to hundreds of tests, and CI was taking over 20 minutes on every PR.

Even for a one-line change.

With the following approach we went from 22 minutes to an average of just 3 minutes — an 85% reduction 💪

👆 Most tests are not affected

Here's the thing: most of the time, your changes don't affect the majority of your test files. If you change Button.tsx, you probably only need to run tests that actually import Button.tsx — directly or indirectly.

Yet, we were running everything on every PR.

✌️ So what?

The basic idea to solve it is:

  1. Find which files changed in the current PR
  2. Build a dependency graph of your codebase
  3. Find test files that depend on the changed files
  4. Run only those tests 
Changed: src/components/Button.tsx

Affected tests found:
Button.tsx → Button.cy.tsx ✓
Button.tsx → Card.tsx → Card.cy.tsx ✓
Button.tsx → Form.tsx → Form.cy.tsx ✓
Enter fullscreen mode Exit fullscreen mode

Running 3 tests instead of 200.

Simple and clean.

But do we really need to build this from scratch?

🚀 affected-tests-runner

We built a tool that does exactly this: affected-tests-runner

npm install affected-tests-runner
Enter fullscreen mode Exit fullscreen mode

Instead of running everything:

npx cypress run --component
Enter fullscreen mode Exit fullscreen mode

we now run only tests affected by the changes we have made:

npx affected-tests run --test-command 'npx cypress run --component --spec "{specs}"'
Enter fullscreen mode Exit fullscreen mode

The tool:

  1. Git diff — Compares your branch against main/master to find changed files
  2. Dependency graph — Uses madge to map all imports in your codebase
  3. Trace affected tests — Finds test files that depend on changed code
  4. Run only those — Passes the filtered spec list to Cypress

⚡ Parallel execution in CI

But there is more we can do.

Because running fewer tests is great, but running them in parallel is even better, when your changes are affecting a lot of tests.

Small change? 1 job, 5 tests, 2 minutes.
Big refactor? 4 parallel jobs split the work.

The basic idea is:

  1. Get the number of groups needed given a max number of tests for each group. For example, the following command might find 30 affected tests, which given a max-tests=10 will output 3: npx affected-tests-runner groups --max-tests 10
  2. Given the number of groups we can execute each group in parallel: npx affected-tests run --group 0 --total-groups 3 npx affected-tests run --group 1 --total-groups 3 npx affected-tests run --group 2 --total-groups 3

In CI, this is typically automated with a matrix strategy. You can find some examples in the README of the package.

🔧 Works with any test runner

The tool isn't Cypress-specific.
It works with any test runner that accepts file paths:

  • Cypress Component --test-command 'npx cypress run --component --spec "{specs}"'
  • Jest --test-command 'npx jest {specs}'
  • Vitest --test-command 'npx vitest run {specs}'
  • Mocha --test-command 'npx mocha {specs}'
  • ...and any other similar test runner

😅 Are there any downsides?

Well, yes. The analysis is file-level, not function-level. This might introduce some false positives:

  • If utils.ts exports 10 functions and you change one, all importers will be considered affected
  • Barrel files (index.ts) create wider blast radiuses

You can mitigate this by using direct imports:

// Avoid
import { Button } from './components'

// Prefer
import { Button } from './components/Button'
Enter fullscreen mode Exit fullscreen mode

But even with barrel files you will see huge improvements.

Running a few extra tests is better than missing a broken one.

Conclusions

If you are struggling with slow test execution in the pipeline, this might be the solution for you.

Please visit the repository page for much more details about how to use and configure it for your use case, and to get to see more examples.

Happy coding! 💻

Links:

Credits:

Top comments (2)

Collapse
 
mantovanig_it profile image
Gabriele Mantovani •

Wow, this is an incredibly clear and practical explanation! It's a very real and interesting topic. I will try it! 👏🏻

Collapse
 
tomma5o profile image
Tomma5o 👨🏻‍💻 •

Finally! Someone has to do it outside NX