Speeding up tests by partitioning

This post is on a practical tip on how to speed up your tests for CI pipelines. If you already followed your framework's official guide and added in the mentioned configurations or flags, this might be useful for you.

The partitioning of the tests works across any testing framework. How does this work? Basically you split the test files into 2 or more sets. The splitting can be done equally. Or a longer running test can be on one set and the rest on another. It can be any combination which helps you decrease the time on CI.

Show me the code

Lets assume you are using GitHub Actions and vitest and have split your tests into 2 partitions.

On GitHub actions, you specify 2 jobs to run on parallel. One of them runs the first partition and the other runs the second. You can use a simple environment variable to indicate which partition is going to be run.

test_server:
    strategy:
      matrix:
        partition:
          - 1
          - 2
# rest of the workflow config
     - run: npm run test
       env:
        PARTITION: ${{ matrix.partition }}

Now, you will need to indicate which files you want to run on which partition. A simple way to do this would be to include specific tests to run on the first partition and the remaining to run on the second partition.

import { defineConfig, UserConfig } from 'vitest/config'
import Config from './vitest.base.config.ts'

const paritionedFileNames = ['src/{analytics,media,social}/**/*.test.ts']

let paritionedConfig: UserConfig

if (process.env.PARTITION === '1') {
  paritionedConfig = defineConfig({
    ...Config,
    test: {
      ...Config.test,
      include: paritionedFileNames
    }
  })
} else {
  paritionedConfig = defineConfig({
    ...Config,
    test: {
      ...Config.test,
      exclude: paritionedFileNames
    }
  })
}

export default paritionedConfig

Why does this work?

Because we want to run them on parallel. The CI runners happen to be less powerful than our development machine in most cases. Most have a single or dual core CPU which make the testing jobs to run longer.
However, most environments allow you to spawn as many virtual runners you want for the mentioned jobs. You can take advantage of this by running them on parallel.

So now instead of your job taking 10 minutes, it will take 5 minutes if you have split it across 2 runners.

Does it work?

Yes it does.

It used to take around 9 minutes to run this test.

After partitioning, it takes around 5 minutes to run the same test.

Thanks for reading all the way and hope you found it useful. Feel free to discuss further optimization techniques you have used and suggest me to implement.