Taking Control of Your Cypress Test Runs With Tags

#cypress #testing

Intro

Cypress is a phenomenal end-to-end testing tool. It's certainly not perfect but it's being updated consistently and it's expandable with plugins. I recently came upon a problem that required more granular control over my test runs than Cypress provides out of the box. Using our imagination, we'll take a similar journey to what I believe is a sweet solution to a problem you'll likely have. Let's go!

Problem

Imagine you have a great set of Cypress tests for your calculator app, perfectly organized by feature:

cypress
└───integration
│   └───add
│   │   │   add.spec.js
│   └───subtract
│   │   │   subtract.spec.js
│   └───multiply
│   │   │   multiply.spec.js
│   └───divide
│   │   │   divide.spec.js

Within each of these spec files are several tests:

// add.spec.js
it('add two numbers', () => {
 ...
})
it('add three fractions', () => {
 ...
})

// subtract.spec.js
it('subtract a negative number from a positive number', () => {
 ...
})
it('subtract two whole numbers', () => {
 ...
})

You get the idea.

The time it takes to test your calculator app end-to-end with Cypress is a lot longer than your unit tests take so you run them nightly. In an effort to catch bugs sooner, you and your team have determined that a certain set of tests need to be executed as part of your continuous integration pipeline. What's the best way to implement this?

Solution

You figure out your best course of action through an iterative process:

Good Solution - Separating Tests

First, you create important-tests.spec.js and move all the relevant tests into that file. You considered copying the tests but didn't want the hassle of keeping them in sync.
Then you run Cypress like this:

$ npx cypress run --spec 'path/to/important-tests.spec.js'

Easy enough, but you soon find out that maintenance is tricky because you now have double the number of files for each feature you need to maintain tests for. While this solution works, the additional complexity and cognitive load are too significant to scale effectively.

Better Solution - Tagging Tests

You discover a great plugin cypress-select-tests developed by Cypress' own Gleb Bahmutov, that provides a way to select or grep the specific tests you want to run.

Setup

Install:

$ npm install --save-dev cypress-select-tests

Update your plugins/index.js file:

import selectTestsWithGrep from 'cypress-select-tests/grep'
export default (on, config) => {
  on('file:preprocessor', selectTestsWithGrep(config))
}

Now running tests like this:

$ npx cypress run --env grep='fraction'

will execute only tests that contain fraction in the title. Sweet!

Tagging

The tests that need to run in CI are titled,

add three fractions
subtract two whole numbers
divide by zero
subtract fractions

You ask yourself,

What search term will result in only these tests running?

and find that there aren't any, so you toy around with the idea of adding one:

ci divide by zero
important: divide by zero
divide by zero, important
divide by zero, #important

Settling on the last format makes sense to you because it's not ambiguous, it's out of the way and, and most importantly, it's distinguishable from the other text, which reduces the risk that it'll be casually edited or a test that happens to have the term important in it won't be included in the run unintentionally. Phew!

Finally

Finally, running Cypress like this:

$ npx cypress run --env grep="#important"

results in a run - containing really important tests - that can be added to your pipeline so you can find critical bugs sooner without adding significant complexity to your workflow.