DEV Community: Filip Hric

Testing frontend performance with Cypress

Filip Hric — Fri, 22 Jul 2022 00:00:00 +0000

There are many ways to measure performance. In today’s post I want to talk about one of the most simple. Imagine a following scenario:

user clicks a button
modal window appears

Our test may look something like this:

  cy.visit('/board/1')

  // wait for loading to finish
  cy.getDataCy('loading')
    .should('not.exist')

  cy.getDataCy('card')
    .click()

This modal window may fetch some data from server, reorder or filter it. Additionally it may perform some other actions such as render images etc. All of these actions take some time and as testers, we want to make sure that the result will not take too long.

performance.mark() API

In all of the current browsers a performance API is available on window object. We can access this API by using cy.window() function and then calling a method. To start measuring the performance, we can create a mark that will label the start of our measurement.

  cy.visit('/board/1')

  // wait for loading to finish
  cy.getDataCy('loading')
    .should('not.exist')

  cy.window()
    .its('performance')
    .invoke('mark', 'modalOpen')

  cy.getDataCy('card')
    .click()

The chain starting with cy.window() actually does the exact same thing as if we were to type window.performance.mark('modalOpen') in our DevTools console. The modalOpen is just a label, and can be named anything.

performance.measure() API

Now that we have labeled the start of our metric, let’s perform the next steps. When we click on the card, it opens modal window. First, we want to make sure that we have reached the desired result. We can check that by making an assertion on the modal window visibility:

  cy.visit('/board/1')

  // wait for loading to finish
  cy.getDataCy('loading')
    .should('not.exist')

  cy.window()
    .its('performance')
    .invoke('mark', 'modalOpen')

  cy.getDataCy('card')
    .click()

  cy.getDataCy('card-detail')
    .should('be.visible')

After that, we can call performance.measure()function to make our measurement. Basically, we are pressing a button on a stopwatch here. The argument of the measure function will be our modalOpen label. The reason for passing this argument is that we can actually add multiple labels into our test and we need to specify which one to measure. To call the measure function we basically perform a very set of Cypress functions as before:

  cy.visit('/board/1')

  // wait for loading to finish
  cy.getDataCy('loading')
    .should('not.exist')

  cy.window()
    .its('performance')
    .invoke('mark', 'modalOpen')

  cy.getDataCy('card')
    .click()

  cy.getDataCy('card-detail')
    .should('be.visible')

  cy.window()
    .its('performance')
    .invoke('measure', 'modalOpen')

The invoke command is going to yield an object with all kinds of results:

Within this command, we can pick a property from this object using .its() command. Since we don’t need retryability, we can set timeout to 0 and make our assertion immediately. Let’s make an assertion that the modal should not load longer than 2 seconds (2000 in milliseconds).

  cy.visit('/board/1')

  // wait for loading to finish
  cy.getDataCy('loading')
    .should('not.exist')

  cy.window()
    .its('performance')
    .invoke('mark', 'modalOpen')

  cy.getDataCy('card')
    .click()

  cy.getDataCy('card-detail')
    .should('be.visible')

  cy.window()
    .its('performance')
    .invoke('measure', 'modalOpen')
    .its('duration', { timeout: 0 })
    .should('be.lessThan', 2000)

Creating a custom command

Now that we know what to do, we can create a custom command out of this. There’s a lot of TypeScript going on, so let me break down what’s happening here. Lines 1-9 is a type declaration. This is how we tell TypeScript compiler that we have added a new cy.mark() command to the library of cy commands. The library is called Chainable, and contains all cy commands. This library is part of a bigger whole - namespace Cypress.

Lines 11 - 29 is a function that contains our chain of commands from previous example. In addition to that, I have hidden the logs of our three commands and added my own log which you can see on lines 15 - 24.

Finally, on line 31, we are adding this function to the Cypress library. While lines 1-9 add our command to the Cypress namespace that our TypeScript compiler can recognize, Cypress.Commands.addAll() function will add it to the Cypress itself. I usually store my custom commands to cypress/support/commands/ folder and do an import ../commands/mark.ts inside cypress/support/index.ts file.

declare namespace Cypress {
  interface Chainable<Subject = any> {
      /**
       * Add a measurment marker. Used with cy.measure() command
       * @example cy.mark('modalWindow')
       */
       mark: typeof mark
  }
}

const mark = (markName: string): Cypress.Chainable<any> => {

  const logFalse = { log: false }

  Cypress.log({
    name: 'mark',
    message: markName,
    consoleProps() {
      return {
        command: 'mark',
        'mark name': markName
      }
    }
  })

  return cy.window(logFalse)
    .its('performance', logFalse)
    .invoke(logFalse, 'mark', markName)
}

Cypress.Commands.addAll({ mark })

Similarly, we can add the cy.measure() command as well:

declare namespace Cypress {
  interface Chainable<Subject = any> {
      /**
       * Add a measurment marker. Used with cy.measure() command
       * @example cy.measure('modalWindow')
       */
       measure: typeof measure
  }
}

const measure = (markName: string): Cypress.Chainable<number> => {

  const logFalse = { log: false }

  let measuredDuration: number
  let log = Cypress.log({
    name: 'measure',
    message: markName,
    autoEnd: false,
    consoleProps() {
      return {
        command: 'measure',
        'mark name': markName,
        yielded: measuredDuration
      }
    }
  })

  return cy.window(logFalse)
    .its('performance', logFalse)
    .invoke(logFalse, 'measure', markName)
    .then( ({ duration }) => {
      measuredDuration = duration
      log.end()
      return duration
    })
}

Cypress.Commands.addAll({ measure })

A small difference from our cy.mark() is that this time our return type will be number, because the our function will return a number. Also, instead of using .its() function, we are returning it from .then() function as we want to use it in our console command detail as well. If this is a lot of new terms, I suggest checking out this post about improving custom Cypress command I’ve made earlier.

Performance testing in Cypress

Whenever we do performance testing of any kind, we need to pay close attention to the environment we are testing on. Are we in production? Is it currently under heavy load? If on staging server, is it 1:1 with production or are we testing a scaled down version? Are we using browser for perfomance testing? Which one? Which version? All of this and more questions need to be asked to provide context for the performance metrics.

In our context, we are running inside a browser that has two iframes opened. One for our application and one for Cypress script. This may have effect on our testing and it is not slight. Cypress docs warn about this in their docs. This doesn’t mean that measuring performance in Cypress is useless. It just means that we need to take the context into account when looking at the metrics.

8 common mistakes in Cypress (and how to avoid them)

Filip Hric — Thu, 14 Jul 2022 00:00:00 +0000

This is a blog post made from a talk I gave at Front end test fest, so if you want to watch a video about it, feel free to do so on this link.

On my Discord server, I sometimes encounter a common pattern when answering questions. There are certain sets of problems that tend to surface repeatedly and for these, I created this blog post. Let’s jump into them!

#1: Using explicit waiting

This first example feels kinda obvious. Whenever you added an explicit wait to your Cypress test, I believe you had an unsettling feeling about this. But what about the cases when our tests fail because the page is too slow? Feels like using cy.wait() is the way to go.

// ❌ incorrect way, don’t use

cy.visit('/')
cy.wait(10000)
cy.get('button') 
  .should('be.visible')

But this makes our test just sit there and hope that the page will get loaded before the next command. Instead, we can make use of Cypress’ built-in retryability.

cy.visit('/')
cy.get('button', { timeout: 10000 })
  .should('be.visible')

So why is this better? Because this way, we will wait maximum 10 seconds for that button to appear. But if the button renders sooner, the test will immediately move on to the next command. This will help you save some time. If you want to read more about this, I recommend checking out my blog on this topic.

#2: Using unreadable selectors

I could write a whole article just on the topic of selectors in fact, I did), since this is one of the most dealt-with topics for testers. Selectors can be the first thing that can give us a clue as to what our test is doing. Because of that, it is worth making them readable.

Cypress has some recommendations as to which selectors should be used. The main purpose of these recommendations is to provide stability for your tests. At the top of the recommendations is to use separate data-* selectors. You should add these to your application.

However (and unfortunately IMHO), testers don’t always have the access to the tested application. This makes selecting elements quite a challenge, especially when the element we are searching for is obscure. Many that find themselves in this situation reach for various strategies for selecting elements.

One of these strategies is using xpath.The big caveat of xpath is that their syntax is very hard to read. By merely looking at your xpath selector, you are not really able to tell what element you are selecting. Moreover, they don’t really add anything to the capabilities of your Cypress tests. Anything xpath can do you can do with Cypress commands, and make it more readable.

// Select an element by text
cy.xpath('//*[text()[contains(.,"My Boards")]]')
// Select an element containing a specific child element
cy.xpath('//div[contains(@class, "list")][.//div[contains(@class, "card")]]')
// Filter an element by index
cy.xpath('(//div[contains(@class, "board")])[1]')
// Select an element after a specific element
cy.xpath('//div[contains(@class, "card")][preceding::div[contains(., "milk")]]')

// Select an element by text
cy.contains('h1', 'My Boards')
// Select an element containing a specific child element
cy.get('.card').parents('.list')
// Filter an element by index
cy.get('.board').eq(0)
// Select an element after a specific element
cy.contains('.card', 'milk').next('.card')

#3: Selecting elements improperly

Consider the following scenario. You want to select a card (the white element on the page) and assert its text.

Notice how both of these elements contain the word "bugs" inside. Can you tell which card are we going to select when using this code?

cy,visit('/board/1')
cy.get('[data-cy=card]')
  .eq(0)
  .should('contain.text', 'bugs')

You might be guessing the first one, with the text "triage found bugs". While that may be a good answer it’s not the most precise one. Correctly it is - whichever card will load first.

It is important to remember that whenever a Cypress command finishes doing its job, it will move on to the next command. So once an element is found by the .get()command, we will move to the.eq(0) command. After that, we will move to our assertion that will fail.

You might wonder why Cypress does not retry at this point, but it actually does. Just not the whole chain. By design, .should() command will retry the previous command, but not the whole chain. This is why it is vital to implement a better test design here and add a "guard" for this test. Before we assert the text of our card, we’ll make sure that all cards are present in DOM:

cy,visit('/board/1')
cy.get('[data-cy=card]')
  .should('have.length', 2)
  .eq(0)
  .should('contain.text', 'bugs')

4#: Ignoring requests in your app

Let’s take a look at this code example:

cy.visit('/board/1')
cy.get('[data-cy=list]')
  .should('not.exist')

When we open our page, multiple requests get fired. Responses from these requests will get digested by the frontend app and rendered into our page. In this example [data-cy=list] elements get rendered after we get a response from /api/lists endpoint.

But the problem with this test is, that we are not telling Cypress to wait for these requests. Because of this, our test may give us a false positive and pass even if there are lists present in our application.

Cypress will not wait for the requests our application does automatically. We need to define this using the intercept command:

cy.intercept('GET', '/api/lists')
  .as('lists')
cy.visit('/board/1')
cy.wait('@lists')
cy.get('[data-cy=list]')
  .should('not.exist')

#5: Overlooking DOM re-rendering

Modern web applications send requests all the time to get information from the database and then render them in DOM. In our next example, we are testing a search bar, where each keystroke will send a new request. Each response will make the content on our page re-render. In this test, we want to take a search result and confirm that after we type the word "for", we will see the first item with the text "search for critical bugs". The test code goes like this:

cy.realPress(['Meta', 'k'])
cy.get('[data-cy=search-input]')
  .type('for')
cy.get('[data-cy=result-item]')
  .eq(0)
  .should('contain.text', 'search for critical bugs')

This test will suffer from "element detached from DOM" error. The reason for this is that while still typing, we will first get 2 results, and when we finish, we’ll get just a single result. In a result, our test will go like this:

"f" key is typed in the search box
request searching for all items with "f" will fire
response comes back and app will render two results
"o" key is typed in the search box
request searching for all items with "fo" will fire
response comes back and app will render two results
"r" key is typed in the search box
request searching for all items with "for" will fire
Cypress is done with typing, so it moves to the next command
Cypress will select [data-cy=result-item] elements and will filter the first one (using .eq(0)) command
Cypress will assert that it has the text "search for critical bugs"
since the text is different, it will run the previous command again (.eq(0))
while Cypress is retrying and going back and forth between .eq(0) and .should() a response from our last request comes back and app will re-render to show single result
the element that we selected in step 10 is no longer present, so we get an error

Remember, .should() command will make the previous command retry, but not the full chain. This means that our cy.get('[data-cy=result-item]') does not get called again. To fix this problem we can again add a guarding assertion to our code to first make sure we get the proper number of results, and then assert the text of the result.

cy.realPress(['Meta', 'k'])
cy.get('[data-cy=search-input]')
  .type('for')
cy.get('[data-cy=result-item]')
  .should('have.length', 1)
  .eq(0)
  .should('contain.text', 'search for critical bugs')

But what if we cannot assert the number of results? I wrote about this in the past, but in short, the solution is to use .should() command with a callback, something like this:

cy.realPress(['Meta', 'k'])
cy.get('[data-cy=search-input]')
  .type('for')
cy.get('[data-cy=result-item]')
  .should( items => {
    expect(items[0].to.have.text('search for critical bugs'))      
  })

#6: Creating inefficient command chains

Cypress has a really cool chaining syntax. Each command passes information to the next one, creating a one-way flow of your test scenario. But even these commands have an internal logic inside them. Cypress commands can be either parent, child or dual. This means, that some of our commands will always start a new chain.

Consider this command chain:

cy.get('[data-cy="create-board"]')
  .click()
  .get('[data-cy="new-board-input"]')
  .type('new board{enter}')
  .location('pathname')
  .should('contain', '/board/')

The problem with writing a chain like this is not only that it is hard to read, but also that it ignores this parent/child command chaining logic. Every .get() command is actually starting a new chain. This means that our .click().get() chain does not really make sense. Correctly using chains can prevent your Cypress tests from unpredictable behavior and can make them more readable:

cy.get('[data-cy="create-board"]') // parent
  .click() // child
cy.get('[data-cy="new-board-input"]') // parent
  .type('new board{enter}') // child
cy.location('pathname') // parent
  .should('contain', '/board/') // child

#7: Overusing UI

I believe that while writing UI tests, you should use UI as little as possible. This strategy can make your test faster and provide you the same (or bigger) confidence with your app. Let’s say you have a navigation bar with links, that looks like this:

<nav>
  <a href="/blog">Blog</a>
  <a href="/about">About</a>
  <a href="/contact">Contact</a>
</nav>

The goal of the test will be to check all the links inside <nav> element, to make sure they are pointing to a live website. The intuitive approach might be using .click() command and then check either location or content of the opened page to see if the page is live.

However, this approach is slow and in fact, can give you false confidence. As I mentioned in one of my previous blogs, this approach can overlook that one of our pages is not live, but returns a 404 error.

Instead of checking your links like this, you can use .request() command to make sure that the page is live:

cy.get('a').each( link => {
  cy.request(page.prop('href'))
})

#8: Repeating the same set of actions

It’s really common to hear that your code should be DRY = don’t repeat yourself. While this is a great principle for your code, it seems like it is slightly ignored during the test run. In an example below, there’s a cy.login() command that will go through the login steps and will be used before every test:

Cypress.Commands.add('login', () => {

  cy.visit('/login')

  cy.get('[type=email]')
    .type('filip+example@gmail.com')

  cy.get('[type=password]')
      .type('i<3slovak1a!')

  cy.get('[data-cy="logged-user"]')
    .should('be.visible')

})

Having this sequence of steps abstracted to a single command is definitely good. It will definitely make our code more "DRY". But as we keep using it in our test, our test execution will go through the same steps over and over, essentially repeating the same set of actions.

With Cypress you can pull up a trick that will help you solve this issue. This set of steps can be cached and reloaded using cy.session() command. This is still in an experimental state, but can be enabled using experimentalSessionAndOrigin: true attribute in your cypress.config.js. You can wrap the sequence in our custom command into .session() function like this:

Cypress.Commands.add('login', () => {

  cy.session('login', () => {

    cy.get('[type=email]')
      .type('filip+example@gmail.com')

    cy.get('[type=password]')
      .type('i<3slovak1a!')

    cy.get('[data-cy="logged-user"]')
      .should('be.visible')

  })

})

This will cause to run the sequence in your custom commands just once per spec. But if you want to cache it throughout your whole test run, you can do that by using cypress-data-session plugin. There are a lot more things you can do this, but caching your steps is probably the most valuable one, as it can easily shave off a couple of minutes from the whole test run. This will of course depend on the test itself. In my own tests, where I just ran 4 tests that logged in, I was able to cut the time in half.

Hopefully, this helped. I’m teaching all this and more in my upcoming workshop. Hope to see you there!

Cypress basics: Where did my cookies disappear?

Filip Hric — Thu, 26 Nov 2020 10:02:00 +0000

You successfully logged into your application. You’ve got your first test! Now to the next one. Click! And now you are logged out. Why is Cypress logging you out after each test?!

The reason is actually simple. Cypress clears out the state of browser in between tests. All your cookies are deleted from your app between each it() block. The reason why you are now logged out may be because your app uses a cookie to store authentication token. When your app makes an http request to your server, that cookie will be sent along with it. This way your server knows that you your authentication is valid and that you have the rights to read or write data. If the cookie is not present, server evaluates this as an unauthorized request and your app will typically log you out.

Clearing the state of the browser is actually a good thing, but you might be in a situation where you want to work around this. E.g. you want to group multiple tests in one spec, where each test requires you to be logged in. Let’s look into what are the options. As is often the case with my blogs, I have a repo set up, so make sure you clone it and play with the code yourself. In our repo, we have a simple app, that lists all of the cookies that are present.

In our first piece of code you can see that we are setting a cookie to our app using .setCookie() command, but in our second test this cookie is not present anymore.

it('should show cookie', () => {
  cy.setCookie('authentication', 'top_secret');
  cy.visit('../../app/index.html');
});

it('opens a page', () => {
  cy.visit('../../app/index.html');
});

Once you’ll run this spec, you’ll see that our page shows "no cookies were found" on our second spec. See video from the test run:

To make sure that our cookies are set in each test, you can set up your cookies before each test using beforeEach hook.

beforeEach(() => {
  cy.setCookie('authentication', 'top_secret');
});

it('first test', () => {
  cy.visit('../../app/index.html');
});

it('second test', () => {
  cy.visit('../../app/index.html');
});

This however may get a little annoying, so instead of using beforeEach hook we can use another approach. Using Cypress’ Cookies API we can set up cookies that we never want to delete. With Cypress.Cookies.defaults we can define which cookies we want to prevent from being cleared before each test:

Cypress.Cookies.defaults({
  preserve: 'authentication'
})

it('first test', () => {
  cy.setCookie('authentication', 'top_secret');
  cy.visit('../../app/index.html');
});

it('second test', () => {
  cy.visit('../../app/index.html');
});

Instead of using the api within out spec, I’d recommend declaring this in support/index.js file in Cypress project. This way you’ll make sure that your cookie is preserved throughout all of your tests. But maybe you don’t want to do that. Instead of keeping the cookie for the whole test suite, you might just want to preserve it for a single spec file.

For this, you can use Cookies API too. Using Cypress.Cookies.preserveOnce will enable you to keep your cookies for your spec. In our following test, we are using before() hook to set up our cookie and then beforeEach() hook to call our preserveOnce function to keep that cookie present for each test:

/// <reference types="cypress" />

before(() => {
  cy.setCookie('authentication', 'top_secret');
});

beforeEach(() => {
  Cypress.Cookies.preserveOnce('authentication')
});

it('first test', () => {
  cy.visit('../../app/index.html');
});

it('second test', () => {
  cy.visit('../../app/index.html');
});

This may not seem too different from just using beforeEach() to add our cookies before tests. However, this example is quite oversimplified. Most of the time the case is not just setting a cookie inside our browser, but actually logging in with a custom command or via API. This may be a more expensive operation, and avoiding repetition might save us some time. I’ll talk about different ways of authenticating to our app in some future blog, so make sure to follow me or subscribe to the newsletter on my homepage to be notified when that comes out.

Create a configuration plugin in Cypress

Filip Hric — Mon, 23 Nov 2020 00:00:00 +0000

Last week I have been playing with configuration in Cypress. I find configuring different environments to be especially hard topic. Especially for people that are beginning. In this blog, I would like to break configuration into small steps, that will help you navigate this topic.

Basics of configuration

Every Cypress project has a cypress.json file. In this file, you have a env attribute, that is usually filled with environment variables. Let’s say I add a variable app to this attribute like this:

{
  "env": {
    "app": "local"
  }
}

When I now open Cypress in GUI mode, I can look into settings tab and see my config file being read, highlighted in blue.

As you can see from the screenshot, there are multiple ways to add your environment variables. You can separate them into cypress.env.json file, add them with a CYPRESS_ prefix to your environment or pass them through CLI as arguments. Take a peek into the documentation, where you can find examples of all these approaches.

Writing a plugin

With more complicated setups, you might want to combine these approaches. Let’s say you have multiple environments, like local, staging, preprod and prod. Each of these has its own home url, its own version of api and its own redirect page. These configurations might look something like this:

// local
baseUrl: 'http://localhost:3000'
api: 'development'
redirect: 'https://www.product-staging.com'

// staging
baseUrl: 'http://dashboard.product-staging.com'
api: 'development'
redirect: 'https://www.product-staging.com'

// preprod
baseUrl: 'http://dashboard-preprod.product.com'
api: 'v2'
redirect: 'https://www.product.com'

// prod
baseUrl: 'http://dashboard.product.com'
api: 'v1'
redirect: 'https://www.product.com'

As you can see this is kind of a mess. But this often happens in real life situations. You have got different combinations of APIs and URLs that differ from environment to environment. More than that, you may want to add some key or secret from environment that should not be in the codebase. Let’s dive into how we can solve this.

First of all, let’s see how the config works. When you add this piece of code to your cypress/plugins/index.ts file, you will see your env variables logged out. If I still have my cypress.json as before, this would write out { app: 'local' } into my terminal after I open Cypress via npx cypress open

module.exports = (on, config) => {

  console.log(config.env) // { app: 'local' }

}

Now let’s clear out our cypress.json and instead of this let’s pass a version flag via CLI to our environment, like this:

npx cypress open --env version="prod"

Upon opening Cypress, you would see { version: 'prod' } logged out to our terminal. Let’s now use this information and change our config according to what we pass in version flag.

module.exports = (on, config) => {

  if (config.env.version === "prod") {
    config.baseUrl = 'http://dashboard.product.com'
    config.env.api = 'v2'
    config.env.redirect = 'https://www.product.com'
  }

  return config

}

Using our plugin, we are now adding some more variables to our env. When we look into our settings in Cypress CLI, you can see that our env variables have been added.

This helps demonstrate how creating a configuration plugin works. Let’s now say that our configuration paths are stored in separate json files, for which we created a separate config folder in our Cypress project:

cypress/
├── config/
│   ├── local.json
│   ├── staging.json
│   ├── preprod.json
│   └── prod.json

Let’s now say, that for each version flag, we want to load a different file and pass it into Cypress as our config and environment. In order to do this, we can rewrite our plugin file like this:

module.exports = (on, config) => {

  // if version not defined, use local
  const version = config.env.version || 'local'

  // load env from json
  config.env = require(`../config/${version}.json`);

  // change baseUrl
  config.baseUrl = config.env.baseUrl

  return config

}

This way we can load whichever file we want and even add some more json files for different configurations. Whichever file we pass to our CLI will be read. Now we can take it a step further, and instead of using CLI flag, we will use Cypress environment variable like this:

cypress_version=preprod npx cypress open

As mentioned, anything passed to our CLI with prefix cypress_ will be added as environment variable to Cypress. Notice how I pass these before npx cypress open as opposed to --env flag that is passed after.

Handling secrets

There are probably couple of keys or secrets that you don’t want to keep in your code base. You usually keep them in your .bashrc or .zshrc file, depending on what shell you use. Or you may use dotenv to store your variables per project. This works too. To read these variables and use them in your tests, you can add following to your configuration:

module.exports = (on, config) => {

  // if version not defined, use local
  const version = config.env.version || 'local'

  // load env from json
  config.env = require(`../config/${version}.json`);

  // change baseUrl
  config.baseUrl = config.env.baseUrl

  // add a secret key
  config.env.SECRET_KEY = process.env.SECRET_KEY

  return config

}

When you open your tests now, you can see the SECRET_KEY being displayed in our project settings.

]

Test grepping in Cypress using Module API

Filip Hric — Sat, 21 Nov 2020 22:06:29 +0000

If you are running hundreds of tests in Cypress, chances are you may want to run just a subset of them. There are several ways you can do this, and in this blog, I’d like to show you mine. If you are here just for the solution, feel free to scroll down to the end end of this blog where you’ll find the code.

As you probably know, to run all of your Cypress tests, you can type following command into your command line:

npx cypress run

This will run all tests inside your current Cypress project. These are usually stored in integration folder. I usually like to create more folders inside to create separate test categories. Let’s say I have an api and ui folder. To run each one of them, I could create a separate npm scripts, so in my package.json I’d have the following:

{
  "scripts": {
    "cy:run": "npx cypress run",
    "cy:run:api": "npx cypress run --spec ./cypress/integration/api/*.ts",
    "cy:run:ui": "npx cypress run --spec ./cypress/integration/ui/*.ts"
  }
}

These commands of course work well, but in order to run each of my test folders, I need to run a separate command. This is not such a big deal when there are only two folders, but if you have multiple of them, things can get complicated.

This is where Module API comes in super handy and I’ll show you how in a second. First, let’s write our run script with Module API. We’ll create a new cypress.js file in the root of our project and add following code inside:

const cypress = require('cypress');

cypress.run();

This is pretty much the same thing as if we ran our npx cypress run command. But instead of this, we will run our command by typing this to our terminal:

node cypress.js

To make things easier for us, let’s add this to our package.json scripts:

{
  "scripts": {
    "cy:run": "node cypress.js"
  }
}

Our cypress.run() function can also take an options parameter. This way we can which tests should be run, similarly as we did with the --spec flag in our previous example. So let’s add options inside our function and specify a spec folder to run:

const cypress = require('cypress');

cypress.run({
  spec: './cypress/integration/api/*.ts',
});

This property can also be an array, so we can run more folders and specify which ones we want to run:

const cypress = require('cypress');

cypress.run({
  spec: ['./cypress/integration/api/*.ts', './cypress/integration/ui/*.ts'],
});

Now that we know all this, we can play inside our cypress.js file and apply any kind of logic we like.

Let’s say that instead of api and ui folder, I have folders named: list, detail, settings, login and signup. I want to be able to pick any number or combination of these, and at the same time be able to run all of them. To do this, we will add a module called yargs. This package enables us to create and work with our own command line options. We are going to add a --grep option, so that if we just want to run tests inside settings and login folders, we will call a script like this:

npm run cy:run -- --grep settings login

To define our --grep option, we will add following to our cypress.js file:

const yargs = require('yargs');

const { grep } = yargs
  .option('grep', {
    type: 'array'
  }).argv;

This will digest out --grep flag. In order to give it multiple arguments, we need do specify the type of the input (see type: 'array'). If you are unfamiliar with the { grep } syntax, go and check out my blog on destructuring, where I explain this in more detail.

Let’s finalize our script and pass these options to our cypress.run() command:

const cypress = require('cypress');
const yargs = require('yargs');

const { grep } = yargs
  .option('grep', {
    type: 'array',
    default: ['*']
  }).argv;

cypress.run({
  spec: grep.map(folder => `./cypress/integration/${folder}/*.ts`),
});

We are mapping out all the folder names, so that when we call npm run cy:run -- --grep settings login our grep variable will be assigned the value of:

["settings", "login"]

and our spec attribute will have the value of:

["./cypress/integration/settings/*.ts", "./cypress/integration/login/*.ts"]

This way we can either pass names of our folders to our --grep argument, or we can omit the argument and run all of our tests.

It’s all just JavaScript so we can apply any logic we want. Instead of --grep we could maybe use --folder as the name of our parameter. We can go even further and create both --folder and --testFile flags to make our pick even more specific.

This has proven to be incredibly useful in my case. I can run just those tests I need to be ran instead of waiting for the whole test suite, but still maintain the option to run everything. Several CI providers enable you to run your pipeline on demand and specify a pipeline variable, which can be used exactly for setting up which tests you want to run.

I write blogs like these every week at filiphric.com and whenever I publish one, I send out an email, so you don’t miss it. You can also follow me on Twitter and reach out to me if you have any questions.

Lesser known Cypress.io tricks

Filip Hric — Mon, 21 Sep 2020 00:00:00 +0000

Cypress.io is pretty amazing, and if you haven’t used it yet and need a jump start, I recommend my super-quick straight-to-the-point course on Udemy. It’s called Cypress test automation for people in a hurry and it’s exactly that. Average video length is 2 minutes, and then it’s practical exercises that help you learn more.

As I create my courses and use Cypress on my own, I often come across undocumented or not so widely used features that you might find helpful. Let’s jump into them.

Routing numbered route

When using .route() command to match your path, you can use wildcards to math the exact api call you need. But sometimes it is just not enough. Your app may call the exact same endpoint twice. To write your test for these types of situations, you can select them both using an array, like this:

cy
  .server()
  .route('POST', '/todos')
  .as('createTodo')

cy
  .visit('/')

// create 2 todos via UI using custom command
cy
  .addTodo()
  .addTodo()

cy
  .wait(['@createTodo', '@createTodo']).then( todos => {

    expect(todos[0].status).to.eq(201)
    expect(todos[1].status).to.eq(201)

  })

Instead of using an array though, you can select just the second instance of routed network request. This can be done by appending the index number on to the alias itself, like this:

cy
  .server()
  .route('POST', '/todos')
  .as('createTodo')

cy
  .visit('/')

// create 2 todos via UI using custom command
cy
  .addTodo()
  .addTodo() // we will wait for a request that happens after this action

cy
  .wait('@createTodo.2').then( todos => {
    expect(todos.status).to.eq(201)
  })

Pro tip: did you know Cypress recently released experimental full-layer network stubbing? Besides xhr requests, you can now route (some) images, static files and fetch requests!

Aliasing DOM element

Routing your network calls is one of the most powerful features in Cypress. To alias them, you can use .as() command, and then use .wait() or .get() command to write a test for that network request.

You can alias your DOM elements in the same way and then use .get() command to select that element later in your test. This is especially useful when you have a list of items on the same level. For example, when using cypress-drag-drop plugin to drag an element onto another.

cy
  .get('.todo')
  .eq(2)
  .as('third')

cy
  .get('.todo')
  .eq(3)
  .as('fourth')

cy
  .get('@third')
  .drag('@fourth')

Custom formatting of .log() messages

I believe that you should write end to end tests as user stories. That is why it is vital that I understand where that story got interrupted on failed test. I have been playing around with ways to create a custom error message for some time now. Mainly because it helps a lot when debugging a failed test from screenshot, or basically captioning an end to end test. There is a way you can customize these messages using markdown formatting syntax.

cy.log('normal')
cy.log('**bold**')
cy.log('_italic_')
cy.log('[blue](<http://example.com>)')

Custom error messages

Beside custom .log() messages, you can customize your error messages too. I was thrilled to find out that expect() function can actually take a second parameter, which will become an error message on failure.

cy
  .get('.todo')
  .then( todo => {
    expect(todo, 'Milk was not found').to.contain.text('Buy milk')
})

Bonus tip: Make your DevTools open automatically in Cypress GUI:

Filip Hric

@filip_hric

I don’t think I have ever used Chrome without opened devtools when writing #cypress tests. I have them auto-open for me via plugins/index.js file, instead of hitting keyboard shortcut every time. #cypresstips

18:04 PM - 19 Mar 2020