DEV Community: Alex Sanzhanov

Cross Browser Testing with Cypress in CI/CD using Docker

Alex Sanzhanov — Thu, 08 Jun 2023 14:18:10 +0000

Greetings to all Cypress enthusiasts!

In this article, we’ll explore one interesting way of running Cypress tests across multiple browsers in CI/CD using Docker. The cool thing here is that with the help of Docker Compose we will set up the simultaneous launch of several Docker containers in which Cypress tests will be executed in parallel in different browsers. The entire workflow will run automatically on the GitHub Actions platform, and as a result, we will get artifacts from running tests in each of the browsers. Get ready to read, it’s going to be very exciting!

Why Cross Browser Testing?

Cross Browser compatibility is one of the most important characteristics of a web application, which implies its equally correct display and functionality in various browsers and their versions.

The modern variety of browsers is determined, among other things, by differences in the ways of rendering web application content, when different browser engines (Blink, WebKit, Gecko, EdgeHTML) perceive and process HTML tags and CSS styles differently, which naturally affects the appearance and application behavior.

In this regard, the importance of cross-browser testing is obvious and extremely clear. Its purpose is to ensure that when a user opens a web application in various browsers and their versions, the correct display of its content is ensured, the integrity of its structure is preserved, there are no functional errors and inconsistencies in performance, layout collapse, overlapping elements on top of each other, etc.

Cypress Features in Cross Browser Testing

If you are not yet familiar with Cypress:

Cypress is a JavaScript-based end-to-end testing tool designed for modern web test automation. Cypress allows conducting both full-fledged end-to-end testing with passing user scenarios on a real product, as well as integration testing of individual front-end components. Cypress has emerged as a popular end-to-end testing tool for web applications due to a bunch of its powerful features, user-friendly interface, fast test execution time, easy installation and debugging, etc.

To my mind, Cypress is a real game changer in end-to-end and component testing and it grows at a rapid pace. Among the many benefits of Cypress, I would like to emphasize the high quality of its documentation, as well as the master classes that the Cypress development team conducts and publishes in the public domain, and also a very friendly and responsive community. Honestly, and as you may have noted from my previous articles I’m a big fan of that wonderful tool!

Cypress has the capability to run tests across multiple browsers. According to official documentation, Cypress has currently support for Chrome-family browsers (including Electron and Chromium-based Microsoft Edge), WebKit (Safari’s browser engine), and Firefox. Excluding Electron, any browser you want to run Cypress tests in needs to be installed on your local system or CI environment.

Obviously, it is often not necessary to run all available test suites in different browsers given the increase in test execution time and the associated cost of the required CI infrastructure. Therefore, Cypress provides different continuous integration strategies for deploying cross-browser testing in CI pipelines depending on the needs of a particular project.

Benefits of Using Cypress to Optimize Cross Browser Testing in CI

In order to balance costs and available CI resources, as well as the optimal level of confidence in testing, Cypress provides the following options for effectively organizing cross-browser testing:

Selecting a specific test suite for a given browser. For example, sometimes it makes sense to run all available tests within Chrome, but within Firefox _execute only the happy or critical path related test files, or a directory of specific _“smoke” test files using --spec flag. In some cases, the priority areas for implementing cross-browser testing may be critical application features or workflows, as well as the most likely user scenarios.
Flexible setting of the periodic frequency of launching individual browsers. For example, running tests within Chrome can be triggered by events in the repository, while within Firefox, tests will run on a set schedule based on release periodic frequency. Modern CI pipelines allow you to set the required time and periodic frequency for running workflows.
Parallel execution of test files for each group, where the groups are based on the browsers being tested. For example, using Cypress Cloud allows you to run each browser at different levels of parallelization, differentiating the amount of allocated CI resources between browsers depending on the importance of each browser in the testing strategy. For example, tests within Chrome can be run conditionally in parallel on up to four machines, while within Firefox on two, minimizing CI costs.
Possibility to configure the launching or exclusion of browsers for a specific test/test suite. Sometimes it makes sense to run or ignore one or more tests in certain browsers to shorten the duration of a test running. To do this, Cypress allows to specify directly in the configuration of a test or test suite a specific browser to run or exclude, for example: { browser: 'firefox' } or { browser: '!chrome' }.
Determination of the software deployment environment. In cases where the project shows consistently stable behavior in different browsers, it is advisable to set up cross-browser testing only before deploying changes to the production environment.

A thoughtful combination of these Cypress advantages will help you build an optimal cross-browser testing strategy based on the needs of a particular project.

Why Use Docker in Cross Browser Testing?

There are various approaches to implementing cross-browser testing with Cypress, one of which is to set up the automatic running of Cypress tests in chosen CI platform using Docker.

Docker makes it much easier to set up and maintain a cross-browser test environment. By encapsulating the entire test stack, including different browsers in isolated Docker containers, it is possible to reproduce a stable and consistent environment for running tests in different browsers, regardless of the servers running the containers.

Using Docker Compose allows us to effortlessly scale our testing infrastructure by distributing our workload across multiple containers. So, it is possible to define several containers for different browsers and run them simultaneously using one command, or rather one configuration file. Parallel execution of tests in different browsers obviously significantly speeds up the overall testing process.

Using the official Cypress Docker images with pre-installed browsers as a base layer eliminates the need to install browsers on servers running Docker containers. You can read more about the benefits of using Docker in testing in my previous article about running Cypress tests in Docker containers.

Moving from theory to practice…

In this article, I use GitHub Actions as my continuous integration platform. It should be noted that Cypress provides many useful examples of configuration files for setting up GitHub Actions workflows, based on which it is quite simple to organize various options for running tests in several browsers. For example, you can create a workflow for each browser separately and activate the created workflows depending on the specified trigger events in the repository. Obviously, this is very convenient.

This article provides a simplified example of creating multiple containers to run tests within different browsers simultaneously using Docker Compose.

The idea is pretty simple — let’s say we need to run a specific Cypress test suite within four browsers — Google Chrome, Firefox, Microsoft Edge, and Electron. Trigger events for automaticall ystarting a workflow in the repository should be a push event to the main branch, an open or reopened pull request, and a scheduled start, for example, every Friday at 2 am.

Also, during the execution of the workflow, it is necessary to obtain artifacts with the results of running tests in each of the browsers — videos and screenshots in case any tests fail.

One possible solution to this task is to create a custom Docker image based on one of the official Cypress Docker images, build and simultaneously run four containers from the created image, in each of which our Cypress tests will be executed in a specific browser.

Briefly about the test project

To demonstrate how to run Cypress tests across multiple browsers, I use the very simple Cypress-Docker project to test my blog “Testing with Cypress” on Medium. The project already has some dependencies installed — Cypress 12.13.0 and Typescript 5.0.4, and also has spec.cy.ts file with a set of three trivial tests for the blog’s homepage:

By running it locally within Chrome browser, we make sure that all tests pass successfully:

Obviously, these tests are only demonstration examples and do not indicate the correct display and functionality of the website in different browsers.

To run tests on the GitHub Actions platform, this project is hosted on GitHub.

Building a Docker image

As a base layer for building the image, the official Docker image Cypress/Browsers was taken, which includes all operating system dependencies and some browsers.

The most current version of the image at the moment — node-18.16.0-chrome-113.0.5672.92–1-ff-113.0-edge-113.0.1774.35–1 includes pre-installed Node.js 18.16.0, as well as three browsers — Google Chrome, Firefox and Microsoft Edge. Given that the Electron browser is pre-installed in Cypress, we will have all the necessary browsers at our disposal to carry out cross-browser test execution in accordance with the initial task.

The final Dockerfile for building the required image will look like this:

Firstly, FROM instruction defines a base image, all of whose dependencies and configurations will be included in the generated image.

The next WORKDIR step creates the /e2e working directory, in which all subsequent commands will be executed.

Next, in COPYinstruction the package.json and cypress.config.ts files, as well as the cypress folder, including the spec file, will be copied from the repository to the working directory inside the image.

Then RUN instruction specifies two commands to run, in particular, npm i — to install the necessary dependencies in the working directory of the image and npx cypress info — to display information about Cypress, current browsers detected by Cypress, and so on.

The last step in ENTRYPOINT instruction the command in exec form is defined to run Cypress in headless mode in containers generated from this image.

Setting up running containers with Docker Compose

As mentioned earlier, after building the Docker image, four containers will be created based on it. In each container, Cypress tests will be run in a specific browser. To simultaneously run containers with a single command, it is advisable to use a tool such as Docker Compose. To describe the process of loading and configuring containers, the YAML configuration file should include the following:

As we can see, in the configuration file we define four services (containers) — e2e-chrome, e2e-firefox, e2e-edge and e2e-electron, each of which uses the Docker image created based on the Dockerfile located in the same directory (build keys).

Next, the values of the command keys set the commands to launch Cypress in browsers with the given names. It is worth noting here that in the e2e-firefox service, the command was supplemented by a configuration change due to an open issue with recording video when using the Firefox browser in Cypress.

The next step is to mount volumes (volumes keys) for each service and set the appropriate mappings to gain access to artifacts outside of containers. In essence, this means that the videos and screenshots generated during the launch of Cypress and written into containers will actually be stored in the GitHub Actions virtual environment at the specified paths relative to the workspace of the launched workflow. This will allow, in case any tests fail, to extract artifacts from the ./artifacts directory during the execution of the GitHub Actions workflow, which is required according to the initial task.

Setting up the Workflow in GitHub Actions

To run the workflow, we’re going to use following workflow e2e.yml file which is located in .github/workflows directory of the project:

Let’s analyze it in more detail. By using on key we define trigger events that automatically activate the workflow according to the initial task conditions:

push — when pushing changes to the main branch
pull_request — when opening or reopening a pull request
schedule — run a scheduled workflow every Friday at 2am

Next, we define one job in the workflow, cypress-run. The job will run in a virtual machine with the latest version of the Ubuntu Linux operating system within the configured timeout of 5 minutes to ensure that an accidental hang does not use up extra CI minutes.

The steps key combines all the necessary steps to complete the task. First, custom application actions/checkout@v3 is launched, which extracts the repository into the virtual machine, sequentially performing the necessary actions, including checking the git version, creating the necessary folders, authorizing, etc.

Next, we see the main step of the job — Run docker-compose, which executes docker-compose up command to build and run four Docker containers with the specified names (e2e-chrome, e2e-firefox, e2e-edge and e2e-electron) defined in Docker Compose configurations. In this case -d flag means running containers in the background to allow the workflow to continue executing subsequent steps.

The purpose of the next step is to provide visibility into the logs of running Docker containers in order to monitor them and control the process of starting services before moving on to subsequent workflow steps. The docker-compose logs -f command displays logs of all running containers in real-time. At this step, it is possible to track the execution of Cypress tests step by step in each of the four browsers.

The next four steps are identical in nature and ensure that the workflow artifacts from each container are downloaded for access after the workflow completes. The custom application actions/upload-artifact@v3 takes the paths provided as input and uploads folders with generated by Cypress videos and screenshots if the test fails. This will make the artifacts from each container available in the workflow summary. In addition, the behavior of the action in case no artifact files are found is also configured here, as well as the storage period for artifacts (5 days).

Starting the Workflow

To activate the workflow, let’s create a trigger event. For this let’s make a small change to one of the existing tests, for example, add “.” at the end of the expected header text (highlighted) for the test to fail:

Next, let’s commit and push the change to the main branch:

Voila, the workflow is started! Let’s go to the project repository on GitHub and open the summary of the last workflow in the Actions tab:

In the log of the completed cypress-run job, we make sure that all steps were completed successfully:

In particular, at the Run docker-compose step, a Docker image was built based on the previously described Dockerfile, from which four containers were generated. During the build of the image, npx cypress info command was executed and information about detected browsers was displayed in the log, as well as other characteristics of the test environment — the operating system, versions of Node.js, Cypress, etc.:

At the next step, processes were simultaneously launched in the created containers:

to be more precise, parallel execution of Cypress tests was launched in four browsers:

We can see logs from each container about the progress of the test execution:

As a result, the first test failed as expected, while the next two passed successfully in each of the four browsers:

Validate Workflow Artifacts

As you may have noticed earlier, the workflow summary contains information about the downloaded artifacts:

After downloading the artifacts from the workflow summary in ZIP format, we verify that we have videos and screenshots of the failing test in each browser (except the video in Firefox):

Let’s check that in case of successful completion of all tests, the workflow artifacts will not be loaded. To do this, let’s fix the first test, make a commit, and push the change to the main branch again:

After the workflow is completed, we make sure that artifacts were not loaded from any container due to their absence:

Final thoughts

In conclusion, it should be noted that the optimization of cross-browser testing with Cypress based on the approach described in this article has a bunch of obvious advantages.

In particular, the simultaneous launch and parallel execution of tests in several browsers are guaranteed. Containerization ensures that the test environment is consistent and reproducible when running Cypress tests on different browsers.

Moreover, it is easier to set up and maintain a test environment based on a single configuration file. The test infrastructure scales easily depending on the desired level of parallelism for each run or the number of browsers being tested, etc.

In general, all this allows you to increase the efficiency of using available CI resources, reduce the total time for executing tests, expand test coverage in several browsers, providing the optimal level of reliability, taking into account the specifics of a particular project.

That’s about it. If you found this useful, share it with a friend or community. Maybe there’s someone who will benefit from it as well. To continue your journey with me and get more information about testing with the awesome Cypress tool, you might be interested in subscribing to my blog “Testing with Cypress” and get notified when there’s a new useful article.

The source code of all examples presented in this article can be found in the repository of the blog on GitHub.

Thank you for your attention! Happy testing!

Running Cypress tests in Docker containers using different Docker images

Alex Sanzhanov — Wed, 07 Jun 2023 01:55:41 +0000

Greetings to all Cypress enthusiasts!

This article describes the benefits of using Docker in Cypress testing, discusses in detail the current official Cypress images, and outlines the mechanism for building custom Docker images to run Cypress tests in Docker containers deployed based on these images.

Firstly, a few words about Cypress

Cypress is a JavaScript-based end-to-end testing tool designed for modern web test automation. Cypress has become a popular end-to-end web application testing tool due to its powerful features, user-friendly interface, and fast test execution.

There are many pros and cons regarding the strength and weaknesses of the most popular web application testing frameworks today (Cypress, Playwright, Selenium, Puppeteer, etc.), but my own preference is Cypress, which I have been using on a daily basis for several years now.

To my mind, Cypress is a real game changer in end-to-end and component testing and it grows at a rapid pace. Among the many benefits of Cypress, I would like to emphasize the high quality of its documentation, as well as the master classes that the Cypress development team conducts and publishes in the public domain, and also a friendly and responsive community. Honestly, and as you may have noted from my previous articles I’m a big fan of that wonderful tool!

Benefits of using Docker in Cypress testing

In modern automated testing, setting up and maintaining a test environment can often become a tedious task, especially when dealing with multiple dependencies and their configurations, different operating systems, libraries, tools, and their versions. Often in practice, we can encounter dependency conflicts, inconsistencies in environments, limitations in scalability and reproduction of errors that occur, etc., which ultimately leads to unpredictable and unreliable test results. Using Docker goes a long way in preventing most of these problems.

Docker is an open-source service for packaging, deploying, and running applications in an isolated containerized environment.

Today it is difficult to imagine the development of any web application without the participation of Docker. The usage of Docker containerization technologies brings a bunch of serious advantages for testing web applications, such as ensuring a consistent and isolated environment for running tests in each new run; independence from the parameters and settings of the system on which the tests are run; reliability, reproducibility, and predictability of test results when they are run in different environments, etc.

In particular, using Docker in Cypress testing can be very useful for several reasons:

It ensures that Cypress tests run in an isolated test environment because the container in which the tests are run is self-contained from other containers and from the external environment. In this case, the tests are essentially independent of what is outside the container, which ensures the reliability and continuity of the tests in each new run. For example, in the case of deploying containers with tests locally, this means that the absence of Node.js, Cypress, any browser, or certain versions of them on the host computer will not become an obstacle to running Cypress tests.
Allows applications and Cypress tests to be run locally on different host machines, as well as deployed to CI/CD pipelines and cloud infrastructure by providing a stable and consistent test environment. When moving a Docker image from one server to another, containers with the application itself and tests will work the same regardless of the operating system used, the presence of Node.js, Cypress, browsers, etc. This ensures reproducibility and predictability when running Cypress tests across different systems.
Frees us from having to install any dependencies outside of containers — on the servers that run the containers. Docker allows us to quickly deploy the containerized environment to run Cypress tests, so we do not need to install operating system dependencies, necessary browsers, and test frameworks every time.
Significantly simplifies the integration of Cypress tests into the web application development process, embedding and deploying tests in various CI/CD pipelines, cloud services, etc., eliminating the need to troubleshoot compatibility issues during installation, reconfiguration, etc. In addition, Docker allows us to safely move tests between software deployment environments, for example, from a tester’s laptop to a stage or production environment.
Speeds up the testing process by reducing the total time for test runs. This is achieved through scaling, i.e. increasing the number of containers, running Cypress tests in different containers in parallel, parallel cross-browser testing capabilities using Docker Compose, etc.

Briefly about official Cypress images

As of today, there are 4 official Cypress Docker images hosted in the public image repository — Docker Hub, as well as in the corresponding “cypress-docker-images” repository of Cypress.io on GitHub:

The Docker repositories of these images include their various versions and tags. The number of downloads of each image (except cypress/factory) at the time of writing was more than 50 million, which is quite impressive.

Now let’s see how each of these images can be interacted with in order to run Cypress tests in containers created from them. It should be noted that often these containers are not generated directly from the four specified images, but based on custom images, which in turn are built from official images. This is possible because Docker allows us to reuse official images as a base layer and build custom images based on them by adding the necessary additional layers.

A short dive into a test project

To demonstrate running tests in Docker containers, I’ll use “Cypress-Docker” — the simplest project with Cypress tests that I have to test my blog “Testing with Cypress” on Medium.

The project already has some dependencies installed — Cypress 12.12.0 and Typescript 5.0.4:

and there is also one spec.cy.ts file with a test suite of three trivial tests for the blog’s homepage:

By running these tests in headless mode, we are convinced of the successful execution of the tests:

Here we should pay attention to the environment settings when running tests locally on the computer — Node.js 19.8.1, Cypress 12.12.0, and browser Electron 106. Next, we will make sure that when running tests inside Docker containers, these settings will change depending on the content of the Docker images used. I also note that Docker Desktop is already installed locally.

Running tests in Docker containers

Now let’s look at how we can run our Cypress tests inside Docker containers using each of the official Cypress Docker images.

CYPRESS/INCLUDED

I would like to start with this image because, unlike the others, it includes all the dependencies of the operating system, pre-installed Cypress, and some browsers. This makes cypress/included available to use out of the box without the need to add additional layers. The image allows us to create containers and run Cypress tests in them using just single command.

To download an image from the Docker Hub, just run the following command in the terminal:

> docker pull cypress/included:12.12.0

It’s important to note here that it’s a good idea to always add a specific image tag rather than relying on the default (latest) tag name because the latest version changes as new versions of the image are released. In the case of cypress/included, the tag name corresponds to the version of Cypress preinstalled inside it, for example, in my case, it is 12.12.0 (stable). Thus, if we do not specify the image version, then there is a chance that the tests will be run in different versions of Cypress since the containers for running the tests will be generated from images with different tags.

The presence of the specified image after it has finished downloading can be checked with a simple command:

> docker images

The downloaded cypress/included image is shown first in the list of downloaded images:

To find out what this image contains, just execute the following command:

> docker run -it --entrypoint=cypress cypress/included:12.12.0 info

As a result, we get information about the version of Node.js preinstalled in the image, Cypress, three browsers, as well as other characteristics:

Next, to run our Cypress tests in a container, we need to initiate the creation of a new container from the downloaded image with the command below:

> docker run -it -v $PWD:/e2e -w /e2e cypress/included:12.12.0

With this command, we have created a Docker container, which essentially means a writable layer on top of existing image layers. Let’s take a look at the structure of used command:

docker run means to create and run a container based on the specified image

-it — interactive terminal

-v $PWD:/e2e — map the contents of the directory inside the container to the contents of the current project directory, where $PWD is the absolute path to the current directory (root of the project), /e2e is the path to the directory inside the container

-w /e2e — set the working directory inside the container, where /e2e is the path to the specified directory

cypress/included:12.12.0 — an indication of the image on the basis of which the container will be created.

This command will create a Docker container in which Cypress will run tests on the built-in Electron browser. It is important to note that the cypress/included image includes ENTRYPOINT ["cypress" "run"] instruction, which automatically starts Cypress inside the container created from this image.

If desired, it is possible to supplement the command with --name flag to specify a specific name for the created container, as well as --rm flag to automatically delete the container after it is stopped:

> docker run -it -v $PWD:/e2e -w /e2e  --name cypress-tests --rm cypress/included:12.12.0

At the same time, any additional arguments can be passed after the image name to control the behavior of Cypress inside Docker container, similar to how we run Cypress in headless mode without using Docker. So, we can specify a specific browser or spec file to run, configure recording parameters, run tests in parallel, etc. For example, to run tests in any of the browsers preinstalled in the image (Chrome, Edge, Firefox), it is enough to add to the command -b flag specifying the name of the browser:

> docker run -it -v $PWD:/e2e -w /e2e cypress/included:12.12.0 -b chrome

Or if we assume that initially, the project has several spec files, then to run a specific file/files, we should pass to the command -s flag specifying the absolute path to the required file/files:

> docker run -it -v $PWD:/e2e -w /e2e cypress/included:12.12.0 -s cypress/e2e/spec.cy.ts

Now that we know all those commands, let’s create a container and run our tests inside it using Chrome browser:

And voila — we can now see that the specified test environment settings (Node.js 18.16.0, Cypress 12.12.0, Chrome 113) match the characteristics of the cypress/included image content while being different from the settings when we initially ran the tests locally. Thus, it took only single command to run the tests inside created Docker container.

Running tests multiple times in the same container

It is important to note that each time we run Cypress tests based on the docker run … command, a new container will be created and run. To avoid this and be able to run tests multiple times inside the same container, we need to connect to a process inside the container to control the running of tests from the container. To do this, it is enough to supplement the used command with --entrypoint flag with /bin/bash argument:

> docker run -it -v $PWD:/e2e -w /e2e --entrypoint=/bin/bash cypress/included:12.12.0

As a result, we have the ability to control the re-running of tests from the container using cypress run command, or to execute any other commands, for example, view files and folders inside the container using the standard ls -la command.

In addition to the above way of running tests using cypress/included image, we can build our own custom image based on it using Dockerfile. However, in the case of cypress/included it often doesn’t make much practical sense (which can’t be said about other images), since this image already includes all the necessary dependencies of the test environment. Moreover, as mentioned earlier, cypress/included already contains ENTRYPOINT ["cypress" "run"] instruction, so there is no need to explicitly define the command to run Cypress tests in Dockerfile.

Next, let’s look at another interesting official Cypress image — cypress/browsers.

CYPRESS/BROWSERS

This image includes all operating system dependencies and some browsers. However, it does not contain preinstalled Cypress, which is similar to cypress/factory, the main difference is that in the latter we can optionally configure certain versions of browsers and operating system dependencies, as well as add Cypress of the desired version.

Each cypress/browsers image is named according to the following general pattern:
cypress/browsers:node-<full Node version>-chrome<Chrome version>-ff-<Firefox version>-edge-<Edge version>

This is a complete template, while in some versions of the image, there may be only one or two browsers from Chrome, Firefox, and Edge in various combinations.

Let’s create a custom image based on cypress/browsers. In doing so, I will deliberately use the current penultimate version of the base image to demonstrate the difference in the version of the browser used.

As you know, Docker can build images automatically by reading the instructions given in Dockerfile. In this case, Dockerfile for building the custom image will look like this:

First, FROM instruction defines a base image, all of whose dependencies and configurations will be included in the generated image. In particular, these are Node.js 18.16.0, as well as pre-installed browsers — Chrome 112.0.5615.121–1, Firefox 112.0.1, and Edge 112.0.1722.48–1. The base image will be picked up from Docker Hub.

At the next step, WORKDIR, a working directory /tests is created, in which all subsequent commands will be executed. If such a directory does not exist, this instruction will create it.

Next, the files and folders specified in COPY command will be copied from the current project directory (in which Dockerfile is located) to the working directory inside the resulting image. It’s worth noting that I’m not copying the entire contents of the project, as it’s obviously not necessary. In particular, there is no point in copying the node-modules directory with the dependencies already installed, as they depend on the test environment settings. Also, to avoid copying, we can create a .dockerignore file by explicitly specifying node-modules inside it.

Then RUN instruction executes commands during the image build process. In our case, it’s a command that installs all the necessary dependencies in the working directory, and in particular Cypress will be installed at this stage.

At the last step, the command to run Cypress is specified in ENTRYPOINT, which will be executed inside the containers generated from this image. It is noteworthy that the instruction is written in exec form.

Let’s initiate an image build based on the instructions above by running the command:

> docker build -t my-cypress-browsers:1.0.0 .

where docker build is the custom image build command

-t my-cypress-browsers:1.0.0 — an addition of arbitrary name and tag for the created image

. means that we are referring to the Dockerfile location as the Docker build context.

As we can see, the successful building of the resulting image ended with FINISHED message:

Let’s check the existence of the created image. We can see that the built image is listed first in the list of available images:

Next, from the created image, we will generate a container and invoke Cypress inside it to execute tests in the Chrome browser:

> docker run -it my-cypress-browsers:1.0.0 -b chrome

As we can see, our tests are automatically run in an environment with the Node.js and Chrome versions specified in the base image:

Next, let’s go over one more image — cypress/base.

CYPRESS/BASE

This image contains all the operating system dependencies necessary to run Cypress but does not contain Cypress itself and pre-installed browsers. Due to this, cypress/base is significantly (almost 2 times) smaller than the cypress/included and cypress/browsers images, which looks very attractive if there is no need to use some of the browsers. As stated in the image description, a specific image tag corresponds to the version of Node.js or the operating system it is built on.

To build our own image based on cypress/base let’s create a new Dockerfile with the following instructions:

Here FROM instruction defines cypress/base:18.15.0 as the base layer, where the 18.15.0 tag name corresponds to the version preinstalled in the Node.js image. In this case, as in the previous case, I specify the penultimate version of the base image to demonstrate the difference in the versions of Node.js used.

At the next step, as before, a working directory is created to run all subsequent commands inside it. The COPY instruction specifies the files and folders to be copied to the working directory within the resulting image. The RUN command installs the necessary dependencies to the working directory. At the final stage, a command is specified to run tests inside containers generated from this image.

Let’s set the name and tag for the resulting image — my-cypress-base:1.0.0 and build it with the already-known command:

> docker build -t my-cypress-base:1.0.0 .

Great, successful build of the image ended with FINISHED message:

As before, let’s check if the built image exists. We can see that the my-cypress-base image is listed first among the available images:

As mentioned earlier, due to the lack of browsers, the created image has a significantly (more than 2 times) smaller size than our previous images.

Next, we will generate a container based on the created image and run Cypress tests inside it:

> docker run -it my-cypress-base:1.0.0

Now we have verified that the tests run in the pre-installed Electron 106 browser. And as expected the specified version of Node.js — 18.15.0 corresponds to the tag name of the base image and differs from the previously used version (18.16.0):

It is important to note that if we try to run the tests in another browser, for example, by adding -b chrome flag to the initial command, then despite the fact that the corresponding container will be generated, the tests will obviously not run due to the absence of the Chrome browser:

As stated in the text of the error, the only available browser is Electron, which is reasonable since the base image cypress/base does not contain any pre-installed browsers.

Well, now let’s dive into the last of the official images — cypress/factory.

CYPRESS/FACTORY

This image appeared relatively recently, the first version of the image with the 1.0.0 tag name was uploaded to Docker Hub about 4 months ago. Cypress/factory allows building custom images with specific versions of:

node
yarn
chrome
firefox
edge
cypress

As stated in the instructions for this image, using cypress/factory provides the following benefits:

Freedom to choose which versions to test against.
No need to wait on an official release to test the latest version of a browser.
Smaller docker sizes especially when not including unused browsers.
Easily test multiple browser versions.
Reduced maintenance and pull requests for the cypress-docker repo.
Ability to offer more variations of docker containers at low cost.

Indeed, one of the main advantages of cypress/factory is the ability to customize the contents of the image, which allows, if necessary, to significantly reduce its size. Required versions of Node.js, Cypress, and browsers can be passed as arguments when building the image.

Suppose we plan to build an image, including the most up-to-date versions of Node.js, as well as Chrome, Edge, and Firefox browsers. At the moment it is:

Node.js 20.1.0
Chrome 113.0.5672.92–1
Edge 113.0.1774.42–1
Firefox 113.0

In this case, we should pass the specified parameters as arguments to build the custom image. There are different ways to declare them:

as instructions in Dockerfile
inside the image build command with --build-arg flag
using Docker Compose and specifying them in the docker-compose.yml file

By the way, if we don’t pass any arguments at all when creating the image, then by default the generated image will contain only Node.js.

Let’s first look at the option of specifying arguments in Dockerfile. To do this, let’s create Dockerfile at the root of our project with the following content:

As we can see, before FROM instruction, the ARG arguments for building the image are declared. ARG is used to set build-time variables with key and value. In our case, it defines the required versions of Node.js and browsers. It’s important to note that the exact version must be used, no wildcards or shorthands are supported.

Further FROM instruction states that the base image for creating a custom image is cypress/factory.

In the next WORKDIR step, the working directory /tests is created inside the resulting image for executing all subsequent commands.

Then as in previous cases, COPY instruction specifies the files and folders from the current project folder (which contains Dockerfile) to be copied to WORKDIR.

Next, RUN instruction will install the necessary dependencies specified in package.json file.

At the last stage, ENTRYPOINT instruction, as already noted, specifies the command to run Cypress inside the containers that will be generated based on this image.

Now let’s create an image following the instructions above:

> docker build -t my-cypress-factory:1.0.0 .

After the image is built, we make sure that it is available:

Now let’s initiate container creation from our resulting image and run Cypress tests inside it:

> docker run -it my-cypress-factory:1.0.0 -b chrome

As we can see, in the created container, the tests are automatically run in an environment with the versions of Node.js and Chrome browser specified in Dockerfile:

Obviously, to control the behavior of Cypress during test execution, we can also pass additional parameters to the container run command using CLI flags. So, in the above command, the tests are run in the Chrome browser using the -b flag.

To build a custom image without adding arguments to Dockerfile, we can run the following command:

> docker build --build-arg NODE_VERSION='20.1.0' --build-arg CHROME_VERSION='113.0.5672.92-1' --build-arg EDGE_VERSION='113.0.1774.42-1' --build-arg FIREFOX_VERSION='113.0' -t my-cypress-factory:1.0.0 .

By the way, this action overrides the ARG values if they were also set in Dockerfile.

Despite the obvious flexibility in building custom images based on cypress/factory, it is possible to combine versions that are incompatible with each other within an image. For example, certain versions of Cypress may not support certain versions of Node.js. Obviously, this will allow us to build the image, but the containers generated from it will not work correctly. According to the developers, both the image itself and containers generated from it are intended for test use only and are not intended for use hosting services in a production environment.

Final thoughts

The official Cypress Docker images discussed in this article, as well as custom images created on their basis, greatly simplify the integration of Cypress tests into the development of modern web applications and increase the reliability and efficiency of the testing process in general. Providing a stable and consistent test environment and reproducibility across different software deployment environments greatly simplifies the setup and maintenance of automated tests, and makes it easier to detect and troubleshoot issues that arise.

I hope this article was helpful for you to understand the basics of using containerization in Cypress testing. Additionally, it is worth noting that outside the article there were cases of running Cypress tests as part of several containers, parallel and cross-browser testing, which certainly deserves a separate article.

That’s about it. If you found this useful, share it with a friend or community. Maybe there’s someone who will benefit from it as well. To continue your journey with me and get more information about testing with the awesome Cypress tool, you might be interested in subscribing to my blog “Testing with Cypress” and get notified when there’s a new useful article.

Dockerfiles used for this article, as well as the test project itself, can be found in the blog’s repository on GitHub.

Thank you for your attention! Happy testing!

Testing Excel Data with Cypress

Alex Sanzhanov — Fri, 21 Apr 2023 03:02:20 +0000

Note: this article was included in the 136th Coding Jag by LambdaTest and also in Issue #82 of Software Testing Notes.

Greetings to all Cypress enthusiasts!

A couple of days ago I was approached by a member of our Cypress-community on Discord with a question about how to validate data from an Excel file using Cypress. This prompted me to write this article.

Looking ahead, I’ll just note that this is a rather simple matter, and is mainly based on setting up the corresponding Node event inside the Cypress configuration file to convert the original Excel file to JSON format and further work with the resulting file. But first things first.

If you are not yet familiar with Cypress:

Cypress is an open-source JavaScript-based testing tool designed for modern web test automation. Cypress has emerged as a popular end-to-end testing tool for web applications due to a bunch of its powerful features, user-friendly interface, fast test execution time, easy installation and debugging, etc.

Cypress is a real game changer in e2e and component testing and it grows at a rapid pace. And as you may have noted from my previous articles I’m a big fan of that wonderful tool!

However, it is obvious that Cypress is not intended for testing Excel data, which in some test scenarios can create some inconvenience if you need to validate data in Excel files. Despite this, Cypress provides a way to do this, as I noted earlier, by converting the Excel file to Cypress-compatible JSON format. This allows you to test your source data using the same test suite that you use to test your web applications.

Well, let’s imagine that in the course of executing some test scenario, we get a certain Excel file in the cypress/downloads directory of our project. In essence, the scenario can be anything, as well as the initial location of the initial file. And now we need to test the data in this file.

For example, I have a companies.xlsx file in the downloads directory

with the data of some 10 non-existing companies:

First, let’s convert the original companies.xlsx file to JSON format. Actually, there are several suitable npm packages for this, the most popular of which today (judging by the number of weekly downloads) is SheetJS. Let’s install it into the project by typing the following command in the terminal:

> npm i xlsx

Next, using the syntax of this package, let’s add a task event to the setupNodeEvents function in the cypress.config.ts file. This will allow us to move from the browser environment to the Node environment and execute some JavaScript function in this environment. For example, let’s call our function convertXlsxToJson, and with its help, we will convert the original Excel file into JSON format:

So, our convertXlsxToJson function takes as an argument the path to the initial Excel file — filePath and uses XLSX.readFile() method to read the file, as a result of which we get a certain workbook object with a rather complex structure containing the data of the original Excel workbook. Let’s analyze it in more detail.

The main element of workbook object is Sheets object, whose properties are also objects containing the individual worksheets in the original Excel workbook, where the keys are the names of the worksheets and the values are the objects that represent each worksheet. In turn, each worksheet object has its own set of properties, whose keys are references to cells in the worksheet (for example, A1, B2, etc.), and the values are objects containing information about the type of data in a particular cell, and also the value of the cell itself. Also, workbook object contains SheetNames array with the names of the worksheets of the initial Excel book, while the order of the names in the array corresponds to the order of the worksheets in the workbook. In addition, workbook object also includes a bunch of other properties that contain data about the original Excel workbook.

To retrieve the name of the first worksheet that includes the companies data table (obviously, it could have been any other worksheet), we access the value of the null element of SheetNames array. Next, using the obtained sheet name as the key of the corresponding property on Sheets object, we get the worksheet object as the value of this property and put it in worksheet variable.

On the specified variable, we apply XLSX.utils.sheet_to_json() method to convert the worksheet data to JSON format. As a result, we have an array of objects jsonData, in which each object corresponds to a specific row of the initial Excel table. The properties in each object include the column headings (keys) and the corresponding data in the cells (values). Ultimately, our convertXlsxToJson function returns jsonData array as the result of the task event.

In general, at this stage, the task can be considered completed, since the jsonData variable contains data in JSON format suitable for use in tests. However, let’s imagine that we want to write the received data as a companies.json file and place it, for example, in the cypress/fixtures directory. To do this, we supplement our convertXlsxToJson function with the following lines of code:

Firstly, we define fileName variable, in which we place the name of the original Excel file (without extension), obtained based on the value of filePath variable using path.basename() method. Using the specified file name, we set jsonFilePath variable to the path to the resulting JSON file in fixtures directory. Finally, we use writeFileSync() method from Node.js built-in fs module, which synchronously writes data to the resulting file as a JSON-formatted string with 2 spaces for each indentation level using JSON.stringify() method.

Thus, we set up our task event convertXlsxToJson in such a way that it accepts any Excel file located at the specified path, reads it and converts it to JSON format, and also writes the resulting JSON file to fixtures directory.

Now, to test the original data of ten companies, let’s create a spec file testExcel.cy.ts in e2e directory. First, let’s add before hook to our test suite, in which we will call our task event, passing it the path to the initial Excel file as an argument:

After calling our task event, the resulting companies.json file will be written in the fixtures directory:

with the following structure:

After adding the tests, finally, our spec file will have the following content:

So, first we define two variables: xlsxPath — the path to the original Excel file and jsonName — the name of the final JSON file, obtained based on the value of the xlsxPath variable using the path.basename() and replace() methods. Next, inside the beforeEach hook, we use the cy.fixture() command to load the contents of the resulting JSON file as a fixture, which we alias companiesData to access the specified content in each test.

As possible examples of tests, I have given three rather primitive demo tests. So, the first one checks if our test file contains data for 10 companies. The second verifies that each company’s data contains non-empty values for the four keys — “Company Name”, “Product”, “City”, and “Email”. And the last test checks if each company’s data contains a unique email address.

Let’s run our tests in the Cypress test runner with the standard command:

> npx cypress open

We click on the name of the spec file in the test runner window and voila — our tests completed successfully in less than 0.1 seconds:

Final thoughts

Although Cypress can be used to validate Excel data, this is obviously not the case for its core functionality. Cypress does not test Excel files “under the hood”. Therefore, you should be aware of possible errors that may occur at the stage of converting an Excel file to JSON format, such as formatting errors, data loss, etc., which can subsequently lead to undesirable results. Despite the relative rarity of the described case and the comparative simplicity of the approach to solving it, I really hope that this article will be useful for improving your testing skills with Cypress.

The source code of all examples as well as the files presented in this article can be found in the repository of the blog on GitHub.

Thank you for your attention! Happy testing!

How to optimize Cypress tests using JavaScript abilities? (Part 2. Recursion)

Alex Sanzhanov — Thu, 20 Apr 2023 20:24:54 +0000

Greetings to all Cypress enthusiasts!

In the last article, we looked at various cases of optimizing Cypress tests based on the use of loops. I would like to delve a bit into the problem of optimizing repetitive tasks and consider another programming construct used to repeatedly repeat a particular piece of code or set of instructions. This construction is called a recursion.

What exactly does recursion mean?

Recursion is a programming technique that involves a function calling itself within its own code. When a function is called recursively, it solves a problem by breaking it down into smaller and simpler subproblems until it reaches the base case, which is the smallest subproblem that can be solved without further recursion.

How can recursion be used in Cypress tests?

In Cypress automated tests, recursion can be used to iterate through complex nested structures or perform actions on dynamic elements that may not be present when the test is first executed. By breaking down the problem into smaller subproblems, recursion allows for more efficient and modular code that can handle a variety of scenarios. One of the most common cases of using recursion in Cypress tests is testing of pagination (see below).

Why use recursion in Cypress tests when we can use loops instead?

Using recursion may be more suitable than using loops in Cypress tests when the problem or task requires repeated execution of a block of code on a nested structure, or when the problem can be divided into smaller, similar sub-problems. Recursion is also useful when the exact number of iterations needed is not known in advance, or when the nesting level of the elements being tested is unknown or variable. Additionally, recursion can make the code more readable and easier to maintain in cases where loops would require complex conditional statements or multiple nested loops.

Now that we have an understanding of the advantages of recursion for optimizing testing processes, let’s consider several common cases of its use in Cypress tests.

1. Iterating through nested elements

In Cypress tests, it is common to iterate through nested elements to perform various actions like searching for elements, clicking or validating their properties, etc. Recursion can be used in this scenario when the elements are nested in an unknown or dynamic depth, meaning that the number of nested elements can vary.

Let’s consider an example where we want to validate the text of all nested elements inside a parent element. We can define a recursive function that traverses through all the nested elements, checks if it has any children, and then recursively calls the same function on those children:

In the above example, the validateNestedElements() function takes a parent element, checks if it has any children, and recursively calls itself on each child element until it reaches the leaf elements without any children. Then expect statement is used to validate the text of each leaf element.

Recursion can also be used to iterate through nested elements in a page and perform actions on them. Therefore, this scenario is very suitable for testing pagination when we are going to test moving to each next/previous page:

Using the visitTextPageIfPossible() function this code repeatedly clicks on the “next” button, waiting for the page to update, until the button becomes disabled, which indicates that there are no more pages to display.

2. Handling asynchronous operations

Recursion can be useful in handling asynchronous operations in Cypress tests when dealing with situations where we need to retry an action until a certain condition is met. For example, suppose we have a test case where we need to wait for an element to appear on the page, but due to network latency or other factors, the element may not appear immediately. In such cases, we can use recursion to retry the action until the element is found or until a certain timeout is reached:

In this example, the clickElement() function uses recursion to handle the case where the element to be clicked is not immediately available on the page. The function first attempts to locate the element using cy.get() command, with a timeout of 5 seconds. If the element is found, the function clicks on it and returns. If the element is not found, the function recursively calls itself with the same selector and one less retry remaining. This continues until either the element is successfully clicked or the maximum number of retries is reached, at which point an error is thrown.

Using recursion with retries is better than multiple increases in the timeout because it allows us to wait for the element to become visible without blocking the execution of the test. Increasing the timeout multiple times would add unnecessary delays to the test and could make it more prone to flakiness if the element takes longer to appear than expected. Recursion with retries allows us to wait for the element without adding unnecessary delays, and if the element never appears, the test fails quickly rather than waiting for an excessive amount of time.

3. Testing complex user interactions

Recursion can be used in Cypress tests to test complex user interactions that involve multiple steps and dynamic elements. In such scenarios, using recursion can be more suitable than loops because it allows for a more elegant and efficient way of navigating through dynamic UI elements.

For example, let’s say we have a form with a dynamic number of fields, and we want to test that all required fields are filled before submitting the form. We can use recursion to navigate through each field, check if it’s required, and fill it if it’s empty:

In this example, the fillForm() function takes an array of FormField() objects and an optional index parameter. It checks if the current field at the given index is required and empty, and fills it if needed using Cypress cy.get() and cy.type() commands. It then calls itself recursively with the next index until all fields are processed. Finally, when all fields are processed, the function submits the form using Cypress submit() command.

4. Building complex test flows

Recursion can be used to build complex test flows that require multiple steps and conditions. For example, if you have a test that requires the user to navigate through a series of pages and perform different actions on each page, you can use recursion to build a flow that navigates through the pages and performs the necessary actions.

Let’s consider an example where we want to test a form that has multiple pages, and we need to fill in various fields on each page before we can submit the form. We can use recursion to navigate through the pages, fill in the fields, and move to the next page until we reach the last page:

In this code, we define an array of FormPage objects, where each object represents a page in the form. Each page has an array of FormField objects, which represent the fields on that page. The next property is a function that is called when the current page is filled out and submitted, which navigates to the next page.

The fillFormPage() function takes an index of the current page and fills out the fields on that page using cy.get() and cy.type() commands. If the current page has a next function defined, it calls that function to move to the next page. The submitForm() function submits the form by clicking the submit button.

5. Dynamic data handling

Recursion can be used to handle dynamic data that changes over time. For example, if you have a list of elements that is updated dynamically, you can use recursion to continuously check for updates and perform actions on the updated elements.

Suppose we have a web page that displays a list of items, where each item has a unique ID and some data associated with it. The data displayed on the page can change dynamically based on various user interactions or other factors. Our Cypress test needs to verify that all the items are displayed correctly on the page, but we don’t know how many items there are or what their IDs are in advance. In this case, we can use recursion to traverse the list of items on the page and check their data one by one until we have checked them all:

In this code, the checkAllItems() function calls itself recursively with the next index until all items have been checked. The function extracts the data for each item from the corresponding DOM element and performs some assertions on the data.

The reason why recursion is a good fit for this scenario is that we don’t know how many items there are in advance, so we can’t use a traditional loop with a fixed number of iterations. Additionally, the Cypress commands used in the function are asynchronous and return promises, so we need to use recursion to ensure that each item is checked in the correct order.

6. Implementing retries

In Cypress tests, sometimes there may be flaky tests, where the test may fail intermittently due to network latency, server issues, or other reasons. In such scenarios, it’s helpful to retry the test multiple times to increase the likelihood of it passing.

To implement retries in Cypress tests, we can use recursion. The basic idea is that we will recursively call the test function until it passes or until a maximum number of attempts have been made.

Let’s say we have a test that checks whether a login form is working properly. We want to retry this test up to three times if it fails due to some intermittent issue. Here’s an example of how we can implement this using recursion:

In this example, we define a testLogin() function that takes two arguments: maxAttempts, which is the maximum number of times the test should be retried, and currentAttempt, which is the current attempt number (defaulted to 1).

Inside the function, we first visit the login page and fill in the form. Then we use cy.url().should() to check whether we have been redirected to the dashboard page. If the assertion passes, we do nothing and the test is considered to have passed. If the assertion fails, we check whether we have exceeded the maximum number of attempts. If we have not exceeded the maximum, we recursively call testLogin() with an incremented attempt number. If we have exceeded the maximum, we use an assertion to fail the test.

Final thoughts

In conclusion, recursion is a powerful tool that can greatly enhance the flexibility and functionality of Cypress tests. As we’ve seen in this article, it can be used in a variety of scenarios, from iterating through nested elements to building complex test flows. However, it’s important to note that while recursion can be a useful approach, it may not always be the most efficient option due to potential performance issues.

It’s crucial to evaluate each use case carefully and consider alternative solutions if necessary. Ultimately, the goal is to create Cypress tests that are reliable, maintainable, and efficient. By understanding the strengths and limitations of recursion in testing, we can make informed decisions and optimize our test suites for success.

The source code of all presented examples can be found in the corresponding repository on GitHub. To continue your journey with me and get even more information about testing with the awesome Cypress tool, I invite you to subscribe to my blog “Testing with Cypress”.

Thank you for your attention! Happy testing!

Selecting elements in Cypress tests: basic + advanced patterns (2 useful Cheatsheets)

Alex Sanzhanov — Thu, 23 Mar 2023 18:52:07 +0000

Greetings to all Cypress enthusiasts!

In this very short note, I would like to share with you one very useful thing for selecting elements in Cypress tests. To be honest, I had plans for a long time to summarize the most common CSS and jQuery selector patterns, which in fact was the reason for creating this extensive table. You can of course use the great Cypress built-in commands like contains(), find(), eq(), etc. to select test elements. However, skillfully combining element attributes, binding to pseudo-classes and the current state of elements, using a hierarchy and combinators, and so on can give you additional advantages and ensure reliability in choosing DOM elements.

Well, I present to you my cheatsheets with basic and advanced selector patterns and examples of their use with the Cypress cy.get() command:

Cheatsheet #1: Basic patterns

Cheatsheet #2: Advanced patterns

I really hope that this will be useful for you and help you improve your skills in testing. To continue your journey with me and get even more information about testing with the awesome Cypress tool, I invite you to subscribe to my blog “Testing with Cypress”.

Thank you for your attention! Happy testing!

How to optimize Cypress tests using JavaScript abilities? (Part 1. Loops)

Alex Sanzhanov — Tue, 28 Feb 2023 07:31:37 +0000

Greetings to all Cypress enthusiasts!

Those who have had experience working with Cypress.io know that it is truly a wonderful tool that provides great opportunities for developing and implementing high-quality and reliable tests in your projects. Therefore, all that remains for us is to be able to make the best use of these opportunities.

The fact is that over the past few years, I have encountered various approaches to writing tests, sometimes with fundamentally different models and templates for building test scenarios and test cases. And it often happened that already written tests needed serious improvement or even complete restructuring, mainly to ensure their reliability and reduce the time it takes to run them. It’s obvious that as a QA engineer, it is crucial to continuously evaluate and optimize testing processes to ensure the highest possible level of quality assurance.

And in my observations, problems were often caused not only by a lack of deep understanding of the built-in commands of the testing framework, but also very often by an inability to apply basic programming language capabilities in tests. This has prompted me to write a series of articles in which I share some of my techniques for optimizing Cypress tests, drawing on simple and mostly familiar JavaScript capabilities.

Among the most common errors, a separate group can be identified for cases where QA engineers repeatedly perform exactly the same actions on different elements, be it form fields, buttons, links, or any other UI elements. Often, depending on the tested elements or actions performed, they are often divided into separate tests. This leads to a sharp increase in the number of tests, which is further compounded by the presence of multiple prerequisites placed in beforeEach and afterEach hooks, which are repeated over and over again as test suites are executed. In some cases, such unjustified omissions ultimately lead to a significant increase in the overall time it takes to run tests.

In general, it is important to note that such erroneous actions do not correspond to the DRY principle, which can make the tests harder to maintain, slower to run, and more prone to errors and inconsistencies.

Therefore, in the first article, I would like to discuss when and how the best way to optimize repeatedly performed actions, events, or assertions in Cypress tests using basic JavaScript loops to avoid the above-mentioned ungrateful situations.

After analyzing the most frequently occurring cases of iterative actions during testing, I identified 8 common scenarios where the use of loops is most effective. This is certainly not an exhaustive list, as other cases may arise depending on the specified test scenarios. Next, I plan to examine each of these scenarios in detail and demonstrate with specific examples how loops can significantly help optimize your Cypress tests.

1. Iterating through a set of items or elements

When iterating through a set of items or elements, loops come in handy to perform repetitive tasks. For instance, if there are multiple checkboxes on a web page, a loop can be used to select or deselect each one of them. The loop can be controlled by setting the start and end values, making the code cleaner and efficient. By using loops in this scenario, you can ensure that all items or elements are verified or interacted with as needed.

In this example, we use the Array.from() method to convert an array-like object returned by Cypress cy.get() command to a standard array. We then use a forEach() loop to iterate over each element and perform some action.

2. Repeating a test with different inputs or expected outputs

For example, if there are multiple forms on a page, each requiring different inputs, a loop can be used to run the same test with each form. The loop can be controlled using conditional statements that verify the expected outputs for each input. This approach saves time and reduces the amount of code needed to test multiple scenarios.

In this example, we use a for…of loop with destructuring to repeat a test with different inputs and expected outputs. We define an array of objects, each with an input and expected output property, and use the loop to iterate over each object and perform the test.

3. Navigating through multiple pages

Cypress test can navigate through multiple pages of a web application by implementing a loop to check the contents of each page. The loop’s value can be regulated with conditional statements that confirm the existence of a next or previous page. Such a technique not only enhances the code’s organization but also improves its readability.

This example demonstrates how to use forEach() loop to navigate through multiple pages. The code creates an array of page URLs, and then uses a forEach() loop to visit each page URL and assert that the page has loaded correctly using Cypress’s should() method.

4. Testing multiple user accounts or roles

If there are multiple roles or user accounts in a web application, a loop can be used to test each role or account by logging in with each set of credentials. The loop can be managed through conditional statements that confirm the anticipated results for every user role or account.

The value of using the for…of loop in this Cypress test is to execute the same test scenario multiple times with different sets of data. In this case, the test scenario is logging in with a specific user's credentials, performing some actions specific to their role, and then logging out. The loop iterates over the users array and performs the test scenario for each user in the array. This saves time and effort in writing individual tests for each user account or role, and allows for more efficient and comprehensive testing of the system under different user scenarios.

5. Running the same test with different configurations

If a web application has multiple configurations, a loop can be used to test the application with each one. The loop can be determined by implementing conditional statements that define the anticipated results for every configuration.

The value of using the for…of loop in this test is to optimize the test execution by reducing code duplication. By iterating over the configs array and using the configuration data to set up the testing environment and execute the tests, we can avoid repeating the same setup and test code for each configuration.

This not only makes the test code more concise and easier to read, but it also reduces the risk of errors and omissions that can occur when copying and pasting test code. Additionally, by separating the configuration data from the test code, we can easily add or remove configurations without having to modify the test code.

6. Dynamically generating test data

Loops can be used to generate test data dynamically. For instance, if there is a need to create multiple user accounts, a loop can be used to generate a set of user data with each iteration. The loop can be controlled by setting the start and end values and by specifying the number of users to create.

The forEach() loop is used here to iterate over each key in the testDataobject and perform a test for each input. This loop enables the code to avoid duplicating the same test case for each input, and instead, it performs the same test for each input dynamically, thus saving time and effort.

By using a loop to iterate over the testDataobject, it becomes easy to add more test cases without duplicating code. This makes the code more efficient and reduces the likelihood of errors when adding new tests.

7. Iterating through a set of test steps

Also, loops can be used to iterate through a set of test steps when testing a web application. For example, if there are multiple steps involved in testing an application, a loop can be used to repeat the steps with each iteration. The loop can be managed using conditional statements that specify the expected outputs for each step.

In this Cypress test each step is an object that contains information about an action that needs to be performed during the test, such as clicking on a button or typing into an input field. By using a loop forEach() to iterate through the steps array, the test can perform each action sequentially and with less duplicated code.

Inside the loop, the test checks the action property of each step object to determine what action should be performed. Depending on the value of action, the test will use a different Cypress command to perform the action. For example, if the action is click, the test will use the cy.get().click() command to simulate a button click.

8. Testing the same functionality or behavior across different environments

Loops can be used to test the same functionality or behavior across different environments, such as multiple screen resolutions or browser types. For example, if a web application needs to be tested on different browsers or screen resolutions, a loop can be used to test the application on each configuration. The loop can be controlled using conditional statements that verify the expected outputs for each environment.

Inside the loop, the test sets the viewport size using the cy.viewport() command to match the width and height properties of the current environment. Using a loop in this way allows the test to be run multiple times with different configurations, without having to duplicate the same test code. This can make the test more efficient and easier to maintain, especially when there are many different environments to test against.

Conclusion

Thus, we have confirmed that there are numerous useful applications of loops in Cypress tests, significantly improving testing efficiency, reducing the time spent on test runs, and enhancing test coverage. Overall, loops are an important tool for automating testing in Cypress, improving the process and saving time and effort. In our next articles, we will explore other JavaScript features that can help optimize your Cypress tests.

The source code of all presented examples can be found in the corresponding repository on GitHub.

Thank you for your attention! Happy testing!

Node Version Manager (NVM): how to install and use (step-by-step guide)

Alex Sanzhanov — Wed, 16 Nov 2022 19:17:39 +0000

As it turns out, the correct installation and usage of NVM on Windows OS often raises some questions due to the fact that there are some inaccuracies in the official repository of this tool. In this regard, in this article, I will tell you how to install NVM on your computer step by step in a more optimal way (MacOS users do not need to read this).

NVM: What's the point? 🤨

Node Version Manager (NVM) is a tool that allows you to have multiple versions of Node.js on your device, switch them quickly, and it is managed from the command line interface (CLI).

Why is this needed? The fact is that different applications you work with can be implemented on different versions of Node.js. In practice, this is usually indicated by the .nvmrc file which is committed with your project and contains a single line with the version of Node.js to use for this project. However, running the application in another environment may not give the expected result or lead to errors.

Can I do without it? The cases when NVM is just an indispensable thing occur very often in the workplace. Let me give you a small example from my past experience. Having come to work for a new company some time ago, I had to quickly get to the bottom of one of their past projects. At that time, we seemed to be using Node.js version 14.16.0, but the project repository required version 10.x. Of course, if I did not know about NVM, I would only have to reinstall from one version to another. And so for each new project! Definitely, it would be a big headache and a waste of working time.
Moreover, when you work on a large team, it's rare that everyone is using the same version of Node.js.

How to install NVM correctly? 🤔

Well, let's get started with NVM installation. Along the way, I will point out the inaccuracies in the official repository that I mentioned at the very beginning of this article. Here I'm only emphasizing that the further explanation is based on the example of the 11th version of Windows OS.

To begin, open the terminal application on your computer. For Windows, I recommend using Git Bash due to some commands may not be available in Windows PowerShell. Ideally, use the built-in Windows Terminal, in which you need to add Git Bash.

Let's make sure NVM is not already installed on your computer. Just enter the command:
```
nvm -v
```
If this is correct, then you will see the following response:
Go to the NVM for Windows releases page and select the latest version (the top one in the list), for example, it is currently 1.1.10. Then download the nvm-setup.exe file:
I assume that most of you already have Node.js installed at this point (if not yet, then NVM will do it for you a little later). And if we turn to the official manual, it states that before installing NVM we need to uninstall any existing versions of Node.js:

However, we will not follow this advice. Instead, run the downloaded setup file. You'll be prompted to agree with the project's terms of use (сhoose 'I accept the agreement' option), then the installer will ask where to install NVM (click on the 'Next' button).
And now attention!
You'll be prompted to indicate where to set the Node.js Symlink. This is the next inaccuracy of the official manual because the default path C:\Program Files\nodejs is not correct:

I strongly recommend to change this path to any other, that does not contain spaces. For example, I specified the following folder in the user's home directory:

In the next window, click the 'Install' button. Immediately after installation, NVM recognizes that you already have Node.js installed and asks the following question:

Select the 'Yes' option and the installation process will complete.
Now restart the terminal application and make sure NVM is installed:
```
nvm -v
```
If everything was successful in the previous step, you will receive the following response:

Usage 😃

First, make sure NVM is enabled:
```
nvm on
```
Expected response:
Check the active version of Node.js:
```
nvm current
```
Expected response (in your case it may be another version):
Download the latest version of Node.js:
```
nvm install latest
```
Expected response (in your case it may be another version):

Also, I can download a specific version of Node.js, for example, 14.18.0:
```
nvm install 14.18.0
```
Expected response:

If you want to remove a specific version use the command:
```
nvm uninstall <version>
```
Open a list of all versions installed on your computer:
```
nvm list
```
Expected response:
Switch to use the specified version:
```
nvm use 14.18.0
```
Expected response:

A complete list of commands can also be seen in the official repository.

Well, that's where I end. I really hope that this article will help you properly install and use this wonderful and very useful tool. If you have any questions, write them in the comments below. 👋

Run API tests with Newman and send reports to Telegram

Alex Sanzhanov — Sun, 06 Nov 2022 20:54:34 +0000

How about running your Postman collections of API requests and automatically getting a meaningful report right in Telegram? In this article, I will show you how to implement this trick in one small script! 🔥

First of all, it would be appropriate to note that this article is only a training example for creating a small part of a continuous integration pipeline, which can be easily done locally using Node.js. That will help you to practice running collections with (or without) executing tests and sending API requests using node-modules and without Postman. But for practical use, there are many great built-in solutions for the most widely known CI tools that you can easily find in the public domain. ♾️

Export of initial collection and environment from Postman 🚀

In order to run any collections of API requests, we first need to have them available as a separate file that can be pulled from Postman, for example. For this case, in my Postman, I had a collection for the CRM Project "ClientBase v5" which contains several dozen of the most likely API requests and more than 400 built-in tests and in some cases also pre-request-scripts.

In all tests, I deliberately used Chai Assertion Library BDD syntax to make it easier to develop automated tests based on them. Some tests contain the same assertions but use a slightly different syntax. The point is that the Postman and Chai Library syntax allows us to implement a specific assertion in different ways, which is certainly great. 🍵

All I needed to do here was to export the collection as a json file and save it locally. Further, it will be important for me to know only the path to this file.

I also exported as a json the set of environment variables. Most of these variables were created as dynamic, in other words, they are generated using the faker library. The values of variables, as you can see in the picture, are not initially set and are filled automatically during the running of requests:

If you are interested, this Postman collection is available here (still growing).

Telegram Bot API 🤖

As I mentioned, аfter running the collection and generating a report, I will send a notification and that report to Telegram. Moreover, it’s important to note here that the receiver, in this case, may not necessarily be Telegram. For example, it can be Slack or any other resource which is available for interaction by API.

Well, for the training example, I chose Telegram because I just wanted to learn more about the capabilities of the Telegram Bot API. To get started, I needed to select an existing or create a new Telegram group to receive notifications and reports there. It could be any Telegram group (public or private). So I created a new group called "API alerts". The most important step was getting my group's chat ID. And I got it very easily with the help of a special bot inside Telegram - the "Get My ID" bot. I just forwarded one message from my newly created group to this bot and already had the chat ID. There were other ways to do this, but this one seemed faster.

Then it was just a matter of creating a bot with which I was going to interact using the request API. Well, I did it even faster and here is a detailed guide on how to create a Telegram bot using @Botfather. The main thing here is to get from Botfather a unique bot authentication token that looks something like 123456:ABC-DEF1234ghIkl-zyx57W2v1u123ew11. Further, I added my new bot to the created "API alerts" group and gave the admin permission to send messages to the group.

It seems there is no point in explaining this step in more detail because everything here is extremely simple and there are a lot of instructions on the Internet on how to do it.

Well, it's time to create a project.

Creating the project and installing the required dependencies 🗂️

Having completed the preparatory steps in Postman and Telegram, I finally created a new project in the IDE, initialized it, and proceeded to install the necessary dependencies. So for running the collection I used Newman which is a command-line Collection Runner for Postman. In order to install Newman locally in the project as a library, used the following command:

npm i newman

As for the report of collection running, in general, we can integrate it with our CI pipeline using the Postman CLI or Newman as a pure command-line Collection Runner and generate some reports for example with Allure or at least JUnit. However, we are also able to use Newman reporters and get at least (and sometimes much more) informative and user-friendly reports with a dashboard-style summary landing page and a set of different tabs which contain the detailed request information.

To be more precise, I often prefer to use Danny Dainton’s html-extra reporter which is an updated version of the standard Newman HTML reporter containing a more in-depth data output and a few helpful extras. 📑

So this reporter was installed in the project using the command:

npm i newman-reporter-htmlextra

In addition to using environment variables in the project, I also needed another module dotenv that loads environment variables from a .env file into process.env. After installing this module with the command:

npm i dotenv

I created a .env file in the root of the project, in which I put the necessary environment variables:

COLLECTION_PATH - the path to a locally hosted json file with my Postman collection;
ENVIRONMENT_PATH - the path to locally hosted json with environment variables which also was exported from Postman;
TOKEN - the token of my Telegram bot;
CHAT_ID - my Telegram group's chat ID.

In my case, it looked like this (the values of sensitive variables have of course been changed):

COLLECTION_PATH=C:/Users/user1/.vscode/Newman-Collection-Run-Reporter-on-CRM-Project-example/CRM_Project_ClientBase_v5_collection.json
ENVIRONMENT_PATH=C:/Users/user1/.vscode/Newman-Collection-Run-Reporter-on-CRM-Project-example/ClientBase_v5_environment.json
TOKEN=270485614:AAHfiqksKZ8WmR2zSjiQ7_v4TMAKdiHm9T0
CHAT_ID=-1002956968

Well, the basic dependencies are installed, it's time for coding. 👨‍💻 In order to do this, I created a runner.js file at the root of the project in which I included the first block of code:

require('dotenv').config()
const newman = require('newman');

//Running collection with Newman, report generation:

newman.run({
    collection: process.env.COLLECTION_PATH.toString(),
    environment: process.env.ENVIRONMENT_PATH.toString(),
    reporters: ['htmlextra'],
    iterationCount: 1,
    reporter: {
        htmlextra: {
            export: 'report' + '/report.html',
            // template: './template.hbs'
            // logs: true,
            // showOnlyFails: true,
            // noSyntaxHighlighting: true,
            // testPaging: true,
            browserTitle: 'My Newman report',
            title: 'My Newman Report',
            titleSize: 4,
            // omitHeaders: true,
            // skipHeaders: "Authorization",
            // omitRequestBodies: true,
            // omitResponseBodies: true,
            // hideRequestBody: ["Login"],
            // hideResponseBody: ["Auth Request"],
            showEnvironmentData: true,
            // skipEnvironmentVars: ["API_KEY"],
            // showGlobalData: true,
            // skipGlobalVars: ["API_TOKEN"],
            // skipSensitiveData: true,
            // showMarkdownLinks: true,
            showFolderDescription: true,
            // timezone: "US/Boston",
            // skipFolders: "folder name with space,folderWithoutSpace",
            // skipRequests: "request name with space,requestNameWithoutSpace",
            displayProgressBar: true
        }
    }
})

As you can see, in the collection and environment fields, I put the COLLECTION_PATH and ENVIRONMENT_PATH as environment variables using the process.env property.

In reporter settings in the export field, I specified the directory where Newman will save my report file. In other words, after Newman finishes its work, a new report directory will appear in the root of the project in which the report.html will be placed.

Also as users, we are able to customize report templates just for our needs. Html-extra reporter allowed us to do this by uncommenting the lines of code inside the htmlextra field.

The problem of asynchronous and its solution 💡

At this stage, we may be in for some problem related to the asynchronous execution of the collection pass and sending the API call to Telegram. It is clear that Newman will take some time to run through all the requests and nested tests, as well as generate a report. Of course, this is only a couple of seconds, but the execution of the code will hasten to send a call to Telegram without waiting for the Newman.

To avoid the problem of asynchrony and ensure sequential execution of script blocks, I had to turn to the help of the module DeAsync which turns the async function into sync. It was installed with the command:

npm i deasync

This package allowed me to use loopWhile(predicateFunc) where predicateFunc is a function that returns a boolean loop condition. So I added the following block to my code:

let done
newman.run({
//...
})
    .on('start', () => {
        console.log(`Running a collection. Please wait a few seconds...`);
    }).on('done', () => {
        done = true;
    })
    require('deasync').loopWhile(() => {
        return !done;
    })

Well, it remains only to set up API calls to Telegram.

Sending files with Fetch API 📨

At this stage, I needed to set up sending two API calls to Telegram: the first one with a notification of the successful completion of the collection running and the second one containing an HTML report. These calls were to go immediately after Newman completed his work. To implement this task, it is certainly possible to use any HTTP client libraries for Node.js which are available as npm packages (Axios, Got, SuperAgent, etc.), but I turned to the Fetch API. The following command was used to install it:

npm i node-fetch

In the code, I used the fetch() method with the usual promise syntax:

const fetch = require('node-fetch')

//Sending an information notification to Telegram:

const url1 = `https://api.telegram.org/bot${process.env.TOKEN}/sendMessage?chat_id=${process.env.CHAT_ID}&text=
Your collection has been successfully run. The results are contained in the attached report below.`

fetch(url1, {
    method: 'GET'
})
    .then((res) => res.json())
    .then((res) => {
        console.log(res);
    })
    .catch((error) => {
        console.log(error);
    });

As you can see, the constant url1 contains a string with the URL to which the request is sent. This is a standard template that is set by the Telegram API. According to this template, I included the token and chat ID inside the URL as environment variables, which I pulled from the .env file using the process.env property.

The next request turned out to be a little more difficult to implement because it had to send a report.html file inside it, which first needed to be converted using the encoding type multipart form-data. Therefore to create readable "multipart/form-data" streams, I needed to use the Form-Data library. In order to install it locally, used the following command:

npm i form-data

Based on the syntax of this library, I created a form with one field containing the file stream of the report.html file and passed it to the fetch() method to submit as the request body. The result was the following block of code:

const FormData = require('form-data');
const fs = require('fs');

//Sending html-report to Telegram:

const url2 = `https://api.telegram.org/bot${process.env.TOKEN}/sendDocument?chat_id=${process.env.CHAT_ID}`

let readStream = fs.createReadStream('report/report.html');
let form = new FormData();

form.append('document', readStream);

fetch(url2, {
        method: 'POST',
        body: form,
    }
)
    .then((res) => res.json())
    .then((res) => {
        console.log(res, 'The report was successfully sent to 
        Telegram');
    })
    .catch((error) => {
        console.log(error);
    })

Well, it's finally time to execute the written script. However, in the package.json file, I previously changed the command to run scripts by writing:

"test": "node runner.js"

When I ran the script using the command:

npm test

immediately in the terminal appeared a special green bar that indicated the progress of running the collection:

After the running was over I received two messages with notification and report.html file directly in my Telegram group "API alerts": ✉️

Great, let's open the resulting report and see what it is:

In addition, in the terminal, I received response bodies from the Telegram API containing detailed information about what was sent and where:

Instead of a conclusion 👋

In this article, I tried to describe the entire process in as much detail as possible, from exporting the Postman collection to receiving a report on its running in Telegram. I hope that repeating the steps I did will help you charge your API skills. 💪

This project with detailed installation and usage instructions is completely available here.