Continuous Integration for End-To-End Tests with Cypress and GitHub Actions

In this article, we are going to learn how to create a continuous integration process for Cypress tests. Github Actions can automate, customize and execute your software development workflows right in your repository and it also supports software project development in a variety of ways, both in Programming Language and Operating System Stack.

Continuous Integration (CI), is a software development practice which requires automating the integration of software changes from multiple contributors into a single software project. Each change is validated by creating a build and running automated tests against the build. Getting fast feedback on the impact and outcome of changes lead to better collaboration and software quality.

This article will cover the following:

• How to create your first GitHub Actions to run Cypress tests on a hosted website.
• Schedule GitHub Actions Jobs to run at a given interval.
• How to setup GitHub Actions to run Cypress tests for a local project.
• Send test results to Cypress dashboard for monitoring.

Cypress team has open sourced an action to make running Cypress end-to-end on GitHub easy. Let’s get started.

Prerequisites

The only required prerequisites is that you should have some experience writing Cypress tests. No prior GitHub Actions experience is required.

Setting up GitHub Actions to Run Cypress Tests on a Hosted Website

A workflow is a configurable automated process made up of one or multiple Jobs (Jobs will be explained further down this post). Workflow files must be stored in .github/workflow directory of your repository and they are defined using YAML syntax (.yml or .yaml file extension). If you are new to YAML and want to learn more, I would recommend this Learn YMAL in five minutes.

Cypress doesn’t recommend running tests on a deployed website Instead, tests should be part of your code project and should live in the same repository. This way it is more efficient to manage and ship code using CI.

First, you need to create a .github/workflows folder with a main.yml file in the project directory and add these code snippets into the main.yml file.

name: This is the name of our workflow. GitHub is going to display this on our repository’s action page.

    name: E2E tests

on: This is the event that triggers the workflow. This can be a single event, an array of events or an event configuration map that schedules. In our case here, this triggers the workflow on push or pull request but only for the develop branch. We are also specifying an array of different activity types for our pull request event.

    on:
      push:
        branches:
          - develop
      pull_request:
        types: [opened, synchronize, reopened]
        branches:
          - develop

jobs: We now know that a workflow can be made up of one job or a collection of jobs. Jobs run at the same time in parallel (default) or sequentially depending on the status of a previous job. Jobs run in their own environments specified by runs-on and here we are using the latest version of ubuntu. For each job, a virtual machine is spawned and other jobs won’t be able to access it.

    jobs:
      cypress-run:
        runs-on: ubuntu-latest
        timeout-minutes: 10

steps: A job is made up of small sub-tasks called steps. In each step, we can run commands, run setup tasks or run an action in our public repository. Not all steps run an action but all actions run as a step. As seen below, we have three steps:

    steps:
        - name: Checkout
          uses: actions/checkout@v1
        - name: Setup npm package
          run: npm install
        - name: Run E2E test
          uses: cypress-io/github-action@v1
          with:
            config: baseUrl=https://seedxb.com

1. The name field is what GitHub displays as the name of the step inside the actions tab.

2. The checkout action is is going to clone the repository inside the workspace directory in GitHub action environment.

3. The Setup npm package as the name implies will run npm install on the cloned repository and install all our dependencies and The run field specifies a Bash command to run for this step.

The last step, Run E2E test will use Cypress GitHub actions to run our tests for us. We do not need to explicitly specify the run command for the test.

With This field specifies options to configure the action. Actions are like functions and they can define parameters to influence their behaviours dynamically. Here, our action parameter is defined using with which is a key/value map. In our case, we are passing a config parameter using with .

we can use public actions such as actions/checkout@v1 or run command line programs, as we do here with npm install.

Now you can push to your repository and you’ll see a new tab under the repository called Actions, click it.

Schedule GH Actions Jobs to run at a given interval

We can also setup our workflow to run on a schedule using POSIX cron syntax independent of any activity on the repository. The schedule event lets us define a schedule for our workflow to run on. As seen below, the scheduled event here gets triggered at 9am daily. GitHub provides detailed documentation on scheduled events here.

    on:
      # push
      # pull_request
      schedule:
        - cron: '0 9 * * *'

Now this is what our code should look like: -

    name: E2E tests
    on:
      push:
        branches:
          - develop
      pull_request:
        types: [opened, synchronize, reopened]
        branches:
          - develop
       schedule:
         - cron: "*/5 * * * *"
    jobs:
      cypress-run:
        timeout-minutes: 10
        runs-on: ubuntu-latest
        steps:
          - name: Checkout
            uses: actions/checkout@v1
          - name: Setup npm package
            run: npm install
          - name: Run E2E test
            uses: cypress-io/github-action@v1
            with:
              config: baseUrl=https://seedxb.com

How to Setup GitHub Actions to Run Cypress Tests for a Local Project (Optional but Recommended)

The workflow is basically the same as seen above the only difference is that we are going to add a wait-on parameter that checks and waits for our local server to be up and running before running tests. By default, wait-on will retry for 60 seconds.

    name: Blog tests
    on: [push]
    jobs:
      cypress-run:
        runs-on: ubuntu-latest
        steps:
          - name: Checkout
            uses: actions/checkout@v1
          - name: Run blog test
            uses: cypress-io/github-action@v1
            with:
              start: npm start
              wait-on: 'http://localhost:3000'
              config: baseUrl=https://localhost:3000

I’ve stripped off some lines so you can focus on what is important for testing locally.

Notice how we’ve updated the baseUrl in the config parameter to point to a localhost endpoint. Ensure that you have an app running at port 3000 which you intend to test.

Send test results to Cypress dashboard for monitoring

Cypress dashboard is an option that provides details and visual representation of our test runs, their reports, status, and events that took place during execution. It basically gives you details of what happened when your tests ran.

The Cypress dashboard service is very useful when you are running test cases in a CI environment. We won’t be covering benefits or features provided by Cypress dashboard service in this article but for more information on that, click here.

How to setup a Cypress dashboard
Open the test runner and click on the runs tab of your project. Of course there will be no record of test runs visible because we are yet to set it up. Now click on the Connect to dashboard button.

You will be prompted to log in to the Cypress dashboard to be able to record your test. To do that, you can log in with your Google or Github account. You should see a success message after a successful login and you will be asked to provide the following details as seen below.

1. You can give your project a name - this can be changed anytime.
2. Select who owns the project - You can personally own it or create an organisation. I have taken ownership as seen above.
3. Finally, you need to decide if you want to make it a private or public project - as seen above, I have provided a public access to mine.

After providing the details, you can click the Set up project button. You should see a view explaining how to record your first run.

Note that your key MUST be confidential and private to you. I left mine for demo purpose and has deactivated it.

Recording Test Runs To your Dashboard
Firstly, Cypress will, by default, auto-populate your cypress.json file with a unique project ID as seen above. You can check your cypress.json file to verify this.

Secondly, we won’t be running the above mentioned command in the terminal, we are going to set the record key as an environmental variable in the project’s repository on Github. Click on the Settings button.

Click on the Secrets tab on the left hand menu to navigate to the Secrets page then click on New secret button on the right hand side of the screen.

You should see a screen as shown below. Type in the name of your secret then copy the value of your record key from the runs tab of your test runner and paste it in the value field then click the Add secret button.

Update your workflow configuration to include a record field and a CYPRESS_RECORD_KEY env variable.

     steps:
        # - name: Checkout
        # - name: Setup npm package
       # - name: Run E2E test
        #  uses: cypress-io/github-action@v1
          with:
             start: npm start
             wait-on: 'http://localhost:3000'
             config: baseUrl=http://localhost:3000
            with:
              record: true
            env:
              CYPRESS_RECORD_KEY: ${{ secrets.CYPRESS_RECORD_KEY }}

record: This tells Cypress to take video recording, screenshots and logs of all our test runs.
We are also passing our dashboard record key as an environment variable here. Cypress action by default runs the cypress run command for us. In addition to that if it sees a CYPRESS_RECORD_KEY and a record property set to true, it will automatically start recording the test.

Now we are all set!! Push to GitHub and as soon as the tests finish running, you should be able to see the details of the tests on the dashboard via the Runs tab of the test runner.

If you click on any record, it’s run will open in chrome. You’ll see a detailed record of test runs, number of test cases that passed or failed, videos, screenshots (for failed test cases), operating system on which the tests ran, duration, etc as shown below.

Yayyyyy!!! We finally have our dashboard up and running. If you made it to this point, CONGRATULATION!!

Comments

[devsrihari4]

Nice article. In my project we are using GitHub self-hosted runner. We have only one VM with self-hosted runner. How can I run Cypress tests in parallel on self-hosted runner. Appreciate your time.