How To Run Browser Tests via Cypress in Your CI/CD Pipeline with AWS CodeBuild

Recently I launched my own blog. During that process, I got interested in how I can automate testing the quality of my new blog. I didn't want a heavy duty framework. I didn't want deep tests because the functionality on my blog isn't very verbose.

A basic thing I wanted to test was the mobile view of my blog versus the tablet/desktop view. I often make the mistake of changing a CSS class or HTML element and breaking the mobile layout in some way I hadn't thought about.

So, I started looking at how to simplify my browser testing. There were a few things I wanted to consider in choosing a framework.

• I wanted it to be lightweight, ideally a single npm install would be ideal.
• It needed to be easy to integrate into my AWS CodePipelie CI/CD pipeline.
• The test setup, assertions, and framework components should be sensible.

A quick evaluation of these led me to not go with the classic Selenium because I found it to be bulky the last time I used it. However, I like the idea of testing components via the browser, but I prefer it to be lightweight.

Enter stage right: Cypress.io

I first came across Cypress via a Tweet from Kent Dodds last year.

If you read the thread you will see that it was in response to users running into browser issues that unit tests just weren't catching. The user, in this case, used Jest to write their tests but were finding browser issues when they went to production.

Unit tests are fantastic, and they are a must-have for front-end and back-end services. But if we recall the testing pyramid from Martin Fowler.

We see that unit tests are the biggest blob. They should be fast and inexpensive.

But, the other blobs, Service, and UI are important as well.

What the tweet above highlights is that unit tests are a large piece of the pie, but not the whole pie.

In my mind, this is where Cypress comes into play.

I can write my unit tests using whatever framework I'm most comfortable with whether that be NUnit, Mocha, Pyunit, etc. But for front-end services like my React based blog, I need to add UI tests or end to end tests that test pieces of functionality or how things are displayed.

Writing the Cypress tests

Getting started with Cypress is very simple. We can just install it as a dev dependency in our project.

$ npm install cypress --save-dev

Once installed we can create a few new directories and our first test fixture.

$ mkdir cypress/integration/blog-verify
$ touch landing.spec.js

For my blog landing page, I am going to write the following cypress test inside of landing.spec.js.

/// <reference types="Cypress" />

context('Landing', () => {
    beforeEach(() => {
        cy.visit('https://blog.kylegalbraith.com')
    })

    it('center header on small screens', () => {
        cy.get('.container > div')
            .should('have.css', 'text-align')
            .and('match', /left/);

        cy.viewport('iphone-6+');

        cy.get('.container > div')
            .should('have.css', 'text-align')
            .and('match', /center/);
    });
});

Here I have a simple cypress test that is validating the position of some header text on my blog landing page. Notice that in the beforeEach we are navigating to my blog before each test in the fixture.

Then in the test center header on small screen I am using the cypress library with Chai like assertions to verify my results. First, I check that the CSS of my header is set to text-align: left; with a normal size browser. Then I set the browser width to a mobile view, cy.viewport('iphone-6+'); in order to verify the text changes to center.

This is a basic test in cypress. It is only scratching the surface of what can be done, so if you are looking to check more complex scenarios check out the documentation on how to interact with various elements.

Now to run the test we can execute Cypress from our node_modules folder.

$ node_modules/.bin/cypress run
Running: blog-verify/landing.spec.js...                                                

  Landing
    √ center header on small screens (4524ms)


  1 passing (7s)

  ┌───────────────────────────────────────────┐
  │ Tests:        1                           │
  │ Passing:      1                           │
  │ Failing:      0                           │
  │ Pending:      0                           │
  │ Skipped:      0                           │
  │ Screenshots:  0                           │
  │ Video:        true                        │
  │ Duration:     7 seconds                   │
  │ Spec Ran:     blog-verify/landing.spec.js │
  └───────────────────────────────────────────┘

Once a test run completes you can also view the video recordings for each test in the videos directory located in your cypress directory.

Pretty cool right? No Selenium, no Chrome browser, and no bulky setup to get going. One package install, write a test and run.

Of course, this is a basic example of what can be done with Cypress. I encourage you to explore adding integration tests that cover the use cases of your user. Examples of these could be:

• Logging into and out of your application.
• Saving and loading application data.
• Checking data is viewable in different browser sizes.
• Validating users with certain permissions can't access other parts of your application.

To wrap things up let's explore how we can incorporate these tests into your continuous integration and deployment pipeline living in AWS.

Incorporating Cypress into AWS CodePipeline and CodeBuild

If you recall from a previous blog post, I developed a Terraform module that provisions a Git repo, AWS CodePipeline, and AWS CodeBuild projects so that I can have an entire end to end CI/CD pipeline for all my new projects. For my blog, I used this module so that I can continuously deploy my blog with any new commits to master.

To run these new tests I simply need to update the buildspec.yml file in my repository. This YAML file is what tells AWS CodeBuild how to build, test, and deploy my static website.

Here is what my buildspec.yml file looks like before I update it to run my tests.

version: 0.2
phases:
  install:
    commands:
      - echo "install cypress dependencies..."
      - npm install -g gatsby-cli
      - npm install -g purgecss
  pre_build:
    commands:
      - echo "pre_build step"
  build:
    commands:
      - cd src
      - cd new-gatsby-blog
      - npm install
      - npm run-script build
      - aws s3 cp public/ "s3://blog.kylegalbraith.com/" --recursive --cache-control 'public, max-age=2628000'
      - aws cloudfront create-invalidation --distribution-id E3SMHPICHT13MG --paths '/*'
  post_build:
    commands:
      - echo "post_build step"

In order to run our tests, I am going to update the install section to add a few dependencies that Cypress relies on. I am also going to update build section to run our tests after we have deployed the site to S3 and invalidated the CloudFront cache. Therefore, think of these tests as post-deployment verifications.

Here is the buildspec.yml file after making these changes.

version: 0.2
phases:
  install:
    commands:
      - echo "install cypress dependencies..."
      - npm install -g gatsby-cli
      - npm install -g purgecss
      - apt-get update
      - apt-get install -y libgtk2.0-0 libnotify-dev libgconf-2-4 libnss3 libxss1 libasound2 xvfb
  pre_build:
    commands:
      - echo "pre_build step"
  build:
    commands:
      - cd src
      - cd new-gatsby-blog
      - npm install
      - npm run-script build
      - aws s3 cp public/ "s3://blog.kylegalbraith.com/" --recursive --cache-control 'public, max-age=2628000'
      - aws cloudfront create-invalidation --distribution-id E3SMHPICHT13MG --paths '/*'
      - node_modules/.bin/cypress run
  post_build:
    commands:
      - echo "post_build step"

Notice here inside of the install step we are running:

apt-get install -y libgtk2.0-0 libnotify-dev libgconf-2-4 libnss3 libxss1 libasound2 xvfb

This is to install the necessary dependencies that Cypress needs in order to run inside of a CodeBuild project.

Then inside of the build step, we execute the tests:

node_modules/.bin/cypress run

Now when I check in the changes my AWS CodePipeline will be triggered.

Once the build completes, I can check my AWS CodeBuild logs and see the tests executed and the build was successful.

  Running: blog-verify/landing.spec.js...                              (2 of 2) 


  Landing

    ✓ center header on small screens (697ms)


  1 passing (2s)


  (Results)

  ┌───────────────────────────────────────────┐
  │ Tests:        1                           │
  │ Passing:      1                           │
  │ Failing:      0                           │
  │ Pending:      0                           │
  │ Skipped:      0                           │
  │ Screenshots:  0                           │
  │ Video:        true                        │
  │ Duration:     2 seconds                   │
  │ Spec Ran:     blog-verify/landing.spec.js │
  └───────────────────────────────────────────┘


  (Video)

  - Started processing:   Compressing to 32 CRF
  - Finished processing:  /codebuild/output/src459391942/src/src/new-gatsby-blog/cypress/videos/blog-verify/landing.spec.js.mp4 (1 second)


================================================================================

  (Run Finished)


      Spec                                    Tests  Pass…  Fail…  Pend…  Skip… 
  ┌────────────────────────────────────────────────────────────────────────────┐
  │ ✔ blog-verify/individual_post.…   00:02      1      1      -      -      - │
  ├────────────────────────────────────────────────────────────────────────────┤
  │ ✔ blog-verify/landing.spec.js     00:02      1      1      -      -      - │
  └────────────────────────────────────────────────────────────────────────────┘
    All specs passed!                 00:05      2      2      -      -      -  

[?25h
[Container] 2018/08/12 00:59:08 Phase complete: BUILD Success: true
[Container] 2018/08/12 00:59:08 Phase context status code:  Message: 
[Container] 2018/08/12 00:59:08 Entering phase POST_BUILD
[Container] 2018/08/12 00:59:08 Running command echo "post_build step - run cypress"
post_build step - run cypress

[Container] 2018/08/12 00:59:08 Phase complete: POST_BUILD Success: true

You can see that we get a detailed breakdown of our test execution. We can even copy the video files out of our CodeBuild run if we wanted to. Very cool!

Conclusion

In this post, we talked about why end to end or integration tests are important. No one is saying you should go all in on integration/UI tests, but you should at least cover the main pieces of functionality for your application with them. They are not a replacement for great unit tests, but they are an excellent complement.

We only scratched the surface of what Cypress has to offer. I intend to explore it further to test more of my complex applications. Hopefully, this has given you an introduction to UI testing and demonstrated how it can be easily incorporated into your CI/CD Pipeline.

As always, if you have any questions please feel free to leave me a comment below or reach out to me Twitter.

Are you hungry to learn more about Amazon Web Services?

Want to learn more about AWS? I recently released an e-book and video course that cuts through the sea of information. It focuses on hosting, securing, and deploying static websites on AWS. The goal is to learn services related to this problem as you are using them. If you have been wanting to learn AWS, but you’re not sure where to start, then check out my course.