DEV Community: Murillo Welsi de Souza Pereira

API Testing with Cypress: Part 3 - CI pipeline on Github Actions

Murillo Welsi de Souza Pereira — Mon, 23 May 2022 06:50:24 +0000

Hello guys!

It's been a while since I posted parts 1 and 2 of this article, and I was owing part 3.

This time our goal will be to put the tests to run in a Continuous Integration pipeline, after all, we want it to be an automated test. So let's dive into that.

Access your GitHub repository

The first step, if you haven't read parts 1 and 2 yet, hurry up, it's super easy and that way we'll make sure we're on the same page.

In part 1, I showed you how you can create your repository on GitHub and some of the most basic Git commands to upload your local data to the repository created in your account.

Assuming you already have this repository created under your account, access it. In my case, the link is:

https://github.com/murillowelsi/cypress-api-tutorial

Make sure your local repository is synced with the remote repository.

Creating your first workflow

Create a .github/workflows directory in your repository on GitHub if it does not already exist.

In the .github/workflows directory, create a file named ci.yml. You can do this manually or simply by executing the following command:

mkdir -p .github/workflows && touch .github/workflows/ci.yml

Now you should be able to see this directory structure in your VsCode:

Editing and understanding the YML file

Each step of our pipeline will be added inside the configuration file, and when the ci.yml file is pushed to the remote repository, GitHub will automatically identify that there is a workflow to be executed and the tests will be run.

This is going to be a bit boring, I know, but let's increase the yml little by little so that you understand what each step is intended to do. Be patient, mate!

name:

We can give a name to the workflow, it will appear in the Actions tab of the GitHub repository.

name: Cypress Tests

on:

It specifies the trigger for this workflow. This example uses the push event, so a workflow run is triggered every time someone pushes a change to the repository or merges a pull request. This is triggered by a push to every branch

name: Cypress Tests

on: [push]

jobs:

Groups all the jobs that run in the ci.yml workflow.

name: Cypress Tests

on: [push]

jobs:

cypress-run:

Defines a job named cypress-run. The keys within it will define the properties of the job.

name: Cypress Tests

on: [push]

jobs:
  cypress-run:

runs-on:

Configures the job to run on the latest version of an Ubuntu Linux runner. This means that the job will execute on a fresh virtual machine hosted by GitHub.

name: Cypress Tests

on: [push]

jobs:
  cypress-run:
    runs-on: ubuntu-latest

steps:

Groups together all the steps that run in the cypress-run job. Each item nested under this section is a separate action or shell script.

name: Cypress Tests

on: [push]

jobs:
  cypress-run:
    runs-on: ubuntu-latest
    steps:

step - Checkout

The uses keyword specifies that this step will run v3 of the actions/checkout action. This is an action that checks out your repository onto the runner, allowing you to run scripts or other actions against your code.

name: Cypress Tests

on: [push]

jobs:
  cypress-run:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout
        uses: actions/checkout@v3

step - Checkout

This step uses the cypress-io/github-action@v2 action. The Cypress team maintains an official Cypress GitHub Action for running Cypress tests. This action provides npm installation, custom caching, additional configuration options and simplifies setup of advanced workflows with Cypress in the GitHub Actions platform.

name: Cypress Tests

on: [push]

jobs:
  cypress-run:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout
        uses: actions/checkout@v3

      - name: Cypress run 🧪
        uses: cypress-io/github-action@v2

For now, this is all we need to run the tests in the pipeline. Let's then commit the new files and upload them to GitHub:

git add .github/workflows/ci.yml
git commit -m "Added github actions workflow"
git push origin main

At this point, if we access the repository on GitHub and click on the Actions tab, we should be able to see that the actions were triggered by the commit as we wanted.

If we click on the running task, we can see the logs of all the commands that were executed

And finally, if we scroll down, we'll see the tests and their status (if everything went well, you'll see green, otherwise I feel sorry for you).

Reporters

To finish this article in style, let's make our CI pipeline even fancier by adding reports to it.

Cypress is built on top of Mocha, so any reporter built for Mocha can be used with Cypress.

Let's install some dependencies that will help us generate these reports. Run this command in your terminal:

npm install cypress-multi-reporters mochawesome mochawesome-merge mochawesome-report-generator rimraf -D

Create a configuration file for the reports in the root folder of your project:

touch reporter-config.json

In this file, paste the following content:

{
  "reporterEnabled": "mochawesome",
  "mochawesomeReporterOptions": {
      "reporterDir": "mochawesome-report",
      "quiet": true,
      "overwrite": false,
      "html": false,
      "json": true
  }
}

Add the following scripts to package.json. It will help us generate the reports:

"scripts": {
    "cy:open": "npx cypress open",
    "cy:run": "npx cypress run",
    "report:merge": "mochawesome-merge > index.json",
    "report:mocha": "marge index.json",
    "report:clean": "rimraf mochawesome-report index.js"
  },

To test Mochawesome reports, first run the tests:

npm run cy:run

Then run the following commands:

npm run report:merge
npm run report:mocha

report:merge -> will merge results from all specs
report:mocha -> will generate the report

To clean the results of the previous run:

npm run report:clean

Note that now the reports were generated inside the mochawesome-report folder, as configured in the json.

Open the index.html file with any browser and see the run results

Publishing the reports to GitHub Pages

Cool, we are already generating the reports locally, but we want more. Let's now publish them on a GitHub Page totally for free.

First let's increment our pipeline to generate the reports:

name: Cypress Tests

on:
  push:
  schedule:
    - cron: '0 0 * * 1'

jobs:
  cypress-run:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout
        uses: actions/checkout@v3

      - name: Cypress run 🧪
        uses: cypress-io/github-action@v2
      - run: npm run report:merge
      - run: npm run report:mocha

      - name: Reports deploy 🚀
        uses: JamesIves/github-pages-deploy-action@v4.3.3
        with:
            BRANCH: gh-pages
            FOLDER: mochawesome-report

Note that we now have a schedule key and cron.

The cron expression '0 0 * * 1' basically means that tests will be run weekly at 00:00 on Mondays.

Check https://crontab.guru/ to better understand how cron expressions work.

Now we also have a new step called Reports deploy. In this step we will use an action responsible for deploying the results of our tests on a GitHub Page in the gh-pages branch.

Let's commit and push this to the repository:

git commit -am "Added step to deploy result to gh-pages"
git push origin main

The pipeline will run, but the reports will not be available yet. Our last step will be to configure the branch in the repository.

In your GitHub repository, under your repository name, click Settings.

In the "Code and automation" section of the sidebar, click on Pages. Select the gh-pages branch and Save.

Check that another action called pages-build-deployment has been run:

To see your published site, under "GitHub Pages", click your site's URL:

...aaand BOOYAHH! Reports are available online, free of charge, and will be generated every time the tests are run:

Thanks for reading this post this far, and who knows, maybe we'll see you in part 4? I accept suggestions for topics for a future article. See ya!!

API Testing with Cypress: Part 2 - Creating your tests

Murillo Welsi de Souza Pereira — Sun, 23 May 2021 18:13:00 +0000

Hello again guys!!

Moving forward with the subject we discussed in the previous post, let's focus on to the part that is most important to us, the tests.

HTTP Methods

The resources of a microservice based applications can be manipulated in several ways. It is possible to create, update, delete them, and do other operations as well.

When we send a request for a service, we need to inform the URI to identify which resources we intend to handle. We also need to inform the type of manipulation we want to do on the resource. For this, we will use the HTTP protocol methods.

The HTTP protocol has several methods, each one has a different function. The most used are:

GET: retrieve data for a resource.
POST: create a new resource.
PUT: make a change to a specific resource that already exists.
PATCH: partially update a given resource.
DELETE: delete a particular resource.

Users Endpoint

Back to our target API - ServeRest, let's start by testing the user's endpoint:

Goals

Use the GET method in /usuarios to retrieve all users data.
Validate query params.
Validate status code.
Validate the content of the response body.

So, move your asses. Let's do it!

We will remove all the contents of the previous post from the Integration folder. That’s where our tests should be, so we’ll just leave what’s needed.

You can remove it manually or use the command below if you are lazy like me:



rm -rf cypress/integration/*

Now we are going to create a folder called Usuarios, which we will use to store all requests related to the User's endpoint. This will keep our code organized.

Inside the Usuarios folder, we will create our first test spec, called GETusuarios.spec.js

Now the project structure should look like this:



├── fixtures
├── integration
│   └── Usuarios
│       └── GETUsuarios.spec.js
├── plugins
│   └── index.js
├── support
│   ├── commands.js
│   └── index.js
└── videos

GET

GETUsuarios.spec.js

Let's add the basic Mocha organization structure. You can use BDD style if you want:

describe - Given -> Test Suite name
context - When -> Test Inner Suite name
it - Then -> You should place your tests here!



/// <reference types="cypress" />

describe('Given the Users api', () => {
  context('When I send GET /usuarios', () => {
    it('Then it should return a list with all registered users', () => {
      // place your tests here
    });
  });

  context('When I send GET /usuarios passing id query param', () => {
    it('Then it should return only the filtered user', () => {
      // place your tests here 
    });
  });
});

First Test Case

Now let's use Cypress method cy.request to make the HTTP request:

method - GET
url - API address + endpoint ! ```javascript

cy.request({
method: 'GET',
url: 'https://serverest.dev/usuarios'
})


After that we will call the `.should` function, which I mentioned in the previous post. It will enable us to make multiple assertions on the yielded subject - **response** in this case.

```javascript


.should((response) => {
  // all your assertions should be placed here!!
});

Let's add a log to see what the 'response' is returning in the body:



cy.log(JSON.stringify(response.body))

In short, cy.log will access the property 'body' of the 'response'. The JSON.stringify function will be used to transform the response body into a string.

Run the command cypress:open and check what the log returns to us.



npm run cypress:open

Awesome! We can conclude that our call is working correctly, as we are receiving the response body correctly and 200 status code (success).

Let's remove cy.log (we don't want trash in our tests) and add some status code and response body assertions.

We can validate the easiest one first, the status code:



expect(response.status).to.eq(200)

Cool! The above code means that we expect the status code of the response to be equal to 200.

We can assert that the quantidade (quantity) key will always have the same number of the usuarios (users) array. Lets add this validation:



expect(response.body.quantidade).to.eq(response.body.usuarios.length)

Let's validate that the email from usuarios field should not be null...

Hmm, but usuarios is a list with three elements. So how do we do that?

We need to pass which index from usuarios list we want to access to do this. For now, let's access the first object in the list, passing 'usuarios[0]':



expect(response.body.usuarios[0].email).to.not.be.null

Great, it worked! But what can we ensure that the 'email' key of all objects within the usuarios array is not null?

Maybe duplicate the previous row, just changing the index?



expect(response.body.usuarios[0].email).to.not.be.null
expect(response.body.usuarios[1].email).to.not.be.null

Yeah, it might even work. BUT what if we had a thousand of users inside this array, would we add a thousand lines to our code?

I don't think so.

To do a smarter assertion, we can use Cypress loadash, which offers us the .each() function:



Cypress._.each(response.body.usuarios, (usuario) => {
  expect(usuario.email).to.not.be.null
})

This function work almost like a forEach(), going through the array and making the assertions now in each 'usuario' object from 'usuarios' array.

Let's take the opportunity to add one last validation to usuarios. We expect each object to have all the keys ('nome', 'email', 'password', 'administrador', '_id'):



expect(usuario).to.have.all.keys('nome', 'email', 'password', 'administrador', '_id')

Second Test Case

Moving to the next test case, we will send the same request as before, but this time passing a query string to filter only one user by _id:

Adding a validation to ensure that the name is always correct:



context('When I send GET /usuarios passing id query param', () => {
  it('Then it should return only the filtered user', () => {
    cy.request({
      method: 'GET',
      url: 'https://serverest.dev/usuarios',
      qs: {
        _id: '0uxuPY0cbmQhpEz1'
      }
    })
      .should((response) => {
        expect(response.status).to.eq(200)
        expect(response.body.usuarios[0].nome).to.eq("Fulano da Silva")
      });
  });
});

cypress.json - baseUrl

We are repeating the url parameter in both cy.request(). Add the following lines to your cypress.json file, so that it is not necessary to repeat this information.

Set video as false as well. We don't want to Cypress record it for us.



{
  "baseUrl": "https://serverest.dev",
  "video": false
}

Okay, that was a good start and now our code looks like this:



/// <reference types="cypress" />

describe('Given the Users api', () => {
  context('When I send GET /usuarios', () => {
    it('Then it should return a list with all registered users', () => {
      cy.request({
        method: 'GET',
        url: '/usuarios'
      })
        .should((response) => {
          expect(response.status).to.eq(200)
          expect(response.body.quantidade).to.eq(response.body.usuarios.length)
          Cypress._.each(response.body.usuarios, (usuario) => {
            expect(usuario.email).to.not.be.null
            expect(usuario).to.have.all.keys('nome', 'email', 'password', 'administrador', '_id')
          })
        });
    });
  });

  context('When I send GET /usuarios passing id query param', () => {
    it('Then it should return only the filtered user', () => {
      cy.request({
        method: 'GET',
        url: '/usuarios',
        qs: {
          _id: '0uxuPY0cbmQhpEz1'
        }
      })
        .should((response) => {
          expect(response.status).to.eq(200)
          expect(response.body.usuarios[0].nome).to.eq("Fulano da Silva")
        });
    });
  });
});

Keep in mind that we are sending two requests to the same endpoint. This will need to be refactored, but we will come back to this case later.

POST

POSTUsuarios.spec.js

Moving on to the next HTTP method, we will now create a new file called POSTUsuarios.spec.js. In this file we will put all tests related to the POST method.

Create the test structure using the Mocha functions, exactly as we did in the GET file. Obviously, make changes to the descriptions according to the scenario describe, context and it.



/// <reference types="cypress" />

describe('Given the Users api', () => {
  context('When I send POST /usuarios', () => {
    it('Then it should create a new user', () => {
    });
  });
});

This time the cy.request() function will be slightly different.

The method will be POST.
The url will remain the same as /users (no need to add the complete url, just add the resource).
In the body param we'll add the necessary information to create a new user. To find out what information is needed, always consult the API documentation at ServeRest.

Our payload will look like this:



body: {
  nome: "Dumb John",
  email: "dumb.john@qa.com.br",
  password: "test",
  administrador: "true"
}

Let's take the opportunity to validate the success message and status code.



/// <reference types="cypress" />

describe('Given the Users api', () => {
  context('When I send POST /usuarios', () => {
    it('Then it should create a new user', () => {
      cy.request({
        method: 'POST',
        url: '/usuarios',
        body: {
          nome: "Dumb Joe",
          email: "dumb.joe@qa.com.br",
          password: "test",
          administrador: "true"
        }
      })
        .should((response) => {
          expect(response.status).eq(201)
          expect(response.body.message).eq("Cadastro realizado com sucesso")
        });
    });
  });
});

Success! Are you sure? Try to run the test again.

When we run the test again it fails, but why? This is because, according to the business rule, it is not allowed to register a user with an already used email. Let's use a strategy to solve this problem.

Let's edit the file located in plugins > index.js

We will use lib Faker to create a different user for each request. Install faker using the npm command:



npm i -D faker

I named this task as freshUser() and inside it we will pass key/value using faker lib:



freshUser() {
  user = {
    nome: faker.name.firstName(),
    email: faker.internet.email(),
    password: faker.internet.password(),
    administrador: "true"
  };
  return user;
}

The complete file will look like this:



/// <reference types="cypress" />

const faker = require("faker");

/**
 * @type {Cypress.PluginConfig}
 */

module.exports = (on, config) => {
  on("task", {
    freshUser() {
      user = {
        nome: faker.name.firstName(),
        email: faker.internet.email(),
        password: faker.internet.password(),
        administrador: "true"
      };
      return user;
    }
  })
  return config
}

Now refactoring POSTUsuarios.spec.js to usefaker. Create a variable called fakeUser.



let fakeUser;

In order to create a different payload before each execution, we will use a hook called beforeEach(). Within this hook, we will call the function cy.task() throwing user created by the task to the fakeUser variable.



beforeEach(() => {
  cy.task('freshUser').then((user) => {
    fakeUser = user;
    cy.log(JSON.stringify(fakeUser))
  });
});

I also added a cy.log() to see the user being created for each request. REMOVE THIS FROM YOUR CODE.

Change the body parameter to receive fakeUser variable as well.



cy.request({
  method: 'POST',
  url: '/usuarios',
  body: fakeUser
})

The refactored file looks like:



/// <reference types="cypress" />

let fakeUser;

describe('Given the Users api', () => {
  beforeEach(() => {
    cy.task('freshUser').then((user) => {
      fakeUser = user;
      cy.log(JSON.stringify(fakeUser))
    });
  });

  context('When I send POST /usuarios', () => {
    it('Then it should create a new user', () => {
      cy.request({
        method: 'POST',
        url: '/usuarios',
        body: fakeUser
      })
        .should((response) => {
          expect(response.status).eq(201)
          expect(response.body.message).eq("Cadastro realizado com sucesso")
        });
    });
  });
});

We can now run the tests as many times as we want.

I will finish this post now, because we already have a lot of information here!

Please keep in mind that just reading this post will not make you learn. Make your own code, practice, fail and fail again until you succeed!

Check this project repository on Github

murillowelsi / cypress-api-tutorial

Repository created in the article REST API Tests with Cypress

cypress-api-tutorial

Repository created in the article REST API Tests with Cypress

View on GitHub

Feel free to add me on LinkendIn. Leave your comments, questions and suggestions.

Thanks for your attention, see you on next post!!

API Testing with Cypress: Part 1 - Setup

Murillo Welsi de Souza Pereira — Mon, 17 May 2021 17:26:33 +0000

Hello folks!!

It’s been a long time since I last wrote something here. A few days ago I was thinking about something that might be interesting to share with you about REST APIs automation testing. So the idea came to me:

Why not use Cypress to create a complete API testing project > from scratch? We can use TDD to create the test, do all the > necessary validations and put this into a CI pipeline!

So lets go for it!

I will split this project into three parts, so that the content does not become too boring.

1st part: we will set up the project and a brief introduction to the integration tests.
2nd part: we will only focus on the tests techniques.
3rd part: we will focus on putting this project in an CI pipeline on Github Actions.

But why should we use Cypress?

Cypress is an user-friendly testing framework, easy to create your automation and no configuration is required to start writing your tests. All the unit testing tools that you are already familiar with are included as well.

With Cypress you can do:

e2e tests
unit tests
component testing
API testing
visual regression testing
and you can even combine them all

Prerequisites

The first thing we need to do is to setup our environment. So here are some things that you should have to start this project:

VSCode: https://code.visualstudio.com/download
Git/Github: https://github.com
NPM: https://www.npmjs.com/get-npm
NodeJS: https://nodejs.org/en/download

Github

First of all, let’s create a repository on github, it will be responsible for storing and handling the versioning of our test code. We should put a repository name, add brief description, a README and a gitignore:

After that, we will clone this repository on our local machine. Fisrt, copy your repository address:

Then type in your terminal the git clone command and paste YOUR repository address. In my case, the command will be:

git clone https://github.com/murillowelsi/cypress-api-tutorial.git

Open the cloned repository in your VSCode and let’s start setting up the project.

Creating a Cypress project

Now we’ll start a node project (default settings) by using the npm command:

npm init -y

Then we will install Cypress in the project as a dev dependency:

npm i cypress -D

Now let’s add the following lines to the package.json file, script section:

"scripts": {
     "cypress: open": "./node_modules/.bin/cypress open",
     "cypress: run": "./node_modules/.bin/cypress run --spec ** / *. spec.js"

The package.json will look like this:

To start the Cypress project run the command:

npm run cypress:open

After executing this command, a sample Cypress project structure will be created:

We will not use this samples in this tutorial, so let's remove the entire examples folder:

rm -rf cypress/integration/examples/

With the Integration folder clean, let’s move on to the next step.

Creating our first test

Within Integration we will create a file called test with the spec.js extension — to be identified and executed as a test spec, and we will start to write our first test.
Mocha offers us the following test organization structure:

describe('Mocha’s interface', () => {
  context('it provides a way to keep tests easier to read and organized.', () => {
    it('This is your test case', () => {
      expect(true).to.eq(true)
    });
  });
});

A spec has three main building blocks that you can see above. This will make it easier to read the tests:

describe() function is a way to group tests in Mocha.
context() function is another level of grouping tests according to the test context.
it() function is used to put it's title in a human-readable way to describe the test case.

Let’s save the test.spec.js file and run the command:

npm run cypress:run

We can also force an error to see how it will be displayed in terminal:

describe('Mocha’s interface', () => {
  context('it provides a way to keep tests easier to read and organized.', () => {
    it('This is your test case', () => {
      expect(true).to.eq(false)
    });
  });
});

Target API

We will use the Serverest API project as the target of our tests. This API is an awesome project idealized by Paulo Gonçalves, and mantained by the brazilian opensource community, which allows us to test the most used HTTP methods.

You can check the documentation in https://serverest.dev/.

Now let’s create a new file called GETProdutos.spec.js and make the first call to the target API, using the GET verb in the /produtos route, in order to get a list of products.
To make an HTTP request we must use the cy.request() function, passing the method and url parameters:

Syntax:

cy.request(method, url)

method (String): GET,POST, PUT, DELETE. If no method is defined, Cypress uses the GET method by default.
url (String): The URL to make the request to.

/// <reference types="cypress" />

describe('Products api', () => {
    context('GET /produtos', () => {
        it('should return a list with all products', () => {
            cy.request({
                method: 'GET',
                url: 'https://serverest.dev/produtos'
            })
                .should((response) => {
                    cy.log(JSON.stringify(response.body))
                });
        });
    });
});

The .should() function is modeled identically to the way Promises work in JavaScript. Whatever is returned from the callback function becomes the new subject and will flow into the next command.
Passing a function to .should() enables you to make multiple assertions on the yielded subject. For now we will not make any assertions, we will just throw the result on the screen using cy.log().

Let’s run the cypress:open command this time to check the results directly in the browser:

npm run cypress:open

The following window will be displayed. Then click on the test spec that you want to run:

Cypress will open the browser, run the tests and show the results on the screen:

Finally, let’s add some assertions inside our it() function, so it becomes a real test.

To validate that the status code we want is actually 200, add the following line:

expect(response.status).to.eq(200)

Access the products endpoint directly in your browser (https://serverest.dev/produtos) to see what kind of assertions we can make in the response body.

We also can say that we expect that the ‘quantidade’ key should always be 2, since the registered data is removed daily in the ServeRest project. So the size of the produtos array will also be 2.

Add these two validations:

expect(response.body.quantidade).to.eq(2)
expect(response.body.produtos.length).to.be.eq(2);

And as a last validation, we can also say that the objects within the product array must contain the keys ‘name’, ‘price’, ‘description’, ‘quantity’, ‘_id’.

expect(response.body.produtos[0]).to.have.all.keys(
  'nome', 'preco', 'descricao', 'quantidade', '_id'
)

Our test will look like this:

/// <reference types="cypress" />

describe('Products api', () => {
    context('GET /produtos', () => {
        it('should return a list with all products', () => {
            cy.request({
                method: 'GET',
                url: 'https://serverest.dev/produtos'
            })
                .should((response) => {
                    expect(response.status).to.eq(200)
                    expect(response.body.quantidade).to.eq(2)
                    expect(response.body.produtos.length).to.be.eq(2);
                    expect(response.body.produtos[0]).to.have.all.keys(
                      'nome', 'preco', 'descricao', 'quantidade', '_id'
                    )
                });
        });
    });
});

If we run the tests again, we will see that all assertions are passing successfully.

Success! We wrote our first API tests using Cypress.

We should not forget to send our code to Github, after all we want to save and version the project.

The purpose of this article is not to show you how to use git but let’s see some commands that can help you push your code to Github.

Let’s add some things to our .gitignore file that we don’t want to send to the remote repository:

# Dependency directories
node_modules/
videos/
screenshots/
package-lock.json

Run the command git add to add all files to the staging area on the terminal.

git add .

Use git status to display the list of changed files to be committed.

git status

Then run the git commit that will create a snapshot of the changes and save it to the git directory. We use the -m parameter to add a message to the commit, describing what was changed.

git commit -m "First cypress project commit"

Finally we are going to push the changes from our local repository to the remote repository using git push.

git push origin main

If we check our repository on Github, we will see all the changes we’ve made in the main branch:

This is the link to my github repository, just in case you want to take a look:

https://github.com/murillowelsi/cypress-api-tutorial

So we’ve just finished the first part of this series of articles that I want to share with you guys. In the next one, we will refactor this test project to make it more reliable and robust.

See ya next time!

So, what do you think? Do you do anything differently when creating your projects?

Leave your comment! Thanks!