DEV Community: David🦉

How we do automated testing on our backend

David🦉 — Thu, 26 Mar 2020 11:11:15 +0000

After sharing how we do automated testing on our fronted in my previous post I wanted to share too how we test our backend services using our API. You can find the extended version of this post in our blog

API/Integration tests must be faster to develop and run since they do not require any user interface to interact with since they directly hit the endpoints designed in your application.

The platform behind

Our backend platform follows the micro-service pattern, decomposing the application in small services that take care of one isolated piece of the whole business. We follow a hexagonal architecture (aka ports and adapters) to avoid having them highly coupled and facilitating testing activities. To expose all the endpoints we make use of APIs which are consumed by our BFFs using GraphQL.

Regarding the base code, it is written in PHP and it follows the Domain driven design (DDD) pattern. With all this what we have in the end are different bounded contexts (BCs) that completely isolate each side of the business.

The test tool

To test this we opted for Gauge which is an open source tool to write and run tests.

Gauge was created by ThoughtWorks and it is supported by a wide community. It can be used in any platform and it supports many languages for writing your test code. Two key features that we liked are that it’s BDD and it supports data-driven executions, making your tests more escalable.

For writing the tests scenarios themselves Gauge uses specification and concept files. Basically, specification files are those containing the scenarios written in a set of steps and concept files are just a collection of steps that you can reuse in the scenarios.

A spec file

Create Booking Request
======================
Tags: booking

Create booking request and charge (OK)
- - - - - - - - - - - - - - - - - - - - - - 
* Login with "test-user"
* Create a booking lead for listing id "999999" with move in "2021–01–01" and move out "2021–01–31"
* Create a booking request from booking lead
* Charge booking
* Wait until the booking request is converted into booking

This one above is an example of a specification file with one scenario written in Markdown language. The scenario just describes what they do in different steps. Each one of these steps has test code behind, but they can also relate to a concept file since they can be a set of steps themselves.

A concept file

# Charge booking
* JSON from file "./bookings/chargeBookingRequest.json" is set as variable "chargeJson"
* Set path "/payment/booking-requests/{id}/chargeNow" as name "charge" replacing path variable "{id}" with variable "bookingRequest"
* An HttpSampler with name "chargeBR" and path in variable "charge" and type "PUT" and port "xxx" and json "chargeJson"
* Response code is equal to "204"

Here we show an example for the step “Charge booking” that we presented in our spec file. You see all the steps behind that one, meaning that every time we call “Charge booking” from a spec file, all these steps will run.
We usually do that when:

There are a lot of small steps required for just one high level task
We do not want to show so much logic at the spec file level
We are going to reuse such step in many test scenarios

Step implementation

Let’s show as an example the implementation for the step “An HttpSampler with name “chargeBR” and path in variable “charge” and type “PUT” and port “xxx” and JSON “chargeJson””

@Step("An httpSampler with name <name> and path <path> and type <type> and port <port> and json <json>")
public void setHttpSamplerNameTypePortBodyGiven(String name, String path, String type, Integer port, String json) {

// Java code to send an http request with the passed on parameters

}

Note the @Step annotation. It denotes that the function is a step used in the spec files.

Build

To manage and build the project we use Maven. Maven is an extensively used build automation tool from Apache used in Java projects, although it can also be used for other languages. We’ve been working with it from the very beginning and it’s been useful to us given the big tree of dependencies that we have to maintain, both in-house and external ones.

Since we’ve got all our API tests in the same place, they are all placed in the same repository. So after updating our test collection, be it adding new tests or refactoring, the build is triggered just after pushing the changes to master branch to make sure all is fine. With that, we make available the latest version of the tests for the BCs to grab and run.

Setting up the CI

If you remember from our first post, we use Brigade as our CI system and our BCs use it in the same way. For every BC, we’ve got a Brigade file to indicate the events that must run in such pipeline.
With that, GitItNow (in-house developed tool), makes quite easy to interact with the pipeline and brings visibility to the whole process.

Above there’s one example of a pipeline for one of our BCs.
For the api-tests event it gets the latest successful build from the test repository and runs it.

Worth mentioning too that we have enabled parallelization in our Ci runs to be able to run them faster and react soon in case anything goes wrong.

Summary

With all commented here, we ended up with a robust framework that enables us to track the quality of the backend code we deploy and react soon in case a red flag shows up. Is it perfect? Nope (I'd say nothing is). We still deal with collateral issues such as stuck queues or false positives due to misleading data or other kind of human errors. But there are also some other aspects such as the escalability that a BDD approach gives us, getting to a point where there are so many steps developed that we almost don't need to develop more but use what exists.

Hope you enjoyed the post 🚀

How we do automated testing on our frontend

David🦉 — Wed, 12 Feb 2020 10:38:06 +0000

And here comes my first post ⭐

In this post I'll share our learnings for testing our frontend platform at Spotahome and how we moved from Selenium to Cypress. You can find the complete post in our blog in case you want to read further.

Our front-end architecture is built up based on the BFF approach using Node + React, that combines the speed of JavaScript and uses new ways of rendering websites, making them highly dynamic and responsive to the users. For it, we make use of GraphQL to request data to the backend. GraphQL was developed to cope with the need for more flexibility and efficiency that developers experience when interacting with REST APIs, making the workflow much more efficient.

The test tool

A while ago we decided to start using Cypress, a tool that intends to overtake Selenium (or to make you forget about it right away), by ripping off all its setbacks such as the set up and debugging. If you want to dive deeper into this comparison, here you can read further.

Cypress is a testing tool built for the modern web. One can see lots of similarities among them, but the benefits and constraints you will face are quite different depending on which one you use. If what you’re seeking for is easiness of setting up and debugging, Cypress can be a pretty good match.

Cypress is an open source test runner that lets you write tests in JavaScript code. You can checkout their repository and contribute to their mission if you feel so.

In our case, we made the shift from Selenium to Cypress for several reasons. The most important one, it seemed to be pretty easy to set up and work with, and that enabled us from QA to scale testing efforts by spreading its use among QAs and devs. Since it uses JavaScript, our dev team welcomed it and started to use it almost friction-less.

The technology underneath

Cypress is written in JavaScript and lets you write tests in JavaScript. One thing to keep in mind is that Cypress is built up on top of Mocha, so if you are already used to write test scripts using Mocha as your test framework, you are likely to find some old friends here.

From Mocha, Cypress adopts its BDD syntax which fits nicely with both integration and unit testing. Another bundled technologies that Cypress comes with are Chai, Chai-jQuery, Sinon, and Sinon-Chai.

Running the tests

When running the test in GUI mode, we see the following:

What Cypress does is open a browser, then on the left you see the ‘live log’ which reflects everything the browser does during the test, and on the right you have the application under test

Worth to mention here that Cypress is not an outsider tool that communicates with the browser through an API, but that it runs directly in the browser, and this makes it to be so close to the web application under test that you can do things that you can’t do in Selenium, such as stubbing DOM APIs.

One of the key features here is the debuggability. When the test finishes or fails, the state is not reset, meaning that you can hover around the ‘live log’ to see what or where was Cypress actually doing at that moment in time -a screenshot is displayed for that moment in time for every action. You can even open the browser console and get a full description of the steps you select on the log. This feature has boosted the whole process of test development so much that we can’t go without it now.

Setting up the CI

For our CI system we use Brigade, which is an event-based tool for running automated tasks on the cloud. Our colleagues from the DevOps team wrote a series of posts about how this works, so in case you are interested you can take a deep dive here.

As commented at the beginning of the post, we split our application in different BFFs, which lets us to build and deploy them independently, making the release process swifter and more easy to scale.

All of these BFFs have at least one e2e test repository attached, that will be triggered as a different event every time a pull request is merged.

This case above illustrates the typical series of events that happen when merging a new Pull Request into our staging environment -BFF pipeline. First comes the push event (including unit tests 😍), then the deployment to the staging environment, and finally a post_deploy_hook triggers the configured e2e tests. We’ve got Slack integrated into this loop to notify us of every step and warn us in case anything went wrong.

The e2e tests are stored in a repository of their own (accounting to more than 15 repositories). We commit every change on e2e test code to those repositories to create a new Docker image that will be grabbed and executed by the BFF whenever the event comes.

Focusing on the e2e test event, the command we use to run the tests is:

$ cypress run --record --group 4x-electron --parallel --ci-build-id ${buildId}

Briefly analyzing the params:

record: Sends the video recordings to Cypress Dashboard to keep track of the runs
group : Group recorded tests together under a single run
parallel: Enables parallelization of the tests
buildId: Necessary to defining a unique build or run

Wait what? Sends the video recordings to a dashboard?

Cypress Dashboard

The last part of the journey. Cypress Dashboard is a service that allows us to access to recorded tests, being pretty useful when we run them on the CI. It gives us information about all the test runs triggered, including logs, screenshots and videos.

Even though we could go without the Dashboard, we found it pretty helpful and use it widely -we are on a paid plan that allows several of our engineers to have access to it and work in a more efficient way.

The big win here is that we can be aware of the state of the application quite fast, and we can address any issue right away whenever anything fails. Once the failure is revealed and the potential impact calculated, we either fix the issue and test it again ASAP or we snooze the failing assertion until it gets fixed, so that we don’t introduce noise in the pipeline.

This is all, I hope you find it useful! In case you had any question or suggestions they are more than welcome 💚