Continuous delivery is widely popular practice today. Product team wants to release new features as soon as possible focusing on the minimum viable product (MVP). The idea is to get faster feedback. Also, any bug reported from the clients should be addressed asap to keep customers happy. Continuous delivery is the ability to make this happen while ensuring high quality, low risk and reliable releases. This allows team to deploy multiple releases in a week or even day.
One of the core foundations of continuous delivery is automated tests. These are the set of tests that are run as part of the continuous pipeline to ensure the code is working as per expectations. Some people consider only functional tests as automation tests. However, automation tests are any kind of tests written that can be run automatically e.g. unit, integration, api, functional, load tests etc. With so many types of tset, one question comes is how should we employ our testing strategy to ensure bug free release and how the strategy is relevant in microservices world.
Microservice Architecture
Microservice architecture is an increasing popular architecture to build software. It proposes to create smaller services with well defined boundaries that allow teams to concentrate on their service only and release faster. The idea is to be able to independently release your service. While microservice is ideal for continuous delivery, one of the major challenge is most of the time any functionality modification depends on changes in multiple services. Thus we need to ensure that services once integrated doesn’t lead to any regression issue. Traditionally, companies have relied on comprehensive functional test suite and manual testing to verify end to end behavior but does that strategy work in case of microservices? Let’s discuss some of the major issues with this and how can we employ continuous testing to achieve continuous delivery.
Issues with large functional suite
Most companies focus on building high quality comprehensive functional suite for regression testing. Functional suite tests user journey and verify that product end to end is working as per expectations. No doubt functional tests provide maximum confidence on the build and are must to have. However, there are couple of issues on heavily relying on them.
- Run at end of cycle: Functional tests run at the end of cycle. Due to this many of the integration and functional issues are identified only at later stage. This leads to delayed release. One of the principle of continuous delivery is to fail early which should be employed for timely and fast releases.
- Time consuming: Functional tests take lot of time to run since they run in browser and cover user journey. Also, many of the actions are async in nature in which case there is lot of wait time involved. I have seen functional suites running even for a day.
- Flakiness: The other issues with functional suite is they can fail due to many reasons outside of our control like slow network etc. and this leads to flakiness in the tests.
Thus relying only on functional tests should be avoided. Instead, teams should employ test pyramid idea.
Test Pyramid
Test Pyramid is not a new concept but still many companies struggle to adopt it. Test pyramid suggests to have tests at different granularity and the more high level you go in the pyramid, the lesser tests you should have. So have lot of unit tests to cover the whole functionality. Then you should have good amount of integration and service tests to test layer integration and finally have smaller number of functional tests to cover end to end functionality. These tests are run at different stages in a pipeline and overall certify build.
When we consider microservices, we first need to ensure that our service is self-tested. Having a good quality unit and integration tests help us in this regard. But usually we are mocking external services in these tests. One of the critical piece in microservice is api tests. Api tests can cover individual service or a cluster of service together. These allows us to ensure that service integration works fine and the contract is not broken. It is also critical when the service is a public service. Finally we have functional tests to cover the end to end user journeys. These tests should ensure that the major use cases are tested when a new release is going live.
In the next part, I will discuss what each of these tests should cover and how they work together to achieve faster and bug free releases.
Top comments (0)