E2E Testing is difficult, definitely one of the most difficult challenges within front-end development. UI tests tend to be flaky, but depending on your risk assessment you probably want to have the confidence E2E tests provide. But have you considered Consumer-Driven-Contract Testing? It is an automated testing approach to test the API integration of distributed systems, promising fast feedback and eliminating flaky integration tests. One tool providing these benefits and enabling you to write Consumer-Driven Contract Tests is Pact.
Spoiler: This is the first part of a series of blog posts taking an in-depth look into Consumer-Driven Contract Testing with Pact particularly for JavaScript Developers.
What is Consumer Driven-Contract Testing?
Splitting the word "Consumer-Driven Contract Testing" apart helps in understanding this testing approach. Let's first have a look at the term "Contract Testing". A Contract in this instance refers to some kind of specification describing the behavior of an HTTP based API. This specification can be in different formats and depends on the concrete implementation of Contract Testing. You can think of it as some kind of swagger.json or an OpenAPI specification if you are familiar with those. Usually such a contract specifies a REST-API, though it is not limited to that. So "Contract Testing" is a testing approach that uses the API specification to test the provider of the API but also the consumer of the API in isolation. We will talk about this more in detail later on in the concrete example of how Pact implements Consumer-Driven Contract Testing.
So let's have a look at the first part of the term "Consumer-Driven Contract Testing". "Consumer-Driven" in this regard means that the API is not dictated by the creator of the API. Usually when it comes to API design, the team that is providing the API is taking care of the design of it. Alternatively in "Consumer-Driven" Contract Testing, the consumer of the API is specifying the API according to its needs. This does not mean that the consumer alone dictates the API. In optimal scenarios, the API specification is the result of an open discussion with having all parties' interests in mind and coming to a solution that works for every party involved. Therefore "Consumer-Driven" just refers to the process of creating an API specification.
What is Pact?
Pact is a code-first tool for testing HTTP and message integrations using contract tests. Contract tests assert that inter-application messages conform to a shared understanding that is documented in a contract.
Taken from the Pact Documentation
In my own words: Pact is a language-agnostic framework implementing Consumer-Driven Contract testing, to verify the behavior and integration of an API by testing all relevant parties in isolation. The API specification is done in a JSON file with its very own schema. We will take a look at such a specification in the next part of this series.
The following image shows how Pact implements Consumer-Driven Contract Testing on a very high level
To run Consumer-Driven Contract tests with Pact on the Consumer-side you are going to spin up a mock server that is simulating the behavior of the API Provider. Therefore the Consumer is performing its request as it would be in the normal program flow. If the request is compliant to the specification, the mock-server is going to reply with the specified response. Part of this test is also to verify that the Consumer works as expected with the returned response.
On the Provider-side, this looks pretty similar. In this case, the mock server Pact spins up will simulate the Consumer of the API. Pact is going to perform the request according to the contract and verifying that the server response is according to the specification.
Knowing this makes it clear that both parties are tested in isolation, but by using the same contract you test the API interaction integratively. Using this testing approach gives you high confidence about the integration of your Provider and the Consumer, you get fast feedback, and you can develop the relative part on its own without being dependant on the progress of the relative other party.
Using Consumer-Driven Contract Testing makes sense when:
- you are having lots of inter-service communication
- microservices
- Provider is developed by a different team than the Consumer
Using Consumer-Driven Contract Testing may not make sense when:
- providing an API for external parties
- consuming third party APIs
- Provider already exists and you are reusing the same API
Does Consumer-Driven Contact Testing replace E2E testing?
TL;DR Nope.
Consumer-Driven Contract Testing is a good approach in an overall testing strategy. It does not replace E2E testing as E2E Testing is business-case driven and process-oriented. E2E tests will give you great insights whether your business use cases are working as expected, but Consumer-Driven Contract Testing is solely focused on API integration. The test execution is very fast and wrong assertions can be pinpointed really well, but E2E Tests have a way bigger scope, giving you more clearance on the functionality of your overall application.
Outlook
I hoped you enjoyed reading this introduction, as this is only the first part of a series of blog posts I will be writing about this topic. Upcoming in this series:
- Taking a look at the Pact Contract specification
- Integrating Pact Consumer tests in an Angular Application
- Implementing Pact Provider tests in a NestJS Application
- Adding Pact API verification to a CI pipeline
Stay tuned for the next part!
Oldest comments (0)