The past 4 years I’ve worked on a test automation library that is primarily used for end-to-end test automation, with some additional uses. In that time it has proven to be a very flexible, easily maintained and extensible piece of software. In this article I’ll share some of the underlying concepts and considerations we’ve used to build the library.
Goals — or “How Should This Thing Work?”
Before setting up this library, there had been some earlier work done that was mostly pieced together JSON snippets, with text concatenation all over the place. Writing a new test meant copy/pasting tons of JSON every time. To top it off, it was written in an tool and language our developers had no knowledge of. Obviously, they wanted no part of it. It was not something I felt was fit to establish a strong foundation of test automation on top of.
Goals we set for the library included:
- It should hide complexity when unnecessary
- It should give control when required
- It should be easy to use, and easy to contribute to
What Are We Automating Again?
The intended use for this library was twofold:
- Use it to automate tests against the public facing API of our platform
- Make it easy to reach certain states with the data it creates
Our public API is a highly configurable one. Each consumer will call the same endpoints, but based on the configuration the required inputs can be very different. Endpoints must be called in a certain order and data most be provided consistently across these endpoints. For example: if you provide an address in call #1 it must be in the same country you provide in call #2.
Completing the series of calls results in an entity being created, which has a complicated lifecycle, with different states, transitions and transition guards. The library should make it easy to create that initial entity, but also modify the state of the entity as desired.
Hiding Complexity
As described above, providing valid inputs to the API across multiple calls requires quite some specific domain and product knowledge. That’s fine if you care about testing the particulars of one set of configuration, but if you simply need an entity of type X to test something else elsewhere on the platform you might not be too enthusiastic about figuring out all those details.
This is what I mean with hiding complexity, the library should provide its users an easy way to create complex data, without them having to know what data results in a valid entity, nor how to orchestrate the calls that are required.
This is where we leverage Kotlin’s concrete interfaces, providing a blueprint for the process and for the pieces of data that must be provided.
The Product interface provides a default implementation for createShoppingCart, while the classes implementing provide specific, valid data through the interface methods.
Sensible Defaults
Creating tests for a new product simply means implementing the methods that provide the data for a valid policy. All that remains to create that policy is calling a single line:
motor.createShoppingCart()
Using Datafaker, we provide localized and randomized data. Akin to James Bach’s galumphing, this helped us to detect issues with inputs “that shouldn’t make a difference” more than once.
Need something specific?
The setup with the Product interface is flexible enough to allow users to create very specific policies where needed, falling back to defaults for values that aren’t specified.
val verySpecificRiskAnswers = setOf(Answer("ALLRISK", "YES"))
motor.createShoppingCart(riskAnswers = verySpecificRiskAnswers)
Mind Your Consumer
With this design for the test data library, we allow users to easily create randomized, valid test data with a one-line statement. No knowledge is required of the underlying API’s nor what data is required for a valid policy. At least…as long as they want to keep away from that complexity. The library is flexible enough to grant that control, and specify every input in detail.
Usage
The Test Data Library powers different types of test and a service
The test data library is used to power both E2E tests and system tests on integrated environments.
Next to test engineers and developers, also business stakeholders often need test data to work with. You’ll probably be familiar with requests on Slack for data like X, Y or Z, and it might be cumbersome to create it manually every time.
For this specific purpose we’ve build a web application with a thin UI layer that allows business stakeholders to easily create test data right from their browser. If they don’t have any specific requirements the sane defaults in the library make this a one click operation. If they have more specific demands, the UI allows for the same level of control as at the library level.
Results
The library has helped us to stay on top of an ever growing number of products, maintaining tests and rolling out changes with ease. A common change only has to be implemented in the Product interface, and all tests automatically follow the updated process.
Adding a new product can be done in a matter of minutes and it’s very clear what information is needed to be able to start testing.
The main thing to keep in mind when working on the library is that there are several consumers, and that the API of the library should remain stable. Breaking changes may complicate the upgrade path of consumers.
As the platform evolves and newer versions of API’s are adopted by production services it’s also important to keep the clients in the library in sync with new endpoints.
Top comments (0)