A microservices update about domain-driven squads, CDC testing, and happily ever after
A couple years ago, Postman CTO and co-founder Ankit Sobti shared Postman’s struggle to escape from a microservices dependency hell. If you want to learn more about how and why Postman Engineering ventured into microservices, check out Ankit’s story.
This is an update — here’s how Postman Engineering does microservices today.
Who is Postman Engineering?
Postman Engineering has 100+ engineers working across eight locations around the world. The company just announced a Series C round of funding, so things are bound to evolve as with any growing startup.
Let’s dive into how these teams are organized. As we’ll soon see, this really impacts the microservices implementation.
With a microservice architecture, how do you organize your teams?
According to Conway’s Law, software starts looking like the organizations that create them. How people communicate, collaborate, and even their tooling has an impact. And the way information flows between related teams influences the output.
Any organization… will produce a design whose structure is a copy of the organization’s communication structure.
This is one reason why organizations like Amazon and Netflix work in small, independent teams. It enables API-first design and development. And is represented as the well-known microservices deathstar.
You can observe Conway’s Law by comparing products from teams grouped by function, by product, or by workflow.
How can you take advantage of this phenomenon?
The Inverse Conway Maneuver proposes that you can change your organizational structure in order to change the technical structure that you desire. By dictating how your organization communicates and collaborates, you begin converging on your ideal shape.
Keeping this in mind, development teams at Postman are called squads. Squads operate on domain-driven design (DDD) principles, which free up each squad to focus on its core domain.
For example, one squad owns the Identity domain. The Identity squad produces services like Create a User and Authenticate a User , among others.
Across our company, there’s 12 squads that produce 40 services for Postman engineering. For the most part, each squad independently manages their roadmap and sprints. They have some freedom to choose their tools and configure their work processes.
Although squads are focused on their specific service, they are cross-functional, with dedicated members from design, product, engineering, security, quality, technical writing, and developer relations (that’s me!).
There’s many ways to implement microservices, but here’s how Postman engineering does it.
With a microservice architecture, how do you create and maintain new services?
Imagine the Identity squad wants to produce a new service — to authenticate an existing user.
Step 1 | Create a team workspace
In Postman, the Identity service owner is the producer of this new service. This producer creates a team workspace as the single source of truth for the new service.
A workspace is the foundation for collaboration in Postman. As we’ll soon see, the workspace contains the expected behavior of the service. Unit and integration tests live here. And it’s the central thread for all the communication during and after development.
Step 2 | Draft a blueprint collection to describe use cases
Within the new workspace, the Identity service owner drafts a collection to describe their service. This collection is like a blueprint with proposed examples of a request and potential server responses to describe various use cases. In fact, the name of the collection includes #blueprint as a tag indicating its purpose.
Step 3 | Negotiate how the proposed service will function
The blueprint collection is a proposal for the service. Potential consumers of the service weigh in on the design, either offline or via threaded comments in the app, to shape what the service will become.
Step 4 | Parallel development with mocks
Once all stakeholders successfully negotiate the design of the service, the collection becomes an agreement, or contract, for the new service. Those examples of requests and responses inform mock servers that enable parallel development.
The mock is a contract between the producer and consumers of this service. Both parties can start developing their code at the same time. Tests are written. Documentation begins. All this activity occurs in parallel and is driven by an agreed-upon vision of the new service.
With a microservice architecture, how do we know when there’s a breaking change?
When we’re ready to deploy the service, we test to ensure that no workflow is broken. One of the most powerful tools that Postman engineering has adopted to manage microservices is consumer-driven contract (CDC) tests.
CDC testing
In the previous section, we talked about how the service owner relies on the blueprint collection to propose, negotiate, and then establish how their service will be used.
Once that contract is agreed upon, consumers write tests using the mocks as a reference point. The consumers only test the endpoints and properties that they need, saving the tests in a collection. Once again, the name of these collections include #contract as a tag indicating its purpose. These test collections are maintained within the producer’s team workspace.
When the producer is ready to deploy their service, the consumer’s contract tests are run against the code as part of the continuous integration (CI) pipeline.
The service owner pulls all the collections within the team workspace using the Postman API, looking for any collections with the #contract tag to run. They use Docker to configure their own environments and deploy their code in the CI/CD pipeline, and rely on Newman to run the collections in the container.
The code changes deploy only if the contract tests pass.
Continuous testing
In addition to running contract tests on demand as part of the CI pipeline, Postman Engineering runs a plethora of other tests on a regular basis.
Our engineers schedule monitors that run test collections from Postman servers. These alerts and summaries are piped into Slack and email to keep tabs on the service throughout the day.
So Postman engineers rely on tests that run on demand and continuously. Incorporating CDC testing in the CI pipeline allows service owners to deploy updates with confidence that they’re not breaking somebody else’s code. Continuously running other tests on monitors also ensures nothing is broken after deployment.
All of this informs an ongoing feedback loop that allows the team to continue evolving and updating their services.
Is that happily ever after?
No blog post about microservices will ever conclude with a tidy bow. All architectures are a continuously evolving work in progress.
As you can probably imagine, Postman engineering uses the Postman product quite a bit. Which means that we’re the first canaries in the coal mine. If something doesn’t work smoothly or it can be done better, we have the opportunity to feel the pain first. This allows us to iterate quickly on new features.
Conclusion
This was a peek at how Postman is working with microservices today. We’re continuing to address some challenges.
- Dependencies on other squads require additional coordination. One squad’s roadmap might be blocked by another squad’s progress, or at least be subject to their service levels.
- Squads are a relatively new, and each squad is independently figuring out what processes work best.
- In domain-driven design, domains require constant evaluation. For example, squads focused on large domains that require a large share of responsibilities debate the trade-offs between managing a microservice vs. amonorepo vs. a monolith.
If you want to learn more about microservices, then check out Ankit’s original blog series about Postman’s initial foray into the microservice architecture. And keep your eyes peeled. We hope to share more updates and stories from the trenches soon.
Top comments (0)