The ultimate guide for Microservices’ design

The Process of defining/designing a microservice's architecture is always painful and flaky. I used to read the word "make it small", "make it simple" and all those kinds of microservices stereotypes. But how to measure how small is small? or how simple it is that simple.

So I started doing research to define a strict guideline for how to define your system as a list of collaborative microservices and it worked for me.

Hint: I believe that if you are starting to build a new service from scratch you should not pick microservices. Microservices come as a step after having your service alive and you totally understand every single part of it. Also, your team grows at that point there's a solid need to go for microservices.

If you can’t build a well-structured monolith, what makes you think you can build a well-structured set of microservices? — distributed big balls of mud by: Simon brown

Also, Martin Fowler has a strong opinion regarding this

Almost all the successful microservice stories have started with a monolith that got too big and was broken up

• Almost all the cases where I’ve heard of a system that was built as a microservice system from scratch, it has ended up in serious trouble. — Monolith first by: Martin Fowler

In this article, we might be applying some concepts from domain-driven-design (DDD). But it’s not a prerequisite to have previous knowledge about it.

So, what is a service?

the best definition that I always liked to define the word service is A stand-alone, independently deployable software component.

We'll be listing the steps to be followed to define your system as a list of services collaborating together.
I’ll be using an example of building a social network.
In my blog post, I will specify the steps/checklist you need to follow, and I will define the output format for each step.

1. Identifying system operations
In this step, your team should collaborate together to define your service’s operation. This should be the starting point at any service’s life cycle.

System operations are usually operations to be done between domain models. So the first step in this process is to define the high-level domain models.
Hight-level domain models are not related to what will be implemented, they are much simpler.

System operations could with either command or query.

• Commands: are those operations related to create, update, delete
• Queries: are those operations related to query/read data. Example queries are: find a user, get pending orders, get paid transactions,…

The easiest way to define system operations is to use the “Given, when, then” format.
Martin Fowler has published a very nice article that has extensive information about that format for specifying system behavior/operation. https://martinfowler.com/bliki/GivenWhenThen.html

To summarise it
Each operation should have an Actor, who will be performing this operation. To define any system operation we should have the state/pre-condition before executing the operation which is defined by the Given block. Given could consist of multiple states/pre-conditions. Then it comes to the When block which defines that operation/behavior that we need to define. The last thing is the Then block, which describes the state of the system after applying/executing that operation we are trying to define.
An example to describe an operation of user reset password

Feature: User password reset
Actor: User
Given that
— I’m logged in to my account
— and I’ve landed on my profile page
When
— I change my password
Then
— I should receive a password reset email.
— And email should contain valid reset password link
— And I could able to use this link for one time

If you need further details you can check Martin Fowler's blog post here, it has more details, https://martinfowler.com/bliki/GivenWhenThen.html

The output format for this step should be two documents:

• High-level domain model diagram with some dummy description of how they are linked together. For example below a high-level diagram for a social network.
• The other document would be a list of system operations written in "Given, When, Then" format.

2. Extract domain models/subdomains
This step comes right away after defining all the system operations. At this step, you can start by defining high-level domain models based on the system operations. For example, in the previous system operation, we can obviously define User as a domain model.

Another example of system operation, if we are designing a social network

Actor: user
Given that
— I’m logged in to my account
When
— I try to change my current home city
Then
— My friends should be able to view the change in their timeline
From the previous operation, we can identify User as a domain model and Address location as a subdomain.

The output format from this shep could be:

• List of domains and their sub-domains

3. Define specification for system commands
After the previous steps, we should be able also to dig deeper and define the system commands and queries.

You can start by listing all the system commands as a high level. For example

Actor: User
Story: write a new group post
Command: postToGroup()
Description: write a post to a group I’m a member of

Afterward, you’ll be able to define the specification for that command. For example:
Operation: postToGroup(user id, group id, post)
Returns: postId
Preconditions:
— User should be logged in
— User should be a member of the group
Postconditions:
— The post was successfully created
— All group members are able to view/interact with the post

Defining system commands
After being able to define the system commands, you might extract some system queries that are needed for these commands. Actually you can extract them from the preconditions. For example,

• You need to able able to authenticate user which is a system query “authenticatUser(email, password)”
• Also, you need to find a user by id “findUserById(user id)”
• You might need to get group details by id “findGroupById”,
• You need to check if a specific user is a member of this group “isGroupMemeber(user id, group id)”

The output format from this step could be:

• A list of system commands and operations with their details.

4. Services decomposition
When it comes to decomposing the services we have two different approaches.

1. By applying business capability decomposition

Business capability
It captures what business does not how. How business could achieve a specific value changes by the time. but the business value never changes.

An example of a business capability
In the old days restaurants used to receive delivery requests via phone calls. But nowadays most of the restaurants handle delivery requests via apps which what we do in Delivery Hero ;)
So business capability is food delivery which never changes, but how it works has been dramatically changed over time.

Let's get back to microservices, so in such a way of decomposition we list all the business capabilities and each capability could be a good candidate to be a separate service. So decomposition by business capability always reflects organization structure, not technical structure.

Let's apply it to an online retailer like Amazon. We can say the list of capability they have could be
— online payment
— shipping
— marketing
Also, some capabilities could have sub-capabilities. For example, the above capabilities could have this hierarchy
- Online payment
— payment processing
— user rewards
— wallet
-Shipping
— fulfillment
— tracking
-Marketing
— coupons and promos
— online campaigns
— user acquisition

So you can also map each sub-domain as a separate service based on the size of each capability and the business itself.

One of the key benefits of decomposition by business capabilities is that they are stable and never changes (but how it works is the one which changes by the time)

For more details about decomposition by business capability you can check this article: https://microservices.io/patterns/decomposition/decompose-by-business-capability.html

2. By subdomain decomposition
In this type of operation, we can start by identify high-level domain models and then extract sub-domains out of each domain. We start by defining the vocabulary or the language which the team uses to define the system operations which we call "ubiquitous language". From this language, we can extract the domain models.

For example, the user will be able to order food from a restaurant which is nearby his location, choose the meals, could add multiple meals to his shopping cart and then submit his order to be paid and processed

If we apply this to our social network example, we can extract those high-level domain models.

• User domain
• Post domain
• Groups domain

And then we can define a service for each subdomain.

So all the previous guidelines might consume much time on the project's timeline. but do it sooner than later. Defining your services before starting in defining each service’s architecture will save you much time.

For more details about decomposition by subdomain: https://microservices.io/patterns/decomposition/decompose-by-subdomain.html

There's also this nice comparison between both ways of decomposition strategies https://stackoverflow.com/questions/45688730/decompose-microservices-business-capability-vs-domain

The output format from this step could be:

• A list of service names and a detailed description of the purpose/goal of each service

5. Defining each service's communication protocols and APIs
The last step is to define how your service will communicate with the external world. By APIs, I do not mean only HTTP requests. I mean defining the communication ports.

For example, you need to define that your service will be able to accept requests via HTTP requests on these specific endpoints, with those headers, this request payload schema and the caller should expect the following response schema. Also, it might accept RPC calls using proto-buffer. Your service might accept these specific events and could also publish those events.

In general, the output of this design step should be:

1. List of services names
2. how these services will be interacting with each other
3. Define the goal of each service
4. Define the domain models/language of each service Interfaces that will be used for communication with each service (APIs, gRPC, events,…). try to use open API/swagger format to define your services. You can add the hosting for the swagger UI client as part of your service’s CI
5. Define communication patterns (IPC) between services (i will be blogging about this later in another blog post)

Last but not least, you can use the points above as your checklist for this process.

In this step of defining microservices' architecture you should ensure the following:

1. That you have involved product/project managers, business people/stakeholders, as much as you can. Involve them in all these high-level design meetings. You can get impressed by how this will reflect your design.
2. Try as much as you can to use diagrams in this step. Data flow diagrams, UML, decision trees, flowcharts, sequence diagrams or even just dummy diagrams that follow no standards but are meaningful.
3. Non-tech people could easily understand the goal of each service.
4. Services' definitions should reflect organizational structure/goals.

By the end of this process, you must ensure that your services could be able to:

1. Deployed independently
2. Enables continuous delivery/deployment
3. Support team independence (teams should not be dependent on each other and changes in one service should not depend/break other services)