Microservices start Here: Chassis Pattern

When we talk about microservices, we have to understand that the number of services within the microservices architecture is significant; in some large organizations where the number of microservices is even in the tens of thousands.

If each service setup has to go through a complicated programming process, then it will be a huge overhead to develop a large number of microservices just to build them. Therefore, we would like to have a service template that can be applied across the organization, so that we can develop a new service without the complicated "first step" and start directly from the functionality.

Such a design pattern is called a microservice chassis.

Nevertheless, there is no standard definition of what should be included in the chassis, but a few essential items are listed in some books.

• Security
• Externalized configuration
• Logging
• Health check
• Metrics
• Distributed tracing

Indeed, these are the keys to running a microservice, but from my point of view, they are far from enough. Because, in an organization, there is a lot of additional works must be handled when setting up a service.

So, this article will describe what my ideal chassis should contain. I will divide these items into three parts: must-have, nice-to-have, and best-effort, and explain the meaning of each item.

Must-have

Let's start with the must-have items, which can be divided into several categories within the must-have items.

• CI/CD related
• Function related
• Troubleshooting related

CI/CD related

Items under this category are when a new source code repository is created and there are established rules based on organizational specifications or team practices, so whether it is a microservice or not, these items should be available as long as the code is managed.

1. Linter: Whatever the programming language is, there are actually corresponding development specifications, for example, Python has PEP8, Javascript has ESLint's Airbnb rules, etc.
2. Formatter: In addition to code writing conventions, there are also conventions for code layout, such as Golang's built-in gofmt.
3. Folder structure: The structure of folders has some fixed conventions in various programming languages or frameworks, for example, Golang has standard project layout.
4. Unit testing: Of course, as long as the software is developed, it should be tested, so each source code repository needs to have a well-defined test framework. In addition to the test suite that must be installed, it is also necessary to define where the test files should be placed, such as in the src/tests directory or on the same level as the source code.
5. Integration testing: With unit testing, integration testing is also important. If you are not sure how to distinguish between the two, please refer to my previous article. Integration testing will need to prepare application dependencies, such as databases, etc. A common practice is to define the entire test environment through docker-compose.yml.
6. Package management: Each language has a corresponding package management tool, such as pip or poetry for Python, npm or yarn for Javascript, etc. In addition to the package management tool, it is also necessary to prepare a lock file to fix the versions of the dependent packages.
7. Dockerfile: In the microservices architecture now mostly exists in the container system, so it is important to prepare a Dockerfile developed based on best practices, in order to make containers more secure and efficient. Here are some best practices for building Python containers.
8. Pipeline template: When the Dockerfile is available, I believe it should be possible to publish at least one of the simplest applications. Then, depending on the version control system or release system used by the organization and the organization's defined release specifications, a basic CI/CD release pipeline can be written. Because each organization uses a different system, it should be prepared to meet the needs of the moment, such as Bitbucket pipeline, Gitlab CI, Github Action, etc. They all use different formats.
9. Static application analysis: The last item is static analysis, which is divided into code level analysis and application level analysis. Common code analysis includes test coverage, memory leaks, package dependencies, etc. Application-level analysis is for possible vulnerabilities, such as the use of Synk.

Function related

There are many items listed above, but they are not finished yet.

Those items focus on CI/CD, but application development is the core, so there will be some preconditions in the software development.

1. Config loader: An application's configuration may come from many different sources, for example, the default configuration file can be overwritten by environment variables, and even specify the path to a new configuration file. Take the Postgres container as an example, it is possible to create a database through POSTGRES_DB, and if we want to create multiple databases, then we can mount the sql files under docker-entrypoint-initdb.d. These rules should be part of the template when setting up a new application, so we don't need to write each once again.
2. Feature toggle loader: The feature toggle and its use are described in my previous article. Feature toggles are already a common strategy in software development, so they should be pre-defined in the similar way as the config loader.
3. Web framework: The most important thing in a microservice chassis is of course to choose the front/back end framework. I believe that the same framework should be used cross the same organization, then the framework should be from the template.

Troubleshooting related

When there is a function that can be published, what happens when there is a problem in the production environment? Especially when there are a large number of microservices under the microservice architecture, it is necessary to provide sufficient observability to identify the root cause of the problem.

1. Distributed tracing: It is a common requirement for microservice architectures to be able to trace the processing rate between each service, and even the time share of database calls within each service, so that the entire service mesh can be mapped. There are many common tools, such as the open sourced Open Telemetry or the commercial New Relic, Elastic APM, etc. These tools require an agent to be installed in the application, and therefore the package and initialization need to be defined.
2. Logger: In order to analyze the behavior of applications, logging plays a significant role. How to record logs, how logs are collected and the format of logs are all related to the team practice, so how logger should be implemented will also be a standard process.
3. Health check: Among the tens of thousands of microservices, how to know each service is still alive? The most common practice is to define a fixed REST API for monitoring, so this fixed API will be defined by the template, but the implementation still has to be developed based on each service's characteristics.
4. Metric exporter: Although distributed tracing is already available to collect general metrics, some application-related behaviors cannot be collected, and some distributed tracing frameworks provide the ability to customize metrics, so how the application's custom metrics are exported also needs to be defined.
5. Local development environment: In the VM era or earlier machine era, we used to debug directly on the production environment and even reproduce the problems on it, but in the microservice era, we should develop and debug locally. After all, with thousands of microservices and many instances of each service, it is impractical to run them in the production environment, so how to develop and debug locally? I believe templates also need to provide the corresponding capabilities.

In fact, all these items are closely related to organizational specifications or team practices, so each different organization should have a service template of its own, and there may be a corresponding template for each programming language.

But the above list is the basic requirements of microservice development, and I will list some more advanced items. However, whether these advanced items are needed depends on the size of the organization and the nature of the service.

Nice-to-have

In order to build a stable, robust and secure service, there are many items to consider.

Robustness related

1. Circuit breaker: One of the issues we need to consider in a microservice architecture is that when a downstream service fails, it will not cause an avalanche, i.e., a cascade failure. Circuit breaker is a common strategy we use to ensure the system will not crash even if the downstream service fails, please refer to the link for more details.
2. Rate limiter: On the other hand, when our service is faced with a large number of requests, can the service handle it? Some services may rely on external gateways to block excess traffic, but I believe that every service should know its capacity and act on excess traffic, so a rate limiter is a worthy option.
3. Authenticator: External communication sessions or JWT tokens, which require specific packages to be installed, and access control for each interface is also a topic that usually requires middlewares or decorators to be added to the API development, which should also be included in the templates. Furthermore, APIs often need to have Role-based access control, which requires domain experts to participate during development.

I believe there are many organizations that have a practice of cutting internal and external services over a physical network, so internal services use a full trust model without any authentication or even using unencrypted channels to communicate, which is not really appropriate.

In recent years, the concept of Zero Trust has emerged that any service should have basic authentication capabilities to avoid unexpected access.

Security related

If the service is for external users also need security mechanisms other than authentication, including CSRF and CORS, the details and principles are not the focus of this article, but all these security features should be provided by the service template.

1. CSRF
2. CORS

Best-effort

In addition to the items listed above, a service usually requires many external dependencies, such as various databases, message queues, and so on. For example, Python with Postgres may be SQLAlchemy and nothing else, and the version number will be fixed.

A complete service template should also include the various drivers used within the organization, although not every service will need to use the full toolset.

Therefore, some large organizations use scaffolding to allow developers to set options when applying templates, such as whether they need databases, message queues, programming languages, etc.

The example above is Netflix's job generator, NEWT.

A complete project can be generated with simple commands and interactive options. This is the ultimate in service templates.

Conclusion

In this article we have introduced the items needed to build a microservice, which should not be built manually by the developer, but rather as a turnkey solution. Ideally, the developer should be able to interactively build the entire project in one shot, depending on the requirements.

Although we have listed many items, these are actually a checklist of production-ready microservices, compiled from my past experience. If there are items that you are not entirely sure what they mean, I suggest looking at the links in the article or the information in the relevant books. After all, the items listed in this article are not super tricky concepts, but rather design patterns for general purposes.

Of course, if you or your organization has other items in the service templates, feel free to share them with me. I'd be glad to know more about design patterns.