A practical guide to microservices

First published on techpilot.dev

Microservices are the natural evolution of monolithic systems in an increasingly demanding, modular, and distributed industry. The strongest argument against them is usually the implied complexity, debugging, and deployment challenges coupled with the poor development experience for small teams/projects.

In practice, most of these problems come from suboptimal implementations, not from the architectural pattern itself. There's still lots of confusion around microservices. Almost every time I bring the topic up, I find someone with a new, unique understanding of it. So, here's my (opinionated) attempt at debunking some of these myths and hopefully help you navigate these stormy waters.

1. Take it step by step. Between a seemingly steep learning curve and the multitude of overlapping tools and frameworks out there, things can quickly become overwhelming
   Evolve your architecture using only tools that solve the problems you know you have, not problems you might think you'll have at a one point.
   It's perfectly fine to start with a "monolith": all actors in one place, but try to design it in such a way that it won't require a tremendous amount of effort to migrate each actor in its own process.
   To achieve this, use standard best practices: inject dependencies, favour composition over inheritance, have a test-driven approach, encapsulate external dependencies etc. I'd argue that by adding a messaging queue to a well designed, modular "monolith" automatically turns it into microservices. Which takes us to our next topic.

2. Use a messaging queue right from the start. Using a messaging queue, something as simple as Redis pubsub or as sophisticated as RabbitMQ will allow you to draw hard lines between your components and stop caring if they run in the same process, on the same machine or even in the same data center.

3. Draw seams based on the problem domain What is a microservice after all? Where do we draw the line between units in our system? Is paramount that the answer to these questions is problem domain-driven. Don't expect a certain count or size, just split them however comes natural for the problem you're trying to solve.

4. Avoid breaking-up your components too early Your enemies here are boilerplate code and lack of domain knowledge. Lay down strong foundations using the best practices you already know and let your system grow.
   When you start a new project you usually don't have enough domain knowledge to correctly define your seams. Breaking-up your system into too many parts early will result in lots of boilerplate code for little functionality.
   Start small, grow steady. For example, an online store, could only have 4 microservices in its first iteration: account, payments, orders and notifications. Later as your solution matures, you could add inventory, affiliates, tracking, suggestions and so on.

5. Validate both the input and the output of each microservice You should reason about each unit of your system in isolation. This is not necessary microservices related, it's architectural 101. Each unit will have an API, a set of functions and capabilities exposed to its peers. Its input and output should always be validated.

6. Develop and debug in-process As much as possible try to take architectural decisions that don't impair your ability to load all your microservices in a debugger within the same process if needed. This is priceless for development speed and bugfixing.
   One of the traditional advantage of microservices is the ability to use different technology stacks for various parts of your system, but this comes with a high price, use it wisely. Having a cohesive development and debugging process is much more important, especially for a small team or solo developers.

7. Don't let the persistence layer drive your architecture There's a misconception that you have to choose between sharing a database among your microservices and have database per microservice. In truth, it doesn't matter, the only thing that matters is that each microservice owns its data store. That means, two microservice should never query or reference the same data store. You can achieve this in many ways, but some make it harder to brake the rules we mentioned above, like having a microservice per database, or per schema (which is what I usually prefer, since it's a design that allows you to deploy both on the same and different database).

8. Replicate data to completely separate data stores Lets imagine we have an online store that has an account and order component, among others. As we learned earlier, the data stores of these two should be owned by each. However, the order needs to know its owner (aka the entity who place it), so how should we approach this?
   In a monolith this would be a foreign key in database, but this breaks the ownership rule, the order service's database should not reference the account's id since they might not even be on the same machine.
   One elegant way to solve this problem is using data replication. The order's database could have one table only with unique owners ids. These would be populated by events in your system: every time a new user is added the account microservice broadcasts their ids and the order microservice (and probably others too) adds them to the owners manifest.

9. Authorize using JWT and similar technologies The simplest way to handle authentication/authorization is to skip the central authority and use technologies like JWT to verify claims without leaving the process (or without calling another microservice). There are many options here and it really depends on the security level your apps requires, but in general for highest security each microservice should check the authorization and permissions before doing anything, no matter how small, while for convenience this can be done at the gateway level only (aka in the component that exposes the public API). I usually go with the former since I think the added security justifies the small overhead of locally checking the identity each time.

10. You don't have to deploy using Docker or Kubernetes right from the start Don't get me wrong, these are great technologies, but both add another layer of complexity to your app. You should consider learning them based on the return of investment rule: does the added complexity and time spent on learning these tools justifies the advantages? Unfortunately, this is a bit difficult to answer. That's why it think it's better to start small.
    If your project uses a popular stack, one of the PaaS out there (e.g. Heroku) is probably a much better fit. There's a strong tendency of overcomplicating things when it comes to microservices deployment.
    Don't forget, microservices are an architectural pattern than can be used no matter how or where you're deploying your app. Your ultimate goal should be to have a clean, scalable solution that doesn't require disproportional effort to build and maintain.

11. Monitor your cluster Logging and monitoring helps you find issues early and react accordingly.
    Avoid third-party or self-made realtime logging libraries (aka you should never make an in-process remote request to log things). Simply log everything using stderr and stdout (with something as simple as debug), then aggregate your logs. If you're using a PaaS, this last step might be already done for you.

12. Testing is a must Write unit test, write integration tests, write end to end tests. In inverse order of their magnitude (i.e. unit tests should come in the largest number). Also, each bug you ever find should have corresponding unit test. With microservices unit testing is not optional, you will never be able to reason about your system as a whole if you can reason about it in isolation.