Terraform and Microservices

Introduction

When I was improving my skills and learning about Microservices, Cloud, and DevOps, I faced a problem when migrating from Monolithic Architecture to Microservices Architecture, specifically in the part where I had to provision the cloud infrastructure with Terraform. The issue was how to write the cloud infrastructure successfully for each service in our microservices program without duplicating code and ensuring that each service’s infrastructure was standalone, without affecting other services. This was challenging because microservices follow the Single Responsibility Principle. Today, I am writing this article to share how I solved this issue for anyone who may wonder how to use Terraform with Microservices.

What is Terraform?

Terraform is an infrastructure-as-code (IaC) tool that allows us to provision cloud infrastructure using code instead of relying on manual configuration through the cloud console. This can be very convenient, especially if you have a coding background. To learn more about Terraform, you can visit their website.

What is Microservices?

Microservices is a backend architecture that follows the Single Responsibility Principle. It involves breaking down a monolithic application into small services, each of which performs a specific function and communicates with each other via APIs and messaging services such as RabbitMQ. The benefits of microservices include the ability to use different programming languages to build these services and the flexibility of having different teams with different backgrounds and technical skills work on them. However, implementing and debugging microservices can be complex due to its distributed nature.

Terraform and Monolith

When it comes to regular backend architecture or monolithic architecture, using Terraform is relatively straightforward. We can write the infrastructure code for one backend API in a single file or use modules if we have a more complex infrastructure. Ultimately, we’re dealing with just one thing, so it’s much simpler compared to implementing microservices architecture.

For example, in my latest project, I built a PDF-to-HTML document converter using a monolithic architecture. I deployed this backend API on AWS infrastructure using Terraform, and I only needed to use AWS S3 and AWS RDS. As a result, my Terraform folder was quite simple, as shown below:

├── aws.tf
├── main.tf
└── modules
    └── aws
        ├── RDS.tf
        ├── S3.tf
        └── main.tf

Terraform and Microservices

On the other hand, when we divide a monolithic service into many microservices, each service has its own infrastructure. This can make things more complicated, especially if we try to do it the old-fashioned way, which can lead to code duplication and difficulties in managing each service’s infrastructure.

The solution to this problem was to use modules to make each service standalone. First, we created a module for the common cloud infrastructure and then created a separate module for each service that used this common module. With this approach, we were able to avoid code duplication and manage services more easily.

For example, in my latest project, I built a form builder using microservices architecture, and I needed to deploy the infrastructure on Google Cloud Platform (GCP). I had three services, each of which needed a Google Storage Bucket and a Google SQL Database. To accomplish this, I created modules for GCP to provision these common GCP services, and then created a separate module for each service under the services’ folder. These modules used the common GCP modules, as shown below:

├── README.md
├── main.tf
└── modules
    ├── gcp
    │   ├── sql_database
    │   │   └── main.tf
    │   └── storage_bucket
    │       └── main.tf
    └── services
        ├── form_builder_service
        │   └── main.tf
        ├── forms_service
        │   └── main.tf
        └── users_service
            └── main.tf

Conclusion

Overall, Terraform is a great tool, and when it comes to microservices, it can become more complicated. However, with Terraform modules, things become easier to implement. I hope this article has been helpful for you, and feel free to follow me for more articles like this or ask me anything related on my LinkedIn.

Comments

[mAmged]

Hell, I explored the code a bit and I couldn't find our how the code get deployed and the instances run on the cloud, or it's done manually?

[Mustafa Elghrib]

Hi @mamged, Thanks for your note, you are right, right now they are done manually as you could see here and the same as in the microservices project as shown here for this service

Even though the porpose of this article is to show how I sloved the problem of using Terraform while building Microservices application, I wrote another article about some ways I used to deploy my applications and you could visit it here

[Wendy Wong]

Thanks Mustafa for writing a great article to introduce us to terraform.

Thanks mAmged for asking a follow up question from this post to further your learning :)

[Bad Request 400]

Hey @mstva ,

Great article nice and concise i really appreciate that :).
Bookmarked! ;)

[Mustafa Elghrib]

You're welcome!

[Saravanan Gnanaguru]

Simple yet very useful article. Thanks for sharing your experience Mustafa Abdallah.

[Mustafa Elghrib]

You're welcome!