Welcome, everyone! This is the first article in a series called "Play Microservices" where I will be implementing a microservices architecture featuring a variety of services and technologies. Throughout this series, I aim to provide concise explanations without being overly verbose. So without further ado let's begin.
First:
As I am not going to get into the details of Microservices architecture, I suggest you to have a glance at microservices.io. I aim to introduce the steps involved in developing a typical microservices software; however, please note that the code and steps detailed throughout this series should not be considered complete nor of production-level quality.
The source code for the project can be found here:
Contents:
- Goal application
- DevOps review
-
Applying DevOps principles to our application: Development-> plan
- Architecture selection
- Microsystems architecture design of our app
- Microsystem patterns
- Choreography diagram
- DevOps diagram
- Tools and technologies
- Next
Our goal application:
A simple task scheduling app with authentication. Users register (Admin or user) and log in to the app. Admins can get the list of all users and schedule email jobs to them. Also admins can query for reports.
DevOps review
Before delving into the specifics, we must first take a broad overview of our product, tools and technologies, and development pipeline. In order to produce a final product, companies must progress through a multi-stage pipeline, each stage operating seamlessly and continuously. Due to the numerous stages involved, we will only briefly cover those that are relevant to full-stack developers. DevOps is a software development approach that emphasizes collaboration and communication between developers and IT operations to speed up software delivery, increase reliability, and improve customer satisfaction. It involves a wide range of tools and practices, including automation, continuous integration and delivery, and agile development. Usually the lifecycle of DevOps is symbolized in the form of an infinity loop.
In DevOps, we continuously perform the following tasks:
To summarize the tasks performed in each phase, we can roughly state the following.
-
Continuous development:
- Plan
- Code
- On each commit integration phase is started.
-
Continuous integration:
- New code review
- Integrate new code into existing code
- Deploy to build server
- Unit testing
- Deploy to test environment
- Feedback
-
Continuous testing:
- Integration tests
- Component tests
- Contract tests
- E2E tests
- UI/UX tests
- Security tests
- Feedback
-
Continuous deployment:
- Deploy to stage environment
- Configure and deploy to production environment
Continuous feedback
Continuous monitoring
Continuous Operations
Applying DevOps principles to our application: Development-> plan
Now we want to apply above approach and go through this pipeline step by step.
- Architecture selection
The first step in our journey is to choose an architectural pattern for our application. Our goal is to utilize a microsystems architecture; however, another approach is the Monolithic architecture. Each architecture has its own set of strengths and weaknesses, as well as unique features at its core.
- Microsystems architecture design of our app
How to design the microsystem architecture for our app? To design the architecture of our Microservice we follow the instructions in assemblage and A pattern language for microservices.
-
Discovering system operations: According to the description of our goal application, our system operations can be listed as:
- signUpUser()
- signInUser()
- refreshToken()
- getUser()
- getAllUsers()
- scheduleJob()
- getJob()
- deleteJob()
- updateJob()
- listJobs()
- ExecuteJob(JobTypeEmail)
- createReport()
- listReport()
-
Services and their collaborations: After conducting further investigations and refactoring, we have compiled a list of our services:
- Authentication service
- Job-Scheduler service
- Email-job Executor service
- Report service
- API gateway service
- Client service
We use the C4 model for visualizing our software architecture. According to this model we have 4 layers of abstraction. If you imagine your software as a map, the 1st layer has the top highest view and then you zoom in and in each layer you provide more details.
In case of our app, we can visualize these layers as follows:
- System context diagram
- Container diagram
- Component diagram : We have 6 microservices each have it's own components.
- Code diagram: We have 6 microservices each have it's own components. Each component has its own code diagram.
- Microsystem patterns
Our application is based on a microservices architecture, and to simplify the design process, we implemented microservices-specific patterns. The following provides an overview of the patterns that were utilized and the ones that should be taken into consideration. Please note that the goal is simply to demonstrate an implementation of the patterns. The fact that we are using these particular patterns does not necessarily mean they are the best fit for our specific use case.
-
Service collaboration patterns:
- Event sourcing: persist aggregates as a sequence of events (instead of saving objects in database)
- CQRS: We define a view database (a new service) which subscribe to events from other services (on data change) and save a view of the desired aggregate that we want. This view is read only and we only query from that service. Reports service uses this pattern to create reports of notification tasks.
- API composition: In this pattern, the result for a specific client request is generated from joining multiple queries to multiple services.
- Saga: In this pattern we design a business transaction as a sequence of local transactions (which occurs in a single service locally, then publish an event). Reverse transactions can occur on failed events. Here task scheduler and notification service use this pattern to update the state of scheduled notifications.
- Domain event
-
Other microservice patterns:
- Database per service.
- Team per service.
- Code source per service.
-
Individual service patterns / principles:
- TDD (Test driven development)
- BDD (Behavior driven development)
- SOLID principles: Following these patterns ensures adherence to SOLID principles.
- Choreography diagram
When we combine all of these components -- the services, their collaborations and connections, and the data flow -- we can represent the architecture of our app in the form of a diagram like this:
- DevOps diagram
We can sketch the DevOps pipeline of our app development as follows:
- Tools and technologies
Since our goal is to demonstrate the flexibility of microservices architecture in terms of independence from specific technologies and systems, we will employ various programming languages for different services of the system. The following are the tools and technologies utilized in our application:
-
Development tools:
- Docker: Containerization.
- Visual studio code: Code editor
- Dev containers: Edit code inside containers
-
Integration / deployment tools:
- Jenkins. Alternatives: circleci, travis-ci, gitlab ci/cd, AWS CodePipeline, Azure Pipelines
-
service collaboration / communication technologies:
-
monitoring technologies:
- Metrices: prometheus
- Tracing: jaeger
- Logging: Elasticsearch
-
Auth service
-
Scheduler service
- Golang : programming language
- gRPC-GO: gRPC framework for golang
- Mongo driver: Query builder for our database communication.
- Kafka go for our message broker communications from go.
- Quartz for scheduling purposes.
-
Email job executor service
- Python : programming language
- Kafka-python: For our message broker communications from go.
-
Reports service
- Python : programming language
- grpc: gRPC framework for python
- Pymongo: Query builder for our database communication.
- Kafka-Python for our message broker communications from python.
- grpcio-tools For compiling .proto files to python.
-
API gateway service
- Golang : programming language
- gRPC-GO: gRPC framework for golang
- Gin: Is a web framework written in Golang
- gin-swagger for our rest api documentation.
-
Client service
- Typescript : programming language
- Next.js: Web framework that can be used for developing backend and front-end applications.
- Tailwind css: A utility-first CSS framework
- react-hook-form
- TanStack Query: A asynchronous state management.
We've covered some of the phases of our DevOps pipeline, with a focus on the development: plan stage. Within the context of our microservices architecture, each service can have its own independent code base and unique technical implementation. However, there are logical connections between services, so collaboration between teams is required.
Once the codebase environments for each service have been prepared, we need to set up a CI/CD pipeline using tools such as Jenkins. Teams write their code and tests, push them to the code base, and trigger the CI/CD pipeline. The CI/CD tool will check out the code base (including any dependent repositories), start the build, test, and deployment pipeline, and automatically provide feedback to the team.
- Next
In the next part of this series, I'll implement the authentication service.
Links to other parts:
Part 1: You are here
Part 2: Play Microservices: Authentication service
Part 3: Microservices: Scheduler
Part 4: Play Microservices: Email service
Part 5: Play Microservices: Report service
Part 6: Play Microservices: Api-gateway service
Part 7: Play Microservices: Client service
Part 8: Play Microservices: Integration via docker-compose
Part 9: Play Microservices: Security
I would love to hear your thoughts. Please comment your opinions. If you found this helpful, let's stay connected on Twitter! khaled11_33.
Top comments (0)