Microservices Architecture in Golang: A Transformative Experience! 🚀

Introduction: As Steve Jobs, co-founder of Apple, once said, “Technology moves the world.” In this article, I will share my experience with microservices architecture in Golang and how it has revolutionized the way I build scalable and flexible systems. Through a solution implemented for a client, I will explore the concepts and technologies involved, providing valuable insights for those looking to adopt this architecture in their projects. Let’s embark on this journey of microservices empowered by Golang! 💡💻

🌐 Building a REST API in Golang: As highlighted by Martin Fowler, a renowned author in the technology field, in his book “Patterns of Enterprise Application Architecture” “Software architecture is the fundamental organization of a system, embodying its key responsibilities and providing a skeleton for its implementation.” My journey started with building a REST API in Golang, leveraging its advanced concurrency and efficiency features. This API served as a service for the front-end, providing structured data from a MySQL database.

example of code:

package main

import (
 "fmt"
 "log"
 "net/http"
)

func main() {
 http.HandleFunc("/users", getUsersHandler)
 log.Fatal(http.ListenAndServe(":8080", nil))
}

func getUsersHandler(w http.ResponseWriter, r *http.Request) {
 // Handle GET request to fetch users from the database
 fmt.Fprintln(w, "Fetching users...")
}

🔒 Managing Sensitive Data: John Allspaw, a renowned author and software reliability expert, mentions in his book “The Art of Capacity Planning” the importance of protecting sensitive data. In the case of the system I developed, the MySQL database stored user information, including login details, access levels, and service history in a queue. These data were managed through a microservices architecture, where security was a priority. As part of this process, I implemented recommended practices for encryption and key management to ensure the confidentiality of user data.

example of code:

package main

import (
 "crypto/aes"
 "crypto/cipher"
 "crypto/rand"
 "encoding/base64"
 "fmt"
 "io"
)

func encryptData(data []byte, key []byte) ([]byte, error) {
 block, err := aes.NewCipher(key)
 if err != nil {
  return nil, err
 }

 encrypted := make([]byte, aes.BlockSize+len(data))
 iv := encrypted[:aes.BlockSize]
 if _, err := io.ReadFull(rand.Reader, iv); err != nil {
  return nil, err
 }

 stream := cipher.NewCFBEncrypter(block, iv)
 stream.XORKeyStream(encrypted[aes.BlockSize:], data)

 return encrypted, nil
}

💬 Using WebSockets for Real-Time Communication: “Effective communication is the key to any successful relationship” said Tony Hsieh, founder of Zappos. Inspired by this statement, I adopted WebSockets technology to provide real-time communication between the API and employees responsible for service handling. WebSockets enable bidirectional communication with support for real-time events. Events such as queue loading, changing service order, and removing a service were managed by the server using Golang’s native WebSocket library.

example of code:

package main

import (
 "log"
 "net/http"

 "github.com/gorilla/websocket"
)

var upgrader = websocket.Upgrader{}

func handleWebSocket(w http.ResponseWriter, r *http.Request) {
 conn, err := upgrader.Upgrade(w, r, nil)
 if err != nil {
  log.Println("Failed to upgrade to WebSocket:", err)
  return
 }

 // Handle WebSocket communication and events
 // ...
}

📊 Integrating Unstructured Data with Redis: To handle real-time unstructured data, I sought an agile and efficient solution. Event-Driven Microservices. When dealing with unstructured data, having an efficient caching storage layer is essential. Based on this principle, I integrated the Redis database into the microservices architecture, using it to store and manage data related to services. To ensure the security and efficiency of accessing Redis data, I developed a dedicated service called the Broker, which performed read, write, and delete operations in the Redis queue.

example of code:

package main

import (
 "fmt"

 "github.com/go-redis/redis"
)

func main() {
 redisClient := redis.NewClient(&redis.Options{
  Addr:     "localhost:6379",
  Password: "", // no password set
  DB:       0,  // use default DB
 })

 // Example of writing data to Redis
 err := redisClient.Set("key", "value", 0).Err()
 if err != nil {
  panic(err)
 }

 // Example of reading data from Redis
 val, err := redisClient.Get("key").Result()
 if err != nil {
  panic(err)
 }
 fmt.Println("key:", val)
}

📨 Asynchronous Communication with RabbitMQ: To retrieve service data from a messaging service, I chose RabbitMQ as the asynchronous interface. As stated by Gregor Hohpe, author of “Enterprise Integration Patterns” “Asynchronous communication is an efficient strategy for integrating distributed systems.” Through a service called the consumer, my system connected to RabbitMQ, captured, analyzed, and filtered service requests, forwarding them to the Broker for insertion into the Redis queue. Thus, I established efficient and reliable communication between microservices.

example of code:

package main

import (
 "log"
 "os"
 "os/signal"

 "github.com/streadway/amqp"
)

func main() {
 conn, err := amqp.Dial("amqp://guest:guest@localhost:5672/")
 if err != nil {
  log.Fatalf("Failed to connect to RabbitMQ: %v", err)
 }
 defer conn.Close()

 ch, err := conn.Channel()
 if err != nil {
  log.Fatalf("Failed to open a channel: %v", err)
 }
 defer ch.Close()

 // Consume messages from RabbitMQ
 msgs, err := ch.Consume(
  "queue_name",
  "",
  true,
  false,
  false,
  false,
  nil,
 )
 if err != nil {
  log.Fatalf("Failed to register a consumer: %v", err)
 }

 forever := make(chan os.Signal, 1)
 signal.Notify(forever, os.Interrupt)

 // Process received messages
 go func() {
  for d := range msgs {
   log.Printf("Received a message: %s", d.Body)
   // Forward the message to the Broker for further processing
  }
 }()

 log.Println("Waiting for messages...")
 <-forever
}

🚀 Enhancing Efficiency with gRPC: “Efficiency is never accidental,” said Elon Musk, founder of Tesla. Inspired by this vision, I opted to use gRPC for communication between microservices. As mentioned by Brendan Burns, one of the creators of Kubernetes, in “Designing Distributed Systems” “gRPC is a high-performance, data-interchange communication framework.” By adopting gRPC, I could accelerate communication between microservices, optimizing performance and efficiency, in contrast to the REST API used for serving the front-end.

example of code:

package main

import (
 "net/http"
 "net/http/httptest"
 "testing"
)

func TestGetUsersHandler(t *testing.T) {
 req, err := http.NewRequest("GET", "/users", nil)
 if err != nil {
  t.Fatal(err)
 }

 rr := httptest.NewRecorder()
 handler := http.HandlerFunc(getUsersHandler)

 handler.ServeHTTP(rr, req)

 if rr.Code != http.StatusOK {
  t.Errorf("Expected status code %v, but got %v", http.StatusOK, rr.Code)
 }

 expected := "Fetching users...\n"
 if rr.Body.String() != expected {
  t.Errorf("Expected response body %v, but got %v", expected, rr.Body.String())
 }
}

🔧 Managing Deployment Lifecycle with ArgoCD: As mentioned by Jez Humble and David Farley in “Continuous Delivery: Reliable Software Releases through Build, Test, and Deployment Automation” an automated deployment cycle is essential to ensure agility and reliability in software development. To manage the deployment lifecycle of services, I utilized ArgoCD. This tool monitored a repository containing YAML configuration files of the services, allowing for automatic deployment to a staging environment whenever a new version was detected.

Here is an example of an ArgoCD Application manifest file:

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: my-app
  namespace: my-namespace
spec:
  destination:
    namespace: my-namespace
    server: https://kubernetes.default.svc
  source:
    repoURL: https://github.com/my-repo
    targetRevision: main
    path: my-app
  project: my-project

By defining the desired state of the application in the manifest file, ArgoCD automatically synchronized the deployment with the Kubernetes cluster, ensuring consistent and reliable application updates.

⚙️ Infrastructure as Code with Kubernetes: In the era of cloud computing, managing infrastructure as code has become crucial. Kubernetes, an industry-leading container orchestration platform, allows for declarative management of infrastructure resources. By defining Kubernetes manifests, I could describe the desired state of the system, including deployments, services, ingress, and more.

Here is an example of a Kubernetes Deployment manifest file:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-app
  labels:
    app: my-app
spec:
  replicas: 3
  selector:
    matchLabels:
      app: my-app
  template:
    metadata:
      labels:
        app: my-app
    spec:
      containers:
      - name: my-app
        image: my-registry/my-app:latest
        ports:
        - containerPort: 8080

With Kubernetes, I could easily manage and scale microservices, ensuring high availability and fault tolerance.

Diagram of architecture:

References:

Fowler, M. (2002). Patterns of Enterprise Application Architecture. Addison-Wesley Professional.
Allspaw, J. (2017), The Art of Capacity Planning: Scaling Web Resources in the Cloud
Gregor H. (2003) Enterprise Integration Patterns
Farley, D.(2010) Continuous Delivery: Reliable Software Releases through Build, Test, and Deployment Automation