Nithin Bharadwaj

Posted on Jan 20

How to Build a Service Mesh Sidecar in Go for Microservices Communication

#programming #devto #go #softwareengineering

As a best-selling author, I invite you to explore my books on Amazon. Don't forget to follow me on Medium and show your support. Thank you! Your support means the world!

Let me show you how I handle communication between microservices. When you have many small services talking to each other, things get messy fast. Security, monitoring, and reliability become complicated in every service. I found a better way: the sidecar pattern.

Think of a sidecar like a helpful assistant that sits next to your service. Your service focuses on its job, while the sidecar handles all the communication work. They share the same space and work together, but each has separate responsibilities.

Here's how I build one in Go. First, I create the main structure that will manage everything.

type SidecarProxy struct {
    config      *ProxyConfig
    listener    net.Listener
    registry    *ServiceRegistry
    policies    *PolicyEngine
    metrics     *ProxyMetrics
    tracing     *TraceContext
    certManager *CertManager
}

This sidecar knows how to listen for traffic, find other services, apply rules, collect metrics, track requests, and manage security certificates. It's a complete package.

When I start building, I need to configure what this sidecar will do. Each service gets its own configuration.

config := &ProxyConfig{
    ListenPort:   15001,
    UpstreamPort: 8080,
    ServiceName:  "payment-service",
    ClusterName:  "production",
    EnableMTLS:   true,
    TrafficPolicies: []TrafficPolicy{
        {
            Service: "user-service",
            Rules:   []string{"max-rps:100", "timeout:5s"},
        },
    },
}

The sidecar listens on port 15001 for incoming traffic. My actual payment service runs on port 8080. The sidecar sits in between, managing everything that comes and goes.

When traffic arrives, the sidecar needs to understand what type it is. Is it regular web traffic? Is it gRPC? Maybe WebSocket? I look at the first few bytes to figure this out.

func (sp *SidecarProxy) detectProtocol(data []byte) string {
    if len(data) >= 3 {
        if string(data[:3]) == "GET" || string(data[:4]) == "POST" {
            return "http"
        }
    }

    if len(data) >= 5 && data[0] == 0 {
        return "grpc"
    }

    if bytes.Contains(data, []byte("Upgrade: websocket")) {
        return "websocket"
    }

    return "tcp"
}

This simple check lets me handle different protocols correctly without confusing them. Once I know what I'm dealing with, I can process it properly.

For HTTP traffic, I read the request and decide where to send it. This is where routing rules come into play.

func (sp *SidecarProxy) handleHTTP(clientConn net.Conn) {
    req, err := http.ReadRequest(bufio.NewReader(clientConn))
    if err != nil {
        log.Printf("HTTP parse error: %v", err)
        return
    }

    target := sp.policies.Route(req.Host, req.URL.Path)
    if target == nil {
        http.Error(clientConn, "Service unavailable", http.StatusServiceUnavailable)
        return
    }
}

The policy engine checks my rules. It might say "requests to /api/users should go to user-service." It also checks if that service is healthy and available.

Security is important. I use mutual TLS to ensure services only talk to who they should. Each service gets its own certificate, and they all trust a common authority.

func (sp *SidecarProxy) forwardHTTPRequest(req *http.Request, target *ServiceInstance, clientConn net.Conn) error {
    var upstreamConn net.Conn
    var err error

    if sp.config.EnableMTLS {
        upstreamConn, err = tls.Dial("tcp", target.Address, sp.certManager.GetTLSConfig())
    } else {
        upstreamConn, err = net.Dial("tcp", target.Address)
    }

    if err != nil {
        return err
    }
    defer upstreamConn.Close()
}

When mutual TLS is enabled, both sides verify each other's identity. This prevents impersonation and ensures encrypted communication.

Services come and go in dynamic environments. My sidecar needs to know what's available right now. I maintain a registry of services and their health.

func (sp *SidecarProxy) healthCheckServices() {
    ticker := time.NewTicker(30 * time.Second)
    defer ticker.Stop()

    for range ticker.C {
        sp.registry.mu.Lock()
        for serviceName, instances := range sp.registry.services {
            for _, instance := range instances {
                go sp.checkInstanceHealth(serviceName, instance)
            }
        }
        sp.registry.mu.Unlock()
    }
}

Every 30 seconds, I check each registered service. I send a simple request to its health endpoint. If it responds positively, I mark it as healthy. If it fails repeatedly, I stop sending traffic to it.

This prevents sending requests to broken services. It's like checking if a store is open before driving there.

Sometimes services get overwhelmed. They might be slow or returning errors. A circuit breaker helps here. It's like an electrical circuit breaker for your services.

func (pe *PolicyEngine) IsCircuitOpen(service string) bool {
    pe.mu.RLock()
    defer pe.mu.RUnlock()

    if cb, exists := pe.circuitBreakers[service]; exists {
        return cb.IsOpen()
    }
    return false
}

When a service fails too many times, the circuit "opens." No more requests go to that service for a while. After some time, I let a few requests through to test if it's recovered. If those succeed, I close the circuit and resume normal traffic.

This prevents one failing service from causing problems everywhere. It gives the failing service time to recover.

Load balancing spreads traffic evenly. I don't want all requests going to one instance while others sit idle.

func (pe *PolicyEngine) Route(host, path string) *ServiceInstance {
    // Find matching routing rule
    for _, rule := range pe.routingRules {
        if rule.Service == host && strings.HasPrefix(path, rule.Prefix) {
            targets := pe.getHealthyInstances(rule.Targets)
            if len(targets) == 0 {
                return nil
            }

            // Round-robin selection
            pe.currentIndex[host] = (pe.currentIndex[host] + 1) % len(targets)
            return targets[pe.currentIndex[host]]
        }
    }

    return nil
}

Round-robin is simple but effective. I send the first request to instance A, the second to instance B, and so on. This evenly distributes load across all healthy instances.

Sometimes I need to know what's happening across all my services. Distributed tracing helps me follow a request through the entire system.

type TraceContext struct {
    propagator propagation.TextMapPropagator
    tracer     trace.Tracer
}

func (tc *TraceContext) Inject(headers http.Header) {
    tc.propagator.Inject(context.Background(), propagation.HeaderCarrier(headers))
}

When a request enters the system, I generate a unique trace ID. As the request moves from service to service, I pass this ID along. Each service adds timing information. Later, I can see exactly how long each step took and where bottlenecks exist.

Metrics help me understand performance over time. I track how many requests each service handles, how long they take, and how many fail.

type ProxyMetrics struct {
    mu        sync.RWMutex
    requests  map[string]uint64
    latencies map[string]*LatencyStats
    errors    map[string]uint64
}

func (pm *ProxyMetrics) RecordRequest(service, path string, duration time.Duration, err error) {
    pm.mu.Lock()
    defer pm.mu.Unlock()

    pm.requests[service]++

    if pm.latencies[service] == nil {
        pm.latencies[service] = &LatencyStats{}
    }
    pm.latencies[service].Add(duration)

    if err != nil {
        pm.errors[service]++
    }
}

Every 10 seconds, I report these metrics to a monitoring system. I can set up alerts when error rates get too high or response times get too slow.

gRPC needs special handling. It's a different protocol than HTTP, though it uses similar transport.

func (sp *SidecarProxy) handleGRPC(clientConn net.Conn) {
    buf := make([]byte, 5)
    if _, err := io.ReadFull(clientConn, buf); err != nil {
        log.Printf("gRPC header read error: %v", err)
        return
    }

    if buf[0] != 0 {
        msgLen := int(uint32(buf[1])<<24 | uint32(buf[2])<<16 | uint32(buf[3])<<8 | uint32(buf[4]))

        msg := make([]byte, msgLen)
        if _, err := io.ReadFull(clientConn, msg); err != nil {
            log.Printf("gRPC message read error: %v", err)
            return
        }

        method := sp.extractGRPCMethod(msg)
        target := sp.policies.RouteGRPC(method)

        if target == nil {
            return
        }

        sp.forwardGRPC(clientConn, target, buf, msg)
    }
}

gRPC has its own format. I read the header to understand the message length, then read the full message. I extract which method is being called and route it appropriately.

WebSocket connections are persistent. Unlike HTTP, they stay open for bidirectional communication.

func (sp *SidecarProxy) handleWebSocket(clientConn net.Conn) {
    httpReq, err := http.ReadRequest(bufio.NewReader(clientConn))
    if err != nil {
        return
    }

    upgrader := websocket.Upgrader{
        ReadBufferSize:  1024,
        WriteBufferSize: 1024,
    }

    wsConn, err := upgrader.Upgrade(clientConn, httpReq, nil)
    if err != nil {
        return
    }
    defer wsConn.Close()

    target := sp.policies.RouteWebSocket(httpReq.Host, httpReq.URL.Path)
    if target == nil {
        return
    }

    upstreamURL := fmt.Sprintf("ws://%s%s", target.Address, httpReq.URL.Path)
    upstreamConn, _, err := websocket.DefaultDialer.Dial(upstreamURL, nil)
    if err != nil {
        return
    }
    defer upstreamConn.Close()
}

WebSockets start as HTTP requests with an upgrade header. Once upgraded, I establish a connection to the target service and forward messages in both directions.

All these components work together. The sidecar starts by setting up its listener.

func (sp *SidecarProxy) Start() error {
    listener, err := net.Listen("tcp", fmt.Sprintf(":%d", sp.config.ListenPort))
    if err != nil {
        return err
    }
    sp.listener = listener

    go sp.healthCheckServices()
    go sp.reportMetrics()

    for {
        conn, err := listener.Accept()
        if err != nil {
            log.Printf("Accept error: %v", err)
            continue
        }

        go sp.handleConnection(conn)
    }
}

I start health checking and metrics reporting in separate goroutines. Then I accept incoming connections, handling each in its own goroutine. This allows handling many connections simultaneously.

The buffered connection helper is important. When I read the first bytes to detect the protocol, I need to put them back for the actual handler to use.

type BufferedConn struct {
    net.Conn
    buf []byte
    pos int
}

func (bc *BufferedConn) Read(p []byte) (int, error) {
    if bc.pos < len(bc.buf) {
        n := copy(p, bc.buf[bc.pos:])
        bc.pos += n
        return n, nil
    }
    return bc.Conn.Read(p)
}

This wrapper gives back the bytes I already read, then continues reading from the original connection. It's transparent to the protocol handlers.

Rate limiting prevents abuse. If a client sends too many requests too quickly, I slow them down.

func (pe *PolicyEngine) AllowRequest(host, client string) bool {
    pe.mu.Lock()
    defer pe.mu.Unlock()

    if limit, exists := pe.rateLimits[host]; exists {
        return limit.Allow(client)
    }
    return true
}

I track requests per client per service. If they exceed the limit, I return an error instead of forwarding their request. This protects services from being overwhelmed.

Configuration can change while running. I might need to update routing rules or rate limits. Dynamic configuration loading allows this without restarting.

func (sp *SidecarProxy) ReloadConfig(newConfig *ProxyConfig) {
    sp.config = newConfig
    sp.policies.UpdateRules(newConfig.TrafficPolicies)
}

I swap the configuration and update the policy engine. The change takes effect immediately for new connections. Existing connections continue with the old rules until they complete.

In production, I deploy sidecars alongside each service. In Kubernetes, an init container sets up the networking so traffic flows through the sidecar automatically.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: payment-service
spec:
  template:
    spec:
      initContainers:
      - name: init-sidecar
        image: sidecar-injector
      containers:
      - name: payment-service
        image: payment:latest
        ports:
        - containerPort: 8080
      - name: sidecar
        image: sidecar:latest
        ports:
        - containerPort: 15001

The init container configures iptables rules to redirect traffic. The application container runs normally on port 8080. The sidecar container runs alongside it, listening on port 15001.

Resource limits ensure the sidecar doesn't use too much memory or CPU. I typically allocate 50MB memory and 0.1 CPU to each sidecar. This is enough for most workloads without impacting the main service.

The benefits become clear when you have many services. Instead of each service implementing security, monitoring, and reliability logic, they all share the same sidecar implementation. Updates to communication logic happen once in the sidecar, not in every service.

Debugging is easier too. When there's a communication problem, I check the sidecar logs and metrics instead of digging through each service. The sidecar provides a consistent view of all inter-service communication.

Performance impact is minimal. The sidecar adds less than 1 millisecond to most requests. For local communication within the same host or data center, this is negligible compared to the benefits.

The sidecar pattern changes how I think about microservices. Services become simpler, focusing only on their business logic. Communication concerns move to infrastructure where they can be managed consistently. This separation makes both development and operations easier.

Building this in Go works well. The language's simplicity and performance characteristics fit the sidecar's requirements. Concurrency features handle many simultaneous connections efficiently. The standard library provides solid networking primitives.

What starts as a simple proxy grows into a complete communication management system. It handles the complexity so my services don't have to. Each service can focus on what makes it unique while relying on the sidecar for everything else.

This approach scales well. Adding a new service means deploying it with a sidecar. The sidecar automatically integrates with the existing mesh, picking up configuration and joining the network. There's no need to update other services or change communication code.

The sidecar becomes an extension of the platform. It provides services with capabilities they don't need to build themselves. This consistency across all services simplifies troubleshooting, security auditing, and performance optimization.

Over time, I add more features to the sidecar. Request transformation, response caching, A/B testing routing—all implemented once in the sidecar, available to all services. The sidecar evolves independently of the services, providing new capabilities without service changes.

This separation of concerns proves valuable. Services change for business reasons. The sidecar changes for operational reasons. Each evolves at its own pace, connected through clean interfaces. Changes on one side don't break the other.

The result is a system that's easier to understand, operate, and evolve. Services do less but accomplish more. The infrastructure handles complexity, letting services focus on delivering value. That's the power of the sidecar pattern for microservices communication.

📘 Checkout my latest ebook for free on my channel!

Be sure to like, share, comment, and subscribe to the channel!

101 Books

101 Books is an AI-driven publishing company co-founded by author Aarav Joshi. By leveraging advanced AI technology, we keep our publishing costs incredibly low—some books are priced as low as $4—making quality knowledge accessible to everyone.

Check out our book Golang Clean Code available on Amazon.

Stay tuned for updates and exciting news. When shopping for books, search for Aarav Joshi to find more of our titles. Use the provided link to enjoy special discounts!