DEV Community: Syed Omair

Creating Pods Without Deployment or Service: The Bare-Metal Kubernetes Experience

Syed Omair — Sat, 06 Dec 2025 17:26:54 +0000

Introduction

In Kubernetes, while Deployments and Services are the recommended approach for managing applications, there are legitimate scenarios where you might want to create a Pod directly without these abstractions. This "bare-metal" approach to Pod management gives you maximum control but comes with important trade-offs. Let's explore when, why, and how to use standalone Pods in Kubernetes.

Creating a Standalone Pod: Basic Examples

Example 1: The Simplest Pod Definition

# simple-pod.yaml
apiVersion: v1
kind: Pod
metadata:
  name: standalone-nginx
  labels:
    app: nginx
spec:
  containers:
  - name: nginx-container
    image: nginx:latest
    ports:
    - containerPort: 80

Create and verify the pod:

kubectl apply -f simple-pod.yaml
kubectl get pods
kubectl describe pod standalone-nginx

Example 2: A Batch Job Pod (Without Job Controller)

# batch-pod.yaml
apiVersion: v1
kind: Pod
metadata:
  name: data-processor
spec:
  restartPolicy: OnFailure  # Note: Not "Always"
  containers:
  - name: processor
    image: busybox
    command: ["/bin/sh"]
    args: ["-c", "echo 'Processing data...'; sleep 30; echo 'Done!'"]

Create and verify the pod:

kubectl apply -f batch-pod.yaml
kubectl get pods
kubectl describe pod data-processor

Why Create Pods Without Deployments or Services?

1. One-Off Tasks and Ephemeral Workloads

# cleanup-job.yaml
apiVersion: v1
kind: Pod
metadata:
  name: temp-cleanup
spec:
  restartPolicy: Never  # We don't want this to restart
  containers:
  - name: cleaner
    image: busybox
    command: ["/bin/sh"]
    args: ["-c", "rm -rf /tmp/old-files/*"]

Use Case: Cleaning up temporary files, running database migrations, or executing a single batch operation where automatic restart would be harmful.

2. Debugging and Troubleshooting

# network-debug-pod.yaml
apiVersion: v1
kind: Pod
metadata:
  name: network-tester
spec:
  containers:
  - name: tester
    image: nicolaka/netshoot  # Popular network troubleshooting image
    command: ["sleep", "infinity"]

Create and debug:

kubectl apply -f network-debug-pod.yaml
kubectl exec -it network-tester -- bash

Now you can run curl, dig, ping, tcpdump, etc.
Advantage: Quick spin-up without deployment overhead. Perfect for temporary diagnostic tools.

3. Init Containers (Within Pods)

# init-container-pod.yaml
apiVersion: v1
kind: Pod
metadata:
  name: app-with-init
spec:
  containers:
  - name: main-app
    image: myapp:latest
    ports:
    - containerPort: 8080
  initContainers:
  - name: init-db
    image: busybox
    command: ['sh', '-c', 'until nslookup database-service; do echo waiting for database; sleep 2; done;']

Note: Init containers are part of Pod spec, not separate resources.

4. Learning and Experimentation

# learning-pod.yaml
apiVersion: v1
kind: Pod
metadata:
  name: learning-experiment
spec:
  containers:
  - name: experiment
    image: nginx
    env:
    - name: LEARNING_MODE
      value: "true"
    resources:
      requests:
        memory: "64Mi"
        cpu: "250m"
      limits:
        memory: "128Mi"
        cpu: "500m"

Benefit: Direct exposure to Pod lifecycle without controller abstractions.

5. Specialized Node Operations

# node-specific-pod.yaml
apiVersion: v1
kind: Pod
metadata:
  name: node-maintenance
spec:
  nodeName: node-01  # Direct node assignment
  containers:
  - name: maintenance
    image: ubuntu
    command: ["/bin/bash"]
    args: ["-c", "apt-get update && apt-get upgrade -y"]

Use Case: Node-specific maintenance tasks where you need to target a specific node.

Advantages of Standalone Pods

Simplicity and Clarity
Direct Control
Quick Iteration
Explicit Failure Handling
Resource Efficiency
- No deployment controller overhead:
- No reconciliation loops
- No status tracking
- Less etcd traffic

Disadvantages and Production Concerns

No Self-Healing
No Rolling Updates
No Scaling Capabilities
Limited Service Integration
Manual Lifecycle Management

When to Use (and When Not to Use)

Use Standalone Pods When:

Temporary debugging or troubleshooting
One-off batch jobs (though consider Job resource)
Learning and experimentation
Init containers within other pods
Simple, stateless utilities with no HA requirements
Node-specific operations requiring direct assignment

Avoid Standalone Pods When:

Production applications requiring high availability
Services needing load balancing
Applications requiring zero-downtime updates
Workloads that need automatic scaling
Stateful applications (use StatefulSet instead)
Long-running services accessed by other pods

Conclusion

Standalone Pods in Kubernetes serve an important niche in the container orchestration ecosystem. They provide the raw, unabstracted access to container execution that's essential for debugging, one-off tasks, and learning. However, they lack the robustness and automation features that make Kubernetes powerful for production workloads.

Key Takeaways:
Use standalone Pods strategically for their intended purposes
Understand the trade-offs between control and automation
Know when to graduate to Deployments and Services
Always clean up temporary standalone Pods
Document the purpose of each standalone Pod for team clarity

Remember: Kubernetes is about abstraction and automation. While standalone Pods give you escape hatches from these abstractions, they should be the exception rather than the rule in a well-architected Kubernetes environment.

Kubernetes Port Forwarding: A Practical Guide with Examples

Syed Omair — Sat, 06 Dec 2025 16:11:37 +0000

Introduction

Kubernetes port forwarding is a powerful feature that allows you to access and debug applications running inside your cluster directly from your local machine. Whether you're a developer testing a service, an operator troubleshooting issues, or someone who needs temporary access to internal resources, port forwarding provides a convenient bridge between your local environment and the Kubernetes cluster.

In this technical deep dive, we'll explore how port forwarding works under the hood, walk through practical examples, and discuss best practices for using this feature effectively.

What is Port Forwarding in Kubernetes?

Port forwarding creates a secure tunnel between your local machine and a specific pod or service running in your Kubernetes cluster. This allows you to access internal cluster resources as if they were running locally on your machine.

How It Works: The Technical Details

When you run kubectl port-forward, here's what happens:

Kubectl Client initiates a request to the Kubernetes API server
API Server authenticates and authorizes the request
Kubelet on the target node establishes the connection
Tunnel is created using SPDY or WebSocket protocol
Traffic flows through the API server proxy to the target pod

The architecture looks like this:
Local Machine → kubectl → API Server → Kubelet → Target Pod

Prerequisites

Before we begin, ensure you have:

kubectl installed and configured
Access to a Kubernetes cluster
Basic understanding of pods, services, and deployments

Basic Port Forwarding Examples

Example 1: Forwarding to a Pod
Let's start with the most common use case - forwarding to a specific pod.

First, let's create a simple nginx pod:

# nginx-pod.yaml
apiVersion: v1
kind: Pod
metadata:
  name: nginx-demo
spec:
  containers:
  - name: nginx
    image: nginx:latest
    ports:
    - containerPort: 80

Create the pod:

kubectl apply -f nginx-pod.yaml

Now, let's forward local port 2224 to the pod's port 80:

kubectl port-forward pod/nginx-demo 2224:80

You'll see output similar to:

Forwarding from 127.0.0.1:2224 -> 80 Forwarding from [::1]:2224 -> 80

Now open your browser and navigate to http://localhost:2224
OR

curl http://localhost:2224

you'll see the nginx welcome page!

Example 2: Forwarding to a Deployment
When working with deployments, you typically forward to a specific pod within the deployment. First, let's create a deployment:

# web-app-deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: web-app
spec:
  replicas: 2
  selector:
    matchLabels:
      app: web-app
  template:
    metadata:
      labels:
        app: web-app
    spec:
      containers:
      - name: web
        image: nginx:latest
        ports:
        - containerPort: 80

Create the deployment:

kubectl apply -f web-app-deployment.yaml

Get the pod name:

kubectl get pods -l app=web-app

Forward to one of the pods:

kubectl port-forward pod/web-app-7d75f47444-abc123 8080:80

Example 3: Forwarding to a Service

You can also forward directly to a service. This is particularly useful when you want to leverage the service's load balancing capabilities.

# web-service.yaml
apiVersion: v1
kind: Service
metadata:
  name: web-service
spec:
  selector:
    app: web-app
  ports:
  - port: 80
    targetPort: 80

Create the service:

kubectl apply -f web-service.yaml

Forward to the service:

kubectl port-forward svc/web-service 8080:80

Advanced Port Forwarding Techniques

Multiple Port Forwarding
You can forward multiple ports simultaneously. This is useful for applications that expose multiple services:

kubectl port-forward pod/multi-port-app 8080:80 8443:443 3000:3000

Background Port Forwarding
For long-running port forwarding sessions, you might want to run it in the background:

# Start port forwarding in background
kubectl port-forward pod/nginx-demo 8080:80 &

Save the process ID for later

PORT_FORWARD_PID=$!

When you're done, kill the process

kill $PORT_FORWARD_PID

Security Considerations

While port forwarding is incredibly useful, it's important to use it securely:

Authentication: Port forwarding uses your kubeconfig credentials
Authorization: RBAC rules apply to port forwarding requests
Network Policies: They don't affect port-forwarded traffic
Temporary Access: Use it for debugging, not for permanent access
Audit Logging: Port forwarding requests are logged in the API server audit logs

Conclusion

Kubernetes port forwarding is an essential tool in the Kubernetes practitioner's toolkit. It provides a simple yet powerful way to bridge the gap between local development and cluster-based services. While perfect for debugging and development scenarios, remember that it's not designed for production traffic or permanent access solutions.

By understanding how to use port forwarding effectively—from basic single-port forwarding to advanced multi-port scenarios—you can significantly improve your development and debugging workflow in Kubernetes environments.

Remember: With great power comes great responsibility. Use port forwarding judiciously, always consider security implications, and clean up after your debugging sessions to maintain a secure and efficient Kubernetes environment.

The Hidden Costs of Inefficient Code: A Case Study

Syed Omair — Wed, 19 Mar 2025 21:10:08 +0000

As developers, we often focus on getting our code to work, but sometimes overlook the performance implications of our design choices. In this post, we'll explore how a seemingly minor difference in code can significantly impact performance, using a real-world example from our own system.

The Scenario

We have two versions of code that achieve the same goal: fetching user points from a point server. The first version is optimized, while the second is less efficient. Let's dive into both and see why one outperforms the other.

Version 1: Optimized Code

userList, count, err := u.repo.GetAllUserDB(limit, offset, orderBy, sort)
if err != nil {
    return err
}

conn, err := u.pointServiceConnectionPool.Get()
if err != nil {
    return fmt.Errorf("failed to get connection from pool: %v", err)
}
defer u.pointServiceConnectionPool.Put(conn)

client := pb.NewPointServerClient(conn)

ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()

userIDs := []string{}
for _, user := range userList {
    userIDs = append(userIDs, user.ID)
}

r, err := client.GetUserListPoints(ctx, &pb.UserListRequest{UserIds: userIDs})
if err != nil {
    u.logger.Error("failed to get points for users", zap.Error(err), zap.Any("userIDs", userIDs))
    return err
}
userPoints := r.GetUserPoints()
for k, v := range userPoints {
    u.logger.Debug("user points", zap.String("user_id", k), zap.Any("points", v))
}

userList = updateUserListWithPoints(userList, userPoints)

return nil

Version 2: Inefficient Code

userList, count, err := u.repo.GetAllUserDB(limit, offset, orderBy, sort)
if err != nil {
    return err
}

conn, err := u.pointServiceConnectionPool.Get()
if err != nil {
    return fmt.Errorf("failed to get connection from pool: %v", err)
}
defer u.pointServiceConnectionPool.Put(conn)

client := pb.NewPointServerClient(conn)

for _, user := range userList {
    u.logger.Debug("fetch user points", zap.String("user_id", user.ID))

    ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
    defer cancel()

    r, err := client.GetUserPoints(ctx, &pb.PointRequest{UserId: user.ID})
    if err != nil {
        u.logger.Error("failed to get user points", zap.Error(err), zap.String("userID", user.ID))
        continue
    }
    u.logger.Debug("user points", zap.String("user_id", user.ID), zap.String("user points", r.GetUserPoint()))

    point, err := strconv.Atoi(r.GetUserPoint())
    if err != nil {
        point = 0
    }
    user.Point = point
}

Prometheus Query:
histogram_quantile(0.95, rate(http_response_time_seconds_bucket[5m]))

What's the Difference?

The key difference between these two versions lies in how they interact with the point server:

Version 1 fetches points for all users in a single request using GetUserListPoints. This approach minimizes the number of requests to the server, reducing overhead and latency.
Version 2 fetches points for each user individually using GetUserPoints. This results in multiple requests to the server, increasing both the number of network calls and the overall processing time.

Performance Impact

When we look at the Prometheus graphs for these two versions, the difference is striking. The response time for Version 1 is significantly lower than Version 2. This is because Version 1 reduces the number of requests to the point server, minimizing network latency and server load.

Lessons Learned

Batching Requests: When possible, batching requests can significantly improve performance by reducing the number of network calls.
Efficient API Usage: Using APIs designed for bulk operations can reduce overhead compared to making individual requests.
Monitoring Performance: Tools like Prometheus are invaluable for identifying performance bottlenecks and optimizing code.

Conclusion

Even seemingly minor differences in code can have a profound impact on performance. By optimizing how we interact with external services and leveraging batching, we can significantly improve response times and reduce system load. Always keep an eye on performance metrics and be mindful of how your code interacts with external systems. Small changes can add up to make a big difference in the efficiency and scalability of your applications.

Improving Performance with Concurrency in Golang: A Case Study

Syed Omair — Mon, 03 Mar 2025 09:01:47 +0000

In modern software development, performance is often a critical factor, especially when dealing with applications that require fetching and processing large amounts of data. Recently, I worked on optimizing a Go method that fetches user data and statistics from a database. By introducing concurrency, I was able to significantly reduce the execution time of the method. In this post, I’ll walk you through the changes I made and the results I achieved.

The Problem

The method in question, GetAllUsersData, is responsible for fetching a list of users along with various statistics such as the highest and lowest age, average age, highest and lowest salary, and average salary. Initially, the method was implemented sequentially, meaning each database query was executed one after the other. Here’s a simplified version of the original implementation:

// GetAllUsersData fetches user data and statistics 
func (c *Controller) GetAllUsersData(limit, offset int, orderBy, sort string) (map[string]interface{}, error) {
    methodName := "GetAllUsersData"
    c.Logger.Debug("method start", zap.String("method", methodName))
    start := time.Now()

    var (
        userList          []*models.User
        count             string
        intUserHighAge    int
        intUserLowAge     int
        fltUserAvgAge     float64
        fltUserAvgSalary  float64
        fltUserLowSalary  float64
        fltUserHighSalary float64
        err               error
    )

    userList, count, err = c.Repo.GetAllUserDB(limit, offset, orderBy, sort)
    if err != nil {
        return nil, err
    }
    intUserHighAge, err = c.Repo.GetUserHighAge()
    if err != nil {
        return nil, err
    }
    intUserLowAge, err = c.Repo.GetUserLowAge()
    if err != nil {
        return nil, err
    }
    fltUserAvgAge, err = c.Repo.GetUserAvgAge()
    if err != nil {
        return nil, err
    }
    fltUserLowSalary, err = c.Repo.GetUserLowSalary()
    if err != nil {
        return nil, err
    }
    fltUserHighSalary, err = c.Repo.GetUserHighSalary()
    if err != nil {
        return nil, err
    }
    fltUserAvgSalary, err = c.Repo.GetUserAvgSalary()
    if err != nil {
        return nil, err
    }
    responseUserObj := models.ResponseUser{
        HighAge:    strconv.Itoa(intUserHighAge),
        LowAge:     strconv.Itoa(intUserLowAge),
        AvgAge:     fmt.Sprintf("%.2f", fltUserAvgAge),
        HighSalary: fmt.Sprintf("%.2f", fltUserHighSalary),
        LowSalary:  fmt.Sprintf("%.2f", fltUserLowSalary),
        AvgSalary:  fmt.Sprintf("%.2f", fltUserAvgSalary),
        Count:      count,
        List:       userList,
    }

    var responseObj map[string]interface{}
    err = mapstructure.Decode(responseUserObj, &responseObj)
    if err != nil {
        return nil, err
    }

    duration := time.Since(start)
    seconds := int(duration.Seconds()) % 60
    milliseconds := duration.Milliseconds() % 1000
    strDuration := fmt.Sprintf("%02d.%03d", seconds, milliseconds)
    c.Logger.Debug("method end", zap.String("method", methodName), zap.String("Time elapsed for this method:", strDuration))
    return responseObj, nil

}

While this approach worked, it was not optimal. Each database call had to wait for the previous one to complete, leading to longer execution times. For example, the method took 64 milliseconds to complete in its sequential form.

The Solution: Introducing Concurrency

To improve performance, I decided to leverage Go’s concurrency model using Goroutines and the errgroup package. The idea was to execute all database queries concurrently, allowing them to run in parallel and reduce the total execution time.

Here’s the updated version of the method:

// GetAllUsersData fetches user data and statistics concurrently.
func (c *Controller) GetAllUsersData(limit, offset int, orderBy, sort string) (map[string]interface{}, error) {
    methodName := "GetAllUsersData"
    c.Logger.Debug("method start", zap.String("method", methodName))
    start := time.Now()
    //ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
    //defer cancel()

    g, _ := errgroup.WithContext(context.Background())

    var (
        userList          []*models.User
        count             string
        intUserHighAge    int
        intUserLowAge     int
        fltUserAvgAge     float64
        fltUserAvgSalary  float64
        fltUserLowSalary  float64
        fltUserHighSalary float64
    )

    g.Go(func() error {
        var err error
        userList, count, err = c.Repo.GetAllUserDB(limit, offset, orderBy, sort)
        if err != nil {
            return err
        }
        return nil
    })

    g.Go(func() error {
        var err error
        intUserHighAge, err = c.Repo.GetUserHighAge()
        if err != nil {
            return err
        }
        return nil
    })

    g.Go(func() error {
        var err error
        intUserLowAge, err = c.Repo.GetUserLowAge()
        if err != nil {
            return err
        }
        return nil
    })

    g.Go(func() error {
        var err error
        fltUserAvgAge, err = c.Repo.GetUserAvgAge()
        if err != nil {
            return err
        }
        return nil
    })

    g.Go(func() error {
        var err error
        fltUserLowSalary, err = c.Repo.GetUserLowSalary()
        if err != nil {
            return err
        }
        return nil
    })

    g.Go(func() error {
        var err error
        fltUserHighSalary, err = c.Repo.GetUserHighSalary()
        if err != nil {
            return err
        }
        return nil
    })

    g.Go(func() error {
        var err error
        fltUserAvgSalary, err = c.Repo.GetUserAvgSalary()
        if err != nil {
            return err
        }
        return nil
    })

    if err := g.Wait(); err != nil {
        return nil, err
    }
    responseUserObj := models.ResponseUser{
        HighAge:    strconv.Itoa(intUserHighAge),
        LowAge:     strconv.Itoa(intUserLowAge),
        AvgAge:     fmt.Sprintf("%.2f", fltUserAvgAge),
        HighSalary: fmt.Sprintf("%.2f", fltUserHighSalary),
        LowSalary:  fmt.Sprintf("%.2f", fltUserLowSalary),
        AvgSalary:  fmt.Sprintf("%.2f", fltUserAvgSalary),
        Count:      count,
        List:       userList,
    }

    var responseObj map[string]interface{}
    err := mapstructure.Decode(responseUserObj, &responseObj)
    if err != nil {
        return nil, err
    }

    duration := time.Since(start)
    seconds := int(duration.Seconds()) % 60
    milliseconds := duration.Milliseconds() % 1000
    strDuration := fmt.Sprintf("%02d.%03d", seconds, milliseconds)
    c.Logger.Debug("method end", zap.String("method", methodName), zap.String("Time elapsed for this method:", strDuration))
    return responseObj, nil
}

In this version, each database query is executed in a separate Goroutine. The errgroup package ensures that all Goroutines complete successfully and handles any errors that may occur.

The Results

After implementing concurrency, I ran the method and compared its performance with the original sequential version. The results were impressive:

Without Concurrency: 64 milliseconds
With Concurrency: 24 milliseconds

By running the database queries concurrently, the method’s execution time was reduced by 62.5%. This is a significant improvement, especially in scenarios where the method is called frequently or where low latency is critical.

Key Takeaways

Concurrency is Powerful: Go’s Goroutines and channels make it easy to implement concurrency, allowing you to execute multiple tasks in parallel and improve performance.
Error Handling Matters: Using errgroup simplifies error handling in concurrent operations, ensuring that any errors are properly propagated and handled.
Measure and Optimize: Always measure the performance of your code before and after optimization to quantify the impact of your changes.

Conclusion

Introducing concurrency in the GetAllUsersData method was a straightforward yet highly effective way to improve its performance. By running database queries concurrently, I was able to reduce the execution time from 64 milliseconds to 24 milliseconds—a significant improvement. If you’re working on similar performance-critical applications, I highly recommend exploring Go’s concurrency features to unlock faster and more efficient code.

The Singleton Pattern in Golang

Syed Omair — Mon, 03 Mar 2025 06:10:43 +0000

In software development, there are times when you need to ensure that only one instance of a particular class exists. This is where the Singleton pattern comes into play. It's a creational design pattern that restricts the instantiation of a class to a single object.

Why Use Singleton?

Resource Management: When dealing with resources like database connections, file systems, or configuration settings, having multiple instances can lead to conflicts and inefficiencies.
Centralized Access: Singletons provide a global point of access, making it easy for different parts of your application to interact with a shared resource.
Configuration: A singleton can hold application-wide configuration settings.

The Go Implementation: Database Connection Example

Let's illustrate the Singleton pattern with a practical example: managing database connections in Go.

type PostgresAdapter struct {
    dbUrl         string
    dbMaxIdle     int
    dbMaxOpen     int
    dbMaxLifeTime int
    dbMaxIdleTime int
    gormConf      string
    db            *gorm.DB
    mu            sync.Mutex
}

func NewPostgresAdapter(url string, dbMaxIdle, dbMaxOpen, dbMaxLifeTime, dbMaxIdleTime int, gormConf string) *PostgresAdapter {
    return &PostgresAdapter{dbUrl: url,
        dbMaxIdle:     dbMaxIdle,
        dbMaxOpen:     dbMaxOpen,
        dbMaxLifeTime: dbMaxLifeTime,
        dbMaxIdleTime: dbMaxIdleTime,
        gormConf:      gormConf,
    }
}

func (p *PostgresAdapter) MakeConnection() (*gorm.DB, error) {
    p.mu.Lock()
    defer p.mu.Unlock()

    if p.db != nil {
        return p.db, nil
    }

    db, err := makeConnection(postgres.Open(p.dbUrl), p.dbUrl, p.dbMaxIdle, p.dbMaxOpen, p.dbMaxLifeTime, p.dbMaxIdleTime, p.gormConf)
    if err != nil {
        return nil, err
    }

    p.db = db
    return db, nil
}

In this code, we have PostgresAdapter that handle database connections using the gorm library. We want to ensure that each adapter creates only one database connection. Here's how the Singleton pattern is implemented:

Private Instance:

Each adapter (PostgresAdapter) has a db *gorm.DB field to store the database connection.

Mutex for Thread Safety:

A sync.Mutex (mu) is used to protect the critical section where the database connection is created. This prevents race conditions in concurrent environments.

Check for Existing Instance:

The MakeConnection() method checks if the db field is already populated. If it is, the existing connection is returned.

Instance Creation:

If the db field is nil, the makeConnection() function is called to create the database connection.
The newly created connection is then stored in the db field.

Key Singleton Elements in Action:

The mu.Lock() and mu.Unlock() calls ensure that only one goroutine can create the database connection at a time.
The if p.db != nil checks prevent redundant database connection creation.
By storing the database connection in the adapter struct, we maintain a single, persistent connection throughout the adapter's lifecycle.

Benefits of This Approach:

Efficiency: We avoid the overhead of creating multiple database connections.
Consistency: All parts of the application using the adapter share the same database connection.
Thread Safety: The mutex ensures that the Singleton implementation is safe for concurrent use.

Important Considerations:

Testability: Singletons can make unit testing more challenging due to their global state. Consider using dependency injection to make your code more testable.
Overuse: Avoid using the Singleton pattern when a simple object instance would suffice.

Conclusion:

The Singleton pattern is a valuable tool for managing resources and ensuring centralized access in your applications. By understanding its principles and implementing it carefully, you can create more efficient and robust software. In our database connection example, the Singleton pattern helps us maintain a single, thread-safe connection, improving performance and resource utilization.

Using a Reverse Proxy to Expose Multiple Microservices Through a Single Port in Docker Compose

Syed Omair — Mon, 03 Mar 2025 05:13:11 +0000

In modern microservices architectures, applications are often broken down into smaller, independent services that communicate with each other. Each microservice typically runs on its own port, which can lead to challenges when exposing these services to the outside world. For example, managing multiple ports, handling cross-origin requests, and ensuring consistent API gateways can become cumbersome.

A reverse proxy is a powerful solution to these challenges. By using a reverse proxy like NGINX, you can expose multiple microservices through a single port, simplifying access and improving manageability. Let’s dive into how this works and the benefits it brings.

How It Works

A reverse proxy acts as an intermediary between clients (e.g., browsers, mobile apps) and your microservices. It listens on a single port (e.g., 80 or 8180) and routes incoming requests to the appropriate microservice based on the request path.

Example Setup
Here’s a simplified example using Docker Compose and NGINX:

1. Docker Compose File:

Define your microservices (user_service, department_service, etc.).
Add an NGINX service as the reverse proxy.

version: "3.8"

services:
  reverse-proxy:
    image: nginx:latest
    ports:
      - "8180:80"  # Expose NGINX on port 8180
    volumes:
      - ./nginx.conf:/etc/nginx/nginx.conf  # Custom NGINX configuration
    depends_on:
      - user_service
      - department_service

  user_service:
    image: your_user_service_image
    ports:
      - "5001:8185"  # Internal port for user_service
    environment:
      - PORT=8185

  department_service:
    image: your_department_service_image
    ports:
      - "5002:8185"  # Internal port for department_service
    environment:
      - PORT=8185

2. NGINX Configuration:

Define routing rules in nginx.conf to map paths to microservices.

events {}

http {
  server {
    listen 80;

    # Route requests to the user_service
    location /api/users/ {
      proxy_pass http://user_service:8185/;
      proxy_set_header Host $host;
      proxy_set_header X-Real-IP $remote_addr;
      proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
      proxy_set_header X-Forwarded-Proto $scheme;
    }

    # Route requests to the department_service
    location /api/departments/ {
      proxy_pass http://department_service:8185/;
      proxy_set_header Host $host;
      proxy_set_header X-Real-IP $remote_addr;
      proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
      proxy_set_header X-Forwarded-Proto $scheme;
    }
  }
}

3. Accessing the Services:

Clients can now access the microservices through the reverse proxy:

http://localhost:8180/api/users → Routes to user_service.
http://localhost:8180/api/departments → Routes to department_service.

How to Deploy a Container from GitHub to AWS ECR and ECS through OIDC

Syed Omair — Thu, 27 Feb 2025 16:15:04 +0000

Deploying a container from GitHub Actions to AWS Elastic Container Registry (ECR) and AWS Elastic Container Service (ECS) can be done securely using OpenID Connect (OIDC). This method eliminates the need to store long-lived AWS credentials, making your CI/CD pipeline more secure. This guide will walk you through setting up OIDC authentication for GitHub Actions to push Docker images to AWS ECR.

Step 1: Enable OIDC Provider in AWS

Sign in to AWS Console and navigate to IAM.
Go to Identity providers > Add provider.
Select OpenID Connect as the provider type.
Enter the Provider URL:

https://token.actions.githubusercontent.com

Click Get thumbprint (AWS will auto-populate this).
Under Audience, enter:

https://github.com

Click Add provider.

Step 2: Create an IAM Role for GitHub Actions

In the AWS Console, go to IAM > Roles > Create Role.
Select Web identity as the trusted entity type.
Choose the OIDC provider you just created.
Under 'Audience' enter https://github.com
Under 'GitHub organization' enter your .
Click Next.
Under 'Name' enter github-role
Click Create

Step 3: Attach Policies for ECR and ECS Access

create a custom policy for more control

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "ecr:GetAuthorizationToken",
                "ecr:BatchCheckLayerAvailability",
                "ecr:PutImage",
                "ecr:InitiateLayerUpload",
                "ecr:UploadLayerPart",
                "ecr:CompleteLayerUpload",
                "ecr:DescribeRepositories",
                "ecr:CreateRepository",
                "ecr:ListImages",
                "ecr:BatchDeleteImage"
            ],
            "Resource": "*"
        }
    ]
}

Click Next, then give the policy a name (e.g., GitHubActionsECR).
Click Create policy. Add another policy name githubECS for ECS Update

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "ecs:UpdateService",
                "ecs:DescribeServices"
            ],
            "Resource": [
                "<ARN of ECS service 1",
                "<ARN of ECS service 2",
                "<ARN of ECS service n"
            ]
        }
    ]
}

Step 4: Update Trust PolicyModify the trust policy to allow GitHub Actions to assume this role:

Go to IAM > Roles > Select your role (GitHubActionsECR).
Click Trust relationships > Edit trust policy.
Replace the existing policy with:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": {
                "Federated": "arn:aws:iam::<AWS_ACCOUNT_ID>:oidc-provider/token.actions.githubusercontent.com"
            },
            "Action": "sts:AssumeRoleWithWebIdentity",
            "Condition": {
                "StringEquals": {
                    "token.actions.githubusercontent.com:aud": "https://github.com"
                },
                "StringLike": {
                    "token.actions.githubusercontent.com:sub": "repo:<GITHUB_ORG_OR_USER>/<REPO_NAME>:*"
                }
            }
        }
    ]
}

(Replace with your AWS account ID)
(Replace / with your GitHub organization and repository name)

Click Update policy.

Step 5: Configure GitHub Actions Workflow In your GitHub repository:

Go to Settings > Secrets and variables > Actions ->Add new repository secret:
Name: AWS_DB_URL
Value: AWS RDS full path for example postgres://db_username:password@rds instance name:5432/DB name
Name: AWS_ECR_URI
Value: ECS repo path for example AWS_ AccountID.dkr.ecr.us-east-2.amazonaws.com/backend-microservices/user-service-stage
Name: AWS_REGION
Value: us-east-2
Name: AWS_ROLE_ARN
Value: The ARN of the IAM role you created (found in AWS IAM).

Step 6: Update GitHub Actions Workflow (.github/workflows/deploy.yml)

Modify your workflow YAML file to assume the IAM role:

name: Deploy services to Stage AWS ECR and ECS

on:
  push:
    branches:
      - stage
jobs:
  deploy:
    runs-on: ubuntu-latest
    permissions:
      id-token: write
      contents: read
    steps:
      - name: Checkout code
        uses: actions/checkout@v2

      - name: Set up Go
        uses: actions/setup-go@v4
        with:
          go-version: 1.23

      - name: Install dependencies
        run: go mod tidy

      - name: Run Integration Tests
        run: go test -v ./integration_test/...

      - name: Configure AWS credentials
        uses: aws-actions/configure-aws-credentials@v3
        with:
          role-to-assume: ${{ secrets.AWS_ROLE_ARN }}
          role-session-name: GitHubActionsSession
          aws-region: ${{ secrets.AWS_REGION}}
          audience: https://github.com

      - name: Load Environment Variables
        run: |
          cat .env_stage.example >> $GITHUB_ENV

      - name: Login to Amazon ECR
        id: login-ecr
        uses: aws-actions/amazon-ecr-login@v2

      - name: Build Docker Image for user-service
        run: |
          docker build \
            --build-arg logLevelEnvVar=${LOG_LEVEL} \
            --build-arg databaseURLEnvVar=${{secrets.AWS_DB_URL}} \
            --build-arg portEnvVar=${PORT} \
            --build-arg dBEnvVar=${DB} \
            --build-arg dBMaxIdleEnvVar=${DB_MAX_IDLE} \
            --build-arg dBMaxOpenEnvVar=${DB_MAX_OPEN} \
            --build-arg dBMaxLifeTimeEnvVar=${DB_MAX_LIFE_TIME} \
            --build-arg dBMaxIdleTimeEnvVar=${DB_MAX_IDLE_TIME} \
            --build-arg zapConf=${ZAP_CONF} \
            --build-arg gormConf=${GORM_CONF} \
            --build-arg pprofEnable=${PPROF_ENABLE}  \
            -t ${{secrets.AWS_ECR_URI}}user-service-stage:latest \
            -f service/user_service/Dockerfile  \
            .
          docker push ${{secrets.AWS_ECR_URI}}user-service-stage:latest

      - name: Deploy user-service to AWS ECS
        run: |
          aws ecs update-service --cluster cluster-backend-microservice --service service-user --force-new-deployment

Step 7: Test the Workflow

Push a commit to the repository.
Navigate to Actions in GitHub and verify that the workflow runs successfully.
Your Docker image should now be pushed to Amazon ECR and ECS

Conclusion

Now your GitHub Actions workflow can push images to AWS ECR and ECS without requiring long-lived AWS credentials. This setup is more secure and efficient, enabling seamless container deployments from GitHub to AWS. 🎉

Understanding Dependency Injection Through a Practical Golang Example

Syed Omair — Tue, 25 Feb 2025 12:19:12 +0000

Dependency Injection (DI) is a design pattern that promotes loose coupling and testability in software applications. It allows you to inject dependencies (e.g., services, configurations, or databases) into a component rather than having the component create or manage them directly. This makes your code more modular, maintainable, and easier to test.

In this post, we’ll explore the Dependency Injection pattern using a practical Golang example. We’ll break down the code and explain how DI is implemented in a real-world scenario.

The Example: A Container for Managing Dependencies
The example consists of three Go files that work together to create a "container" for managing dependencies like a logger, database connection, and configuration. Let’s dive into the code and see how DI is applied.

1. Logger Setup

The first file sets up a logger using the zap logging library. The logger is initialized using a configuration file, and the NewLogger function is responsible for creating the logger instance.

func NewLogger(zapConfig string) (*zap.Logger, error) {
    file, err := os.Open(zapConfig)
    if err != nil {
        return nil, fmt.Errorf("failed to open logger config file")
    }
    defer file.Close()

    var cfg zap.Config
    if err := json.NewDecoder(file).Decode(&cfg); err != nil {
        return nil, fmt.Errorf("failed to parse logger config json")
    }

    logger, err := cfg.Build()
    if err != nil {
        return nil, err
    }

    defer logger.Sync()
    logger.Debug("logger construction succeeded")
    return logger, nil
}

Here, the NewLogger function takes a configuration file path (zapConfig) as input and returns a zap.Logger instance. This is an example of constructor injection, where the dependency (logger configuration) is injected into the function.

2. Database Connection Setup

The second file handles database connections using the gorm library. It defines an interface Db and two implementations (PostgresAdapter and MySQLAdapter) for connecting to PostgreSQL and MySQL databases.

type Db interface {
    MakeConnection() (*gorm.DB, error)
}

func NewDBConnectionAdapter(dbName, url string, dbMaxIdle, dbMaxOpen, dbMaxLifeTime, dbMaxIdleTime int, gormConf string) Db {
    switch dbName {
    case Postgres:
        return &PostgresAdapter{dbUrl: url, dbMaxIdle: dbMaxIdle, dbMaxOpen: dbMaxOpen, dbMaxLifeTime: dbMaxLifeTime, dbMaxIdleTime: dbMaxIdleTime, gormConf: gormConf}
    case Mysql:
        return &MySQLAdapter{dbUrl: url, dbMaxIdle: dbMaxIdle, dbMaxOpen: dbMaxOpen, dbMaxLifeTime: dbMaxLifeTime, dbMaxIdleTime: dbMaxIdleTime, gormConf: gormConf}
    }
    return &PostgresAdapter{dbUrl: url, dbMaxIdle: dbMaxIdle, dbMaxOpen: dbMaxOpen, dbMaxLifeTime: dbMaxLifeTime, dbMaxIdleTime: dbMaxIdleTime, gormConf: gormConf}
}

The NewDBConnectionAdapter function acts as a factory, creating the appropriate database adapter based on the dbName parameter. This is an example of factory injection, where the factory decides which implementation to inject.

3. Container for Managing Dependencies

The third file defines a Container interface and its implementation. The container is responsible for managing all dependencies (logger, database, etc.) and injecting them where needed.

type Container interface {
    Logger() *zap.Logger
    Db() *gorm.DB
    Port() string
    PprofEnable() string
}

type container struct {
    logger               *zap.Logger
    db                   *gorm.DB
    port                 string
    pprofEnable          string
    environmentVariables map[string]string
}

func New(envVars map[string]string) (Container, error) {
    c := &container{environmentVariables: envVars}

    var err error
    c.db, err = c.dbSetup()
    if err != nil {
        return c, err
    }
    c.logger, err = c.loggerSetup()
    if err != nil {
        return c, err
    }
    c.port, err = c.portSetup()
    if err != nil {
        return c, err
    }
    c.pprofEnable, err = c.pprofEnableSetup()
    if err != nil {
        return c, err
    }
    return c, nil
}

The New function initializes the container by setting up all dependencies. It uses constructor injection to pass environment variables and configuration to the container. Each dependency (logger, database, etc.) is initialized separately, making the code modular and easy to test.

Key Benefits of Dependency Injection in This Example

Loose Coupling:

The Container does not directly create its dependencies. Instead, it relies on external configurations and factories to provide them. This makes the code more flexible and easier to modify.

Testability:

Since dependencies are injected, you can easily mock them during testing. For example, you can replace the real database connection with a mock database for unit tests.

Single Responsibility Principle:

Each component (logger, database adapter, etc.) has a single responsibility. The Container is only responsible for managing dependencies, not for creating them.

Reusability:

The Db interface and its implementations can be reused across different parts of the application. You can switch between PostgreSQL and MySQL without changing the core logic.

How to Use the Container

Here’s how you can use the Container in your application:


func main() {
        c, err := container.New(map[string]string{
        container.LogLevelEnvVar:      os.Getenv(container.LogLevelEnvVar),
        container.DatabaseURLEnvVar:   os.Getenv(container.DatabaseURLEnvVar),
        container.PortEnvVar:          os.Getenv(container.PortEnvVar),
        container.DBMaxIdleEnvVar:     os.Getenv(container.DBMaxIdleEnvVar),
        container.DBMaxOpenEnvVar:     os.Getenv(container.DBMaxOpenEnvVar),
        container.DBMaxLifeTimeEnvVar: os.Getenv(container.DBMaxLifeTimeEnvVar),
        container.DBMaxIdleTimeEnvVar: os.Getenv(container.DBMaxIdleTimeEnvVar),
        container.ZapConf:             os.Getenv(container.ZapConf),
        container.GormConf:            os.Getenv(container.GormConf),
        container.PprofEnable:         os.Getenv(container.PprofEnable),
    })
    if err != nil {
        defer func() {
            fmt.Println("server initialization failed error: %w", err)
        }()
        panic("server initialization failed")
    }


    logger := c.Logger()
    db := c.Db()

    logger.Info("Application started", zap.String("port", c.Port()))
    // Use db and logger in your application...
}

Conclusion

The Dependency Injection pattern is a powerful tool for building modular, testable, and maintainable applications. In this example, we saw how DI can be implemented in Go using interfaces, factories, and a container to manage dependencies.

By adopting DI, you can:

Decouple your application’s components.
Improve testability.
Make your code more reusable and maintainable.

If you’re new to Dependency Injection, I encourage you to try implementing it in your own projects. Start small, and gradually refactor your code to use DI where it makes sense. Happy coding!