DEV Community: Deepanshu

Building a Production-Grade E-Commerce Platform on GCP: A Complete DevOps Journey

Deepanshu — Thu, 08 Jan 2026 20:59:16 +0000

Hey there, fellow developers!

Have you ever wondered what it takes to build and deploy a real production-grade application on the cloud? Not just a simple "Hello World" app, but a full-fledged microservices platform with proper CI/CD, monitoring, security, and all the bells and whistles that make it truly production-ready?

Well, you're in the right place! In this comprehensive guide, I'll walk you through my journey of building and deploying a complete e-commerce platform on Google Cloud Platform (GCP) using modern DevOps practices. Whether you're a beginner taking your first steps into cloud-native development or an experienced engineer looking to level up your skills, this guide has something for everyone.

What You'll Learn

By the end of this guide, you'll understand:

How to architect a microservices-based application from scratch
Setting up Google Kubernetes Engine (GKE) for production workloads
Implementing GitOps with ArgoCD for automated deployments
Building a complete CI/CD pipeline with GitHub Actions
Adding full observability with Datadog and SonarQube
Managing infrastructure as code with Terraform
Implementing security best practices (SSL/TLS, secrets management, vulnerability scanning)
Setting up automated monitoring and alerting

Why This Project?

I created this project to bridge the gap between simple tutorials and real-world production systems. Most tutorials show you how to deploy a single container, but they don't show you:

How to manage multiple microservices
How to set up proper CI/CD pipelines
How to monitor and debug issues in production
How to handle secrets and security
How to make your infrastructure reproducible

This project addresses all of these challenges and more!

Application Souce Code: Link

Project Overview

This is a cloud-native e-commerce platform built with:

5 microservices written in Java, Go, and Node.js
Kubernetes (GKE) for container orchestration
ArgoCD for GitOps-based deployments
GitHub Actions for CI/CD automation
Datadog for monitoring, logging, and APM
SonarQube for code quality analysis
Terraform for infrastructure provisioning
Helm for Kubernetes package management

The Architecture

Let me show you the high-level architecture of what we're building:

┌─────────────────────────────────────────────────────────────┐
│                    Internet Traffic (HTTPS)                  │
└────────────────────┬────────────────────────────────────────┘
                     │
            ┌────────▼────────┐
            │  GCP Load       │
            │  Balancer       │
            └────────┬────────┘
                     │
            ┌────────▼────────┐
            │  NGINX Ingress  │
            │  Controller     │
            │  (SSL/TLS)      │
            └────────┬────────┘
                     │
        ┌────────────┼────────────┐
        │            │            │
   ┌────▼───┐  ┌────▼───┐  ┌────▼───┐
   │   UI   │  │  Cart  │  │Checkout│
   │ Service│  │ Service│  │ Service│
   └────┬───┘  └────┬───┘  └────┬───┘
        │           │            │
        └───────┬───┴────────┬───┘
                │            │
           ┌────▼───┐   ┌───▼────┐
           │Catalog │   │ Orders │
           │ Service│   │ Service│
           └────┬───┘   └───┬────┘
                │           │
                └─────┬─────┘
                      │
                ┌─────▼─────┐
                │   MySQL   │
                │  Database │
                └───────────┘

Key Components:

Frontend (UI Service): Java-based web interface for customers
Cart Service: Manages shopping cart operations (Java)
Catalog Service: Product catalog and inventory (Go)
Checkout Service: Handles order processing (Node.js)
Orders Service: Order management and history (Java)
MySQL Database: Persistent data storage

All of this runs on Google Kubernetes Engine, managed by ArgoCD, monitored by Datadog, and deployed automatically via GitHub Actions.

Key Features

DevOps Excellence

** GitOps Workflow**: Every deployment is version-controlled and declarative
** Automated CI/CD**: Push code → Build → Test → Scan → Deploy (all automatic!)
** Full Observability**: See everything happening in your cluster in real-time
Security First: Automated vulnerability scanning, secrets management, SSL/TLS
** Infrastructure as Code**: Entire infrastructure defined in Terraform
** Self-Healing**: Kubernetes automatically restarts failed containers

Production-Ready Features

Auto-scaling: Automatically scales based on traffic
Zero-downtime deployments: Update without affecting users
Multi-zone deployment: High availability across multiple zones
Automated SSL certificates: Let's Encrypt integration
Centralized logging: All logs in one place
Distributed tracing: Track requests across microservices

Technology Stack

Let me break down the technologies we'll be using:

Cloud & Infrastructure

Technology	Purpose	Why We Use It
Google Cloud Platform	Cloud provider	Excellent Kubernetes support, competitive pricing
GKE (Google Kubernetes Engine)	Container orchestration	Managed Kubernetes, auto-updates, built-in monitoring
Terraform	Infrastructure as Code	Reproducible infrastructure, version control
Helm	Kubernetes package manager	Simplifies complex deployments

CI/CD & GitOps

Technology	Purpose	Why We Use It
GitHub Actions	CI/CD pipeline	Free for public repos, easy to configure
ArgoCD	GitOps controller	Automated deployments, easy rollbacks
GCP Artifact Registry	Container registry	Native GCP integration, secure

Monitoring & Quality

Technology	Purpose	Why We Use It
Datadog	Monitoring & APM	Best-in-class observability platform
SonarQube	Code quality	Catches bugs and security issues early
Trivy	Security scanning	Finds vulnerabilities in container images

Networking & Security

Technology	Purpose	Why We Use It
NGINX Ingress	Traffic routing	Industry standard, highly configurable
Cert-Manager	SSL/TLS automation	Free SSL certificates from Let's Encrypt
Kubernetes Secrets	Secrets management	Secure credential storage

Prerequisites

Before we dive in, make sure you have:

Required Tools

Google Cloud Account (with billing enabled)

Sign up here
You get $300 in free credits!

gcloud CLI - Google Cloud command-line tool

   # Install on Linux/macOS
   curl https://sdk.cloud.google.com | bash

   # Or use package manager
   # macOS: brew install google-cloud-sdk
   # Windows: Download installer from cloud.google.com

kubectl - Kubernetes command-line tool

   # Install via gcloud
   gcloud components install kubectl

   # Or standalone
   # macOS: brew install kubectl
   # Linux: snap install kubectl --classic

Terraform - Infrastructure as Code tool

   # macOS
   brew install terraform

   # Linux
   wget https://releases.hashicorp.com/terraform/1.6.0/terraform_1.6.0_linux_amd64.zip
   unzip terraform_1.6.0_linux_amd64.zip
   sudo mv terraform /usr/local/bin/

Docker - For building container images

   # Install from docker.com/get-started

Helm - Kubernetes package manager

   # macOS
   brew install helm

   # Linux
   curl https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3 | bash

Git - Version control

   # Most systems have this pre-installed
   git --version

Required Accounts

GitHub Account (for code hosting and CI/CD)
Datadog Account (free trial available at datadoghq.com)
Domain Name (optional, but recommended for production)

Knowledge Prerequisites

Beginner-Friendly! You should have:

Basic understanding of Docker containers
Familiarity with command-line interfaces
Basic Git knowledge (clone, commit, push)
Willingness to learn!

Nice to Have (but not required):

Understanding of Kubernetes concepts
Experience with cloud platforms
Knowledge of CI/CD pipelines

Don't worry if you're missing some of these - I'll explain everything as we go!

Part 1: Getting Started & Infrastructure Setup

Step 1: Initial Setup & Authentication

First, let's set up your local environment and authenticate with Google Cloud.

1.1 Clone the Repository

# Clone the repository
git clone https://github.com/YOUR_USERNAME/Ecommerce-K8s.git
cd Ecommerce-K8s

1.2 Authenticate with Google Cloud

# Login to Google Cloud
gcloud auth login

# This will open a browser window - sign in with your Google account

1.3 Create a GCP Project

# Create a new project (replace PROJECT_ID with your desired ID)
gcloud projects create YOUR_PROJECT_ID --name="E-Commerce Platform"

# Set as default project
gcloud config set project YOUR_PROJECT_ID

# Enable billing (required for GKE)
# You'll need to do this in the GCP Console: console.cloud.google.com

1.4 Enable Required APIs

# Enable all necessary GCP APIs
gcloud services enable container.googleapis.com
gcloud services enable compute.googleapis.com
gcloud services enable artifactregistry.googleapis.com
gcloud services enable cloudresourcemanager.googleapis.com
gcloud services enable iam.googleapis.com
gcloud services enable dns.googleapis.com

** Pro Tip**: This step might take 2-3 minutes. Grab a coffee! ☕

1.5 Set Your Region and Zone

# Choose a region close to your users
# Popular choices: us-central1, europe-west1, asia-southeast1
gcloud config set compute/region us-central1
gcloud config set compute/zone us-central1-a

Step 2: Infrastructure Provisioning with Terraform

Now we'll use Terraform to create all the infrastructure we need.

2.1 Configure Terraform Variables

cd terraform

# Create a terraform.tfvars file
cat > terraform.tfvars <<EOF
project_id = "YOUR_PROJECT_ID"
region     = "us-central1"
zone       = "us-central1-a"
cluster_name = "ecommerce-gke-cluster"
EOF

2.2 Initialize Terraform

# Initialize Terraform (downloads required providers)
terraform init

You should see output like:

Initializing the backend...
Initializing provider plugins...
- Finding latest version of hashicorp/google...
- Installing hashicorp/google v5.x.x...

Terraform has been successfully initialized!

2.3 Review the Infrastructure Plan

# See what Terraform will create
terraform plan

This shows you:

VPC network and subnets
GKE cluster with node pools
Artifact Registry for container images
IAM roles and service accounts
Firewall rules

** What's Being Created?**

VPC Network: Isolated network for your resources
GKE Cluster: Kubernetes cluster with 3 nodes (auto-scaling enabled)
Artifact Registry: Private container registry
Service Account: For GitHub Actions to deploy
Firewall Rules: Security rules for your cluster

2.4 Apply the Configuration

# Create the infrastructure (this takes 10-15 minutes)
terraform apply

# Type 'yes' when prompted

** Time for a break!** Creating a GKE cluster takes about 10-15 minutes. This is a good time to:

Read through the ArgoCD documentation
Set up your Datadog account
Grab another coffee ☕

2.5 Verify the Cluster

# Configure kubectl to use your new cluster
gcloud container clusters get-credentials ecommerce-gke-cluster \
    --region us-central1

# Verify connection
kubectl get nodes

You should see something like:

NAME                                          STATUS   ROLES    AGE   VERSION
gke-ecommerce-default-pool-xxxxx-xxxx        Ready    <none>   5m    v1.28.x
gke-ecommerce-default-pool-xxxxx-yyyy        Ready    <none>   5m    v1.28.x
gke-ecommerce-default-pool-xxxxx-zzzz        Ready    <none>   5m    v1.28.x

Congratulations! Your Kubernetes cluster is ready!

The GKE cluster running in Google Cloud Console, showing nodes and cluster configuration

🔧 Step 3: Installing Core Components

Now let's install the essential components: NGINX Ingress, Cert-Manager, and ArgoCD.

3.1 Install NGINX Ingress Controller

# Add Helm repository
helm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx
helm repo update

# Install NGINX Ingress
helm install ingress-nginx ingress-nginx/ingress-nginx \
  --namespace ingress-nginx \
  --create-namespace \
  --set controller.service.type=LoadBalancer

Wait for the Load Balancer to be ready:

# This might take 2-3 minutes
kubectl get svc -n ingress-nginx -w

Press Ctrl+C when you see an EXTERNAL-IP assigned.

Load balancers created for external traffic routing

3.2 Install Cert-Manager (for SSL/TLS)

# Add Helm repository
helm repo add jetstack https://charts.jetstack.io
helm repo update

# Install Cert-Manager
helm install cert-manager jetstack/cert-manager \
  --namespace cert-manager \
  --create-namespace \
  --set installCRDs=true

Verify installation:

kubectl get pods -n cert-manager

All pods should be in Running state.

3.3 Configure Let's Encrypt

# Create ClusterIssuer for SSL certificates
kubectl apply -f - <<EOF
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: letsencrypt-prod
spec:
  acme:
    server: https://acme-v02.api.letsencrypt.org/directory
    email: your-email@example.com  # Change this!
    privateKeySecretRef:
      name: letsencrypt-prod
    solvers:
    - http01:
        ingress:
          class: nginx
EOF

** Important**: Replace your-email@example.com with your actual email!

3.4 Install ArgoCD

# Create namespace
kubectl create namespace argocd

# Install ArgoCD
kubectl apply -n argocd -f https://raw.githubusercontent.com/argoproj/argo-cd/stable/manifests/install.yaml

# Wait for pods to be ready
kubectl wait --for=condition=Ready pods --all -n argocd --timeout=300s

Get ArgoCD admin password:

# Retrieve the initial admin password
kubectl -n argocd get secret argocd-initial-admin-secret \
  -o jsonpath="{.data.password}" | base64 -d && echo

** Save this password!** You'll need it to login to ArgoCD.

Access ArgoCD UI:

# Port-forward to access locally
kubectl port-forward svc/argocd-server -n argocd 8080:443

Now open your browser to: https://localhost:8080

Username: admin
Password: (the password you just retrieved)

You now have ArgoCD running!

ArgoCD dashboard showing all microservices managed via GitOps

Step 4: Setting Up the Database

Before deploying microservices, we need a database.

4.1 Create Namespace

# Create namespace for our application
kubectl create namespace retail-store

4.2 Create MySQL Secret

# Create secret for MySQL password
kubectl create secret generic mysql-secret \
  --from-literal=mysql-root-password=YOUR_SECURE_PASSWORD \
  --from-literal=mysql-password=YOUR_SECURE_PASSWORD \
  -n retail-store

4.3 Deploy MySQL

# Deploy MySQL
kubectl apply -f k8s/catalog-mysql.yaml -n retail-store
kubectl apply -f k8s/catalog-mysql-service.yaml -n retail-store

Verify MySQL is running:

kubectl get pods -n retail-store

Wait until the MySQL pod shows Running status.

Initialize the database:

# Run database initialization job
kubectl apply -f k8s/mysql-setup-job.yaml -n retail-store

Step 5: Setting Up Monitoring with Datadog

5.1 Get Your Datadog API Key

Sign up at datadoghq.com (free trial available)
Go to: Organization Settings → API Keys
Copy your API key

5.2 Create Datadog Secret

# Create secret with your Datadog API key
kubectl create secret generic datadog-secret \
  --from-literal=api-key=YOUR_DATADOG_API_KEY \
  -n default

5.3 Deploy Datadog Agent

# Deploy the Datadog agent
kubectl apply -f k8s/datadog-agent.yaml

Verify Datadog is collecting data:

kubectl get pods -l app=datadog-agent

Within 5 minutes, you should see data in your Datadog dashboard!

Real-time monitoring of Kubernetes pod restarts and alerts in Datadog

Detailed view of all pods with resource usage and status

Step 6: Setting Up SonarQube for Code Quality

6.1 Deploy SonarQube

# Deploy SonarQube
kubectl apply -f terraform/sonarqube-production.yaml

6.2 Expose SonarQube

# Change service type to LoadBalancer
kubectl patch svc sonarqube -n default -p '{"spec": {"type": "LoadBalancer"}}'

# Get the external IP
kubectl get svc sonarqube -n default -w

6.3 Access SonarQube

Wait for the EXTERNAL-IP to be assigned, then:

Open browser to: http://EXTERNAL-IP:9000
Default login: admin / admin
Change password when prompted

6.4 Create SonarQube Projects

In the SonarQube UI, create 5 projects:

retail-store-cart
retail-store-catalog
retail-store-checkout
retail-store-orders
retail-store-ui

For each project:

Click "Create Project" → "Manually"
Enter project key and name
Click "Set Up"
Choose "With GitHub Actions"
Copy the token generated

SonarQube code quality metrics showing all microservices passing quality gates

6.5 Configure GitHub Secrets

Go to your GitHub repository → Settings → Secrets and variables → Actions

Add these secrets:

SONAR_TOKEN: Your SonarQube token
SONAR_HOST_URL: http://YOUR_SONARQUBE_IP:9000
GCP_PROJECT_ID: Your GCP project ID
GCP_REGION: us-central1
ARTIFACT_REGISTRY: us-central1-docker.pkg.dev
GCP_SA_KEY: Service account JSON key (from Terraform output)
DATADOG_API_KEY: Your Datadog API key

Part 2: Deploying Microservices & CI/CD

Step 7: Deploying Microservices with ArgoCD

7.1 Configure ArgoCD Applications

# Apply ArgoCD project definition
kubectl apply -f argocd/projects/retail-store-project.yaml

# Apply all ArgoCD application definitions
kubectl apply -f argocd/applications/retail-store-cart.yaml
kubectl apply -f argocd/applications/retail-store-catalog.yaml
kubectl apply -f argocd/applications/retail-store-checkout.yaml
kubectl apply -f argocd/applications/retail-store-orders.yaml
kubectl apply -f argocd/applications/retail-store-ui.yaml

7.2 Sync Applications

In the ArgoCD UI:

Click on each application
Click "Sync" → "Synchronize"

Or use CLI:

# Install ArgoCD CLI
brew install argocd  # macOS
# or download from: https://argo-cd.readthedocs.io/en/stable/cli_installation/

# Login
argocd login localhost:8080

# Sync all applications
argocd app sync retail-store-cart
argocd app sync retail-store-catalog
argocd app sync retail-store-checkout
argocd app sync retail-store-orders
argocd app sync retail-store-ui

7.3 Monitor Deployment

# Watch pods being created
kubectl get pods -n retail-store -w

You should see all services starting up:

cart-xxxxx
catalog-xxxxx
checkout-xxxxx
orders-xxxxx
ui-xxxxx

Visualized Kubernetes resource relationships for the deployed application

All running pods across namespaces including ArgoCD, cert-manager, and microservices

Step 8: Accessing Your Application

8.1 Create Ingress

# Apply ingress configuration
kubectl apply -f terraform/retail-store-ingress.yaml -n retail-store

8.2 Get the Ingress IP

# Get the external IP
kubectl get ingress -n retail-store

8.3 Configure DNS (Optional)

If you have a domain:

Create an A record pointing to the Ingress IP
Wait for DNS propagation (5-30 minutes)

Example DNS configuration:

store.example.com -> <INGRESS_IP>

8.4 Access the Application

Open your browser to:

With domain: https://store.example.com
Without domain: http://<INGRESS_IP>

Your e-commerce platform is live!

Live e-commerce application accessible via the configured domain

Understanding the CI/CD Pipeline

Let's dive deep into how the automated CI/CD pipeline works!

Pipeline Overview

Every time you push code to the main branch, GitHub Actions automatically:

Detects Changes - Identifies which microservices were modified
Runs SonarQube Analysis - Checks code quality and security
Builds Docker Images - Creates container images
Scans for Vulnerabilities - Uses Trivy to find security issues
Pushes to Registry - Uploads images to GCP Artifact Registry
Updates Helm Values - Modifies deployment configurations
ArgoCD Deploys - Automatically deploys to Kubernetes

Pipeline Workflow Diagram

┌─────────────┐
│   Git Push  │
└──────┬──────┘
       │
       ▼
┌─────────────────┐
│ Detect Changes  │ (Which services changed?)
└──────┬──────────┘
       │
       ▼
┌─────────────────┐
│ SonarQube Scan  │ (Code quality check)
└──────┬──────────┘
       │
       ▼
┌─────────────────┐
│  Build Image    │ (Docker build)
└──────┬──────────┘
       │
       ▼
┌─────────────────┐
│  Security Scan  │ (Trivy)
└──────┬──────────┘
       │
       ▼
┌─────────────────┐
│ Push to Registry│ (Artifact Registry)
└──────┬──────────┘
       │
       ▼
┌─────────────────┐
│ Update Helm Vals│ (Git commit)
└──────┬──────────┘
       │
       ▼
┌─────────────────┐
│ ArgoCD Deploys  │ (Automatic)
└─────────────────┘

Stage 1: Detect Changes

The pipeline only builds services that have changed:

detect-changes:
  runs-on: ubuntu-latest
  outputs:
    cart: ${{ steps.filter.outputs.cart }}
    catalog: ${{ steps.filter.outputs.catalog }}
    checkout: ${{ steps.filter.outputs.checkout }}
    orders: ${{ steps.filter.outputs.orders }}
    ui: ${{ steps.filter.outputs.ui }}
  steps:
    - uses: actions/checkout@v4
    - uses: dorny/paths-filter@v2
      id: filter
      with:
        filters: |
          cart: 'src/cart/**'
          catalog: 'src/catalog/**'
          checkout: 'src/checkout/**'
          orders: 'src/orders/**'
          ui: 'src/ui/**'

What this does: Only builds services that actually changed. Saves time and resources!

Stage 2: Code Quality Analysis

sonarqube:
  needs: detect-changes
  runs-on: ubuntu-latest
  strategy:
    matrix:
      service: [cart, catalog, checkout, orders, ui]
  steps:
    - name: Checkout code
      uses: actions/checkout@v4

    - name: Run SonarQube Scan
      # Different scan methods for Java, Go, and Node.js
      # Checks for bugs, vulnerabilities, code smells

What this does:

Analyzes code for bugs and security issues
Checks code coverage
Enforces quality gates
Fails the build if quality standards aren't met

Stage 3: Build and Deploy

build-deploy:
  needs: [detect-changes, sonarqube]
  runs-on: ubuntu-latest
  strategy:
    matrix:
      service: [cart, catalog, checkout, orders, ui]
  steps:
    - name: Build Docker Image
      run: |
        docker build -t $IMAGE_NAME:$TAG .

    - name: Scan with Trivy
      uses: aquasecurity/trivy-action@master
      with:
        image-ref: $IMAGE_NAME:$TAG
        severity: "CRITICAL,HIGH"

    - name: Push to Artifact Registry
      run: docker push $IMAGE_NAME:$TAG

    - name: Update Helm Values
      run: |
        # Updates image tag in Helm chart
        # ArgoCD detects this change and deploys

What this does:

Builds optimized Docker images
Scans for security vulnerabilities
Pushes to private registry
Triggers GitOps deployment

How to Trigger the Pipeline

Method 1: Push Code Changes

# Make changes to a service
cd src/cart
# Edit some files...

# Commit and push
git add .
git commit -m "feat: add new cart feature"
git push origin main

The pipeline automatically runs!

Method 2: Manual Trigger

In GitHub:

Go to Actions tab
Select "Build and Deploy"
Click "Run workflow"

Part 3: Monitoring, Security & Operations

Monitoring and Observability

Datadog Dashboard Setup

1. Kubernetes Overview Dashboard

In Datadog, go to Dashboards → New Dashboard

Key Metrics to Monitor:

Pod CPU Usage
Pod Memory Usage
Pod Restart Count
Container Status
Network Traffic
Disk I/O

Overview of pod health, resource usage, and container states in the GKE cluster

GCP Console showing resource usage, container restarts, and namespace metrics

2. Application Performance Monitoring (APM)

Enable APM in your services:

Each microservice is already configured with Datadog APM environment variables:

env:
  - name: DD_AGENT_HOST
    valueFrom:
      fieldRef:
        fieldPath: status.hostIP
  - name: DD_SERVICE
    value: "cart-service"
  - name: DD_ENV
    value: "production"
  - name: DD_VERSION
    value: "1.0.0"
  - name: DD_TRACE_ENABLED
    value: "true"

View traces in Datadog:

Go to APM → Traces
See request flow across microservices
Identify slow queries and bottlenecks

3. Log Management

View logs:

Go to Logs → Explorer
Filter by service: service:cart-service
Search for errors: status:error

Create log alerts:

Alert when: Error count > 10 in 5 minutes
Notify: Your email or Slack

Security Best Practices

1. Container Security

Image Scanning with Trivy

Every image is scanned for vulnerabilities before deployment:

- name: Run Trivy vulnerability scanner
  uses: aquasecurity/trivy-action@master
  with:
    image-ref: ${{ env.IMAGE_NAME }}:${{ env.IMAGE_TAG }}
    format: "sarif"
    severity: "CRITICAL,HIGH"
    exit-code: "1" # Fail build if vulnerabilities found

Best Practices:

Use minimal base images (Alpine, Distroless)
Regularly update dependencies
Don't run containers as root
Scan images in CI/CD pipeline

Dockerfile Security

Bad Example:

FROM ubuntu:latest
RUN apt-get update && apt-get install -y python3
COPY . /app
CMD ["python3", "app.py"]

Good Example:

FROM python:3.11-alpine
RUN addgroup -g 1000 appuser && \
    adduser -D -u 1000 -G appuser appuser
WORKDIR /app
COPY --chown=appuser:appuser requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
COPY --chown=appuser:appuser . .
USER appuser
CMD ["python", "app.py"]

2. Secrets Management

Never Commit Secrets!

Bad:

env:
  - name: DB_PASSWORD
    value: "mypassword123" # Never do this!

Good:

env:
  - name: DB_PASSWORD
    valueFrom:
      secretKeyRef:
        name: mysql-secret
        key: password

Creating Secrets

# From literal values
kubectl create secret generic mysql-secret \
  --from-literal=password=YOUR_PASSWORD \
  -n retail-store

# From files
kubectl create secret generic tls-secret \
  --from-file=tls.crt=cert.pem \
  --from-file=tls.key=key.pem \
  -n retail-store

3. Network Security

Network Policies

Restrict traffic between pods:

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: cart-network-policy
  namespace: retail-store
spec:
  podSelector:
    matchLabels:
      app: cart
  policyTypes:
    - Ingress
    - Egress
  ingress:
    - from:
        - podSelector:
            matchLabels:
              app: ui
      ports:
        - protocol: TCP
          port: 8080
  egress:
    - to:
        - podSelector:
            matchLabels:
              app: mysql
      ports:
        - protocol: TCP
          port: 3306

Troubleshooting Guide

Common Issues and Solutions

Issue 1: Pods Not Starting

Symptom:

kubectl get pods -n retail-store
NAME                    READY   STATUS             RESTARTS   AGE
cart-xxxxx              0/1     ImagePullBackOff   0          2m

Diagnosis:

kubectl describe pod cart-xxxxx -n retail-store
kubectl logs cart-xxxxx -n retail-store

Common Causes:

Image doesn't exist - Check Artifact Registry
Authentication issues - Verify image pull secrets
Wrong image tag - Check Helm values

Issue 2: Service Not Accessible

Symptom:

curl http://INGRESS_IP
# Connection refused or 404

Diagnosis:

# Check ingress
kubectl get ingress -n retail-store

# Check service
kubectl get svc -n retail-store

# Check endpoints
kubectl get endpoints -n retail-store

Solutions:

Verify ingress configuration
Check service selector matches pod labels
Ensure pods are ready

Issue 3: Database Connection Errors

Symptom:

Error: Unable to connect to MySQL

Diagnosis:

# Check MySQL pod
kubectl get pods -n retail-store | grep mysql

# Check MySQL logs
kubectl logs mysql-xxxxx -n retail-store

# Test connection
kubectl run -it --rm debug --image=mysql:8.0 --restart=Never -n retail-store -- \
  mysql -h mysql -u root -p

Debugging Commands Cheat Sheet

# View all resources
kubectl get all -n retail-store

# Describe a resource
kubectl describe pod POD_NAME -n retail-store

# View logs
kubectl logs POD_NAME -n retail-store
kubectl logs POD_NAME -n retail-store --previous

# Follow logs in real-time
kubectl logs -f POD_NAME -n retail-store

# Execute commands in a pod
kubectl exec -it POD_NAME -n retail-store -- /bin/bash

# Port forward to a service
kubectl port-forward svc/cart 8080:80 -n retail-store

# View events
kubectl get events -n retail-store --sort-by='.lastTimestamp'

# Check resource usage
kubectl top nodes
kubectl top pods -n retail-store

Cost Optimization

1. Right-Size Your Resources

Monitor Resource Usage

# Check actual resource usage
kubectl top pods -n retail-store

# Compare with requested resources
kubectl describe deployment cart -n retail-store | grep -A 5 Requests

Adjust based on actual usage:

resources:
  requests:
    cpu: "100m" # Start small
    memory: "256Mi"
  limits:
    cpu: "500m" # Set reasonable limits
    memory: "512Mi"

2. Use Preemptible/Spot Instances

Create a preemptible node pool:

gcloud container node-pools create preemptible-pool \
  --cluster=ecommerce-gke-cluster \
  --preemptible \
  --num-nodes=2 \
  --machine-type=e2-medium \
  --region=us-central1

3. Implement Autoscaling

Horizontal Pod Autoscaler (HPA)

# Create HPA
kubectl autoscale deployment cart \
  --cpu-percent=70 \
  --min=2 \
  --max=10 \
  -n retail-store

Cluster Autoscaler

Already enabled in Terraform configuration:

autoscaling {
  min_node_count = 1
  max_node_count = 10
}

4. Estimated Monthly Costs

Component	Estimated Cost	Optimization Tips
GKE Cluster	$150-300	Use Preemptible nodes for dev/test
Compute Instances	$200-500	Right-size node pools, enable autoscaling
Artifact Registry	$10-30	Implement image retention policies
Load Balancers	$20-40	Consolidate ingresses where possible
Cloud Storage	$5-15	Lifecycle policies for old backups
Datadog	$0-100+	Depends on plan/usage
Total	$385-985+

Learning Resources

Official Documentation

Recommended Books

"Kubernetes in Action" by Marko Lukša
"The DevOps Handbook" by Gene Kim
"Site Reliability Engineering" by Google
"Terraform: Up & Running" by Yevgeniy Brikman

Online Courses

Conclusion

Congratulations! 🎊 You've just learned how to build and deploy a production-grade microservices platform on Google Cloud Platform!

What We've Accomplished

Infrastructure: Set up a complete GKE cluster with Terraform
Microservices: Deployed 5 microservices in multiple languages
GitOps: Implemented automated deployments with ArgoCD
CI/CD: Built a complete pipeline with GitHub Actions
Monitoring: Added full observability with Datadog
Security: Implemented security best practices
Quality: Integrated code quality checks with SonarQube

Next Steps

Now that you have a solid foundation, consider:

Add More Features

Implement caching with Redis
Add message queuing with Pub/Sub
Integrate with Cloud CDN

Enhance Security

Implement service mesh (Istio)
Add OAuth2/OIDC authentication
Enable Cloud Armor WAF

Improve Observability

Add custom business metrics
Implement distributed tracing
Create SLO/SLI dashboards

Scale Further
- Multi-region deployment
- Global load balancing
- Database sharding

Final Thoughts

Building production systems is a journey, not a destination. This project gives you a solid foundation, but there's always more to learn and improve. Keep experimenting, keep learning, and most importantly, keep building!

If you found this guide helpful, please:

⭐ Star the repository
🔄 Share with your network
💬 Leave a comment with your experience
🐛 Report issues or suggest improvements

Connect With Me

GitHub: [https://github.com/deepanshub9]
LinkedIn: [https://www.linkedin.com/in/deepanshub/]
Twitter: [https://x.com/realthetechguy]
Blog: [https://dev.to/deepanshub09]

Happy coding! 🚀

License

This project is licensed under the Apache License 2.0 - see the LICENSE file for details.

Acknowledgments

Original AWS Retail Store Sample App team
Google Cloud Platform documentation
Kubernetes community
ArgoCD project
All open-source contributors

Did you deploy this successfully? I'd love to hear about your experience! Drop a comment below! 👇

kubernetes #gcp #devops #microservices #gitops #cloudnative #terraform #argocd #cicd #production

Building a Complete AWS VPC with Load Balancer: A Step-by-Step Journey

Deepanshu — Sat, 04 Oct 2025 21:01:35 +0000

A comprehensive guide to creating a production-ready AWS infrastructure including all the roadblocks I hit and how to solve them

Introduction

Hey there, fellow cloud explorer! Have you ever tried setting up an AWS infrastructure and found yourself scratching your head, wondering, “Why isn’t this working?!” I’ve been there too! In this down-to-earth guide, I’ll walk you through building a full AWS Virtual Private Cloud (VPC) with an Application Load Balancer (ALB), EC2 instances, and more while sharing every stumble I made along the way and how I fixed them.

This isn’t your typical polished tutorial. It’s the real deal complete with those “uh-oh” moments and their practical solutions. By the end, you’ll have a production ready setup and the know how to troubleshoot like a pro. Let’s dive in!

What We’re Building

Imagine we’re setting up “Dublin Delights,” a small online store selling Irish goodies. Our final architecture will include:

A VPC with public and private subnets
An Internet Gateway for public access
A NAT Gateway for private subnet internet access
An Application Load Balancer to distribute traffic
3 EC2 instances (1 public, 2 private)
Tight security groups and routing
Working web servers behind the load balancer

Ready? Let’s get started!

Phase 1: Setting Up the VPC Foundation

Creating the VPC

First, let’s build the foundation our Virtual Private Cloud:

Head to the AWS Console > Search “VPC” > Click “VPC” > “Create VPC”.
Choose “VPC and more” (this sets up everything in one go).
Configuration:
- Name: dublin-delights-vpc
- IPv4 CIDR: 10.0.0.0/16 (gives us 65,536 IP addresses)
- Number of Availability Zones (AZs): 2 (e.g., US-east-1a, US-east-1b)
- Public subnets: 1 (e.g., 10.0.1.0/24)
- Private subnets: 2 (e.g., 10.0.2.0/24, 10.0.3.0/24)
- NAT gateways: 1 (in the public subnet)
- VPC endpoints: None
Hit “Create VPC” and wait a minute for it to spin up.

What This Creates:

A VPC with the 10.0.0.0/16 CIDR block
1 public subnet (10.0.1.0/24)
2 private subnets (10.0.2.0/24, 10.0.3.0/24)
An Internet Gateway
A NAT Gateway in the public subnet
Route tables (we’ll tweak these later)

The First Problem: DNS Resolution Woes

After creating the VPC, I launched instances but hit a snag DNS wasn’t working! Updates failed with “temporary failure resolving” errors. Here’s the fix:

VPC Console > “Your VPCs” > Select dublin-delights-vpc.
“Actions” > “Edit VPC settings”.
Enable “DNS hostnames” and “DNS resolution” (both checkboxes).
Save changes.

Why This Matters: Without DNS hostnames, instances can’t resolve domain names (e.g., sudo-apt-get update fails). This little toggle saved my day!

Phase 2: Launching EC2 Instances

Creating the Instances

Let’s launch our three servers:

Public Instance (public-web-1):
1. EC2 Console > “Launch Instance”.
2. Configuration:
  - Name: public-web-1
  - AMI: Ubuntu Server 22.04 LTS
  - Instance type: t2.micro (Free Tier eligible)
  - Key pair: Select or create devops-key (download .pem file)
  - VPC: dublin-delights-vpc
  - Subnet: public-subnet-1
  - Auto-assign Public IP: Enable
  - Security group: Create new “public-sg”
3. Launch and wait for “Running”.
Private Instances (private-web-2, private-web-3):
- Repeat the process, but:
- Name: private-web-2 and private-web-3
- Subnet: private-subnet-1 and private-subnet-2
- Auto-assign Public IP: Disable
- Security group: Create new “private-sg”
- Launch both.

The Second Problem: Connection Timeouts

I couldn’t SSH to my instances got “ssh: connect to host 52.23.157.242 port 22: Connection timed out.” Ouch! The culprit? Security groups.

Solution - Fix Security Groups:
- public-sg:
- Inbound: SSH (22) from “My IP”, HTTP (80) from 0.0.0.0/0
- Outbound: All traffic to 0.0.0.0/0
- private-sg:
- Inbound: SSH (22) from public-sg, HTTP (80) from ALB security group (later)
- Outbound: All traffic to 0.0.0.0/0
- Apply and test SSH: ssh -i devops-key.pem ubuntu@public-ip.

The Third Problem: Session Manager Not Working

Tried AWS Session Manager got “SSM Agent is not online.” The route table was the issue my “public” subnet wasn’t really public!

Solution - Fix Route Tables:
1. VPC Console > “Route Tables” > Find the one for public-subnet-1.
2. “Routes” tab > “Edit routes” > Add: Destination 0.0.0.0/0, Target “Internet Gateway” (dublin-igw).
3. Save.
Result: SSH, Session Manager, and internet access worked!

Phase 3: Setting Up the Application Load Balancer

Creating the ALB

EC2 Console > “Load Balancers” > “Create Load Balancer” > “Application Load Balancer”.
Configuration:
- Name: dublin-alb
- Scheme: Internet-facing
- IP address type: IPv4
- VPC: dublin-delights-vpc
- Availability Zones: Select both AZs with public-subnet-1
- Security group: Create new “alb-sg”
Create and wait 5 minutes.

Creating the Target Group

EC2 Console > “Target Groups” > “Create target group”.
Configuration:
- Target type: Instances
- Name: web-targets
- Protocol: HTTP
- Port: 80
- VPC: dublin-delights-vpc
- Health check path: /
Register targets: Add all 3 instances.
Create.

The Fourth Problem: 504 Gateway Timeout

Accessing the ALB gave “504 Gateway Time-out,” with targets “Unhealthy” (Target.Timeout).

Root Cause #1: No Web Servers

Solution: Install Apache on each instance:

sudo apt update
sudo apt install -y apache2
sudo systemctl start apache2
sudo systemctl enable apache2
echo "<h1>Web Server - $(hostname)</h1>" | sudo tee /var/www/html/index.html
curl http://localhost

Root Cause #2: ALB Security Group Outbound Rules Missing
- Even with web servers, 504 persisted. The ALB’s security group had no outbound rules!
- Critical Fix:
- EC2 Console > “Security Groups” > alb-sg.
- “Outbound rules” > “Edit outbound rules” > Add: Type “All traffic”, Destination 0.0.0.0/0.
- Save.
Why This Matters: ALB needs outbound rules to send health checks and traffic.

Phase 4: Fine-Tuning and Testing

Proper Security Group Configuration

alb-sg:
- Inbound: HTTP (80) from 0.0.0.0/0
- Outbound: All traffic to 0.0.0.0/0
public-sg:
- Inbound: SSH (22) from “My IP”, HTTP (80) from alb-sg
- Outbound: All traffic to 0.0.0.0/0
private-sg:
- Inbound: SSH (22) from public-sg, HTTP (80) from alb-sg
- Outbound: All traffic to 0.0.0.0/0

Testing the Complete Setup

Test 1: Individual Instance Access
- From public instance: curl http://10.0.2.135 and curl http://10.0.3.150.
Test 2: Load Balancer
- curl http://dublin-alb-689675767.eu-west-1.elb.amazonaws.com
- Loop: for i in {1..5}; do curl http://dublin-alb-689675767.eu-west-1.elb.amazonaws.com; done (shows rotation).
Test 3: NAT Gateway Functionality
- From private instances: curl ifconfig.me (shows NAT IP), sudo apt update (works).

Phase 5: Common Issues and Solutions

Issue 1: “Instance is not in public subnet”
- Fix: Add 0.0.0.0/0 → Internet Gateway in route table.
Issue 2: “Referenced group id for existing IPv4 CIDR rule”
- Fix: Add new rules instead of editing existing ones.
Issue 3: Private instances can’t reach internet
- Fix: Route 0.0.0.0/0 → NAT Gateway in private route table.
Issue 4: Health checks failing with “Target.Timeout”
- Fix: Install web server, fix security groups, ensure /var/www/html/index.html.

Final Architecture Overview

                    🌐 INTERNET
                         |
                    ┌────▼────┐
                    │   IGW   │ dublin-igw
                    └────┬────┘
                         |
              ┌──────────▼──────────┐
              │  Application LB     │ dublin-alb
              │     (alb-sg)        │
              └──┬────────┬────────┬┘
                 |        |        |
    ┌────────────▼─┐  ┌───▼───┐  ┌─▼────────────┐
    │ AZ: us-east-1a│  │us-east│  │ AZ: us-east-1c│
    │               │  │  -1b  │  │               │
    │ ┌───────────┐ │  │┌─────┐│  │ ┌───────────┐ │
    │ │Public Sub │ │  ││Priv ││  │ │Private Sub│ │
    │ │10.0.1.0/24│ │  ││Sub 1││  │ │10.0.3.0/24│ │
    │ │           │ │  ││10.0.││  │ │           │ │
    │ │┌─────────┐│ │  ││2.0/ ││  │ │┌─────────┐│ │
    │ ││public-  ││ │  ││24  ││  │ ││private- ││ │
    │ ││web-1    ││ │  ││     ││  │ ││web-3    ││ │
    │ ││(public- ││ │  ││┌───┐││  │ ││(private-││ │
    │ ││sg)      ││ │  │││web│││  │ ││sg)      ││ │
    │ │└─────────┘│ │  │││-2 │││  │ │└─────────┘│ │
    │ │           │ │  ││└───┘││  │ │           │ │
    │ │┌─────────┐│ │  │└─────┘│  │ │           │ │
    │ ││NAT GW   ││ │  └───────┘  │ │           │ │
    │ │└─────────┘│ │             │ │           │ │
    │ └───────────┘ │             │ └───────────┘ │
    └───────────────┘             └───────────────┘
                |                         |
                └─────────┬─────────────────┘
                         |
                    ┌────▼────┐
                    │   IGW   │ (for NAT traffic)
                    └────┬────┘
                         |
                    🌐 INTERNET

Network Flow: Inbound: Internet → IGW → ALB → Instances. Outbound: Public via IGW, Private via NAT.
Security: Public SSH from specific IP, private via public, ALB open on 80.

Key Lessons Learned

Security Groups Are Stateful: Inbound allows return traffic, but outbound rules are needed for initiated connections.
“Public” Subnet Needs Routing: Name doesn’t matter route tables make it public.
ALB Needs Outbound Rules: Missing rules cause 504 errors.
NAT Gateway Placement: Must be in a public subnet.
Health Checks Are Critical: Unhealthy targets stop traffic.

Cost Optimization Tips

Costs: 3 t2.micro (~$25/month), NAT Gateway (~$32/month), ALB (~$16/month), data variable. Total ~$73/month.
Savings: Use NAT Instance (~$0), Network Load Balancer, terminate unused instances, or Reserved Instances (30-60% off).

Production Readiness Checklist

Security: Restrict SSH, enable VPC Flow Logs, CloudTrail, IAM roles, encryption.
Monitoring: CloudWatch alarms, ALB logs, target health, SNS notifications.
High Availability: Multi-AZ, Auto Scaling, health checks, disaster recovery.
Performance: Right-size instances, ALB stickiness, CloudFront CDN.

Troubleshooting Commands Reference

Connectivity Issues:
- telnet <ip> <port>
- aws ec2 describe-security-groups --group-ids sg-xxxxxxxxx
- nslookup google.com
- ip route show
Web Server Issues:
- sudo systemctl status apache2
- sudo netstat -tlnp | grep :80
- curl http://localhost
- sudo tail -f /var/log/apache2/error.log
ALB Issues:
- curl -I http://your-alb-dns-name.elb.amazonaws.com
- aws elbv2 describe-target-health --target-group-arn <arn>

Conclusion

Building AWS infrastructure is like assembling a puzzle every piece (VPC, subnets, security groups, ALB) must fit. My mistakes taught me to start with basics, test incrementally, and read error messages closely. Each “why isn’t this working?” turned into a “now I get it!” moment.

What’s Next?
Explore Auto Scaling Groups, RDS, CloudFront, Route 53, ACM, or Infrastructure as Code (CloudFormation/Terraform). Share your AWS adventures in the comments let’s learn together!

The OSI Model Explained: Your Complete Guide to Network Communication 🌐

Deepanshu — Tue, 23 Sep 2025 11:18:12 +0000

Ever wondered how your message travels from your phone to your friend's computer across the globe? Let's dive into the fascinating world of network communication through the OSI model!

What is the OSI Model?

Imagine you're sending a letter to a friend in another country. You write the message, put it in an envelope, add the address, take it to the post office, and it magically reaches your friend. The OSI (Open Systems Interconnection) model works similarly - it's like a postal system for digital communication.

The OSI model is a conceptual framework that standardizes how different network devices communicate. Think of it as a universal language that ensures your iPhone can talk to a Windows laptop, which can then communicate with a Linux server.

Why Do We Need the OSI Model?

Before the OSI model, different manufacturers created networking equipment that couldn't talk to each other. It was like having people speaking different languages with no translator. The OSI model solved this by creating seven standardized layers that work together seamlessly.

The 7 Layers of the OSI Model

Let's explore each layer from bottom to top, using real-world analogies that make sense:

Layer 1: Physical Layer - The Highway

What it does: Handles the actual physical connection between devices.

Think of this as the roads and highways where data travels. Just like cars need roads to move from one place to another, data needs physical pathways.

Examples:

Ethernet cables (the copper wires)
Fiber optic cables (light signals)
Wi-Fi radio waves
Bluetooth signals

Real-world scenario: When you plug an Ethernet cable into your laptop, the Physical Layer ensures the electrical signals can flow through the copper wires.

Physical Layer Components:
├── Cables (Ethernet, Fiber)
├── Hubs
├── Repeaters
└── Network Interface Cards (NICs)

Layer 2: Data Link Layer - The Traffic Rules

What it does: Manages how data moves between directly connected devices and handles error detection.

This layer is like traffic rules and road signs. It ensures data doesn't crash into each other and reaches the right destination on the local network.

Key concepts:

MAC Addresses: Like license plates for network devices
Frames: Data packets with addressing information
Error Detection: Checking if data arrived correctly

Real-world scenario: When your laptop sends data to your router, the Data Link Layer adds your laptop's MAC address as the sender and your router's MAC address as the receiver.

Example MAC Address: 00:1B:44:11:3A:B7

Layer 3: Network Layer - The GPS Navigation

What it does: Determines the best path for data to travel across multiple networks.

Think of this as your GPS navigation system. It figures out the best route from your house to your destination, even if it involves multiple highways and cities.

Key player: IP Addresses (like postal addresses for the internet)

Real-world scenario: When you visit google.com, the Network Layer uses IP addresses to route your request through multiple routers across the internet to reach Google's servers.

Your Computer (192.168.1.100) 
    ↓
Router 1 (ISP Gateway)
    ↓
Router 2 (Regional Hub)
    ↓
Router 3 (Google's Network)
    ↓
Google Server (172.217.164.110)

Layer 4: Transport Layer - The Delivery Service

What it does: Ensures reliable data delivery and manages data flow.

This is like FedEx or UPS - it makes sure your package (data) arrives complete, in the right order, and handles any delivery issues.

Two main protocols:

TCP (Transmission Control Protocol): Reliable delivery (like registered mail)
UDP (User Datagram Protocol): Fast delivery (like regular mail)

Real-world scenario: When you download a file, TCP ensures all pieces arrive correctly. If a piece is missing, it asks for it to be sent again.

TCP vs UDP Example:

TCP: "Did you get packet 1? Good. Did you get packet 2? Resending packet 3..."
UDP: "Here's packet 1, 2, 3, 4... Hope you got them all!"

Layer 5: Session Layer - The Phone Call Manager

What it does: Manages connections between applications.

Think of this as a phone call manager that establishes, maintains, and terminates conversations between applications.

Key functions:

Starting conversations (sessions)
Keeping track of who's talking
Ending conversations properly

Real-world scenario: When you log into your online banking, the Session Layer establishes a secure session, keeps it active while you browse, and properly closes it when you log out.

Layer 6: Presentation Layer - The Translator

What it does: Handles data formatting, encryption, and compression.

This layer is like a universal translator that ensures data is in the right format for the receiving application.

Key functions:

Encryption/Decryption: Scrambling data for security
Compression: Making files smaller for faster transfer
Format conversion: Converting between different data formats

Real-world scenario: When you visit an HTTPS website, the Presentation Layer encrypts your data before sending it and decrypts incoming data.

Example:

Original: "Hello World"
Encrypted: "8b7df143d91c716ecfa5fc1730022f6b"
Compressed: "H3ll0 W0rld" (simplified example)

Layer 7: Application Layer - The User Interface

What it does: Provides network services directly to end-users.

This is the layer you actually see and interact with - your web browser, email client, or messaging app.

Examples:

Web browsers (HTTP/HTTPS)
Email clients (SMTP, POP3, IMAP)
File transfer (FTP)
Remote access (SSH)

Real-world scenario: When you type www.example.com in your browser, you're interacting with the Application Layer.

How All Layers Work Together

Let's trace what happens when you send a message on WhatsApp:

Sending Data (Top to Bottom):

Application Layer: You type "Hello!" in WhatsApp
Presentation Layer: Encrypts and compresses your message
Session Layer: Establishes connection with WhatsApp servers
Transport Layer: Breaks message into packets, adds sequence numbers
Network Layer: Adds IP addresses (your phone → WhatsApp server)
Data Link Layer: Adds MAC addresses for local network routing
Physical Layer: Converts to electrical/radio signals and transmits

Receiving Data (Bottom to Top):

Physical Layer: Receives electrical/radio signals
Data Link Layer: Checks MAC addresses, verifies data integrity
Network Layer: Checks IP addresses, routes to correct device
Transport Layer: Reassembles packets in correct order
Session Layer: Manages the connection session
Presentation Layer: Decrypts and decompresses the message
Application Layer: WhatsApp displays "Hello!" on your friend's phone

Memory Tricks to Remember the Layers

Here are some popular mnemonics (from Layer 1 to 7):

"Please Do Not Throw Sausage Pizza Away"

Physical
Data Link
Network
Transport
Session
Presentation
Application

"All People Seem To Need Data Processing" (Layer 7 to 1)

Real-World Examples

Example 1: Watching Netflix

Application Layer: Netflix app interface
Presentation Layer: Video compression (H.264), HTTPS encryption
Session Layer: Maintains streaming session
Transport Layer: TCP ensures all video packets arrive
Network Layer: Routes data from Netflix servers to your device
Data Link Layer: Wi-Fi manages local network communication
Physical Layer: Radio waves carry the data

Example 2: Sending an Email

Application Layer: Gmail interface
Presentation Layer: Email formatting, attachments encoding
Session Layer: SMTP session with email server
Transport Layer: TCP reliable delivery
Network Layer: IP routing across internet
Data Link Layer: Ethernet frame handling
Physical Layer: Fiber optic cables

Common Protocols by Layer

Layer	Common Protocols
Application	HTTP, HTTPS, FTP, SMTP, DNS
Presentation	SSL/TLS, JPEG, MPEG, ASCII
Session	NetBIOS, RPC, SQL
Transport	TCP, UDP
Network	IP, ICMP, OSPF, BGP
Data Link	Ethernet, Wi-Fi, PPP
Physical	Ethernet cables, Fiber optic

Troubleshooting with the OSI Model

When network issues occur, the OSI model helps you troubleshoot systematically:

Problem: Can't access a website

Layer-by-layer troubleshooting:

Physical: Is the cable plugged in? Is Wi-Fi connected?
Data Link: Can you see other devices on the local network?
Network: Can you ping the router? Can you ping external IPs?
Transport: Are the right ports open?
Session: Is the connection being established?
Presentation: Are there encryption/certificate issues?
Application: Is the web browser working correctly?

OSI vs TCP/IP Model

While we've focused on the OSI model, it's worth noting that the internet actually uses the TCP/IP model, which has 4 layers:

OSI Model	TCP/IP Model
Application, Presentation, Session	Application
Transport	Transport
Network	Internet
Data Link, Physical	Network Access

The OSI model is more detailed and better for understanding concepts, while TCP/IP is what's actually implemented.

Why This Matters for Developers

Understanding the OSI model helps you:

Debug network issues more effectively
Choose the right protocols for your applications
Optimize performance by understanding where bottlenecks occur
Design better architectures with proper separation of concerns
Communicate effectively with network administrators

Conclusion

The OSI model might seem complex at first, but it's really just a logical way to organize how networks communicate. Each layer has a specific job, and they all work together like a well-orchestrated symphony.

Remember:

Physical Layer: The roads (cables, wireless)
Data Link Layer: Traffic rules (local network management)
Network Layer: GPS navigation (routing across networks)
Transport Layer: Delivery service (reliable data transfer)
Session Layer: Call manager (connection management)
Presentation Layer: Translator (data formatting)
Application Layer: User interface (what you interact with)

Next time you send a message, browse the web, or stream a video, you'll know the incredible journey your data takes through these seven layers!

Found this helpful? Give it a ❤️ and share your thoughts in the comments! What networking concept would you like me to explain next?

networking #osi #tcp #protocols #webdev #computerscience #tutorial

ThreeTierApplication: A Modern DevOps Todo Deployment using DevSecOps on AWS EKS

Deepanshu — Wed, 13 Aug 2025 13:33:00 +0000

Welcome to the ultimate hands-on guide for deploying a production-ready three-tier web application! This comprehensive tutorial will walk you through the ThreeTierAppChallenge - a modern todo application built with React, Node.js, and MongoDB, complete with Docker containerization, Kubernetes manifests, and CI/CD pipelines. Whether you're looking to learn modern DevOps practices or deploy a scalable web application, this guide has everything you need to get started and master the deployment pipeline.

The ThreeTierAppChallenge is a full-stack todo application featuring a React frontend, Node.js backend, and MongoDB database, designed for cloud-native deployment on AWS EKS. Clone the repo, run with Docker Compose in under 5 minutes, or dive deep into Kubernetes deployment with included manifests and Jenkins CI/CD pipelines. Perfect for learning modern DevOps practices including containerization, orchestration, and infrastructure as code.

Prerequisites

Before diving in, make sure you have these tools installed:

Git (for version control)
Node.js (v18+ recommended) and npm
Docker and Docker Compose (for containerization)
Optional but recommended:
- kubectl (for Kubernetes management)
- AWS CLI (for cloud deployment)
- Terraform (for infrastructure provisioning)

🚀 Quick Local Setup

Let's get the application running on your local machine in just a few minutes!

Step 1: Clone the Repository

git clone https://github.com/YOUR_USERNAME/TWSThreeTierAppChallenge.git
cd TWSThreeTierAppChallenge

Step 2: Run with Docker Compose (Fastest Method)

The easiest way to get everything running is with Docker Compose:

docker-compose up --build

Sample Output:

[+] Building 45.2s (23/23) FINISHED
[+] Running 3/3
 ✔ Container mongodb    Started
 ✔ Container backend    Started  
 ✔ Container frontend   Started

After a few minutes, you'll have:

Frontend: http://localhost:3000
Backend API: http://localhost:3500
MongoDB: localhost:27017

Step 3: Manual Setup (Development Mode)

For development, you might want to run services individually:

Backend Setup:

cd Application-Code/backend
npm install
npm start

Frontend Setup (in a new terminal):

cd Application-Code/frontend
npm install
npm start

Sample Output:

Compiled successfully!

You can now view client in the browser.

  Local:            http://localhost:3000
  On Your Network:  http://192.168.1.100:3000

webpack compiled with 0 errors

Step 4: Run Tests

# Backend tests
cd Application-Code/backend
npm test

# Frontend tests  
cd Application-Code/frontend
npm test

📦 Annotated Dockerfile Analysis

Let's examine the frontend Dockerfile to understand the containerization strategy:

FROM node:18-alpine AS build        # 1. Multi-stage build for optimization
WORKDIR /app
COPY package*.json ./               # 2. Copy package files first for layer caching
RUN npm install                     # 3. Install dependencies
COPY . .                            # 4. Copy application source code
RUN npm run build                   # 5. Build the React production bundle

FROM nginx:alpine                   # 6. Use lightweight Nginx for serving
COPY --from=build /app/build /usr/share/nginx/html  # 7. Copy built assets
EXPOSE 80
CMD ["nginx", "-g", "daemon off;"]  # 8. Start Nginx in foreground

Key Benefits:

Multi-stage build reduces final image size by ~70%
Layer caching speeds up subsequent builds
Nginx provides production-grade static file serving
Alpine base minimizes security attack surface

🏗️ Architecture Overview

Diagram Description:
Create a diagram showing three tiers connected vertically:

┌─────────────────┐
│   Frontend      │ ← React.js (Port 3000)
│   (React)       │
└─────────┬───────┘
          │ HTTP/REST API
┌─────────▼───────┐
│   Backend       │ ← Node.js/Express (Port 3500)  
│   (Node.js)     │
└─────────┬───────┘
          │ MongoDB Connection
┌─────────▼───────┐
│   Database      │ ← MongoDB (Port 27017)
│   (MongoDB)     │
└─────────────────┘

Additional Infrastructure:

Docker Containers around each tier
Kubernetes Pods for orchestration
AWS EKS for managed Kubernetes
Jenkins for CI/CD automation

💡 Three Essential Code Snippets

1. Smart Database Connection with Fallback

// Application-Code/backend/db.js
module.exports = async () => {
    // Skip MongoDB connection if not using MongoDB
    if (process.env.USE_MONGODB !== 'true') {
        console.log("Using file-based storage (MongoDB disabled)");
        return;
    }

    try {
        const connectionParams = {
            useNewUrlParser: true,
            useUnifiedTopology: true,
        };
        const useDBAuth = process.env.USE_DB_AUTH || false;
        if(useDBAuth){
            connectionParams.user = process.env.MONGO_USERNAME;
            connectionParams.pass = process.env.MONGO_PASSWORD;
        }

        await mongoose.connect(process.env.MONGO_CONN_STR, connectionParams);
        console.log("Connected to MongoDB database.");
    } catch (error) {
        console.log("Could not connect to database. Falling back to file-based storage...");
        process.env.USE_MONGODB = 'false';
    }
};

Line-by-line breakdown:

Lines 3-6: Check environment variable to enable/disable MongoDB
Lines 8-16: Configure connection parameters with optional authentication
Lines 18: Attempt MongoDB connection using environment variables
Lines 21-23: Graceful fallback to file-based storage on connection failure

This pattern ensures your application works in any environment!

2. Frontend API Service with Error Handling

// Application-Code/frontend/src/services/taskServices.js
export const taskService = {
    getTasks: async (filters = {}) => {
        const params = new URLSearchParams();
        Object.entries(filters).forEach(([key, value]) => {
            if (value !== undefined && value !== null && value !== '') {
                params.append(key, value);
            }
        });

        const response = await axios.get(`${apiUrl}?${params}`);
        return response.data;
    },

    addTask: async (taskData) => {
        try {
            const response = await axios.post(apiUrl, taskData);
            return response.data;
        } catch (error) {
            console.error('Add task error:', error);
            throw error;
        }
    }
};

What's happening:

Lines 3-9: Dynamic query parameter building for flexible filtering
Lines 11-12: Clean GET request with constructed parameters
Lines 15-22: POST request with comprehensive error handling
Lines 19-20: Proper error logging and re-throwing for upstream handling

3. Production-Ready Kubernetes Deployment

# Kubernetes-Manifests-file/Backend/deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata: 
  name: api
  namespace: three-tier
spec: 
  replicas: 2                        # High availability with 2 instances
  strategy: 
    type: RollingUpdate
    rollingUpdate:
      maxSurge: 1                    # Allow 1 extra pod during updates
      maxUnavailable: 25%            # Maximum 25% downtime during updates
  template:
    spec:
      containers:
      - name: api
        env:
          - name: MONGO_CONN_STR
            value: mongodb://mongodb-svc:27017/todo?directConnection=true
        ports:
        - containerPort: 3500
        livenessProbe:               # Health check configuration
          httpGet:
            path: /ok
            port: 3500
          initialDelaySeconds: 2
          periodSeconds: 5

Deployment highlights:

Line 8: Two replicas ensure high availability
Lines 10-13: Rolling updates with zero downtime
Lines 18-19: Service discovery using Kubernetes DNS
Lines 22-27: Health checks for automatic recovery

🚨 Common Issues & Solutions

Issue 1: Backend Using File Storage Instead of MongoDB

Symptoms: API works but data doesn't persist between container restarts

Solution:

# Check backend logs
docker logs backend

# Set environment variable
echo "USE_MONGODB=true" >> Application-Code/backend/.env
echo "MONGO_CONN_STR=mongodb://mongodb:27017/todo" >> Application-Code/backend/.env

# Restart containers
docker-compose restart backend

Issue 2: Frontend Can't Connect to Backend

Symptoms: Frontend loads but no tasks appear, network errors in browser console

Solution:

# Check if backend is running
curl http://localhost:3500/health

# Verify environment variable
echo "REACT_APP_BACKEND_URL=http://localhost:3500/api/tasks" >> Application-Code/frontend/.env

# Rebuild frontend
docker-compose build frontend

Issue 3: Port Conflicts

Symptoms: docker-compose up fails with "port already in use"

Solution:

# Check what's using the ports
lsof -i :3000,3500,27017

# Kill conflicting processes or change ports in docker-compose.yml
# Example: Change frontend port to 3001
ports:
  - "3001:3000"

🔄 CI/CD Pipeline Overview

The project includes comprehensive Jenkins pipelines for both frontend and backend:

Pipeline Features:

Code Quality: SonarQube analysis
Security: OWASP dependency checks and Trivy scanning
Containerization: Docker image building and ECR pushing
GitOps: Automatic deployment manifest updates

To set up CI/CD:

Deploy Jenkins using the provided Terraform scripts
Configure the provided Jenkinsfile pipelines
Set up AWS ECR repositories for image storage
Watch automated deployments to EKS!

☸️ Kubernetes Deployment

Ready to go production? Deploy to Kubernetes:

# Create namespace
kubectl create namespace three-tier

# Deploy database
kubectl apply -f Kubernetes-Manifests-file/Database/

# Create MongoDB credentials
kubectl create secret generic mongo-sec \
  --from-literal=username=admin \
  --from-literal=password=password123 \
  -n three-tier

# Deploy backend and frontend
kubectl apply -f Kubernetes-Manifests-file/Backend/
kubectl apply -f Kubernetes-Manifests-file/Frontend/

# Set up ingress (requires AWS Load Balancer Controller)
kubectl apply -f Kubernetes-Manifests-file/ingress.yaml

# Check deployment status
kubectl get all -n three-tier

🚀 What's Next?

Congratulations! You've successfully deployed a modern three-tier application. Here are your next steps:

🎯 Immediate Next Steps:

Explore the Code: Dive deeper into the Application-Code directory
Try Kubernetes: Use the manifests in Kubernetes-Manifests-file
Set Up CI/CD: Implement the Jenkins pipelines from Jenkins-Pipeline-Code

📚 Learning Opportunities:

Infrastructure as Code: Study the Terraform scripts in Jenkins-Server-TF
Security: Implement the security best practices shown in the code
Monitoring: Add Prometheus and Grafana (hint: it's in the challenge levels!)

🤝 Community & Contributing:

Report Issues: Found a bug? Open an issue
Contribute: Check the contribution guidelines
Share Your Experience: Write about your deployment journey!

🏆 Challenge Yourself:

Bronze: Get the basic deployment working ✅
Silver: Add SSL/TLS and monitoring
Gold: Implement GitOps with ArgoCD
Platinum: Add your own innovative features!

📋 Quick Reference Commands

# Development
npm start                          # Start local development
docker-compose up                  # Run full stack locally
kubectl get pods -n three-tier     # Check Kubernetes status

# Troubleshooting  
docker logs container_name         # Check container logs
kubectl describe pod pod_name      # Debug Kubernetes issues
curl localhost:3500/health         # Test backend health

# Cleanup
docker-compose down               # Stop local environment
kubectl delete namespace three-tier  # Remove Kubernetes deployment

Recommended dev.to tags: tutorial, javascript, react, nodejs, docker, kubernetes, devops, aws, ci-cd, mongodb

Ready to build something amazing? Clone the repo and start your DevOps journey today! 🚀

Have questions or run into issues? Drop a comment below or reach out to the community. Happy coding!

Spring Boot Banking Application Deployment using DevSecOps on AWS EKS

Deepanshu — Sat, 09 Aug 2025 11:14:33 +0000

A comprehensive, secure banking web application built with Spring Boot, featuring modern web technologies and enterprise-grade security. This application provides essential banking operations with a user-friendly interface and robust backend architecture.

This is a multi-tier bank an application written in Java (Springboot).

Tech stack used in this project:

GitHub (Code)
Docker (Containerization)
Jenkins (CI)
OWASP (Dependency check)
SonarQube (Quality)
Snyk (AI vulnerability)
ArgoCD (CD)
AWS EKS (Kubernetes)
Helm (Monitoring using grafana and prometheus)

Steps to deploy:

Pre-requisites:

root user access

sudo su

[!Note]
This project will be implemented on North Virginia region (us-east-1).

Create 1 Master machine on AWS (t3.large) and 25 GB of storage.

Open the below ports in security group
Create EKS Cluster on AWS
IAM user with access keys and secret access keys
AWSCLI should be configured

  curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"
  sudo apt install unzip
  unzip awscliv2.zip
  sudo ./aws/install
  aws configure

Install kubectl

  curl -o kubectl https://amazon-eks.s3.us-west-2.amazonaws.com/1.19.6/2021-01-05/bin/linux/amd64/kubectl
  chmod +x ./kubectl
  sudo mv ./kubectl /usr/local/bin
  kubectl version --short --client

Install eksctl

  curl --silent --location "https://github.com/weaveworks/eksctl/releases/latest/download/eksctl_$(uname -s)_amd64.tar.gz" | tar xz -C /tmp
  sudo mv /tmp/eksctl /usr/local/bin
  eksctl version

Create EKS Cluster

  eksctl create cluster --name=bankapp \
                      --region=us-west-1 \
                      --version=1.30 \
                      --without-nodegroup

Associate IAM OIDC Provider

  eksctl utils associate-iam-oidc-provider \
    --region us-west-1 \
    --cluster bankapp \
    --approve

Create Nodegroup

  eksctl create nodegroup --cluster=bankapp \
                       --region=us-east-1 \
                       --name=bankapp \
                       --node-type=t3.medium \
                       --nodes=2 \
                       --nodes-min=2 \
                       --nodes-max=2 \
                       --node-volume-size=25 \
                       --ssh-access \
                       --ssh-public-key=eks-nodegroup-key

[!Note]
Make sure the ssh-public-key "eks-nodegroup-key is available in your aws account"

Install Jenkins

sudo apt update -y
sudo apt install fontconfig openjdk-17-jre -y

sudo wget -O /usr/share/keyrings/jenkins-keyring.asc \
  https://pkg.jenkins.io/debian-stable/jenkins.io-2023.key

echo "deb [signed-by=/usr/share/keyrings/jenkins-keyring.asc]" \
  https://pkg.jenkins.io/debian-stable binary/ | sudo tee \
  /etc/apt/sources.list.d/jenkins.list > /dev/null

sudo apt-get update -y
sudo apt-get install jenkins -y

After installing Jenkins, change the default port of jenkins from 8080 to 8081. Because our bankapp application will be running on 8080.
- Open /usr/lib/systemd/system/jenkins.service file and change JENKINS_PORT environment variable
- Reload daemon

  sudo systemctl daemon-reload

Restart Jenkins

  sudo systemctl restart jenkins

Install docker

sudo apt install docker.io -y
sudo usermod -aG docker ubuntu && newgrp docker

Install and configure SonarQube

docker pull sonarqube

docker run -d --name sonarqube -e SONAR_ES_BOOTSTRAP_CHECKS_DISABLE=true -p 9000:9000 sonarqube:latest

Install OWASP

docker pull owasp/dependency-check

[!Note]
OWASP setup take around 10-20min first its download all the vulnerability from there database into you system. With API key your process becomes quit faster. "

You can request you OWASP API key (FREE) Link

Install and Configure ArgoCD
- Create argocd namespace

  kubectl create namespace argocd

Apply argocd manifest

  kubectl apply -n argocd -f https://raw.githubusercontent.com/argoproj/argo-cd/stable/manifests/install.yaml

Make sure all pods are running in argocd namespace

  watch kubectl get pods -n argocd

Install argocd CLI

  curl --silent --location -o /usr/local/bin/argocd https://github.com/argoproj/argo-cd/releases/download/v2.4.7/argocd-linux-amd64

Provide executable permission

  chmod +x /usr/local/bin/argocd

Check argocd services

  kubectl get svc -n argocd

Change argocd server's service from ClusterIP to NodePort

  kubectl patch svc argocd-server -n argocd -p '{"spec": {"type": "NodePort"}}'

Confirm service is patched or not

  kubectl get svc -n argocd

Check the port where ArgoCD server is running and expose it on security groups of a k8s worker node
Access it on browser, click on advance and proceed with

  <public-ip-worker>:<port>

Fetch the initial password of argocd server

kubectl -n argocd get secret argocd-initial-admin-secret -o jsonpath="{.data.password}" | base64 -d; echo

Username: admin
Now, go to User Info and update your argocd password

Go to Jenkins and click on Manage Jenkins --> Plugins --> Available plugins install the below plugins:
- OWASP
- SonarQube Scanner
- Synk
- Docker
- Pipeline: Stage View

Configure OWASP, move to Manage Jenkins --> Plugins --> Available pluginsb>
After OWASP plugin is installed, Now move to Manage jenkins --> Tools and configure it.
After Synk plugin is installed, Now move to Manage --> Tools and configure it.

Login to SonarQube server and create the credentials for jenkins to integrate with SonarQube
- Navigate to Administration --> Security --> Users --> Token and paste into inside your jenkins credentials

Now, go to Manage Jenkins --> credentials and add Sonarqube credentials:

Go to Manage Jenkins --> Tools and search for SonarQube Scanner installations:

Go to Manage Jenkins --> credentials and add Docker credentials to push updated the updated docker image to dockerhub.

Go to Manage Jenkins --> System and search for SonarQube installations:

Login to SonarQube server, go to Administration --> Webhook and click on create

Go to Master Machine and add our own eks cluster to argocd for application deployment using cli
- Login to argoCD from CLI

   argocd login 52.53.156.187:32738 --username admin

[!Tip]
52.53.156.187:32738 --> This should be your argocd url

Check how many clusters are available in argocd

  argocd cluster list

Get your cluster name

  kubectl config get-contexts

Add your cluster to argocd

  argocd cluster add bankapp-cluster.us-east-1.eksctl.io --name bankapp-eks-cluster

[!Tip] > bankapp-cluster.us-east-1.eksctl.io --> This should be your EKS Cluster Name.

Once your cluster is added to argocd, go to argocd console Settings --> Clusters and verify it

Go to Settings --> Repositories and click on Connect repo

[!Note]
Connection should be successful

Create BankApp-CI job

Create BankApp-CD job, same as CI job.

Provide permission to docker socket so that docker build and push command do not fail

chmod 777 /var/run/docker.sock

Now, go to Applications and click on New App

[!Important]
Make sure to click on the Auto-Create Namespace option while creating argocd application

Congratulations, your application is deployed on AWS EKS Cluster
Open port 30080 on worker node and Access it on browser

<worker-public-ip>:30080

How to monitor EKS cluster, kubernetes components and workloads using prometheus and grafana via HELM (On Master machine)

Install Helm Chart

curl -fsSL -o get_helm.sh https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3

chmod 700 get_helm.sh

./get_helm.sh

Add Helm Stable Charts for Your Local Client

helm repo add stable https://charts.helm.sh/stable

Add Prometheus Helm Repository

helm repo add prometheus-community https://prometheus-community.github.io/helm-charts

Create Prometheus Namespace

kubectl create namespace prometheus

kubectl get ns

Install Prometheus using Helm

helm install stable prometheus-community/kube-prometheus-stack -n prometheus

Verify prometheus installation

kubectl get pods -n prometheus

Check the services file (svc) of the Prometheus

kubectl get svc -n prometheus

Expose Prometheus and Grafana to the external world through Node Port > [!Important] > change it from Cluster IP to NodePort after changing make sure you save the file and open the assigned nodeport to the service.

kubectl edit svc stable-kube-prometheus-sta-prometheus -n prometheus

Verify service

kubectl get svc -n prometheus

Now,let’s change the SVC file of the Grafana and expose it to the outer world

kubectl edit svc stable-grafana -n prometheus

Check grafana service

kubectl get svc -n prometheus

Get a password for grafana

kubectl get secret --namespace prometheus stable-grafana -o jsonpath="{.data.admin-password}" | base64 --decode ; echo

[!Note]
Username: admin

Now, view the Dashboard in Grafana

Clean Up

Delete eks cluster

eksctl delete cluster --name=bankapp --region=us-east-1

📚 Additional Resources

Happy Coding! 🏦💳

Terraform Modules: Your Infrastructure's Building Blocks (The Complete Guide)

Deepanshu — Thu, 31 Jul 2025 19:25:07 +0000

Imagine if every time you wanted to build a house, you had to start from scratch - making your own bricks, mixing cement, crafting doors. Sounds exhausting, right? That's exactly what writing Terraform without modules feels like. Let me show you how modules can change your infrastructure game forever.

🤔 What Are Terraform Modules?

Think of Terraform modules as LEGO blocks for your infrastructure. Just like how you can build a castle, spaceship, or car using the same LEGO pieces, modules let you create reusable infrastructure components that you can use across different projects.

In simple terms: A module is a collection of Terraform files that create a specific piece of infrastructure.

🏗️ Traditional Way (Without Modules):
Every project = Build everything from scratch

🧱 With Modules:
Every project = Assemble pre-built, tested components

😫 The Pain Without Modules

Let me paint you a picture of life before modules:

Scenario 1: The Copy-Paste Nightmare

Developer A: "I need to create a VPC for the new project"
Developer B: "Just copy the VPC code from the last project"
Developer A: "Which version? The one from 3 months ago or the updated one?"
Developer B: "Ummm... good question 🤷‍♂️"

Scenario 2: The "Find and Replace" Horror

# Project 1
resource "aws_instance" "web_server_project_alpha" {
  # 50 lines of configuration
}

# Project 2 (copy-pasted and modified)
resource "aws_instance" "web_server_project_beta" {
  # Same 50 lines, but with different names
}

# Project 3... Project 4... Project N...
# 😱 Now you have to maintain the same logic in 20 different places!

Scenario 3: The "Oops, I Broke Everything" Story

DevOps Engineer: "I found a security bug in our EC2 configuration"
Team Lead: "Great! How many projects do we need to update?"
DevOps Engineer: "Only... 15 different repositories... 😰"
*2 weeks later, still updating projects*

🎯 What Problems Do Modules Solve?

1. DRY Principle (Don't Repeat Yourself)

Instead of copying the same configuration everywhere, write it once and reuse it.

2. Consistency Across Projects

Everyone uses the same, tested infrastructure patterns.

3. Easier Maintenance

Fix a bug once, and it's fixed everywhere.

4. Team Collaboration

Share infrastructure patterns across teams and organizations.

5. Testing and Quality

Test your modules once, trust them everywhere.

🏗️ Module Architecture

Here's how modules work in the real world:

📁 Your Organization
├── 🏢 Projects
│   ├── Project A (uses modules)
│   ├── Project B (uses modules)
│   └── Project C (uses modules)
│
└── 📦 Module Library
    ├── 🌐 VPC Module
    ├── 🖥️ EC2 Module
    ├── 🗄️ RDS Module
    └── 🔒 Security Group Module

Instead of each project building everything from scratch,
they all use the same tested, standardized modules!

🛠️ Creating Your First Module

Let's build a real module together! We'll create a "Web Server" module that you can reuse across projects.

Step 1: Module Structure

modules/
└── web-server/
    ├── main.tf        # Main resources
    ├── variables.tf   # Input variables
    ├── outputs.tf     # Output values
    └── README.md      # Documentation

Step 2: Define the Module

File: modules/web-server/main.tf

# Security group for web server
resource "aws_security_group" "web_sg" {
  name_prefix = "${var.name_prefix}-web-sg"
  vpc_id      = var.vpc_id

  ingress {
    description = "HTTP"
    from_port   = 80
    to_port     = 80
    protocol    = "tcp"
    cidr_blocks = ["0.0.0.0/0"]
  }

  ingress {
    description = "HTTPS"
    from_port   = 443
    to_port     = 443
    protocol    = "tcp"
    cidr_blocks = ["0.0.0.0/0"]
  }

  ingress {
    description = "SSH"
    from_port   = 22
    to_port     = 22
    protocol    = "tcp"
    cidr_blocks = [var.ssh_cidr]
  }

  egress {
    from_port   = 0
    to_port     = 0
    protocol    = "-1"
    cidr_blocks = ["0.0.0.0/0"]
  }

  tags = merge(var.tags, {
    Name = "${var.name_prefix}-web-sg"
  })
}

# EC2 instance for web server
resource "aws_instance" "web_server" {
  count                  = var.instance_count
  ami                    = var.ami_id
  instance_type         = var.instance_type
  key_name              = var.key_name
  vpc_security_group_ids = [aws_security_group.web_sg.id]
  subnet_id             = var.subnet_id

  user_data = var.user_data_script

  tags = merge(var.tags, {
    Name = "${var.name_prefix}-web-${count.index + 1}"
  })
}

# Application Load Balancer (optional)
resource "aws_lb" "web_lb" {
  count              = var.create_load_balancer ? 1 : 0
  name               = "${var.name_prefix}-web-lb"
  internal           = false
  load_balancer_type = "application"
  security_groups    = [aws_security_group.web_sg.id]
  subnets           = var.lb_subnets

  tags = var.tags
}

File: modules/web-server/variables.tf

variable "name_prefix" {
  description = "Prefix for all resource names"
  type        = string
}

variable "vpc_id" {
  description = "VPC ID where resources will be created"
  type        = string
}

variable "subnet_id" {
  description = "Subnet ID for EC2 instances"
  type        = string
}

variable "ami_id" {
  description = "AMI ID for EC2 instances"
  type        = string
  default     = "ami-0c02fb55956c7d316" # Amazon Linux 2
}

variable "instance_type" {
  description = "EC2 instance type"
  type        = string
  default     = "t3.micro"
}

variable "instance_count" {
  description = "Number of EC2 instances to create"
  type        = number
  default     = 1
}

variable "key_name" {
  description = "AWS Key Pair name for SSH access"
  type        = string
}

variable "ssh_cidr" {
  description = "CIDR block allowed for SSH access"
  type        = string
  default     = "10.0.0.0/8"
}

variable "user_data_script" {
  description = "User data script for EC2 initialization"
  type        = string
  default     = <<-EOF
    #!/bin/bash
    yum update -y
    yum install -y httpd
    systemctl start httpd
    systemctl enable httpd
    echo "<h1>Hello from Terraform Module!</h1>" > /var/www/html/index.html
  EOF
}

variable "create_load_balancer" {
  description = "Whether to create an Application Load Balancer"
  type        = bool
  default     = false
}

variable "lb_subnets" {
  description = "Subnet IDs for Load Balancer (required if create_load_balancer is true)"
  type        = list(string)
  default     = []
}

variable "tags" {
  description = "Tags to apply to all resources"
  type        = map(string)
  default     = {}
}

File: modules/web-server/outputs.tf

output "instance_ids" {
  description = "IDs of the created EC2 instances"
  value       = aws_instance.web_server[*].id
}

output "instance_public_ips" {
  description = "Public IP addresses of the instances"
  value       = aws_instance.web_server[*].public_ip
}

output "instance_private_ips" {
  description = "Private IP addresses of the instances"
  value       = aws_instance.web_server[*].private_ip
}

output "security_group_id" {
  description = "ID of the created security group"
  value       = aws_security_group.web_sg.id
}

output "load_balancer_dns" {
  description = "DNS name of the load balancer (if created)"
  value       = var.create_load_balancer ? aws_lb.web_lb[0].dns_name : null
}

output "load_balancer_arn" {
  description = "ARN of the load balancer (if created)"
  value       = var.create_load_balancer ? aws_lb.web_lb[0].arn : null
}

🚀 Using Your Module

Now that we've created our module, let's use it in different projects:

Project 1: Simple Web Server

File: projects/simple-web/main.tf

# Call our web-server module
module "simple_web" {
  source = "../../modules/web-server"

  name_prefix   = "simple-web"
  vpc_id        = "vpc-12345678"
  subnet_id     = "subnet-12345678"
  key_name      = "my-key-pair"
  instance_type = "t3.micro"

  tags = {
    Environment = "development"
    Project     = "simple-web"
    Owner       = "dev-team"
  }
}

# Use module outputs
output "web_server_ip" {
  value = module.simple_web.instance_public_ips[0]
}

Project 2: High-Availability Web Application

File: projects/ha-web-app/main.tf

# Call the same module with different configuration
module "ha_web_app" {
  source = "../../modules/web-server"

  name_prefix          = "ha-web-app"
  vpc_id              = "vpc-87654321"
  subnet_id           = "subnet-87654321"
  key_name            = "prod-key-pair"
  instance_type       = "t3.small"
  instance_count      = 3
  create_load_balancer = true
  lb_subnets          = ["subnet-11111111", "subnet-22222222"]

  user_data_script = <<-EOF
    #!/bin/bash
    yum update -y
    yum install -y httpd
    systemctl start httpd
    systemctl enable httpd
    echo "<h1>Production Web App - Server $(hostname)</h1>" > /var/www/html/index.html
  EOF

  tags = {
    Environment = "production"
    Project     = "ha-web-app"
    Owner       = "ops-team"
    Backup      = "required"
  }
}

output "load_balancer_url" {
  value = "http://${module.ha_web_app.load_balancer_dns}"
}

🎯 Real-World Module Examples

Here are some modules you'll commonly see in organizations:

1. VPC Module

module "vpc" {
  source = "./modules/vpc"

  cidr_block           = "10.0.0.0/16"
  availability_zones   = ["us-east-1a", "us-east-1b"]
  public_subnet_cidrs  = ["10.0.1.0/24", "10.0.2.0/24"]
  private_subnet_cidrs = ["10.0.10.0/24", "10.0.20.0/24"]

  enable_nat_gateway = true
  enable_vpn_gateway = false

  tags = {
    Environment = "production"
  }
}

2. RDS Database Module

module "database" {
  source = "./modules/rds"

  identifier     = "myapp-db"
  engine         = "postgres"
  engine_version = "13.7"
  instance_class = "db.t3.micro"

  allocated_storage = 20
  storage_encrypted = true

  db_name  = "myapp"
  username = "admin"
  password = var.db_password

  vpc_security_group_ids = [module.vpc.database_security_group_id]
  subnet_group_name      = module.vpc.database_subnet_group_name

  backup_retention_period = 7
  backup_window          = "03:00-04:00"
  maintenance_window     = "sun:04:00-sun:05:00"
}

3. S3 Bucket Module

module "app_storage" {
  source = "./modules/s3-bucket"

  bucket_name = "myapp-storage-${random_string.bucket_suffix.result}"

  enable_versioning = true
  enable_encryption = true

  lifecycle_rules = [
    {
      id     = "transition_to_ia"
      status = "Enabled"

      transition = {
        days          = 30
        storage_class = "STANDARD_IA"
      }
    }
  ]

  tags = {
    Environment = "production"
    Purpose     = "application-storage"
  }
}

🌟 Advanced Module Patterns

1. Conditional Resources

# Create resources only when needed
resource "aws_lb" "this" {
  count = var.create_load_balancer ? 1 : 0
  # ... configuration
}

2. Dynamic Blocks

# Create multiple similar blocks dynamically
resource "aws_security_group" "this" {
  name = var.name

  dynamic "ingress" {
    for_each = var.ingress_rules
    content {
      from_port   = ingress.value.from_port
      to_port     = ingress.value.to_port
      protocol    = ingress.value.protocol
      cidr_blocks = ingress.value.cidr_blocks
    }
  }
}

3. Module Composition

# Use multiple modules together
module "vpc" {
  source = "./modules/vpc"
  # ... configuration
}

module "web_servers" {
  source = "./modules/web-server"

  vpc_id    = module.vpc.vpc_id
  subnet_id = module.vpc.public_subnet_ids[0]
  # ... other configuration
}

module "database" {
  source = "./modules/rds"

  vpc_security_group_ids = [module.vpc.database_security_group_id]
  subnet_group_name      = module.vpc.database_subnet_group_name
  # ... other configuration
}

📦 Module Sources and Versioning

1. Local Modules

module "web_server" {
  source = "./modules/web-server"
}

2. Git Repository Modules

module "web_server" {
  source = "git::https://github.com/your-org/terraform-modules.git//web-server?ref=v1.2.0"
}

3. Terraform Registry Modules

module "vpc" {
  source  = "terraform-aws-modules/vpc/aws"
  version = "~> 3.0"
}

4. Private Registry Modules

module "internal_module" {
  source  = "company.registry.io/team/module-name/aws"
  version = "1.0.0"
}

🔧 Best Practices for Module Development

1. Follow Naming Conventions

✅ Good:
- terraform-aws-vpc
- terraform-azure-storage
- terraform-gcp-compute

❌ Bad:
- my-module
- stuff
- vpc-thing

2. Use Semantic Versioning

v1.0.0 - Initial release
v1.1.0 - Added new feature
v1.1.1 - Bug fix
v2.0.0 - Breaking change

3. Provide Good Documentation

# Web Server Module

## Usage

hcl
module "web_server" {
source = "./modules/web-server"

name_prefix = "my-app"
vpc_id = "vpc-12345"
# ...
}


## Requirements
- Terraform >= 1.0
- AWS Provider >= 4.0

## Inputs
| Name | Description | Type | Default | Required |
|------|-------------|------|---------|----------|
| name_prefix | Prefix for resource names | string | n/a | yes |

4. Include Examples

modules/
└── web-server/
    ├── main.tf
    ├── variables.tf
    ├── outputs.tf
    ├── README.md
    └── examples/
        ├── simple/
        │   └── main.tf
        └── advanced/
            └── main.tf

🚨 Common Pitfalls and How to Avoid Them

1. Overly Complex Modules

# ❌ Bad: One module that does everything
module "everything" {
  source = "./modules/entire-infrastructure"
  # 50+ variables
}

# ✅ Good: Focused, single-purpose modules
module "vpc" {
  source = "./modules/vpc"
}

module "web_servers" {
  source = "./modules/web-server"
}

2. Hard-coded Values

# ❌ Bad: Hard-coded values
resource "aws_instance" "web" {
  ami           = "ami-12345678"  # This will break in other regions!
  instance_type = "t2.micro"     # No flexibility
}

# ✅ Good: Use variables and data sources
data "aws_ami" "amazon_linux" {
  most_recent = true
  owners      = ["amazon"]

  filter {
    name   = "name"
    values = ["amzn2-ami-hvm-*-x86_64-gp2"]
  }
}

resource "aws_instance" "web" {
  ami           = data.aws_ami.amazon_linux.id
  instance_type = var.instance_type
}

3. Not Planning for Outputs

# ✅ Always think about what consumers might need
output "instance_ids" {
  value = aws_instance.web[*].id
}

output "security_group_id" {
  value = aws_security_group.web.id
}

output "load_balancer_dns" {
  value = aws_lb.web.dns_name
}

📊 Real-World Success Story

Here's how a 100-person engineering organization transformed their infrastructure:

Before Modules:

🔥 6 hours to set up a new environment
😓 Configuration drift across 20+ projects
🐛 Same bugs appearing in multiple places
📧 "Can you share your VPC configuration?" messages daily

After Modules:

⚡ 30 minutes to set up a new environment
🎯 100% consistent configurations
🔒 Security fixes deployed everywhere instantly
🤝 Self-service infrastructure for all teams

The Numbers:

Development Speed: 10x faster environment setup
Bug Reduction: 85% fewer infrastructure-related bugs
Team Productivity: 3 hours/week saved per developer
Security Compliance: 100% consistent security posture

🎯 Module Development Workflow

1. Plan Your Module

What problem does this solve?
Who will use this module?
What should be configurable?
What are the outputs?

2. Start Simple

# Version 1.0: Basic functionality
resource "aws_instance" "web" {
  ami           = var.ami_id
  instance_type = var.instance_type
}

3. Iterate and Improve

# Version 1.1: Add security group
# Version 1.2: Add load balancer option
# Version 2.0: Support multiple instances

4. Test and Document

- Create example configurations
- Test in multiple environments
- Document all variables and outputs
- Add troubleshooting guide

💡 Module Testing Strategy

1. Unit Testing with Terratest

func TestWebServerModule(t *testing.T) {
    terraformOptions := &terraform.Options{
        TerraformDir: "../examples/simple",
    }

    defer terraform.Destroy(t, terraformOptions)
    terraform.InitAndApply(t, terraformOptions)

    instanceId := terraform.Output(t, terraformOptions, "instance_id")
    assert.NotEmpty(t, instanceId)
}

2. Integration Testing

# Test module in different scenarios
cd examples/simple && terraform apply
cd examples/with-load-balancer && terraform apply
cd examples/multi-instance && terraform apply

3. Security Scanning

# Use tools like tfsec, checkov, or terrascan
tfsec modules/web-server/
checkov -d modules/web-server/

🔮 Advanced Module Topics

1. Module Registry

Host modules in private registries
Version management and automated testing
Module discovery and documentation

2. Module Composition Patterns

Parent modules that combine child modules
Environment-specific module configurations
Cross-module data sharing

3. Module Governance

Approval processes for module changes
Module lifecycle management
Security and compliance scanning

🎯 Implementation Checklist

[ ] Identify repetitive infrastructure patterns
[ ] Design your first module with clear inputs/outputs
[ ] Create module documentation and examples
[ ] Set up module versioning strategy
[ ] Implement testing for your modules
[ ] Create a module registry (internal or external)
[ ] Train your team on module usage
[ ] Establish module governance processes
[ ] Monitor module usage and gather feedback

💡 Key Takeaways

Modules are LEGO blocks for your infrastructure
Start simple, then add complexity as needed
Documentation and examples are crucial for adoption
Versioning prevents breaking changes from affecting consumers
Testing ensures your modules work reliably
Collaboration improves when everyone uses the same building blocks

🤝 Wrapping Up

Terraform modules are like having a team of infrastructure experts who've already solved common problems and packaged their solutions for you to use. They transform Infrastructure as Code from a chore into a joy.

The initial investment in creating good modules pays dividends forever. Your future self (and your teammates) will thank you for the consistency, reliability, and speed that modules bring to your infrastructure.

Start small - pick one repetitive pattern in your infrastructure and modularize it. Then watch as your team's productivity and infrastructure quality improve dramatically.

What infrastructure patterns are you planning to modularize first? Share your module ideas in the comments below!

Tags: #terraform #modules #devops #infrastructure #iac #aws #automation #best-practices

Found this helpful? Give it a ❤️ and follow me for more Infrastructure as Code content!

Building your first module? Drop your questions in the comments - I love helping fellow DevOps engineers!

Terraform Remote Backend: Why Your Team Needs It (And How to Set It Up)

Deepanshu — Wed, 30 Jul 2025 15:59:00 +0000

Ever had that moment when you and your teammate both run terraform apply at the same time and suddenly your infrastructure is in a weird state? Yeah, we've all been there. Let's talk about how Terraform remote backend can save your sanity (and your infrastructure).

What is Terraform Remote Backend?

Think of Terraform's state file as your infrastructure's memory - it remembers what resources exist, their current configuration, and how they're connected. By default, this "memory" is stored locally on your computer in a file called terraform.tfstate.

But here's the problem: What happens when you work in a team?

Imagine you're building a house with your friends, but each person has their own blueprint and nobody shares what they've built. Chaos, right? That's exactly what happens with local state files.

Remote backend is like having a shared blueprint stored in the cloud that everyone can access and update safely.

The Horror Stories of Local State

Let me paint you a picture of what can go wrong:

Scenario 1: The "Oops, I Deleted Everything" Story

Developer A: "I just ran terraform destroy to clean up my test environment"
Developer B: "Wait... wasn't that the production database?! "

Scenario 2: The "Conflict of Interest" Story

Developer A: Creates 5 EC2 instances
Developer B: (at the same time) Creates 3 RDS databases
Terraform: "Error: State file conflict. Good luck figuring this out! "

Scenario 3: The "Lost in Space" Story

Developer: "My laptop crashed and I lost the state file"
Infrastructure: *exists but is now unmanaged*
Team: "So... do we rebuild everything or manually import 100 resources?"

Remote Backend Architecture

Here's how remote backend solves these problems:

    👩‍💻 Developer 1        👨‍💻 Developer 2        👩‍💻 Developer 3
         │                    │                    │
         └─────────────┬──────┴─────────┬─────────┘
                       │                │
                       ▼                ▼
              ┌─────────────────────────────┐
              │       ☁️ AWS S3 Bucket       │
              │    (Shared State Storage)   │
              └─────────────┬───────────────┘
                            │
                            ▼
              ┌─────────────────────────────┐
              │      🔒 DynamoDB Table      │
              │      (State Locking)       │
              └─────────────────────────────┘

Real-World Benefits for Organizations

1. Team Collaboration Made Easy

Instead of emailing state files around (please tell me you're not doing this), everyone accesses the same centralized state.

2. State Locking Prevents Disasters

When someone runs terraform apply, the state gets locked. If another team member tries to run Terraform simultaneously, they get a friendly "Hey, wait your turn!" message instead of corrupting the state.

3. Backup and Versioning

S3 automatically versions your state files. Accidentally destroyed production? Roll back to the previous state version.

4. Security and Compliance

State files often contain sensitive information. Remote backend with proper IAM policies ensures only authorized team members can access it.

5. CI/CD Integration

Your deployment pipelines can access the state without needing to store it in your code repository.

Setting Up Remote Backend (Step by Step)

Let's build this together! I'll show you exactly how to set up a remote backend with AWS S3 and DynamoDB.

Step 1: Create the Backend Infrastructure

First, we need to create the S3 bucket and DynamoDB table that will store our state:

File: backend-setup/resource.tf

S3 bucket for storing state files

resource "aws_s3_bucket" "terraform_state" {
  bucket = "your-company-terraform-state-bucket"

  tags = {
    Name        = "Terraform State Bucket"
    Environment = "Production"
    Purpose     = "Infrastructure State Management"
  }
}

Enable versioning for state file backup

resource "aws_s3_bucket_versioning" "terraform_state_versioning" {
  bucket = aws_s3_bucket.terraform_state.id
  versioning_configuration {
    status = "Enabled"
  }
}

Encrypt the state files

resource "aws_s3_bucket_server_side_encryption_configuration" 
"terraform_state_encryption" {
  bucket = aws_s3_bucket.terraform_state.id

  rule {
    apply_server_side_encryption_by_default {
      sse_algorithm = "AES256"
    }
  }
}

DynamoDB table for state locking

resource "aws_dynamodb_table" "terraform_state_lock" {
  name           = "terraform-state-lock"
  billing_mode   = "PAY_PER_REQUEST"
  hash_key       = "LockID"

  attribute {
    name = "LockID"
    type = "S"
  }

  tags = {
    Name = "Terraform State Lock Table"
  }
}

File: backend-setup/provider.tf

provider "aws" {
  region = "us-east-1"
}

File: backend-setup/terraform.tf

terraform {
  required_version = ">= 1.0"

  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 5.0"
    }
  }
}

Run this setup first:

cd backend-setup/
terraform init
terraform plan
terraform apply

Step 2: Configure Your Project to Use Remote Backend

Now, for any Terraform project, add this backend configuration:

File: your-project/terraform.tf

terraform {
  required_version = ">= 1.0"

  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 5.0"
    }
  }

  # Here's the magic!
  backend "s3" {
    bucket         = "your-company-terraform-state-bucket"
    key            = "projects/my-awesome-project/terraform.tfstate"
    region         = "us-east-1"
    dynamodb_table = "terraform-state-lock"
    encrypt        = true
  }
}

Step 3: Initialize and Migrate

cd your-project/
terraform init

# If you have existing local state, Terraform will ask:
# "Do you want to copy existing state to the new backend?"
# Answer: yes

Pro Tips for Production Use

1. Organize Your State Files

Use meaningful key paths:

# Good
key = "environments/production/vpc/terraform.tfstate"
key = "environments/staging/databases/terraform.tfstate"

# Bad
key = "terraform.tfstate"
key = "stuff.tfstate"

2. Use Different Buckets for Different Environments

# Production
bucket = "company-terraform-prod-state"

# Staging
bucket = "company-terraform-staging-state"

3. Set Up Proper IAM Permissions

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "s3:ListBucket",
        "s3:GetObject",
        "s3:PutObject",
        "s3:DeleteObject"
      ],
      "Resource": [
        "arn:aws:s3:::your-terraform-state-bucket",
        "arn:aws:s3:::your-terraform-state-bucket/*"
      ]
    },
    {
      "Effect": "Allow",
      "Action": [
        "dynamodb:GetItem",
        "dynamodb:PutItem",
        "dynamodb:DeleteItem"
      ],
      "Resource": "arn:aws:dynamodb:*:*:table/terraform-state-lock"
    }
  ]
}

Common Gotchas and How to Avoid Them

1. The Bootstrap Problem

You can't use remote backend to create the remote backend! Create the S3 bucket and DynamoDB table first with local state, then migrate other projects.

2. State File Naming Conflicts

Always use unique key values for different projects:

# This will cause conflicts if multiple projects use it
key = "terraform.tfstate"  

# This is unique and descriptive
key = "projects/web-app/production/terraform.tfstate"

3. Lock File Stuck

If Terraform crashes, the lock might remain. Force unlock with:

terraform force-unlock LOCK_ID

Real-World Success Story

Here's how a 50-person engineering team I worked with transformed their infrastructure management:

Before Remote Backend:

3 production outages due to state conflicts
2 hours average time to resolve state issues
State files shared via Slack/email
One developer accidentally destroyed staging DB

After Remote Backend:

Zero state-related outages in 6 months
5 minutes to onboard new team members
Seamless collaboration across teams
Audit trail of all infrastructure changes

Implementation Checklist

[ ] Create S3 bucket for state storage
[ ] Set up DynamoDB table for locking
[ ] Enable S3 versioning and encryption
[ ] Configure IAM permissions
[ ] Update terraform.tf with backend config
[ ] Run terraform init to migrate
[ ] Test with team members
[ ] Document the setup for your team
[ ] Set up monitoring and alerts

Next Steps

Once you have remote backend set up, consider these advanced topics:

Terraform Workspaces for environment management
Remote state data sources for cross-project dependencies
Terraform Cloud/Enterprise for advanced features
State file encryption with customer-managed keys

Key Takeaways

Remote backend is essential for any team using Terraform
It prevents state conflicts and enables collaboration
S3 + DynamoDB is a solid, cost-effective solution
Set it up once, enjoy peace of mind forever
Your future self (and teammates) will thank you

Learn more about terraform with practical knowledge check out GitHub

Wrapping Up

Setting up remote backend might seem like extra work initially, but it's an investment that pays dividends. Think of it as infrastructure insurance - you hope you never need it, but you'll be incredibly grateful when you do.

Have you implemented remote backend in your organization? What challenges did you face? Share your experiences in the comments below!

Tags: #terraform #aws #devops #infrastructure #backend #s3 #dynamodb #iac

Found this helpful? Give it a ❤️ and follow me for more practical DevOps content!

Got questions? Drop them in the comments - I read and respond to every one!