DEV Community: Rahul Joshi

Day 29 — 🔭 Monitoring & Observability Part Two

Rahul Joshi — Tue, 09 Jun 2026 04:13:37 +0000

In Part 1, we covered:

Observability Fundamentals
Monitoring
Metrics
Prometheus
Grafana
Alerting

Monitoring tells us:

Something is wrong

But monitoring alone cannot answer:

Why is it wrong?
Which service failed?
Which request caused the problem?

This is where the remaining pillars of observability become critical:

Logging
     +
Tracing

Together they help engineers perform:

Root Cause Analysis
Incident Investigation
Distributed System Debugging
Performance Optimization

🔗 Resources

** Support the Journey on GitHub: If you're following along, consider starring and forking the repo:** https://github.com/17J/30-Days-Cloud-DevSecOps-Journey

The Three Pillars Revisited

Observability consists of:

Metrics
Logs
Traces

Metrics answer:

What is happening?

Logs answer:

What happened?

Traces answer:

Why did it happen?

Why Monitoring Alone Is Not Enough

Example:

Prometheus Alert:

API Error Rate = 30%

Monitoring tells us:

Problem Exists

But not:

Which API?
Which User?
Which Request?
Which Database Query?

Logs and traces provide those answers.

What is Logging?

Logging is the process of recording events generated by applications, operating systems, and infrastructure.

Examples:

User Login Success
Payment Processed
Database Connection Failed
Pod Restarted
API Timeout

Logs are detailed records of system behavior.

Why Logging Matters

Imagine an application crash.

Monitoring shows:

CPU = Normal
Memory = Normal
Error Rate = High

But logs reveal:

Database Authentication Failed

Root cause found.

Types of Logs

Modern environments generate multiple log types.

Application Logs

Generated by application code.

Example:

{
  "timestamp":"2026-01-01T10:00:00Z",
  "service":"payment-api",
  "level":"ERROR",
  "message":"Payment processing failed"
}

System Logs

Generated by operating systems.

Examples:

Kernel Events
Service Start
Authentication Events
System Reboots

Container Logs

Generated by containers.

Example:

kubectl logs pod-name

Kubernetes Logs

Generated by Kubernetes components.

Examples:

Kubelet
API Server
Scheduler
Controller Manager

Security Logs

Examples:

Failed Login Attempts
Privilege Escalation
Unauthorized Access

Very important for SOC teams.

Challenges with Logging

Modern environments generate huge volumes.

Example:

100 Microservices
      ↓
10 Pods Each
      ↓
Millions of Log Lines

Problems:

Storage
Search
Correlation
Cost

This is why centralized logging exists.

What is Centralized Logging?

Instead of:

Application A Logs
Application B Logs
Application C Logs

stored separately,

we collect everything into a central platform.

Applications
      ↓
Log Collector
      ↓
Central Storage
      ↓
Search & Analysis

Popular Logging Platforms

Today most organizations use:

ELK Stack
EFK Stack
Loki
Splunk
Datadog

Understanding ELK Stack

ELK stands for:

Elasticsearch
Logstash
Kibana

One of the most popular logging solutions.

ELK Architecture

Applications
      ↓
Logstash
      ↓
Elasticsearch
      ↓
Kibana

Elasticsearch

Stores logs.

Think of it as:

Searchable Log Database

Capabilities:

Full-text search
Indexing
Analytics
Aggregation

Logstash

Processes logs.

Responsibilities:

Collect
Transform
Parse
Enrich
Forward

Example:

Raw Log
     ↓
Structured JSON

Kibana

Visualization layer.

Provides:

Dashboards
Search
Analytics
Visualizations

Example ELK Workflow

Application Log
      ↓
Logstash
      ↓
Elasticsearch
      ↓
Kibana Dashboard

What is EFK Stack?

Kubernetes-focused version of ELK.

EFK:

Elasticsearch
Fluent Bit
Kibana

Fluent Bit replaces Logstash.

Why Fluent Bit?

Benefits:

Lightweight
Fast
Kubernetes Native
Lower Resource Usage

EFK Architecture

Pods
 ↓
Fluent Bit
 ↓
Elasticsearch
 ↓
Kibana

What is Grafana Loki?

Loki is a modern log aggregation system developed by Grafana Labs.

Designed specifically for cloud-native environments.

Why Loki Became Popular

ELK is powerful but expensive.

Loki offers:

Simpler Architecture
Lower Storage Cost
Grafana Integration
Kubernetes Friendly

Loki Architecture

Applications
      ↓
Promtail
      ↓
Loki
      ↓
Grafana

Promtail

Log collector.

Collects:

Container Logs
Pod Logs
System Logs

and sends them to Loki.

Loki Advantages

Benefits:

Low Cost
Easy Deployment
Cloud Native
Grafana Native

What is Distributed Tracing?

Logs tell us:

What happened?

But not:

Where did latency occur?

Tracing solves this problem.

Why Tracing Exists

In microservices:

User Request
      ↓
Frontend
      ↓
API Gateway
      ↓
Service A
      ↓
Service B
      ↓
Database

One slow service impacts the entire request.

Tracing helps locate it.

What is a Trace?

A trace represents a complete request journey.

Example:

Request Start
      ↓
Service A
      ↓
Service B
      ↓
Database
      ↓
Response

What is a Span?

A trace contains multiple spans.

Example:

Trace
 ├─ API Call
 ├─ Database Query
 ├─ Cache Lookup
 └─ External API Call

Each span measures duration.

What is OpenTelemetry?

OpenTelemetry (OTel) is the industry standard for observability instrumentation.

Supported by CNCF.

Provides:

Metrics
Logs
Traces

through one framework.

Why OpenTelemetry Matters

Before OpenTelemetry:

Vendor Specific Agents

After OpenTelemetry:

Single Standard

for observability.

OpenTelemetry Components

SDK

Embedded in applications.

Collects:

Metrics
Logs
Traces

Collector

Receives telemetry.

Processes:

Filter
Transform
Route
Export

Exporters

Send data to:

Prometheus
Jaeger
Loki
Elastic
Datadog

What is Jaeger?

Jaeger is an open-source distributed tracing platform.

Originally developed by Uber.

Now maintained by CNCF.

Jaeger Architecture

Application
      ↓
OTel Collector
      ↓
Jaeger
      ↓
UI

Jaeger Features

Provides:

Trace Visualization
Dependency Mapping
Latency Analysis
Performance Troubleshooting

Example Trace

User Request
      ↓ 50ms
Frontend

      ↓ 20ms
API Gateway

      ↓ 300ms
Payment Service

      ↓ 10ms
Database

Problem identified:

Payment Service

Installing Loki in Development

Add repository:

helm repo add grafana \
https://grafana.github.io/helm-charts

Install:

helm install loki grafana/loki-stack

Installing Jaeger in Development

docker run -d \
--name jaeger \
-p 16686:16686 \
-p 4317:4317 \
jaegertracing/all-in-one

Access:

http://localhost:16686

Installing Loki in Pre-Prod Kubernetes

helm install loki \
grafana/loki-stack \
-n observability \
--create-namespace

Installing Jaeger in Kubernetes

helm repo add jaegertracing \
https://jaegertracing.github.io/helm-charts

Install:

helm install jaeger \
jaegertracing/jaeger \
-n observability

Modern Kubernetes Observability Stack

Prometheus
+
Grafana
+
Loki
+
Jaeger
+
OpenTelemetry

This combination is currently one of the most popular cloud-native observability platforms.

Final Thoughts

Observability is much more than monitoring.

Monitoring tells you:

Something is wrong

Logging tells you:

What happened

Tracing tells you:

Why it happened

Modern cloud-native platforms achieve full observability by combining:

Prometheus
+
Grafana
+
Loki
+
Jaeger
+
OpenTelemetry

Together these tools provide:

Metrics
Logs
Traces
Root Cause Analysis
Performance Optimization
Incident Response

which are essential for operating reliable Kubernetes and microservices platforms at scale.

Day 28 — 🔭 Monitoring & Observability Part One

Rahul Joshi — Mon, 08 Jun 2026 03:50:09 +0000

In Modern Time applications are no longer simple monolithic systems.

Today organizations run:

Microservices
Kubernetes
Containers
Serverless Functions
Multi-Cloud Platforms
Distributed Systems

As infrastructure becomes more distributed, troubleshooting becomes significantly harder.

A single user request may travel through:

Frontend
    ↓
API Gateway
    ↓
Microservice A
    ↓
Microservice B
    ↓
Database

When something breaks, the biggest challenge becomes:

"What exactly happened?"

This is where Observability becomes critical.

🔗 Resources

** Support the Journey on GitHub: If you're following along, consider starring and forking the repo:** https://github.com/17J/30-Days-Cloud-DevSecOps-Journey

What is Observability?

Observability is the ability to understand the internal state of a system by analyzing the data it produces.

In simple words:

Can we understand
what is happening
inside our systems?

Observability helps engineers answer:

Why is the application slow?
Which service is failing?
Which request caused the issue?
What changed recently?
Where is latency occurring?

Without observability:

Problem Exists
      ↓
Guessing Begins

With observability:

Problem Exists
      ↓
Evidence Available
      ↓
Faster Resolution

Why Observability Matters

Modern cloud-native systems generate enormous amounts of data.

Example:

100 Microservices
      ↓
Millions of Requests
      ↓
Thousands of Containers

Traditional monitoring alone is no longer sufficient.

Organizations need:

Visibility
Insights
Correlation
Root Cause Analysis

Observability provides all of them.

Monitoring vs Observability

Many people confuse monitoring and observability.

Monitoring asks:

What is wrong?

Observability asks:

Why is it wrong?

Example:

Monitoring:

CPU Usage = 95%

Observability:

Which service?
Which request?
Which dependency?
Which deployment caused it?

Observability provides context.

The Three Pillars of Observability

Modern observability is built on three primary pillars.

Metrics
Logs
Traces

Or:

Monitoring
Logging
Tracing

Together they provide a complete picture of system behavior.

Pillar 1: Monitoring (Metrics)

Monitoring focuses on numerical measurements.

Examples:

CPU Usage
Memory Usage
Request Rate
Error Rate
Latency
Disk Usage

Metrics answer:

How much?
How often?
How fast?

Pillar 2: Logging

Logs provide detailed event information.

Example:

User Login Success
Database Connection Failed
API Request Received

Logs answer:

What happened?

Pillar 3: Tracing

Tracing follows a request across multiple services.

Example:

User Request
      ↓
Frontend
      ↓
API
      ↓
Payment Service
      ↓
Database

Tracing answers:

Where did the request spend time?

Why Metrics Matter First

Among all observability signals:

Metrics

are usually the first thing engineers implement.

Reasons:

Lightweight
Efficient
Fast alerting
Low storage cost
Easy visualization

This is why Prometheus became the industry standard.

What is Prometheus?

Prometheus is an open-source monitoring and alerting system originally developed at SoundCloud and now maintained by CNCF.

Prometheus collects:

Metrics

from applications and infrastructure.

Example:

CPU
Memory
Network
Latency
Errors

Why Prometheus Became Popular

Before Prometheus:

Monitoring Tools
      ↓
Complex
Expensive
Difficult Scaling

Prometheus introduced:

Pull-Based Collection
Powerful Query Language
Kubernetes Integration
Open Source

Understanding Prometheus Components

Prometheus Server

Core component.

Responsible for:

Metric collection
Storage
Query processing
Alerting

Exporters

Prometheus collects metrics through exporters.

Examples:

Node Exporter
MySQL Exporter
MongoDB Exporter
Redis Exporter
Blackbox Exporter

Alertmanager

Handles alerts.

Example:

CPU > 90%
      ↓
Alertmanager
      ↓
Email
Slack
Teams
PagerDuty

Time-Series Database

Prometheus stores metrics as:

Timestamp + Value

Example:

10:00 CPU=45%
10:01 CPU=48%
10:02 CPU=51%

What is Grafana?

Grafana is a visualization platform used to create dashboards from Prometheus metrics.

Prometheus stores data.

Grafana visualizes data.

Relationship:

Prometheus
      ↓
Metrics
      ↓
Grafana
      ↓
Dashboards

Why Grafana is Popular

Grafana provides:

Beautiful dashboards
Alerting
Multiple data sources
Real-time visualization

Supported sources:

Prometheus
Elasticsearch
Loki
InfluxDB
CloudWatch
Azure Monitor

Prometheus + Grafana Architecture

Applications
      ↓
Exporters
      ↓
Prometheus
      ↓
Grafana
      ↓
Engineers

Common Metrics Monitored

Infrastructure:

CPU
Memory
Disk
Network

Application:

Request Rate
Response Time
Error Rate

Kubernetes:

Pod Count
Node Status
Container CPU
Container Memory

Installing Prometheus in Development Environment

For local development, Docker is easiest.

Run Prometheus Container

docker run -d \
--name prometheus \
-p 9090:9090 \
prom/prometheus

Verify:

http://localhost:9090

Check Targets

Navigate:

Status
   ↓
Targets

Installing Node Exporter

docker run -d \
--name node-exporter \
-p 9100:9100 \
prom/node-exporter

This exposes:

CPU Metrics
Memory Metrics
Disk Metrics

Configure Prometheus

Example:

global:
  scrape_interval: 15s

scrape_configs:
  - job_name: node
    static_configs:
      - targets:
        - localhost:9100

Restart Prometheus.

Installing Grafana in Development Environment

Run Grafana:

docker run -d \
--name grafana \
-p 3000:3000 \
grafana/grafana

Access:

http://localhost:3000

Default:

admin/admin

Connect Grafana to Prometheus

Add Data Source:

Grafana
    ↓
Connections
    ↓
Data Sources
    ↓
Prometheus

URL:

http://prometheus:9090

Save and Test.

Creating First Dashboard

Example panel:

rate(node_cpu_seconds_total[5m])

Shows CPU usage.

Installing Prometheus in Pre-Production Kubernetes

Production-like environments typically use Helm.

Add Prometheus Community Repo

helm repo add prometheus-community \
https://prometheus-community.github.io/helm-charts

Update:

helm repo update

Install kube-prometheus-stack

helm install monitoring \
prometheus-community/kube-prometheus-stack \
-n monitoring \
--create-namespace

This installs:

Prometheus
Grafana
Alertmanager
Node Exporter
Kube State Metrics

in one deployment.

Verify Installation

kubectl get pods -n monitoring

Expected:

prometheus
grafana
alertmanager
node-exporter

Access Grafana

kubectl port-forward svc/monitoring-grafana \
3000:80 \
-n monitoring

Open:

http://localhost:3000

Access Prometheus

kubectl port-forward svc/monitoring-kube-prometheus-prometheus \
9090:9090 \
-n monitoring

Open:

http://localhost:9090

Production Monitoring Stack

A typical enterprise monitoring stack looks like:

Kubernetes Cluster
       ↓
Node Exporter
       ↓
Prometheus
       ↓
Alertmanager
       ↓
Grafana
       ↓
Operations Team

Example Alert Rule

CPU Alert:

groups:
- name: cpu-alerts

  rules:
  - alert: HighCPUUsage

    expr: node_cpu_seconds_total > 90

    for: 5m

Grafana Dashboard Examples

Infrastructure Dashboard:

CPU Usage
Memory Usage
Disk Usage
Network Traffic

Kubernetes Dashboard:

Nodes
Pods
Deployments
Namespaces

Application Dashboard:

Request Rate
Error Rate
Latency
Availability

Monitoring Best Practices

Use Labels Properly

Good:

environment=prod
team=platform
service=payment

Retain Metrics Wisely

Avoid storing metrics forever.

Create Actionable Alerts

Bad:

CPU > 80%

Good:

CPU > 90% for 10 minutes

Separate Environments

Dev
QA
PreProd
Prod

should have independent monitoring.

Observability Tools Landscape

Monitoring:

Prometheus
Grafana
Datadog
New Relic
CloudWatch
Azure Monitor

Logging:

ELK Stack
EFK Stack
Loki
Splunk

Tracing:

Jaeger
Zipkin
Tempo
OpenTelemetry

What We'll Cover in Part Two

This article focused on:

Observability Fundamentals
Monitoring
Prometheus
Grafana

In Part Two we'll cover:

Logging
Centralized Log Management
ELK Stack
EFK Stack
Loki
Tracing
Jaeger
OpenTelemetry
Distributed Tracing
End-to-End Observability

Final Thoughts

Observability is one of the most important capabilities in modern cloud-native platforms.

Without observability:

Failures Become Guesswork

With observability:

Metrics
Logs
Traces
      ↓
Faster Troubleshooting
Better Reliability
Improved User Experience

For most organizations, the journey starts with:

Prometheus
+
Grafana

because they provide a powerful, scalable, and Kubernetes-native monitoring platform.

Once monitoring is established, the next step is adding:

Logging
+
Tracing

to achieve full-stack observability.

Day 27 — Container & Runtime Security

Rahul Joshi — Sun, 07 Jun 2026 03:55:37 +0000

Containers have transformed modern software delivery.

Today almost every cloud-native platform uses:

Docker
Kubernetes
Amazon EKS
Azure AKS
Google GKE
OpenShift

Containers allow organizations to deploy applications faster, scale efficiently, and maintain consistency across environments.

However, containers also introduce new security challenges.

Many teams focus only on:

Container Build
       ↓
Image Scan
       ↓
Deploy

and assume they are secure.

Unfortunately, security doesn't stop after deployment.

Modern attackers target:

Container runtimes
Kubernetes clusters
Misconfigured containers
Exposed secrets
Privileged workloads

This is where Container Security and Runtime Security become critical.

🔗 Resources

** Support the Journey on GitHub: If you're following along, consider starring and forking the repo:** https://github.com/17J/30-Days-Cloud-DevSecOps-Journey

Why Container Security Matters

Modern applications are increasingly deployed as containers.

Example:

Application
      ↓
Container
      ↓
Kubernetes
      ↓
Production

If an attacker compromises a container:

Container
      ↓
Host Node
      ↓
Kubernetes Cluster
      ↓
Cloud Environment

The impact can be enormous.

What is Container Security?

Container Security is the practice of protecting containers throughout their entire lifecycle.

This includes:

Build Phase
      ↓
Registry
      ↓
Deployment
      ↓
Runtime

Container security covers:

Secure image creation
Vulnerability scanning
Image signing
Access control
Runtime protection
Compliance monitoring

Container Lifecycle Security

A secure container journey looks like:

Developer Writes Code
          ↓
Build Container
          ↓
Image Scan
          ↓
Push to Registry
          ↓
Deploy to Kubernetes
          ↓
Runtime Monitoring
          ↓
Threat Detection

Security should exist at every stage.

Why Traditional Security Is Not Enough

Traditional security tools focus on:

Servers
Virtual Machines
Networks

Containers introduce:

Ephemeral Workloads
Shared Kernel
Microservices
Dynamic Scaling

which require specialized security approaches.

Understanding Container Architecture

A container consists of:

Unlike virtual machines:

Multiple Containers
      ↓
Shared Kernel

This creates unique attack vectors.

Common Container Security Risks

1. Vulnerable Base Images

Many developers pull images directly from public registries.

Example:

FROM ubuntu:latest

Problem:

Unknown Vulnerabilities
Unknown Dependencies
Unknown Configuration

2. Running Containers as Root

Dangerous example:

USER root

Risk:

Container Escape
Privilege Escalation
Host Compromise

3. Hardcoded Secrets

Bad practice:

DATABASE_PASSWORD=password123

inside:

Dockerfile
Source Code
Environment Variables

4. Excessive Linux Capabilities

Containers often receive permissions they don't need.

Example:

NET_ADMIN
SYS_ADMIN

These capabilities increase attack surface.

5. Untrusted Container Images

Downloading random images from Docker Hub can introduce:

Malware
Crypto miners
Backdoors

What is Runtime Security?

Container security before deployment is important.

Runtime security focuses on what happens after deployment.

Runtime Security means:

Monitor
Detect
Respond

to suspicious container behavior while applications are running.

Why Runtime Security Matters

Even a perfectly scanned image can be compromised.

Example:

Image Clean
      ↓
Application Vulnerability
      ↓
Remote Code Execution
      ↓
Runtime Attack

Image scanning cannot detect runtime behavior.

Runtime Threat Example

Container Running Normally
          ↓
Attacker Exploits Vulnerability
          ↓
Shell Spawned
          ↓
Sensitive Data Accessed

Image scans won't catch this.

Runtime security will.

Understanding Runtime Threats

Reverse Shells

One of the most common attacks.

Example:

Container
      ↓
Attacker
      ↓
Reverse Shell

Now the attacker has interactive access.

Cryptocurrency Mining

Compromised containers are often used for:

Cryptocurrency Mining

Symptoms:

High CPU Usage
Unexpected Processes
Resource Exhaustion

Privilege Escalation

Attackers attempt:

Container
      ↓
Root Access
      ↓
Host Access

to escape the container boundary.

Suspicious Process Execution

Example:

Nginx Container
      ↓
Unexpected Bash Process

This should trigger an alert.

File Tampering

Attackers may modify:

Application Files
System Files
Configurations

inside running containers.

Container Escape

One of the most dangerous attacks.

Goal:

Container
      ↓
Host Node

If successful:

Entire Kubernetes Node Compromised

What is Container Hardening?

Container Hardening reduces the attack surface before deployment.

Think of it as:

Removing Everything Unnecessary

from the container.

Why Container Hardening?

Default containers often include:

Extra Packages
Unused Tools
Shells
Package Managers

All of these increase risk.

Container Hardening Best Practices

Use Minimal Base Images

Bad:

FROM ubuntu

Better:

FROM alpine

Even better:

FROM gcr.io/distroless/static

Benefits:

Smaller Images
Fewer Vulnerabilities
Reduced Attack Surface

Run as Non-Root

Bad:

USER root

Good:

RUN adduser appuser
USER appuser

Benefits:

Reduced Privilege Escalation Risk

Remove Unnecessary Packages

Avoid installing:

curl
wget
vim
bash
gcc

unless absolutely required.

Use Read-Only File Systems

Example:

securityContext:
  readOnlyRootFilesystem: true

Benefits:

Prevents File Modification

Drop Linux Capabilities

Example:

capabilities:
  drop:
    - ALL

Grant only required capabilities.

Set Resource Limits

Example:

resources:
  limits:
    cpu: "500m"
    memory: "512Mi"

Protects against:

DoS
Crypto Mining
Resource Abuse

What is Image Scanning?

Image scanning analyzes container images for:

Vulnerabilities
Misconfigurations
Secrets
Malware

before deployment.

Why Image Scanning Matters

Applications often contain:

Open Source Libraries
Operating System Packages
Framework Dependencies

Some may have known vulnerabilities.

Example Vulnerability

Application
      ↓
Old Log4j Version
      ↓
Remote Code Execution

This could compromise the entire environment.

Image Scanning Workflow

Developer Builds Image
          ↓
Image Scanner
          ↓
Vulnerability Report
          ↓
Fix Issues
          ↓
Deploy

Popular Image Scanning Tools

Trivy

One of the most popular scanners.

Features:

Image scanning
Filesystem scanning
IaC scanning
Kubernetes scanning

Example:

trivy image nginx:latest

Grype

Container vulnerability scanner.

Benefits:

Fast
Open Source
Accurate

Snyk Container

Enterprise-focused platform.

Features:

Vulnerability detection
Fix recommendations
Continuous monitoring

Clair

Open-source container scanner.

Often integrated into registries.

Example Trivy Output

Critical: 2
High: 8
Medium: 14
Low: 20

Organizations often block deployments if:

Critical > 0

Runtime Security Tools

Image scanning alone is not enough.

You need runtime visibility.

Falco

One of the most popular runtime security tools.

Created by:

Sysdig

Now a CNCF project.

How Falco Works

Container Activity
        ↓
Kernel Events
        ↓
Falco Rules
        ↓
Alert

Example Falco Detection

Detect:

Shell Spawned in Container

Alert:

Unexpected Shell Execution

Falco Use Cases

Detect:

Reverse shells
Privilege escalation
Crypto miners
Suspicious file access
Container escape attempts

Tetragon

Modern eBPF-based runtime security platform.

Developed by:

Isovalent

Features:

Process Monitoring
Network Monitoring
Security Enforcement

Sysdig Secure

Enterprise runtime security platform.

Provides:

Runtime detection
Compliance
Threat intelligence

Runtime Security in Kubernetes

A secure Kubernetes deployment looks like:

Pod
 ↓
Security Context
 ↓
Network Policy
 ↓
Runtime Monitoring
 ↓
Alerting

Multiple security layers are required.

Secure Container Pipeline

Modern DevSecOps pipeline:

Developer Commit
        ↓
SAST
        ↓
SCA
        ↓
Container Build
        ↓
Image Scan
        ↓
Registry
        ↓
Kubernetes Deployment
        ↓
Runtime Monitoring
        ↓
Threat Detection

Container Security Best Practices

Use Trusted Images

Only pull images from approved registries.

Scan Every Image

Integrate scanners into CI/CD.

Example:

Trivy
Grype
Snyk

Run as Non-Root

Avoid privileged containers.

Use Read-Only Filesystems

Prevent file tampering.

Sign Container Images

Use:

Cosign
Notary

to verify image authenticity.

Enforce Kubernetes Policies

Use:

Kyverno
OPA Gatekeeper

to prevent insecure deployments.

Monitor Runtime Activity

Use:

Falco
Tetragon
Sysdig

for continuous visibility.

Real-World Attack Scenario

Vulnerable Application
        ↓
Remote Code Execution
        ↓
Shell Spawned
        ↓
Credential Theft
        ↓
Cloud Access
        ↓
Infrastructure Compromise

Without runtime security:

Attack Goes Undetected

With runtime security:

Falco Alert
      ↓
SOC Investigation
      ↓
Threat Contained

Enterprise Container Security Architecture

Developer
      ↓
Git Repository
      ↓
CI/CD Pipeline
      ↓
Trivy Scan
      ↓
Container Registry
      ↓
Kubernetes Cluster
      ↓
Falco Runtime Monitoring
      ↓
SIEM
      ↓
Security Team

Final Thoughts

Container security is no longer optional.

As organizations adopt:

Kubernetes
Microservices
Cloud Native Platforms
DevSecOpscode

they must secure containers at every stage.

A mature security strategy includes:

Secure Images
      +
Container Hardening
      +
Image Scanning
      +
Runtime Monitoring
      +
Threat Detection

Because the most dangerous attacks often happen after deployment, not before.

The strongest container security programs combine preventive controls, runtime visibility, and continuous monitoring to protect modern cloud-native environments.

Day 26 - HashiCorp Vault & Secrets Management

Rahul Joshi — Sat, 06 Jun 2026 03:39:02 +0000

Modern applications depend on secrets.

Every application requires:

Database Passwords
API Keys
SSH Keys
TLS Certificates
Cloud Credentials
OAuth Tokens
Service Account Keys

The biggest question is:

Where should we store them securely?

Unfortunately many organizations still store secrets in:

Git Repository
Docker Image
Application Config Files
Environment Variables
Shared Documents
Excel Sheets

This creates a massive security risk.

This is why Secret Management platforms like HashiCorp Vault became critical in modern cloud-native environments.

🔗 Resources

** Support the Journey on GitHub: If you're following along, consider starring and forking the repo:** https://github.com/17J/30-Days-Cloud-DevSecOps-Journey

What is a Secret?

A secret is any sensitive piece of information used to authenticate or authorize access.

Examples:

Database Password
AWS Access Key
JWT Signing Key
API Token
TLS Certificate
Private Key
OAuth Secret

If a secret gets exposed:

Attacker
      ↓
Application Access
      ↓
Database Access
      ↓
Infrastructure Compromise

What is Secrets Management?

Secrets Management is the process of:

Store
Protect
Rotate
Control
Audit

sensitive credentials securely.

A modern secrets management platform provides:

Centralized storage
Encryption
Access control
Secret rotation
Audit logs
Dynamic credentials

Why Secrets Management Matters

Imagine this scenario:

database:
  username: admin
  password: Password123

committed into GitHub.

Result:

Developer Pushes Code
          ↓
GitHub Repository
          ↓
Credential Leak
          ↓
Database Breach

This happens more often than people realize.

The Problem with Traditional Secret Storage

Many teams use:

.env Files
Kubernetes Secrets
Configuration Files
Hardcoded Passwords

Problems:

Difficult rotation
No audit trail
Poor access control
Risk of accidental exposure
Compliance failures

What is HashiCorp Vault?

HashiCorp Vault is a centralized secrets management platform designed to securely store, access, and manage secrets.

Think of Vault as:

Central Secret Bank

for your infrastructure and applications.

Instead of:

Application
     ↓
Database Password

stored locally,

you use:

Application
      ↓
Vault
      ↓
Database Credentials

Why HashiCorp Created Vault

Modern infrastructure became increasingly complex.

Organizations adopted:

Kubernetes
Multi-cloud
Microservices
Containers
CI/CD Pipelines

Suddenly there were thousands of secrets.

Example:

50 Microservices
     ↓
20 Secrets Each
     ↓
1000 Secrets

Managing them manually became impossible.

Vault was created to solve this problem.

Core Features of HashiCorp Vault

1. Centralized Secret Storage

All secrets stored in one location.

Applications
       ↓
HashiCorp Vault
       ↓
Secrets

2. Encryption as a Service

Vault encrypts sensitive data.

Plain Text
     ↓
Encryption
     ↓
Encrypted Secret

3. Dynamic Secrets

One of Vault's most powerful features.

Instead of:

Static Password

Vault generates temporary credentials.

Example:

Application
      ↓
Vault
      ↓
Temporary Database User
      ↓
Expires Automatically

4. Secret Rotation

Vault automatically rotates secrets.

Example:

Old Password
      ↓
Vault Rotation
      ↓
New Password

No manual work required.

5. Audit Logging

Every secret access is logged.

Example:

Who accessed?
When?
What secret?
From where?

Critical for compliance.

6. Fine-Grained Access Control

Not everyone should access every secret.

Vault provides:

Policy-Based Access

Example:

Developer
     ↓
Read Dev Secrets

Production Secrets
     ✗ Denied

Main Vault Components

Vault Server

Core service responsible for:

Authentication
Authorization
Secret storage
Encryption

Storage Backend

Stores encrypted secrets.

Examples:

Integrated Storage (Raft)
Consul
AWS DynamoDB
PostgreSQL

Authentication Methods

Vault supports:

Userpass
LDAP
GitHub
Kubernetes
AWS IAM
Azure AD
OIDC

Example:

Developer
     ↓
GitHub Login
     ↓
Vault

Policies

Vault policies define access permissions.

Example:

path "secret/data/dev/*" {
  capabilities = ["read"]
}

Meaning:

Can read dev secrets only

What are Secrets Engines?

Secrets Engines are plugins that generate or store secrets.

Vault ships with many.

KV Secrets Engine

Most common.

Stores:

Username
Password
API Keys
Tokens

Example:

vault kv put secret/app \
username=admin \
password=secret123

Database Secrets Engine

Creates temporary database users.

Example:

Application
      ↓
Vault
      ↓
Temporary PostgreSQL User

Automatically expires later.

PKI Secrets Engine

Issues certificates dynamically.

Example:

Vault
      ↓
TLS Certificate

instead of manually creating certificates.

AWS Secrets Engine

Generates temporary AWS credentials.

Example:

Application
      ↓
Vault
      ↓
AWS IAM Credentials

Dynamic Secrets vs Static Secrets

Static Secret

password123

Exists forever.

Dynamic Secret

Generated
     ↓
Used
     ↓
Automatically Expired

Much safer.

Why Dynamic Secrets Are Important

Static credentials are often stolen.

Dynamic credentials reduce risk because:

Credential Expires
       ↓
Attack Window Reduced

Installing Vault in Development Environment

Development mode is useful for learning.

Run Vault Using Docker

docker run \
--cap-add=IPC_LOCK \
-e VAULT_DEV_ROOT_TOKEN_ID=root \
-p 8200:8200 \
hashicorp/vault

Access:

http://localhost:8200

Token: root

Verify Vault

vault status

Expected output:

Initialized: true
Sealed: false

Store First Secret

vault kv put secret/app \
username=admin \
password=password123

Retrieve:

vault kv get secret/app

Installing Vault in Kubernetes

Most production environments run Vault inside Kubernetes.

Add Helm Repository

helm repo add hashicorp \
https://helm.releases.hashicorp.com

Update Repository

helm repo update

Install Vault

helm install vault hashicorp/vault

Verify:

kubectl get pods

Enable UI

server:
  ui:
    enabled: true

Production Vault Architecture

Recommended architecture:

Load Balancer
       ↓
Vault Cluster
       ↓
Raft Storage

Multiple replicas:

Vault-1
Vault-2
Vault-3

for high availability.

Vault Auto-Unseal

Without Auto-Unseal:

Vault Restart
      ↓
Manual Unseal Required

Production clusters use:

AWS KMS
Azure Key Vault
GCP KMS

for automatic unsealing.

Vault + Kubernetes Integration

Vault can inject secrets directly into Pods.

Traditional:

env:
  DB_PASSWORD: password123

Vault:

Pod
     ↓
Vault Agent
     ↓
Secret Injection

No hardcoded secrets.

Vault Agent Injector

Automatically injects secrets into Pods.

Application Pod
      ↓
Vault Sidecar
      ↓
Secret Available

without storing secrets in Git.

Vault in CI/CD Pipelines

Modern CI/CD:

GitHub Actions
      ↓
Vault Authentication
      ↓
Temporary Secrets
      ↓
Deployment

Benefits:

No hardcoded credentials
Automatic rotation
Auditability

Vault Security Best Practices

Enable TLS

Never expose Vault without HTTPS.

Use Auto-Unseal

Avoid manual operations.

Use Least Privilege Policies

Grant minimum access.

Enable Audit Logs

Track every access.

Use Dynamic Secrets

Avoid static passwords.

Integrate with Identity Provider

Examples:

Azure AD
Okta
GitHub
LDAP

Common Use Cases

Kubernetes Secrets Management

Pods
 ↓
Vault
 ↓
Secrets

Database Credentials

Application
 ↓
Vault
 ↓
Temporary PostgreSQL User

Cloud Credentials

Application
 ↓
Vault
 ↓
AWS IAM Credentials

PKI Certificates

Vault
 ↓
Generate TLS Certificates

Enterprise Vault Architecture

Developers
       ↓
Applications
       ↓
Vault Cluster
       ↓
Policies
       ↓
Secrets Engines
       ↓
Database / Cloud / Certificates

Final Thoughts

Modern infrastructure depends on secrets.

As organizations adopt:

Kubernetes
Multi-cloud
GitOps
Platform Engineering
DevSecOps

traditional secret management approaches are no longer sufficient.

HashiCorp Vault solves this problem by providing:

Centralized Storage
Dynamic Secrets
Secret Rotation
Audit Logging
Encryption
Fine-Grained Access Control

For small AWS-only workloads, AWS Secrets Manager may be enough.

For Azure-only environments, Azure Key Vault works well.

But for organizations needing:

Multi-Cloud
Kubernetes
Hybrid Cloud
Advanced Security

HashiCorp Vault remains one of the most powerful and widely adopted secrets management platforms available today.

Day 25 - Helm Chart

Rahul Joshi — Fri, 05 Jun 2026 04:23:16 +0000

Kubernetes is powerful, but managing Kubernetes YAML files manually becomes difficult very quickly.

A simple application may need:

Deployment
Service
ConfigMap
Secret
Ingress
HPA
ServiceAccount
RBAC
NetworkPolicy

If you write everything manually, your project can easily become messy.

This is where Helm comes in.

Helm is often called:

The package manager for Kubernetes

Just like:

apt installs packages on Ubuntu
yum installs packages on RHEL
npm installs packages for Node.js
Helm installs applications on Kubernetes

🔗 Resources

** Support the Journey on GitHub: If you're following along, consider starring and forking the repo:** https://github.com/17J/30-Days-Cloud-DevSecOps-Journey

What is Helm?

Helm is a Kubernetes package manager that helps you define, install, upgrade, and manage Kubernetes applications.

Instead of applying many YAML files one by one:

kubectl apply -f deployment.yaml
kubectl apply -f service.yaml
kubectl apply -f configmap.yaml
kubectl apply -f ingress.yaml

You can install everything using one Helm command:

helm install my-app ./my-chart

Helm packages Kubernetes manifests into a reusable unit called a Helm Chart.

What is a Helm Chart?

A Helm Chart is a collection of files that describe a Kubernetes application.

A chart contains:

Kubernetes templates
Default configuration values
Metadata
Helper templates
Dependency definitions

Think of a Helm Chart as:

Reusable Kubernetes application package

Example:

nginx-chart
 ├── Deployment
 ├── Service
 ├── ConfigMap
 ├── Ingress
 └── Values

Why Do We Need Helm?

Without Helm, Kubernetes YAML becomes repetitive.

Example problem:

You have three environments:

dev
qa
prod

Each environment needs the same application but different values.

Dev:

replicas: 1
image: app:dev

Prod:

replicas: 5
image: app:v1.0.0

Without Helm, you may create separate YAML files for every environment.

With Helm, you create one reusable chart and change only values.

Helm Solves These Problems

1. Reusability

One chart can be reused across environments.

Same Chart
   ↓
dev-values.yaml
qa-values.yaml
prod-values.yaml

2. Easy Configuration

You can customize deployments using values.yaml.

3. Version Control

Helm charts can be versioned.

Example:

my-app-chart-1.0.0
my-app-chart-1.1.0
my-app-chart-2.0.0

4. Easy Upgrade and Rollback

Upgrade:

helm upgrade my-app ./my-chart

Rollback:

helm rollback my-app 1

5. Better Release Management

Helm tracks every release.

helm list

When you run:

helm install

Helm renders templates into Kubernetes YAML and sends them to the Kubernetes API server.

Helm Chart File Structure

When you create a chart:

helm create my-app

Helm creates this structure:

my-app/
├── Chart.yaml
├── values.yaml
├── charts/
├── templates/
│   ├── deployment.yaml
│   ├── service.yaml
│   ├── ingress.yaml
│   ├── serviceaccount.yaml
│   ├── hpa.yaml
│   ├── _helpers.tpl
│   └── NOTES.txt
└── .helmignore

Now let’s understand every important file.

Chart.yaml

Chart.yaml contains metadata about the chart.

Example:

apiVersion: v2
name: my-app
description: A Helm chart for deploying my application
type: application
version: 1.0.0
appVersion: "1.0.0"

Explanation:

apiVersion: Helm chart API version
name: Chart name
description: Short chart description
type: application or library
version: Chart version
appVersion: Application version

Important difference:

version     = Helm chart version
appVersion  = Application version

values.yaml

values.yaml stores default values for the chart.

Example:

replicaCount: 2

image:
  repository: nginx
  tag: "1.25"
  pullPolicy: IfNotPresent

service:
  type: ClusterIP
  port: 80

resources:
  limits:
    cpu: 500m
    memory: 512Mi
  requests:
    cpu: 100m
    memory: 128Mi

This file makes your chart configurable.

Instead of hardcoding values inside Kubernetes YAML, Helm reads them from values.yaml.

templates/ Directory

The templates/ folder contains Kubernetes YAML templates.

These are not normal YAML files.

They contain template expressions like:

{{ .Values.replicaCount }}

Helm replaces these values during rendering.

Deployment Template Example

apiVersion: apps/v1
kind: Deployment

metadata:
  name: {{ .Release.Name }}-app

spec:
  replicas: {{ .Values.replicaCount }}

  selector:
    matchLabels:
      app: {{ .Chart.Name }}

  template:
    metadata:
      labels:
        app: {{ .Chart.Name }}

    spec:
      containers:
        - name: {{ .Chart.Name }}
          image: "{{ .Values.image.repository }}:{{ .Values.image.tag }}"
          ports:
            - containerPort: 80

If values.yaml contains:

replicaCount: 3

image:
  repository: nginx
  tag: "1.25"

Helm renders:

replicas: 3
image: nginx:1.25

Service Template Example

apiVersion: v1
kind: Service

metadata:
  name: {{ .Release.Name }}-service

spec:
  type: {{ .Values.service.type }}

  ports:
    - port: {{ .Values.service.port }}
      targetPort: 80

  selector:
    app: {{ .Chart.Name }}

This means the same chart can create different service types:

service:
  type: ClusterIP

or:

service:
  type: LoadBalancer

What is Helm Templating?

Helm templating allows you to dynamically generate Kubernetes YAML.

Example:

replicas: {{ .Values.replicaCount }}

This means:

Take replicaCount value from values.yaml

Helm templates are powered by Go templating.

Common Helm Template Objects

.Values

Reads values from values.yaml.

{{ .Values.image.repository }}

.Release

Provides release information.

{{ .Release.Name }}

Example:

helm install dev-app ./my-chart

Then:

{{ .Release.Name }}

becomes:

dev-app

.Chart

Provides chart metadata.

{{ .Chart.Name }}

.Namespace

Returns release namespace.

{{ .Release.Namespace }}

_helpers.tpl

_helpers.tpl stores reusable template snippets.

Example:

{{- define "my-app.fullname" -}}
{{ .Release.Name }}-{{ .Chart.Name }}
{{- end }}

Use it like:

name: {{ include "my-app.fullname" . }}

This keeps templates clean and avoids duplication.

NOTES.txt

NOTES.txt displays information after chart installation.

Example:

Application installed successfully.

Access your app using:

kubectl port-forward svc/{{ .Release.Name }}-service 8080:80

After install, Helm prints this message.

.helmignore

.helmignore works like .gitignore.

It tells Helm which files to ignore while packaging the chart.

Example:

.git/
README.md
*.log

Helm Install Command

Install a chart:

helm install my-app ./my-app

Check release:

helm list

Check Kubernetes resources:

kubectl get all

Helm Upgrade Command

If you update image tag or replicas:

helm upgrade my-app ./my-app

Example:

helm upgrade my-app ./my-app \
  --set image.tag=1.26

Helm Rollback Command

Check release history:

helm history my-app

Rollback:

helm rollback my-app 1

This is one of Helm’s most useful features.

Helm Uninstall Command

Remove release:

helm uninstall my-app

This deletes all Kubernetes resources created by that Helm release.

Using Multiple Values Files

You can create different values files:

values-dev.yaml
values-qa.yaml
values-prod.yaml

Example dev:

replicaCount: 1
image:
  tag: dev
service:
  type: ClusterIP

Example prod:

replicaCount: 5
image:
  tag: v1.0.0
service:
  type: LoadBalancer

Install dev:

helm install my-app-dev ./my-app -f values-dev.yaml

Install prod:

helm install my-app-prod ./my-app -f values-prod.yaml

Same chart, different environments.

Helm Dry Run

Before installing, test the chart:

helm install my-app ./my-app --dry-run --debug

This shows rendered YAML without applying it.

Useful for debugging.

Helm Template Command

Render YAML locally:

helm template my-app ./my-app

This is useful when you want to inspect final Kubernetes manifests.

Helm Lint

Check chart issues:

helm lint ./my-app

It validates chart structure and common problems.

Popular chart repositories:

ArtifactHub
JFrog Artifactory
Sonatype Nexus
GitHub Pages
OCI Registry

Packaging a Helm Chart

Package chart:

helm package my-app

Output:

my-app-1.0.0.tgz

This package can be uploaded to a chart repository.

Installing Public Helm Charts

Add repo:

helm repo add bitnami https://charts.bitnami.com/bitnami

Update repo:

helm repo update

Install chart:

helm install my-nginx bitnami/nginx

Helm for Kubernetes Deployments

Kubernetes deployment without Helm:

deployment.yaml
service.yaml
ingress.yaml
configmap.yaml
secret.yaml

Kubernetes deployment with Helm:

helm install my-app ./chart

Helm makes Kubernetes deployment cleaner, reusable, and easier to manage.

Complete Example: NGINX Helm Chart

Create chart:

helm create nginx-chart

Update values.yaml:

replicaCount: 2

image:
  repository: nginx
  tag: "1.25"

service:
  type: ClusterIP
  port: 80

Update templates/deployment.yaml:

apiVersion: apps/v1
kind: Deployment

metadata:
  name: {{ .Release.Name }}-nginx

spec:
  replicas: {{ .Values.replicaCount }}

  selector:
    matchLabels:
      app: nginx

  template:
    metadata:
      labels:
        app: nginx

    spec:
      containers:
        - name: nginx
          image: "{{ .Values.image.repository }}:{{ .Values.image.tag }}"
          ports:
            - containerPort: 80

Update templates/service.yaml:

apiVersion: v1
kind: Service

metadata:
  name: {{ .Release.Name }}-service

spec:
  type: {{ .Values.service.type }}

  selector:
    app: nginx

  ports:
    - port: {{ .Values.service.port }}
      targetPort: 80

Install:

helm install nginx-demo ./nginx-chart

Verify:

kubectl get pods
kubectl get svc

Reusable Deployments with Helm

Helm helps platform teams create reusable templates.

Example:

Frontend Team
Backend Team
Payment Team
Order Team

All can use the same base chart.

Only values change:

image.repository
image.tag
replicaCount
resources
env
ingress.host

This creates standardization across teams.

Helm in CI/CD Pipeline

Modern CI/CD pipeline:

Developer Commit
      ↓
Build Docker Image
      ↓
Push Image to Registry
      ↓
Update Helm values
      ↓
helm upgrade
      ↓
Kubernetes Deployment

Example:

helm upgrade my-app ./chart \
  --set image.tag=$GITHUB_SHA

Helm with GitOps

In modern Kubernetes, Helm is often used with GitOps tools.

Example:

Git Repository
      ↓
Helm Chart
      ↓
ArgoCD / Flux
      ↓
Kubernetes Cluster

ArgoCD can deploy Helm charts directly.

Helm Best Practices

1. Keep Charts Reusable

Avoid hardcoding environment-specific values.

Bad:

replicas: 5

Good:

replicas: {{ .Values.replicaCount }}

2. Use values.yaml Properly

Keep all configurable values in values.yaml.

3. Use Separate Values Files

Use:

values-dev.yaml
values-prod.yaml

4. Run helm lint

Always validate charts.

helm lint ./chart

5. Use Versioning

Always version charts properly.

version: 1.0.0
appVersion: "1.0.0"

6. Avoid Storing Secrets Directly

Do not store plain secrets in values files.

Use:

External Secrets Operator
Sealed Secrets
HashiCorp Vault
AWS Secrets Manager
Azure Key Vault

Common Helm Commands

helm create my-chart
helm install my-app ./my-chart
helm upgrade my-app ./my-chart
helm rollback my-app 1
helm uninstall my-app
helm list
helm history my-app
helm template my-app ./my-chart
helm lint ./my-chart
helm package my-chart

Final Thoughts

Helm is one of the most important tools in the Kubernetes ecosystem.

It solves a very real problem:

Too many YAML files
Too much repetition
Too much manual configuration

With Helm, you get:

Reusable deployments
Environment-based configuration
Template-based Kubernetes manifests
Easy upgrades
Easy rollbacks
Better release management

If Kubernetes is the platform for running containers, Helm is the tool that makes Kubernetes applications easier to package, reuse, and operate.

For DevOps Engineers, Cloud Engineers, SREs, and Platform Engineers, Helm is not optional anymore.

It is a must-have Kubernetes skill.

Day 24 - Kubernetes Fundamentals and Security

Rahul Joshi — Thu, 04 Jun 2026 03:19:03 +0000

In Current Time applications are no longer deployed as a single server.

Today organizations run:

Microservices
Containers
Kubernetes
Service Mesh
GitOps
Cloud Native Platforms

At the center of this transformation is Kubernetes.

Kubernetes has become the de-facto standard for container orchestration.

According to CNCF surveys, Kubernetes adoption continues to grow rapidly across enterprises, startups, cloud providers, and platform engineering teams.

But Kubernetes is much more than deploying containers.

A production Kubernetes engineer must understand:

Pods
Services
ConfigMaps
Secrets
Deployments
RBAC
Network Policies
Pod Security
Runtime Security
Policy Enforcement

🔗 Resources

** Support the Journey on GitHub: If you're following along, consider starring and forking the repo:** https://github.com/17J/30-Days-Cloud-DevSecOps-Journey

What is Kubernetes?

Kubernetes (K8s) is an open-source container orchestration platform originally developed by Google and now maintained by the CNCF.

Its purpose is to automate:

Container Deployment
        ↓
Scaling
        ↓
Networking
        ↓
Self-Healing
        ↓
Availability

Instead of manually managing containers:

Docker Container
Docker Container
Docker Container

Kubernetes manages them automatically.

Why Kubernetes?

Before Kubernetes:

Application
      ↓
Virtual Machine
      ↓
Manual Scaling
      ↓
Manual Recovery

Problems:

Downtime
Scaling issues
Operational complexity
Resource wastage

Kubernetes solves these problems.

Benefits:

Self-healing
Auto-scaling
Service discovery
Rolling updates
Multi-cloud portability
High availability

Control Plane manages the cluster.

Worker Nodes run workloads.

Understanding Pods

Pods are the smallest deployable unit in Kubernetes.

Think of a Pod as:

Container Wrapper

Example:

Pod
 └─ NGINX Container

A Pod can also contain multiple containers.

Example:

Pod
 ├─ Application Container
 └─ Logging Sidecar

NGINX Pod Example

apiVersion: v1
kind: Pod

metadata:
  name: nginx-pod

spec:
  containers:
  - name: nginx
    image: nginx:latest

Create Pod:

kubectl apply -f pod.yaml

Why Pods Matter

Pods provide:

Shared network
Shared storage
Shared lifecycle

Containers inside a Pod communicate using:

localhost

What is a Deployment?

Managing Pods directly is not recommended.

Instead use Deployments.

Deployment provides:

Scaling
Rollbacks
Rolling Updates
Self-healing

Deployment Example

apiVersion: apps/v1
kind: Deployment

metadata:
  name: nginx

spec:
  replicas: 3

  selector:
    matchLabels:
      app: nginx

  template:
    metadata:
      labels:
        app: nginx

    spec:
      containers:
      - name: nginx
        image: nginx:latest

Deployment Benefits

Pod Crashes
      ↓
Deployment Detects Failure
      ↓
New Pod Created

This is Kubernetes self-healing.

What is a Service?

Pods are ephemeral.

Their IP addresses change.

Service provides a stable endpoint.

Without Service:

Client
   ↓
Pod IP

Pod restart breaks communication.

With Service:

Client
   ↓
Service
   ↓
Pods

Applications always connect to Service.

Service Types

ClusterIP

Default internal service.

Inside Cluster Only

NodePort

Exposes service through node port.

NodeIP:30080

LoadBalancer

Creates cloud load balancer.

AWS ELB
Azure LB
GCP LB

ExternalName

Maps service to external DNS.

Service Example

apiVersion: v1
kind: Service

metadata:
  name: nginx-service

spec:
  selector:
    app: nginx

  ports:
  - port: 80
    targetPort: 80

  type: ClusterIP

What is ConfigMap?

Applications need configuration.

Examples:

Database Host
API URL
Feature Flags

Hardcoding these values is bad practice.

ConfigMaps store non-sensitive configuration.

ConfigMap Example

apiVersion: v1
kind: ConfigMap

metadata:
  name: app-config

data:
  APP_ENV: production
  LOG_LEVEL: info

Use inside Pod:

envFrom:
- configMapRef:
    name: app-config

What are Kubernetes Secrets?

Secrets store sensitive information.

Examples:

Database Password
API Keys
Tokens
Certificates

Unlike ConfigMaps, Secrets are intended for sensitive data.

Secret Example

apiVersion: v1
kind: Secret

metadata:
  name: db-secret

type: Opaque

stringData:
  username: admin
  password: Password123

Important Security Warning

Secrets are Base64 encoded.

They are NOT automatically encrypted.

Bad assumption:

Base64 = Encryption

Wrong.

Production clusters should enable:

Encryption at Rest

for Secrets.

Kubernetes Security Fundamentals

Many organizations secure:

Cloud
Network
Applications

but forget Kubernetes security.

A compromised cluster can expose:

Customer data
Internal services
Cloud credentials
Secrets

Kubernetes security must be built in from the beginning.

What is RBAC?

RBAC stands for:

Role-Based Access Control

RBAC controls:

Who
Can Do What
Inside Kubernetes

RBAC Components

Role

Defines permissions.

Example:

Read Pods
Read Services

RoleBinding

Assigns Role to User.

User
   ↓
RoleBinding
   ↓
Role

RBAC Example

apiVersion: rbac.authorization.k8s.io/v1
kind: Role

metadata:
  name: pod-reader

rules:
- apiGroups: [""]
  resources: ["pods"]
  verbs: ["get","list","watch"]

Why RBAC Matters

Without RBAC:

Developer
     ↓
Cluster Admin

Huge security risk.

Use least privilege.

What are Network Policies?

By default:

Pod A
      ↓
Pod B

Communication is often allowed.

This violates Zero Trust principles.

Network Policy Purpose

Control:

Pod-to-Pod Traffic
Namespace Traffic
Ingress
Egress

Example Policy

Allow traffic only from frontend Pods.

kind: NetworkPolicy

Result:

Frontend
    ↓
Backend

Other Pods
    ✗ Blocked

Pod Security

Pods themselves must be secured.

Bad Pod:

privileged: true
runAsUser: 0

This means:

Root Access

inside container.

Huge risk.

Secure Pod Example

securityContext:

  runAsNonRoot: true

  allowPrivilegeEscalation: false

  readOnlyRootFilesystem: true

Benefits:

Reduced attack surface
Better compliance
Least privilege

Secrets Management Best Practices

Never:

Store Passwords in Git
Hardcode API Keys
Commit Secrets to Repository

Use:

Kubernetes Secrets
External Secrets Operator
HashiCorp Vault
AWS Secrets Manager
Azure Key Vault

Runtime Security with Falco

Kubernetes security doesn't stop at deployment.

You must monitor runtime behavior.

This is where Falco comes in.

What is Falco?

Falco is a CNCF runtime security tool.

It detects suspicious behavior inside Kubernetes.

Example:

Container Starts Shell

Falco Alert:

Unexpected Shell Execution

Falco Architecture

Container Activity
        ↓
Falco Rules Engine
        ↓
Alert Generated

Falco Detection Examples

Detect:

Reverse shell
Privilege escalation
Crypto miners
Suspicious processes
Unexpected file access
Sensitive mounts

Example Falco Alert

Terminal shell in container

This could indicate compromise.

Policy Enforcement with Kyverno

Security should be automated.

Developers make mistakes.

Kyverno prevents insecure workloads from being deployed.

What is Kyverno?

Kyverno is a Kubernetes-native policy engine.

It validates Kubernetes resources before deployment.

Example Use Case

Block privileged containers.

Bad Deployment:

privileged: true

Kyverno:

Deployment Rejected

Kyverno Benefits

Enforce:

Non-root containers
Resource limits
Approved registries
Label requirements
Security standards

Automatically.

Example Security Policy

Require:

runAsNonRoot=true

Any Pod violating policy:

Rejected

before reaching production.

Production Kubernetes Security Checklist

Identity Security

RBAC
Least Privilege
Service Accounts

Network Security

Network Policies
Service Mesh
mTLS

Secrets Security

Vault
Secrets Manager
Encryption at Rest

Pod Security

Non-root containers
Read-only filesystems
Drop Linux capabilities

Runtime Security

Falco
Monitoring
Audit Logs

Policy Security

Kyverno
OPA Gatekeeper

Modern Secure Kubernetes Architecture

Developer
      ↓
Git Repository
      ↓
CI Pipeline
      ↓
Container Scan
      ↓
Kyverno Validation
      ↓
Kubernetes Cluster
      ↓
Network Policies
      ↓
RBAC
      ↓
Falco Runtime Monitoring
      ↓
Alerts

Final Thoughts

Learning Kubernetes means more than learning Pods and Deployments.

A production Kubernetes engineer must understand both:

Workload Management
        +
Security

Core Fundamentals:

Pods
Deployments
Services
ConfigMaps
Secrets

Core Security:

RBAC
Network Policies
Pod Security
Secrets Management
Falco
Kyverno

Organizations that treat Kubernetes security as an afterthought often face misconfigurations, exposed workloads, and compliance failures.

The most successful Kubernetes platforms combine:

Automation
Security
Observability
Governance

Day 23 - Github Actions CI/CD Pipeline

Rahul Joshi — Wed, 03 Jun 2026 03:59:55 +0000

In Present Time software teams need fast, secure, and automated delivery.

Earlier, release flow looked like this:

Developer writes code
        ↓
Manual build
        ↓
Manual test
        ↓
Manual deployment
        ↓
Production issue

Today, GitHub Actions can automate this entire process directly from your GitHub repository.

🔗 Resources

** Support the Journey on GitHub: If you're following along, consider starring and forking the repo:** https://github.com/17J/30-Days-Cloud-DevSecOps-Journey

What is GitHub Actions?

GitHub Actions is GitHub’s automation platform for building, testing, scanning, packaging, and deploying applications.

You define automation using YAML files inside:

.github/workflows/

Example:

.github/workflows/ci-cd.yml

A workflow can run when:

Code is pushed
Pull request is opened
Tag is created
Manual trigger is clicked
Schedule runs
External webhook/event triggers it

GitHub supports workflow triggers for repository activity, schedules, and external events.

Why GitHub Actions?

GitHub Actions is powerful because it is close to the source code.

Benefits:

Code + CI/CD + Security + Packages + Deployments

inside one platform.

It helps teams:

Build automatically
Test every pull request
Run security scans
Build Docker images
Push artifacts
Deploy to cloud
Deploy to Kubernetes
Use approval gates
Use OIDC instead of long-lived cloud keys

GitHub Actions Core Concepts

Workflow

A workflow is the complete automation file.

Example:

name: CI/CD Pipeline

It contains triggers, jobs, permissions, and steps.

Event

An event starts the workflow.

Example:

on:
  push:
    branches: [ main ]
  pull_request:

This means the workflow runs on push to main and pull requests.

Job

A job is a group of steps.

Example:

jobs:
  build:
    runs-on: ubuntu-latest

Step

A step is a single command or action.

Example:

steps:
  - name: Checkout Code
    uses: actions/checkout@v4

Action

An action is a reusable task.

Example:

uses: actions/setup-node@v4

Actions help you reuse community or official automation components.

Runner

A runner is the machine that executes your job.

There are two main types:

GitHub-hosted runner
Self-hosted runner

GitHub-hosted runners are managed by GitHub, while self-hosted runners are machines you manage yourself.

What are Private / Self-Hosted Runners?

A self-hosted runner is a machine deployed and managed by you to execute GitHub Actions jobs.

It can run on:

EC2
Azure VM
GCP VM
Kubernetes
On-prem server
Private subnet

Use self-hosted runners when your pipeline needs access to:

Private Kubernetes Cluster
Private Database
Internal Nexus
Private SonarQube
Internal APIs
Private VPC Resources

Self-Hosted Runner Labels

When added, self-hosted runners automatically receive labels like:

self-hosted
linux
windows
macOS
x64
ARM
ARM64

GitHub uses these labels to route jobs.

Example:

runs-on: [self-hosted, linux, x64]

This job will run on a private Linux x64 runner.

Private Runner Example

jobs:
  deploy:
    runs-on: [self-hosted, linux, x64]

    steps:
      - name: Checkout
        uses: actions/checkout@v4

      - name: Deploy to Private Kubernetes
        run: |
          kubectl get nodes
          kubectl apply -f k8s/

This is useful when your Kubernetes API is not public.

Self-Hosted Runner Security Best Practices

Use private runners carefully.

Best practices:

Use ephemeral runners
Restrict repository access
Use runner groups
Avoid running untrusted fork PRs
Use least privilege
Patch runners regularly
Do not store secrets on runner disk

For Kubernetes-based autoscaling runners, GitHub identifies Actions Runner Controller as the recommended Kubernetes solution for autoscaling self-hosted runners.

What is Pipeline YAML?

GitHub Actions pipeline is written in YAML.

Basic structure:

name: Pipeline Name

on:
  push:

jobs:
  job-name:
    runs-on: ubuntu-latest

    steps:
      - name: Step Name
        run: echo "Hello CI/CD"

Important YAML Sections

name

name: Node.js CI/CD

Pipeline display name.

on

on:
  push:
    branches: [ main ]

Defines when pipeline runs.

permissions

permissions:
  contents: read
  id-token: write

Defines workflow token permissions.

id-token: write is required for OIDC-based cloud authentication.

env

env:
  APP_NAME: my-app

Defines environment variables.

jobs

jobs:
  build:

Defines pipeline jobs.

runs-on

runs-on: ubuntu-latest

Defines runner machine.

steps

steps:
  - run: npm install

Commands or reusable actions.

Secrets in GitHub Actions

Secrets are encrypted sensitive values.

Examples:

AWS_ACCOUNT_ID
SONAR_TOKEN
DOCKERHUB_TOKEN
SLACK_WEBHOOK
DATABASE_PASSWORD

Access secrets like this:

env:
  SONAR_TOKEN: ${{ secrets.SONAR_TOKEN }}

Never hardcode secrets in YAML.

Variables in GitHub Actions

Variables are non-sensitive configuration values.

Examples:

AWS_REGION=ap-south-1
APP_NAME=todo-api
ENVIRONMENT=dev

GitHub supports variables and exposes them through the vars context.

Example:

env:
  AWS_REGION: ${{ vars.AWS_REGION }}

What is GITHUB_TOKEN?

GITHUB_TOKEN is an automatically generated token available in workflows.

It can be used for GitHub API operations like:

Checkout
Comment on PR
Create releases
Push tags
Update repo content

GitHub provides documentation explaining how GITHUB_TOKEN works for secure automation.

Example:

permissions:
  contents: read
  packages: write

What is OIDC in GitHub Actions?

OIDC means OpenID Connect.

It allows GitHub Actions to authenticate with cloud providers without storing long-lived access keys.

Old approach:

Store AWS_ACCESS_KEY_ID
Store AWS_SECRET_ACCESS_KEY

Better approach:

GitHub Actions
      ↓
OIDC Token
      ↓
AWS IAM Role
      ↓
Temporary Credentials

Benefits:

No long-lived cloud keys
Short-lived credentials
Better security
Easier rotation
Least privilege

AWS OIDC Example

permissions:
  id-token: write
  contents: read

- name: Configure AWS Credentials using OIDC
  uses: aws-actions/configure-aws-credentials@v4
  with:
    role-to-assume: arn:aws:iam::123456789012:role/github-actions-deploy-role
    aws-region: ap-south-1

What is a Webhook?

A webhook is an event notification sent from GitHub to another system.

Example:

GitHub Push Event
        ↓
Webhook
        ↓
External System

Use cases:

Trigger Jenkins pipeline
Notify Slack
Trigger deployment platform
Send events to security tools

Branch Rules and Rulesets

Rules protect important branches.

Example:

main branch

should not allow direct push.

Common rules:

Require pull request
Require approvals
Require status checks
Require signed commits
Restrict force pushes
Restrict deletions
Require linear history

Why Rulesets Matter

Rulesets enforce governance.

Example:

Developer opens PR
        ↓
CI pipeline runs
        ↓
Tests pass
        ↓
Security scan passes
        ↓
Approval received
        ↓
Merge allowed

Without rulesets, someone may directly push insecure code to production branch.

Environment Protection Rules

GitHub Actions can control deployments using environments, concurrency groups, and protection rules.

Example:

environment:
  name: production

You can configure:

Required reviewers
Wait timer
Deployment branches
Environment secrets

Environment secrets and protection rules are available depending on repository type and plan.

Full GitHub Actions CI/CD Pipeline Example

This example does:

Checkout
Install dependencies
Run tests
Run SAST
Build Docker image
Push to Amazon ECR
Deploy to Kubernetes

name: GitHub Actions CI/CD Pipeline

on:
  push:
    branches: [ main ]
  pull_request:
    branches: [ main ]

permissions:
  contents: read
  id-token: write
  packages: write

env:
  AWS_REGION: ap-south-1
  ECR_REPOSITORY: my-node-app
  IMAGE_TAG: ${{ github.sha }}

jobs:
  ci:
    name: Build, Test and Scan
    runs-on: ubuntu-latest

    steps:
      - name: Checkout Code
        uses: actions/checkout@v4

      - name: Setup Node.js
        uses: actions/setup-node@v4
        with:
          node-version: 20

      - name: Install Dependencies
        run: npm ci

      - name: Run Unit Tests
        run: npm test

      - name: Run Semgrep SAST
        uses: semgrep/semgrep-action@v1
        with:
          config: auto

      - name: Build Docker Image
        run: |
          docker build -t $ECR_REPOSITORY:$IMAGE_TAG .

  deploy:
    name: Build Image and Deploy
    needs: ci
    runs-on: [self-hosted, linux, x64]
    if: github.ref == 'refs/heads/main'

    environment:
      name: production

    steps:
      - name: Checkout Code
        uses: actions/checkout@v4

      - name: Configure AWS Credentials using OIDC
        uses: aws-actions/configure-aws-credentials@v4
        with:
          role-to-assume: arn:aws:iam::123456789012:role/github-actions-deploy-role
          aws-region: ${{ env.AWS_REGION }}

      - name: Login to Amazon ECR
        run: |
          aws ecr get-login-password --region $AWS_REGION | \
          docker login --username AWS --password-stdin \
          123456789012.dkr.ecr.$AWS_REGION.amazonaws.com

      - name: Build and Push Docker Image
        run: |
          IMAGE_URI=123456789012.dkr.ecr.$AWS_REGION.amazonaws.com/$ECR_REPOSITORY:$IMAGE_TAG
          docker build -t $IMAGE_URI .
          docker push $IMAGE_URI
          echo "IMAGE_URI=$IMAGE_URI" >> $GITHUB_ENV

      - name: Deploy to Kubernetes
        run: |
          kubectl set image deployment/my-node-app \
            my-node-app=$IMAGE_URI \
            -n production

          kubectl rollout status deployment/my-node-app -n production

Pipeline Flow Explained

Developer Pushes Code
        ↓
GitHub Actions Triggered
        ↓
CI Job Runs on GitHub Runner
        ↓
Tests + SAST
        ↓
Deploy Job Runs on Private Runner
        ↓
OIDC Authenticates to AWS
        ↓
Docker Image Pushed to ECR
        ↓
Kubernetes Deployment Updated

Build Automation

Build automation means converting source code into a deployable artifact.

Examples:

Java → JAR/WAR
Node.js → Bundle
Dockerfile → Docker Image
Helm Chart → Versioned Package

Example:

- name: Build Docker Image
  run: docker build -t my-app:${{ github.sha }} .

Deploy Automation

Deploy automation means moving the artifact to the target environment.

Examples:

ECR → EKS
ACR → AKS
Docker Hub → Kubernetes
S3 → CloudFront
Lambda ZIP → AWS Lambda

Example:

- name: Deploy to Kubernetes
  run: kubectl apply -f k8s/

GitOps Deployment Alternative

In modern Kubernetes setups, GitHub Actions should often do only CI.

CD should be handled by ArgoCD or Flux.

Flow:

GitHub Actions
      ↓
Build Image
      ↓
Push to Registry
      ↓
Update GitOps Repo
      ↓
ArgoCD / Flux Deploys

This avoids giving CI pipeline direct cluster-admin deployment access.

GitOps Example Step

- name: Update GitOps Manifest
  run: |
    git config user.name "github-actions"
    git config user.email "actions@github.com"

    sed -i "s|image: .*|image: $IMAGE_URI|g" k8s/deployment.yaml

    git add k8s/deployment.yaml
    git commit -m "Update image to $IMAGE_TAG"
    git push

Then ArgoCD or Flux detects the manifest change and deploys it.

Recommended Pre Production Pipeline

GitHub Actions Best Practices

Use:

OIDC instead of access keys
Environment approvals for production
Branch rulesets
Private runners for private infra
Least privilege permissions
Pinned action versions
Secrets only for sensitive values
Variables for non-sensitive config
Concurrency control
Artifact retention policies

Final Thoughts

GitHub Actions is more than a CI/CD tool.

It is an automation platform tightly integrated with GitHub.

It can handle:

CI pipelines
Security scanning
Docker builds
Cloud authentication
Kubernetes deployment
Release automation
GitOps workflows

For modern DevOps and DevSecOps teams, GitHub Actions becomes even more powerful when combined with:

Private runners
OIDC
Rulesets
Environment approvals
ArgoCD / Flux
Security scanning

A strong production pipeline is not only about deploying fast.

It is about deploying:

Fast
Securely
Repeatably
With control

Day 22 - Artifact Repository Management

Rahul Joshi — Tue, 02 Jun 2026 04:24:38 +0000

In Present Time software development produces far more than just source code.

Every build generates artifacts such as:

JAR files
WAR files
NPM packages
Python packages
Docker images
Helm charts
NuGet packages
Maven dependencies

Without proper management, these artifacts become difficult to track, secure, and distribute.

This is where Artifact Repository Management becomes critical.

What is an Artifact Repository?

An Artifact Repository is a centralized storage system that stores, manages, versions, and distributes software build artifacts.

Think of it as:

Git stores source code
        ↓
Artifact Repository stores build outputs

Example:

Source Code
      ↓
CI Build
      ↓
app-1.0.jar
      ↓
Artifact Repository
      ↓
Deployment

Instead of rebuilding software every time, teams store generated artifacts and reuse them.

What is a Software Artifact?

An artifact is any file generated during the software build process.

Examples:

Artifact Type	Example
Maven Package	app-1.0.jar
Java WAR	app.war
Docker Image	myapp:v1
Helm Chart	app-chart-1.0.0
NPM Package	package.tgz
Python Package	wheel (.whl)
NuGet Package	.nupkg

Why Artifact Repositories Matter Today

Modern applications use:

Microservices
Containers
Kubernetes
CI/CD Pipelines
GitOps
Multi-cloud deployments

Organizations may build:

100 Developers
       ↓
500 Commits Daily
       ↓
Thousands of Build Artifacts

Managing these manually becomes impossible.

Problems Without Artifact Repositories

Without a repository:

Developer Machine
      ↓
Local Build
      ↓
Manual Sharing

Problems:

No version control
Lost packages
Security risks
Inconsistent deployments
No audit trail

Benefits of Artifact Repositories

Centralized Storage

All artifacts stored in one location.

Developers
      ↓
Repository
      ↓
CI/CD

Version Control

Store multiple versions.

Example:

app-1.0.jar
app-1.1.jar
app-1.2.jar

Security

Provides:

Authentication
Authorization
Package scanning
Audit logging

Faster Builds

Instead of downloading dependencies repeatedly:

Internet
     ↓
Repository Cache

Builds become faster.

Supply Chain Security

Modern repositories help secure:

Dependencies
Containers
Packages

against supply chain attacks.

Where Artifact Repositories Fit in CI/CD

Developer Commit
        ↓
CI Pipeline
        ↓
Build Application
        ↓
Create Artifact
        ↓
Artifact Repository
        ↓
Deployment

The repository becomes the source of truth for deployable software.

Popular Artifact Repository Platforms

1. Sonatype Nexus Repository

One of the most widely used artifact repositories.

Supports:

Maven
Docker
Helm
NPM
NuGet
PyPI
Yum
Raw artifacts

Architecture:

Developers
      ↓
Nexus
      ↓
Package Storage

Why Nexus is Popular

Benefits:

Free Community Edition
Enterprise Edition
Easy setup
Strong Maven support
Docker registry support

Popular in:

DevOps
Enterprise Java environments
Kubernetes platforms

2. JFrog Artifactory

Enterprise-grade repository management platform.

Supports:

Maven
Docker
Helm
NPM
PyPI
OCI Artifacts

Architecture:

Build
     ↓
Artifactory
     ↓
Deploy

Strong enterprise features include:

Xray security scanning
Distribution
Federated repositories

3. AWS CodeArtifact

AWS-managed artifact repository.

Supports:

Maven
NPM
NuGet
Python

Benefits:

Fully managed
IAM integration
No infrastructure management

Architecture:

AWS Build
      ↓
CodeArtifact
      ↓
Deployments

4. GitHub Packages

Native package management within GitHub.

Supports:

Docker
Maven
NPM
NuGet

Best for teams already using GitHub.

5. GitLab Package Registry

Integrated into GitLab.

Supports:

Maven
NPM
Helm
Generic packages

Benefits:

Single Platform
Code + CI + Packages

Understanding Maven Repositories

Maven uses three repository types.

Local Repository

Stored on developer machine.

~/.m2/repository

Central Repository

Public repository.

Example:

repo.maven.apache.org

Enterprise Repository

Example:

Nexus
Artifactory

Used by organizations.

Maven Release Repository

Stores stable releases.

Example:

app-1.0.jar
app-1.1.jar
app-2.0.jar

Immutable.

Once released:

Never Changed

Maven Snapshot Repository

Stores development versions.

Example:

app-1.0-SNAPSHOT

Can change frequently.

Useful during development.

Snapshot Example

Developer updates code:

v1
 ↓
app-1.0-SNAPSHOT

New commit:

v2
 ↓
app-1.0-SNAPSHOT

Same version but newer build.

Snapshots help teams continuously test ongoing development.

Maven Project Example

pom.xml

<groupId>com.company</groupId>
<artifactId>employee-service</artifactId>
<version>1.0-SNAPSHOT</version>

Development build:

employee-service-1.0-SNAPSHOT.jar

Production Release Example

<version>1.0.0</version>

Artifact:

employee-service-1.0.0.jar

Published to Release Repository.

Installing Nexus in Development Environment

The easiest approach is Docker.

Run Nexus Container

docker run -d \
--name nexus \
-p 8081:8081 \
sonatype/nexus3

Verify:

docker ps

Access:

http://localhost:8081

Initial Login

Default username:

admin

Password stored inside container:

docker exec nexus cat /nexus-data/admin.password

Development Architecture

Developer
      ↓
Nexus Docker Container
      ↓
Local Storage

Perfect for learning and testing.

Nexus Repository Types to Create

Typical repositories:

maven-releases
maven-snapshots
docker-hosted
helm-hosted
npm-hosted

Nexus in Pre-Production Environment

For pre-production, Docker alone is not enough.

Recommended architecture:

Load Balancer
      ↓
Nexus
      ↓
Persistent Volume
      ↓
Database Storage

Kubernetes Deployment Example

Kubernetes
      ↓
Nexus Deployment
      ↓
Persistent Volume
      ↓
Ingress

Recommended Pre-Prod Components

Use:

Persistent Volumes
Backup strategy
TLS certificates
Ingress Controller
Monitoring

Example Kubernetes Storage

storageClassName: gp3

For AWS EKS.

Nexus Production Best Practices

Use Persistent Storage

Never store repository data inside ephemeral containers.

Enable HTTPS

Always secure repositories.

Backup Regularly

Protect:

Artifacts
Configurations
Metadata

Integrate with LDAP/SSO

Enterprise user management.

Restrict Anonymous Access

Avoid public exposure.

Artifact Repository in Modern GitOps

Modern deployment flow:

Artifacts become immutable deployment units.

Security Considerations

Artifact repositories are now part of the software supply chain.

Protect them carefully.

Use:

RBAC
TLS
Vulnerability Scanning
Audit Logging
Repository Policies

Why Artifact Repositories Are Critical in 2026

Modern organizations deploy software continuously.

Artifact repositories provide:

Versioning
Security
Traceability
Reproducibility
Compliance
Supply Chain Protection

Without them, reliable software delivery becomes extremely difficult.

Final Thoughts

Artifact Repository Management is a foundational component of modern DevOps and Platform Engineering.

As organizations adopt:

Kubernetes
Microservices
GitOps
Cloud-native architectures

artifact repositories become the backbone of software delivery.

Whether you choose:

Sonatype Nexus
JFrog Artifactory
AWS CodeArtifact
GitHub Packages
GitLab Package Registry

the goal remains the same:

Store Once
Version Properly
Deploy Reliably

Because in modern software engineering, source code alone is not enough—the artifact is what actually gets deployed.

Day 21 - CI/CD Fundamentals

Rahul Joshi — Mon, 01 Jun 2026 04:21:22 +0000

Modern software development moves incredibly fast.

A single application may receive:

Hundreds of commits daily
Multiple releases per week
Thousands of automated tests
Continuous security scans

Imagine manually building, testing, and deploying every code change.

It would look like:

Developer Writes Code
        ↓
Manual Build
        ↓
Manual Testing
        ↓
Manual Deployment
        ↓
Production Issues

This approach doesn't scale.

This is why CI/CD became one of the most important practices in modern DevOps.

🔗 Resources

** Support the Journey on GitHub: If you're following along, consider starring and forking the repo:** https://github.com/17J/30-Days-Cloud-DevSecOps-Journey

What is CI/CD?

CI/CD stands for:

Continuous Integration
Continuous Delivery / Continuous Deployment

CI/CD is a software engineering practice that automates:

Building applications
Running tests
Security scanning
Packaging artifacts
Deploying applications

The goal is simple:

Deliver Software Faster
        +
Safer
        +
More Reliably

Why CI/CD Matters

Before CI/CD, software releases often looked like:

Developers Work for Weeks
        ↓
Massive Release
        ↓
Unexpected Issues
        ↓
Rollback

Common problems:

Human errors
Slow deployments
Integration conflicts
Unstable releases
Delayed feedback

CI/CD solves these challenges through automation.

The Evolution of Software Delivery

Manual Deployments
        ↓
Build Automation
        ↓
Continuous Integration
        ↓
Continuous Delivery
        ↓
GitOps
        ↓
Platform Engineering

Modern organizations rely heavily on CI/CD.

What is Continuous Integration (CI)?

Continuous Integration is the practice of frequently merging code into a shared repository.

Every code change triggers:

Git Commit
      ↓
Build
      ↓
Tests
      ↓
Security Scans
      ↓
Feedback

Developers receive immediate feedback.

Why Continuous Integration?

Without CI:

Developer A
       ↓
Developer B
       ↓
Developer C
       ↓
Massive Merge Conflict

With CI:

Small Changes
       ↓
Frequent Integration
       ↓
Early Problem Detection

Benefits of Continuous Integration

Faster Feedback

Developers immediately know if code breaks.

Better Code Quality

Automated testing catches bugs earlier.

Fewer Integration Problems

Small changes are easier to merge.

Improved Team Collaboration

Everyone works from a shared codebase.

What is Continuous Delivery?

Continuous Delivery ensures applications are always ready for release.

Pipeline example:

Code Commit
       ↓
Build
       ↓
Test
       ↓
Security Scan
       ↓
Package
       ↓
Deploy to Staging
       ↓
Ready for Production

A human approval step may still exist before production deployment.

What is Continuous Deployment?

Continuous Deployment goes one step further.

Code Commit
       ↓
Build
       ↓
Test
       ↓
Security Scan
       ↓
Production Deployment

No manual approval required.

Every successful change reaches production automatically.

Modern CI/CD Architecture

Today's delivery pipelines look different from five years ago.

Traditional:

CI Tool
      ↓
Build
      ↓
Deploy

Modern:

CI Pipeline
      ↓
Container Registry
      ↓
GitOps Repository
      ↓
ArgoCD / Flux
      ↓
Kubernetes

GitOps has fundamentally changed CD.

Mostly CI Pipeline Stages

Modern CI pipelines usually contain:

Stage 1: Source Code

Developer Commit
       ↓
GitHub / GitLab / Bitbucket

Pipeline starts.

Stage 2: Secret Scanning

Detect:

API Keys
Tokens
Passwords

Popular tools:

Gitleaks
TruffleHog

Stage 3: Build

Compile application.

Examples:

mvn package
npm run build
dotnet build

Stage 4: Unit Testing

Verify application logic.

Examples:

JUnit
PyTest
Jest
NUnit

Stage 5: Static Security Testing

SAST Scans:

Source Code

Popular tools:

SonarQube
Semgrep
Checkmarx

Stage 6: Software Composition Analysis

Checks dependencies.

Popular tools:

Snyk
Trivy
OWASP Dependency Check

Stage 7: Container Build

Docker Build

Creates application image.

Stage 8: Container Security Scan

Scan images for vulnerabilities.

Popular tools:

Trivy
Grype
Dockle

Stage 9: Push Artifact

Push:

Docker Hub
ECR
ACR
GCR
Harbor
Nexus

Stage 10: Update GitOps Repository

Instead of deploying directly:

Update Manifest Repository

Example:

image:
  tag: v1.5.0

Commit changes.

Modern CD with GitOps

Today many organizations use:

CI Tool
      ↓
Build
      ↓
Push Image
      ↓
Update Git Repository
      ↓
GitOps Controller
      ↓
Deploy

This separates CI from CD.

Why GitOps Became Popular

Traditional CD:

Pipeline Pushes Changes

GitOps:

Cluster Pulls Changes

Benefits:

Better security
Auditability
Rollback simplicity
Declarative deployments

Popular GitOps Tools

ArgoCD

One of the most popular GitOps platforms.

Features:

Kubernetes-native
Visual dashboard
Automatic synchronization
Rollbacks
Multi-cluster support

Architecture:

Git Repository
       ↓
ArgoCD
       ↓
Kubernetes Cluster

FluxCD

Another GitOps platform.

Features:

Lightweight
Kubernetes-native
CNCF graduated project

Architecture:

Git Repository
       ↓
Flux Controller
       ↓
Kubernetes

Popular CI/CD Platforms

Jenkins

The most popular traditional CI/CD platform.

Benefits:

Open source
Huge plugin ecosystem
Highly customizable

Best for:

Complex Enterprise Pipelines

GitHub Actions

Native CI/CD for GitHub.

Example:

name: Build

on:
  push:

jobs:
  build:
    runs-on: ubuntu-latest

Benefits:

Easy setup
GitHub integration
Marketplace actions

GitLab CI/CD

Built directly into GitLab.

Example:

stages:
  - build
  - test

Benefits:

Integrated DevOps platform
Strong Kubernetes support

Azure DevOps

Microsoft's enterprise DevOps platform.

Features:

Pipelines
Boards
Repositories
Artifacts

Popular in enterprise environments.

Bitbucket Pipelines

Integrated with Bitbucket repositories.

Example:

pipelines:
  default:
    - step:
        script:
          - npm install

CircleCI

Cloud-native CI/CD platform.

Popular among startups.

TeamCity

JetBrains CI/CD solution.

Common in enterprise Java environments.

Bamboo

Atlassian's CI/CD platform.

Used alongside:

Jira
Bitbucket
Confluence

CI/CD in Kubernetes Era

Modern Kubernetes pipelines often look like:

Developer Commit
       ↓
GitHub Actions / Jenkins
       ↓
Build Container
       ↓
Security Scans
       ↓
Push Image
       ↓
Update GitOps Repository
       ↓
ArgoCD / Flux
       ↓
Kubernetes Deployment

This model has become the industry standard.

DevSecOps in CI/CD

Modern pipelines integrate security at every stage.

Example:

Commit
      ↓
Secret Scan
      ↓
SAST
      ↓
SCA
      ↓
Container Scan
      ↓
IaC Scan
      ↓
Deploy

Security is no longer a separate phase.

CI/CD Best Practices

Keep Pipelines Fast

Developers should receive feedback quickly.

Automate Testing

Manual testing does not scale.

Integrate Security Early

Shift security left.

Use GitOps for CD

Prefer:

ArgoCD
FluxCD

for Kubernetes deployments.

Version Everything

Store:

Source code
Infrastructure
Kubernetes manifests

inside Git.

Monitor Deployments

Deployment success is not enough.

Monitor:

Performance
Errors
Availability

Real-World Pre Prod Enterprise Pipeline

Future of CI/CD

The industry is moving toward:

GitOps
Platform Engineering
AI-assisted pipelines
Policy as Code
Progressive Delivery
Automated Compliance
Zero-Touch Deployments

The future isn't just CI/CD.

It's:

Secure
Automated
GitOps-Driven
Cloud-Native Delivery

Final Thoughts

CI/CD has transformed how software is delivered.

Instead of:

Manual Builds
Manual Tests
Manual Deployments

we now have:

Automated Builds
Automated Testing
Automated Security
Automated Delivery

Modern organizations typically use:

CI

Jenkins
GitHub Actions
GitLab CI
Azure DevOps
Bitbucket Pipelines

CD (GitOps)

ArgoCD
FluxCD

Together, these tools enable faster releases, higher quality software, stronger security, and reliable cloud-native deployments.

Whether you're a:

DevOps Engineer
Platform Engineer
Cloud Engineer
SRE
Software Developer

understanding CI/CD fundamentals is one of the most valuable skills in modern software engineering.

Day 20 - AWS Lambda

Rahul Joshi — Sun, 31 May 2026 05:22:16 +0000

Cloud computing has evolved dramatically over the last decade.

The journey looked something like this:

Physical Servers
        ↓
Virtual Machines
        ↓
Containers
        ↓
Serverless Computing

One of the biggest innovations in cloud computing is AWS Lambda.

Instead of managing:

Servers
Operating Systems
Patching
Scaling
Capacity Planning

You simply upload code and AWS runs it.

This is the foundation of Serverless Computing.

What is AWS Lambda?

AWS Lambda is a serverless compute service that allows you to run code without provisioning or managing servers.

You upload a function and AWS executes it whenever an event occurs.

Example:

User Uploads Image
        ↓
S3 Event Triggered
        ↓
Lambda Function Runs
        ↓
Image Processed

You only pay for execution time.

No execution means:

No Cost

Why AWS Introduced Lambda

Before Lambda, deploying applications looked like this:

Provision EC2
       ↓
Install Runtime
       ↓
Deploy Application
       ↓
Monitor Servers
       ↓
Scale Infrastructure
       ↓
Patch OS

Even small applications required infrastructure management.

AWS wanted developers to focus on:

Business Logic

Instead of:

Infrastructure Management

Thus Lambda was introduced in 2014.

What is Serverless?

Serverless does NOT mean servers don't exist.

Servers still exist.

AWS manages them for you.

Instead of:

You Manage Servers

Lambda provides:

AWS Manages Servers
You Manage Code

Example:

API Gateway
       ↓
Lambda
       ↓
DynamoDB

Benefits of AWS Lambda

1. No Server Management

No:

EC2
OS updates
Capacity planning

2. Automatic Scaling

AWS automatically scales functions.

1 Request
      ↓
1 Lambda Instance

1000 Requests
      ↓
1000 Lambda Instances

3. Pay Per Use

You only pay for:

Requests
+
Execution Duration

4. Event Driven

Lambda reacts to events.

Examples:

API requests
S3 uploads
SNS notifications
SQS messages
DynamoDB streams

5. High Availability

AWS automatically distributes Lambda execution across Availability Zones.

Supported Programming Languages

AWS Lambda supports multiple runtimes.

Python

Popular for:

Automation
AI/ML
Data processing

Example:

import json

def lambda_handler(event, context):
    # TODO implement
    return {
        'statusCode': 200,
        'body': json.dumps('Hello from Lambda!')
    }

Node.js

Popular for:

APIs
Web applications

Java

Popular for:

Enterprise workloads

.NET

Popular for:

Microsoft environments

Go

Popular for:

High performance
Fast startup

Custom Runtime

Using Custom Runtime API, you can run:

Rust
PHP
Other languages

Lambda Function Components

Every Lambda contains:

Function Code

Your business logic.

Runtime

Language execution environment.

Example:

Python 3.12
Node.js 20
Java 21

Handler

Entry point of Lambda.

Example:

lambda_handler(event, context)

AWS invokes this function.

Event

Input to the Lambda.

Example:

{
  "bucket": "images"
}

Context

Runtime information.

Contains:

Request ID
Timeout
Memory

Lambda Execution Lifecycle

Request Arrives
        ↓
Environment Created
        ↓
Function Runs
        ↓
Response Returned

Understanding Cold Starts

One of the most important Lambda concepts.

Cold Start

When Lambda has no running execution environment:

Request Arrives
       ↓
Create Environment
       ↓
Load Runtime
       ↓
Execute Function

Extra startup time occurs.

Warm Start

If environment already exists:

Request Arrives
       ↓
Execute Function Immediately

Faster response.

What is Concurrency?

Concurrency means:

How Many Functions
Can Run Simultaneously

Example:

100 Requests
       ↓
100 Concurrent Executions

Reserved Concurrency

Reserve capacity for critical workloads.

Example:

Payment Function
Reserved = 100

Always guaranteed.

Provisioned Concurrency

Used to eliminate cold starts.

AWS keeps execution environments warm.

Useful for:

APIs
User-facing workloads

What is Lambda Scaling?

Lambda automatically scales horizontally.

1 Request
       ↓
1 Environment

10000 Requests
       ↓
10000 Environments

No manual scaling required.

What is a Lambda Layer?

One of the most important Lambda concepts.

Lambda Layers allow sharing code across multiple functions.

Without Layers:

Function A
 └─ boto3

Function B
 └─ boto3

Function C
 └─ boto3

Duplication occurs.

With Layers

Layer
 └─ boto3

Function A
Function B
Function C

All functions share the same dependency.

Why Lambda Layers Matter

Benefits:

Smaller deployment packages
Reusability
Easier maintenance
Faster deployments

Common Layer Use Cases

Python Libraries

numpy
pandas
requests

Monitoring Agents

Datadog
New Relic
OpenTelemetry

Lambda Storage Options

Temporary Storage

/tmp

Default:

512 MB

Can be increased.

Amazon S3

Persistent object storage.

Used for:

Files
Images
Backups

Amazon EFS

Network file system for Lambda.

Useful for:

Shared storage
Large datasets

Event Sources for Lambda

API Gateway

User Request
      ↓
API Gateway
      ↓
Lambda

Most common pattern.

Amazon S3

File Uploaded
      ↓
Lambda Triggered

Lambda and VPC

By default:

Lambda
     ↓
AWS Managed Network

For private resources:

Lambda
     ↓
VPC
     ↓
RDS

Lambda can connect to:

RDS
ElastiCache
Private APIs

Lambda with RDS

Common architecture:

API Gateway
      ↓
Lambda
      ↓
Aurora MySQL

Challenges:

Connection management
Database scaling

Solution:

RDS Proxy

Lambda Monitoring

CloudWatch Logs

Automatically captures:

stdout
stderr
application logs

CloudWatch Metrics

Monitor:

Invocations
Duration
Errors
Throttles
Concurrent executions

AWS X-Ray

Distributed tracing for Lambda applications.

Useful for:

Performance analysis
Bottleneck detection

Lambda Security

IAM Roles

Lambda should never use hardcoded credentials.

Use:

IAM Execution Role

Secrets Manager

Store:

Database passwords
API keys
Tokens

KMS Encryption

Encrypt:

Environment variables
Data

Lambda Limits

Some important limits:

Feature	Limit
Timeout	15 Minutes
Memory	128 MB – 10 GB
Ephemeral Storage	Up to 10 GB
Deployment Package	50 MB ZIP
Container Image	10 GB

Real-World Lambda Use Cases

Image Processing

S3 Upload
      ↓
Lambda
      ↓
Resize Image

Serverless APIs

API Gateway
      ↓
Lambda
      ↓
DynamoDB

Log Processing

CloudWatch Logs
       ↓
Lambda
       ↓
Elasticsearch/OpenSearch

Scheduled Jobs

EventBridge
      ↓
Lambda
      ↓
Daily Report

Production Best Practices

Keep Functions Small

One function = one responsibility.

Use Layers

Avoid dependency duplication.

Enable Monitoring

Use:

CloudWatch
X-Ray

Use Provisioned Concurrency

For latency-sensitive APIs.

Use RDS Proxy

For database-heavy workloads.

Secure Secrets

Use:

Secrets Manager
Parameter Store

Never hardcode credentials.

Final Thoughts

AWS Lambda transformed cloud computing by allowing developers to focus entirely on code.

Instead of managing:

Servers
Operating Systems
Scaling
Patching

you simply write functions and AWS handles the infrastructure.

Lambda is ideal for:

APIs
Event-driven applications
Automation
Data processing
Serverless architectures

Understanding concepts like:

Layers
Cold Starts
Concurrency
Provisioned Concurrency
Event Sources
RDS Proxy

is essential for designing production-grade serverless applications.

For modern cloud engineers, AWS Lambda is no longer optional—it is one of the most important services in the AWS ecosystem.

Day 19 - Relational Database Service

Rahul Joshi — Sat, 30 May 2026 04:21:10 +0000

Latest applications rely heavily on databases.

Whether you're building:

E-commerce platforms
Banking applications
SaaS products
Mobile apps
Enterprise systems
AI-powered applications

You need a reliable database.

Traditionally, managing databases meant:

Installing database servers
Managing backups
Configuring replication
Applying patches
Monitoring performance
Handling failovers
Scaling storage

All manually.

This creates operational overhead and increases risk.

To solve this problem, AWS introduced Amazon Relational Database Service (RDS).

🔗 Resources

** Support the Journey on GitHub:
If you're following along, consider starring and forking the repo:**
https://github.com/17J/30-Days-Cloud-DevSecOps-Journey
AWS Command Sheet:
https://aws-command.vercel.app/

What is AWS RDS?

Amazon Relational Database Service (RDS) is a fully managed database service provided by AWS.

RDS allows you to run relational databases without worrying about underlying infrastructure management.

Instead of managing:

Operating System
Database Installation
Patching
Backups
Failover
Monitoring
Storage Scaling

AWS manages them for you.

You focus on:

Applications
Queries
Schema Design
Business Logic

Why AWS Created RDS

Before RDS, database administrators spent significant time managing infrastructure.

Typical workflow:

Provision Server
       ↓
Install Database
       ↓
Configure Security
       ↓
Setup Backups
       ↓
Configure Replication
       ↓
Monitor Health
       ↓
Apply Patches

RDS automates most of these tasks.

Why Use Amazon RDS?

1. Managed Service

AWS handles:

OS maintenance
Database patching
Monitoring
Automated backups
High availability

2. Automated Backups

RDS automatically creates backups.

Benefits:

Point-in-time recovery
Reduced operational effort
Disaster recovery support

3. High Availability

Using Multi-AZ deployment:

Primary Database
        ↓
Synchronous Replication
        ↓
Standby Database

If primary fails:

Automatic Failover

4. Easy Scaling

You can scale:

Compute

db.t3.medium
      ↓
db.r6g.large

Storage

Increase storage without rebuilding infrastructure.

5. Security

Integration with:

IAM
KMS Encryption
Security Groups
VPC
CloudTrail

RDS Architecture

Application
       ↓
Load Balancer
       ↓
EC2 / ECS / EKS
       ↓
Amazon RDS
       ↓
Storage Layer

Applications connect to the RDS endpoint.

Example:

mydb.xxxxxx.us-east-1.rds.amazonaws.com

Core Components of Amazon RDS

DB Instance

A DB Instance is the database server.

Example:

MySQL Instance
db.t3.medium
50 GB Storage

This is where the database engine runs.

Storage Layer

RDS supports:

General Purpose SSD
Provisioned IOPS SSD
Magnetic Storage (legacy)

Endpoint

Applications never connect directly to servers.

Instead they connect using:

Database Endpoint

Example:

mysql-prod.abc123.us-east-1.rds.amazonaws.com

Security Groups

Control inbound database traffic.

Example:

EC2 → MySQL Port 3306 → RDS

Parameter Groups

Used to customize database settings.

Examples:

max_connections
query_cache_size
log settings

Option Groups

Used for engine-specific features.

Examples:

Oracle Enterprise Options
SQL Server Features

How to Create Amazon RDS MySQL Database - Step by Step Guide

Understanding Multi-AZ Deployments

One of the most important RDS features.

Without Multi-AZ:

Application
      ↓
Single Database

Database failure means downtime.

With Multi-AZ:

Primary DB
      ↓
Synchronous Replication
      ↓
Standby DB

Benefits:

High availability
Automatic failover
Better resilience

Read Replicas

Read Replicas improve read performance.

Architecture:

Primary Database
      ↓
Asynchronous Replication
      ↓
Read Replica 1
      ↓
Read Replica 2

Used for:

Reporting
Analytics
Read-heavy applications

Automated Backups

RDS automatically creates backups.

Features:

Daily snapshots
Transaction logs
Point-in-time recovery

Example:

Restore database
to 11:42 AM yesterday

Manual Snapshots

Manual snapshots remain until deleted.

Useful for:

Migration
Upgrades
Disaster recovery

Amazon Aurora Explained

Aurora is AWS's cloud-native relational database.

It is compatible with:

MySQL
PostgreSQL

But internally it is architected differently.

Why Aurora Exists

Traditional databases face limitations:

Storage Bottlenecks
Replication Delays
Scaling Challenges

Aurora solves these issues.

Aurora Architecture

Application
       ↓
Aurora Cluster
       ↓
Writer Instance
       ↓
Shared Distributed Storage
       ↓
Reader Instances

Storage and compute are separated.

Aurora MySQL vs Standard MySQL

Aurora Performance Advantage

Aurora's storage system replicates data across multiple Availability Zones.

Traditional MySQL:

Database
      ↓
Single Storage Layer

Aurora:

Database
      ↓
Distributed Storage
      ↓
Multiple AZs

This improves:

Throughput
Durability
Recovery

Aurora Cluster Components

Writer Instance

Handles:

INSERT
UPDATE
DELETE

Reader Instance

Handles:

SELECT Queries

Used for scaling reads.

Shared Storage Layer

Aurora automatically replicates data across multiple AZs.

RDS vs Aurora

Feature	RDS MySQL	Aurora MySQL
Simplicity	Easier	More Advanced
Cost	Lower	Higher
Performance	Good	Excellent
Scaling	Good	Better
Availability	High	Very High
Enterprise Workloads	Moderate	Excellent

Security Features in RDS

Encryption at Rest

Uses AWS KMS.

Database Storage
Snapshots
Read Replicas

can be encrypted.

Encryption in Transit

Uses SSL/TLS.

Application
      ↓ TLS
Amazon RDS

IAM Authentication

Instead of passwords:

IAM Token Authentication

can be used.

VPC Integration

RDS runs inside VPC.

Benefits:

Isolation
Network security
Controlled access

Monitoring RDS

AWS provides:

CloudWatch Metrics

Monitor:

CPU Utilization
Free Storage
Memory
Connections
IOPS

Enhanced Monitoring

Provides OS-level visibility.

Performance Insights

Shows:

Slow queries
Wait events
Database bottlenecks

Very useful for troubleshooting.

RDS Scaling

Vertical Scaling

Increase instance size.

db.t3.medium
      ↓
db.r6g.large

Storage Scaling

Increase storage without downtime.

Read Scaling

Add Read Replicas.

Common RDS Use Cases

Web Applications

Application
      ↓
RDS MySQL

E-commerce Platforms

Store:

Orders
Products
Customers

SaaS Platforms

Store:

Users
Billing data
Application metadata

Enterprise Applications

Store:

ERP data
CRM data
Business records

Cost Optimization Tips

Use Right Instance Types

Avoid oversized databases.

Use Reserved Instances

Can significantly reduce cost.

Enable Storage Auto Scaling

Avoid over-provisioning.

Remove Unused Read Replicas

Unused replicas generate cost.

Best Practices for Production

Enable Multi-AZ

Never run critical production workloads on single-AZ databases.

Enable Backups

Always configure backup retention.

Monitor Performance

Use:

CloudWatch
Performance Insights

Encrypt Everything

Enable:

Storage Encryption
TLS Connections

Use Read Replicas

For read-heavy applications.

Test Restores

Backups are useless if restore procedures are never tested.

Example Production Architecture

Final Thoughts

Amazon RDS removes much of the operational complexity involved in running databases.

Instead of spending time managing:

Servers
Backups
Failovers
Patching

teams can focus on building applications.

For most workloads:

RDS MySQL
RDS PostgreSQL

are excellent choices.

For high-performance enterprise workloads:

Amazon Aurora

is often the preferred option.

Whether you're a:

Cloud Engineer
DevOps Engineer
Backend Developer
Platform Engineer
Solutions Architect

understanding RDS is a fundamental AWS skill because databases remain at the heart of nearly every modern application.

Day 18 - Infrastructure as Code (IaC) with Terraform

Rahul Joshi — Fri, 29 May 2026 04:08:52 +0000

Modern cloud infrastructure is too complex to manage manually.

Imagine creating:

10 EC2 instances
5 VPCs
20 Security Groups
15 IAM Roles
3 Load Balancers
Kubernetes Clusters

using only a cloud console.

It quickly becomes:

Slow
Error-Prone
Difficult to Scale
Impossible to Audit

This is why Infrastructure as Code (IaC) became one of the most important practices in modern DevOps and Cloud Engineering.

🔗 Resources

** Support the Journey on GitHub: If you're following along, consider starring and forking the repo:** https://github.com/17J/30-Days-Cloud-DevSecOps-Journey

What is Infrastructure as Code (IaC)?

Infrastructure as Code (IaC) is the practice of managing infrastructure through code instead of manually creating resources.

Instead of clicking buttons:

AWS Console
      ↓
Create EC2
      ↓
Create Security Group
      ↓
Create VPC

You write:

resource "aws_instance" "web" {
  ami           = "ami-123456"
  instance_type = "t2.micro"
}

And infrastructure gets created automatically.

Why Infrastructure as Code Matters

Before IaC, infrastructure management was painful.

Common problems included:

Manual mistakes
Configuration drift
Poor documentation
Difficult disaster recovery
Inconsistent environments

Example:

Developer Environment
        ↓
Works Perfectly
        ↓
Production Environment
        ↓
Different Configuration
        ↓
Application Fails

IaC solves this problem by making environments reproducible.

Benefits of Infrastructure as Code

1. Consistency

Every environment is identical.

Dev
 ↓
QA
 ↓
Staging
 ↓
Production

All built from the same code.

2. Version Control

Infrastructure becomes:

Git Commit
Pull Request
Code Review
Rollback
Audit Trail

Infrastructure changes become trackable.

3. Automation

Entire environments can be created in minutes.

4. Disaster Recovery

If infrastructure is lost:

Git Repository
        ↓
terraform apply
        ↓
Infrastructure Restored

5. Scalability

Large organizations can manage thousands of resources through code.

Infrastructure as Code Market Growth

Infrastructure automation has become a standard practice.

Today IaC is used by:

Cloud Engineers
DevOps Engineers
Platform Engineers
SRE Teams
Security Teams

Organizations running:

AWS
Azure
GCP
Kubernetes

almost always adopt some form of IaC.

Types of Infrastructure as Code

Declarative

You describe the desired state.

Example:

resource "aws_instance" "web" {
  instance_type = "t2.micro"
}

Tool decides how to create it.

Examples:

Terraform
CloudFormation
Bicep

Imperative

You define step-by-step instructions.

Example:

create_vpc()
create_subnet()
create_ec2()

Examples:

Pulumi
Custom automation scripts

Popular Infrastructure as Code Tools

1. Terraform

2. AWS CloudFormation (CFT)

AWS-native IaC service.

Supports:

VPC
EC2
IAM
S3
RDS
Lambda

Example:

Resources:
  MyBucket:
    Type: AWS::S3::Bucket

3. Azure Bicep

Microsoft's modern IaC language.

Simplifies Azure Resource Manager templates.

Example:

resource storage 'Microsoft.Storage/storageAccounts@2022-09-01' = {
  name: 'mystorage'
}

4. Pulumi

Modern Infrastructure as Code.

Uses programming languages:

Python
Go
TypeScript
C#
Java

Example:

import pulumi_aws as aws

bucket = aws.s3.Bucket("my-bucket")

Why Terraform Dominates IaC

Terraform became the industry standard because:

One Language
        ↓
Multiple Clouds
        ↓
Single Workflow

Engineers can manage:

AWS
Azure
GCP
Kubernetes

using one tool.

Terraform Architecture

Terraform Basics

Understanding Terraform starts with four key concepts:

Providers
Resources
Variables
State

Terraform Providers

Providers allow Terraform to communicate with platforms.

Examples:

AWS Provider
Azure Provider
Google Provider
Kubernetes Provider
GitHub Provider

Example:

provider "aws" {
  region = "us-east-1"
}

Terraform now knows where to create resources.

Terraform Resources

Resources are actual infrastructure components.

Examples:

EC2 Instance
S3 Bucket
VPC
Security Group
IAM Role

Example:

resource "aws_s3_bucket" "demo" {
  bucket = "my-demo-bucket"
}

Terraform will create:

AWS S3 Bucket

Terraform Variables

Variables make code reusable.

Without variables:

instance_type = "t2.micro"

With variables:

variable "instance_type" {}

instance_type = var.instance_type

Now different environments can use:

Dev → t2.micro
QA → t3.small
Prod → t3.large

Terraform State

Terraform keeps track of infrastructure using:

terraform.tfstate

This file stores:

Resource IDs
Current state
Dependency mapping

Terraform compares:

Current State
        vs
Desired State

and calculates required changes.

Terraform Workflow

Step 1

Write Code

resource "aws_instance" "web" {
  ami           = "ami-123456"
  instance_type = "t2.micro"
}

Step 2

Initialize

terraform init

Downloads providers.

Step 3

Validate

terraform validate

Checks syntax.

Step 4

Preview

terraform plan

Shows changes before execution.

Step 5

Apply

terraform apply

Creates infrastructure.

Deep Terraform Example

Let's create a simple AWS infrastructure.

Provider

provider "aws" {
  region = "us-east-1"
}

Variable

variable "instance_type" {
  default = "t2.micro"
}

Security Group

resource "aws_security_group" "web_sg" {

  name = "web-sg"

  ingress {
    from_port   = 80
    to_port     = 80
    protocol    = "tcp"
    cidr_blocks = ["0.0.0.0/0"]
  }
}

EC2 Instance

resource "aws_instance" "web" {

  ami           = "ami-123456"
  instance_type = var.instance_type

  vpc_security_group_ids = [
    aws_security_group.web_sg.id
  ]

  tags = {
    Name = "Terraform-Web"
  }
}

What Happens Behind the Scenes?

Terraform File Structure

Typical project:

terraform-project/

├── main.tf
├── variables.tf
├── outputs.tf
├── terraform.tfvars
└── providers.tf

Best Practices

Use Remote State

Store state in:

S3
Azure Storage
GCS
Terraform Cloud

Never store production state locally.

Use Modules

Avoid repeating code.

module "vpc" {
  source = "./modules/vpc"
}

Use Version Control

Infrastructure should always live in Git.

Enable Code Reviews

Treat infrastructure like application code.

Separate Environments

Dev
QA
Staging
Production

should have separate state files.

Infrastructure as Code in DevOps Pipeline

Developer Pushes Terraform
          ↓
Pull Request
          ↓
Code Review
          ↓
terraform validate
          ↓
terraform plan
          ↓
Security Scan
          ↓
terraform apply
          ↓
Infrastructure Created

Security Considerations

Never store:

AWS Keys
Passwords
Tokens
Secrets

inside Terraform code.

Use:

AWS Secrets Manager
Azure Key Vault
HashiCorp Vault

instead.

Final Thoughts

Infrastructure as Code transformed how cloud infrastructure is managed.

Instead of:

Manual Infrastructure

we now have:

Version Controlled Infrastructure

Among all IaC tools:

Terraform dominates multi-cloud environments
CloudFormation is ideal for AWS-centric teams
Bicep is excellent for Azure
Pulumi is attractive for developers who prefer real programming languages

For anyone pursuing:

DevOps
Cloud Engineering
Platform Engineering
Site Reliability Engineering

Infrastructure as Code is no longer optional—it is a fundamental skill of modern cloud operations.