DEV Community: Darshan Vasani

🐳 Docker Multi-Stage Build - Complete Documentation

Darshan Vasani — Sat, 26 Jul 2025 07:59:22 +0000

🐳 Docker Multi-Stage Build - Complete Documentation

📋 Table of Contents

🎯 What is Docker Multi-Stage Build?
🚀 Why Use Multi-Stage Builds?
🏗️ How Multi-Stage Builds Work
📦 Step-by-Step Tutorial
🔄 Multi-Stage Build Workflow
📊 Architecture Diagrams
🛠️ Commands and Best Practices
⚡ Performance Comparison
🔧 Debugging and Troubleshooting
✨ Best Practices

🎯 What is Docker Multi-Stage Build?

Docker Multi-Stage Build is a powerful feature that allows you to use multiple FROM statements in a single Dockerfile[1]. Each FROM instruction creates a new stage in the build process, enabling you to optimize image size and improve security by separating build dependencies from runtime requirements[2].

🔤 Key Concepts

🏗️ Multiple Stages: Each stage has its own base image and purpose
📦 Selective Copying: Copy only necessary artifacts between stages
🗑️ Artifact Exclusion: Build tools and dependencies are left behind
🎯 Production-Ready: Final image contains only runtime requirements

🚀 Why Use Multi-Stage Builds?

Multi-stage builds solve several critical problems in containerized application development[3]:

🎯 Benefit	📋 Description	💡 Impact
📉 Smaller Image Size	Excludes build tools and dependencies from final image	🚀 Faster deployments
🔒 Enhanced Security	Reduces attack surface by removing unnecessary components	🛡️ Lower vulnerability risk
⚡ Better Performance	Lighter images load and start faster	🏃 Improved runtime speed
🧹 Cleaner Workflow	Single Dockerfile for entire build process	🛠️ Simplified maintenance
💰 Cost Optimization	Reduced storage and bandwidth usage	💵 Lower infrastructure costs

🏗️ How Multi-Stage Builds Work

🔍 Single-Stage vs Multi-Stage Comparison

graph TB
    subgraph "❌ Single-Stage Build Problems"
        SINGLE[📦 Single Stage]
        SINGLE --> BUILD_TOOLS[🔧 Build Tools]
        SINGLE --> SOURCE[📄 Source Code]
        SINGLE --> DEPS[📚 All Dependencies]
        SINGLE --> ARTIFACTS[⚡ Build Artifacts]
        SINGLE --> FINAL1[📦 Final Image: 200MB+]
    end

    subgraph "✅ Multi-Stage Build Solution"
        STAGE1[🏗️ Build Stage]
        STAGE2[🚀 Runtime Stage]

        STAGE1 --> BUILD_TOOLS2[🔧 Build Tools]
        STAGE1 --> SOURCE2[📄 Source Code]  
        STAGE1 --> BUILD_DEPS[📚 Build Dependencies]
        STAGE1 --> COMPILE[⚙️ Compile/Build]

        STAGE2 --> RUNTIME_BASE[🏃 Runtime Base Image]
        STAGE2 --> COPY_ARTIFACTS[📋 Copy Build Artifacts]
        STAGE2 --> FINAL2[📦 Final Image: 50MB]

        COMPILE -.->|Copy Only Artifacts| COPY_ARTIFACTS
    end

🔄 Build Process Flow

sequenceDiagram
    participant User as 👤 Developer
    participant Docker as 🐳 Docker Engine
    participant Stage1 as 🏗️ Build Stage (installer)
    participant Stage2 as 🚀 Runtime Stage (deployer)
    participant Registry as 📦 Image Registry

    User->>Docker: docker build -t multistage .
    Docker->>Stage1: FROM node:18-alpine AS installer
    Stage1->>Stage1: WORKDIR /app
    Stage1->>Stage1: COPY package*.json ./
    Stage1->>Stage1: RUN npm install
    Stage1->>Stage1: COPY . .
    Stage1->>Stage1: RUN npm run build

    Docker->>Stage2: FROM nginx:latest AS deployer
    Stage2->>Stage1: COPY --from=installer /app/build /usr/share/nginx/html

    Docker->>User: 📦 Optimized Image Ready (50MB)
    User->>Registry: docker push multistage

📦 Step-by-Step Tutorial

🛠️ Project Setup

1. Clone the Application

git clone 
cd react-app-docker
ls  # Check project structure

Project Structure:

react-app-docker/
├── src/
├── public/
├── package.json
├── package-lock.json
└── README.md

📝 Creating Multi-Stage Dockerfile

2. Create Dockerfile

touch Dockerfile
vi Dockerfile

3. Multi-Stage Dockerfile Content

# 🏗️ Stage 1: Build Stage (installer)
FROM node:18-alpine AS installer
WORKDIR /app

# Copy package files for dependency installation
COPY package*.json ./

# Install all dependencies (including devDependencies)
RUN npm install

# Copy source code
COPY . .

# Build the application
RUN npm run build

# 🚀 Stage 2: Runtime Stage (deployer)
FROM nginx:latest AS deployer

# Copy only build artifacts from previous stage
COPY --from=installer /app/build /usr/share/nginx/html

# Nginx will serve the static files
EXPOSE 80

🔧 Build Process

4. Build the Multi-Stage Image

docker build -t multistage .

📊 Build Output Analysis

graph LR
    subgraph "🏗️ Build Stage Process"
        A[📄 package.json] --> B[📦 npm install]
        C[📁 Source Code] --> B
        B --> D[⚙️ npm run build]
        D --> E[📁 /app/build]
    end

    subgraph "🚀 Runtime Stage Process"
        F[🌐 nginx:latest] --> G[📋 Copy build artifacts]
        E -.->|COPY --from=installer| G
        G --> H[🎯 Final Image]
    end

    style E fill:#90EE90
    style H fill:#87CEEB

🔄 Multi-Stage Build Workflow

🎯 Complete Workflow Diagram

flowchart TD
    START([🚀 Start Build Process]) --> STAGE1{🏗️ Build Stage}

    STAGE1 --> NODE[📦 FROM node:18-alpine AS installer]
    NODE --> WORKDIR[📁 WORKDIR /app]
    WORKDIR --> COPY_PKG[📋 COPY package*.json ./]
    COPY_PKG --> NPM_INSTALL[⬇️ RUN npm install]
    NPM_INSTALL --> COPY_SRC[📄 COPY . .]
    COPY_SRC --> NPM_BUILD[⚙️ RUN npm run build]

    NPM_BUILD --> STAGE2{🚀 Runtime Stage}

    STAGE2 --> NGINX[🌐 FROM nginx:latest AS deployer]
    NGINX --> COPY_BUILD[📋 COPY --from=installer /app/build /usr/share/nginx/html]
    COPY_BUILD --> FINAL[✨ Optimized Final Image]

    FINAL --> SIZE_CHECK{📏 Size Check}
    SIZE_CHECK -->|Before: 200MB+| BEFORE[❌ Single Stage: Bloated]
    SIZE_CHECK -->|After: ~50MB| AFTER[✅ Multi-Stage: Optimized]

    style STAGE1 fill:#FFE4B5
    style STAGE2 fill:#E0FFFF
    style FINAL fill:#90EE90
    style AFTER fill:#98FB98

🔍 Stage Dependency Graph

graph TB
    subgraph "📦 Base Images"
        NODE18[🟢 node:18-alpine]
        NGINX[🔵 nginx:latest]
    end

    subgraph "🏗️ Build Stage (installer)"
        INSTALLER[installer stage]
        BUILD_DEPS[📚 Build Dependencies]
        SOURCE_CODE[📄 Source Code]
        BUILD_ARTIFACTS[⚡ Build Artifacts]

        NODE18 --> INSTALLER
        INSTALLER --> BUILD_DEPS
        INSTALLER --> SOURCE_CODE
        BUILD_DEPS --> BUILD_ARTIFACTS
        SOURCE_CODE --> BUILD_ARTIFACTS
    end

    subgraph "🚀 Runtime Stage (deployer)"
        DEPLOYER[deployer stage]
        STATIC_FILES[📁 Static Files Only]

        NGINX --> DEPLOYER
        BUILD_ARTIFACTS -.->|COPY --from=installer| DEPLOYER
        DEPLOYER --> STATIC_FILES
    end

    subgraph "🗑️ Excluded from Final Image"
        EXCLUDED[❌ node_modules❌ Source code❌ Build tools❌ Dev dependencies]
    end

    style BUILD_ARTIFACTS fill:#90EE90
    style STATIC_FILES fill:#87CEEB
    style EXCLUDED fill:#FFB6C1

🛠️ Commands and Best Practices

🔧 Essential Docker Commands

Build Commands

# Build multi-stage image
docker build -t multistage .

# Build with specific target stage
docker build --target installer -t build-stage .

# Build with build arguments
docker build --build-arg NODE_VERSION=18 -t multistage .

Image Management

# List all images
docker images

# Remove specific image
docker image rm multistage

# Remove dangling images
docker image prune

# Check image size
docker images --format "table {{.Repository}}\t{{.Tag}}\t{{.Size}}"

Container Operations

# Run container
docker run -d -p 3000:80 --name app-container multistage

# Check running containers
docker ps

# View container logs
docker logs 

# Execute commands in container
docker exec -it  /bin/sh

🔍 Debugging and Inspection Commands

# Inspect container configuration
docker inspect 

# Check container filesystem
docker exec -it  ls -la /usr/share/nginx/html

# Monitor container resource usage
docker stats 

# View container port mappings
docker port

📊 Performance Comparison

📏 Size Comparison

graph LR
    subgraph "📊 Image Size Comparison"
        SINGLE[❌ Single-Stage200MB+]
        MULTI[✅ Multi-Stage~50MB]

        SINGLE --> REDUCTION[75% Size Reduction]
        REDUCTION --> MULTI
    end

    subgraph "⚡ Performance Impact"
        FASTER[🚀 3x Faster Pull]
        SECURE[🔒 Lower Attack Surface]
        COST[💰 Reduced Storage Cost]
    end

    MULTI --> FASTER
    MULTI --> SECURE
    MULTI --> COST

    style SINGLE fill:#FFB6C1
    style MULTI fill:#90EE90
    style REDUCTION fill:#FFD700

📈 Benefits Breakdown

📊 Metric	🔴 Single-Stage	🟢 Multi-Stage	📈 Improvement
📦 Image Size	200MB+	~50MB	75% reduction
⬇️ Pull Time	30 seconds	10 seconds	3x faster
🚀 Startup Time	15 seconds	8 seconds	2x faster
🔒 Security Vulnerabilities	High	Low	60% fewer
💾 Storage Cost	High	Low	75% savings

🔧 Debugging and Troubleshooting

🔍 Container Investigation Commands

# Check container logs
docker logs 

# Access container shell
docker exec -it  /bin/sh

# Inspect container details
docker inspect

🕵️ Inside Container Exploration

graph TB
    CONTAINER[🐳 Running Container]

    subgraph "📁 Container Filesystem"
        ROOT[/ (root directory)]
        USR[/usr]
        SHARE[/usr/share]
        NGINX[/usr/share/nginx]
        HTML[/usr/share/nginx/html]
        FILES[📄 Static Files]

        ROOT --> USR
        USR --> SHARE
        SHARE --> NGINX
        NGINX --> HTML
        HTML --> FILES
    end

    subgraph "🔍 Inspection Commands"
        LS[ls -la]
        CAT[cat index.html]
        PS[ps aux]
        TOP[top]
    end

    CONTAINER --> ROOT
    FILES --> LS
    FILES --> CAT

    style FILES fill:#90EE90
    style HTML fill:#87CEEB

🚨 Common Issues and Solutions

❌ Problem	🔍 Symptom	✅ Solution
Build fails	`unknown instruction WORKDIR`	Check Dockerfile syntax
Large image size	Image still 200MB+	Verify multi-stage is working
Container won't start	Exit code 125	Check port conflicts
Files not found	404 errors	Verify COPY paths
Permission issues	Access denied	Use non-root user

✨ Best Practices

🎯 Multi-Stage Build Best Practices

1. 🏷️ Use Named Stages

# ✅ Good: Named stages
FROM node:18-alpine AS installer
FROM nginx:latest AS deployer

# ❌ Bad: Unnamed stages
FROM node:18-alpine
FROM nginx:latest

2. 📦 Choose Optimal Base Images

# ✅ Good: Lightweight base images
FROM node:18-alpine AS installer    # Small Alpine-based
FROM nginx:alpine AS deployer       # Lightweight nginx

# ❌ Bad: Heavy base images  
FROM node:18 AS installer           # Ubuntu-based (larger)
FROM nginx:latest AS deployer       # Full nginx image

3. 🎯 Copy Only What's Needed

# ✅ Good: Selective copying
COPY --from=installer /app/build /usr/share/nginx/html

# ❌ Bad: Copying everything
COPY --from=installer /app /usr/share/nginx/html

4. 📋 Optimize Layer Caching

# ✅ Good: Copy package files first
COPY package*.json ./
RUN npm install
COPY . .

# ❌ Bad: Copy everything first
COPY . .
RUN npm install

🔒 Security Best Practices

5. 👤 Use Non-Root User

# ✅ Good: Non-root user
FROM nginx:alpine AS deployer
RUN addgroup -g 1001 -S nodejs
RUN adduser -S nextjs -u 1001
USER nextjs

# ❌ Bad: Running as root (default)
FROM nginx:alpine AS deployer
# No user specified - runs as root

6. 🗑️ Remove Unnecessary Packages

# ✅ Good: Clean up after installation
RUN apt-get update && apt-get install -y \
    package1 \
    package2 \
    && apt-get clean \
    && rm -rf /var/lib/apt/lists/*

# ❌ Bad: Leave package cache
RUN apt-get update && apt-get install -y package1 package2

🚀 Performance Best Practices

7. 🎯 Use Specific Targets

# Build only specific stage for testing
docker build --target installer -t build-stage .

# Build final production image
docker build -t production-app .

8. 📦 Multi-Architecture Support

# Support multiple architectures
FROM --platform=$BUILDPLATFORM node:18-alpine AS installer
# Build process...

FROM --platform=$TARGETPLATFORM nginx:alpine AS deployer
# Runtime setup...

🔄 Advanced Multi-Stage Patterns

9. 🧪 Testing Stage

# Build stage
FROM node:18-alpine AS installer
WORKDIR /app
COPY package*.json ./
RUN npm install
COPY . .
RUN npm run build

# Test stage
FROM installer AS tester
RUN npm test

# Production stage
FROM nginx:alpine AS deployer
COPY --from=installer /app/build /usr/share/nginx/html

10. 🔀 Parallel Builds

# Base dependencies
FROM node:18-alpine AS base
WORKDIR /app
COPY package*.json ./
RUN npm install

# Frontend build
FROM base AS frontend
COPY frontend/ ./
RUN npm run build:frontend

# Backend build  
FROM base AS backend
COPY backend/ ./
RUN npm run build:backend

# Final stage
FROM nginx:alpine AS final
COPY --from=frontend /app/dist /usr/share/nginx/html
COPY --from=backend /app/build /app/api

📊 Additional Best Practices

mindmap
  root((🎯 Multi-StageBest Practices))
    🏗️ Build Optimization
      📦 Use Alpine images
      🎯 Named stages
      📋 Layer caching
      🗂️ .dockerignore file
    🔒 Security
      👤 Non-root user
      🧹 Clean package cache
      🔍 Minimal attack surface
      🚫 No secrets in layers
    ⚡ Performance
      🔄 Parallel builds
      📏 Smaller final image
      🚀 Faster deployments
      💾 Reduced storage
    🛠️ Maintenance
      📝 Clear documentation
      🏷️ Consistent naming
      🧪 Testing stages
      🔄 CI/CD integration

🎓 Summary

Docker Multi-Stage Builds are a game-changing feature that revolutionizes container image optimization[1][2]. By separating build and runtime environments, you can achieve:

🔑 Key Achievements

📉 75% smaller images - From 200MB+ to ~50MB
🔒 Enhanced security - Reduced attack surface
⚡ Faster deployments - 3x faster pull times
💰 Cost savings - Lower storage and bandwidth costs
🧹 Cleaner workflow - Single Dockerfile for entire process

🚀 Implementation Steps

🏗️ Design stages - Separate build and runtime concerns
📦 Choose base images - Use lightweight Alpine variants
🎯 Copy selectively - Only production artifacts
🔒 Apply security - Non-root users, clean packages
📊 Monitor results - Measure size and performance improvements

Multi-stage builds represent a fundamental shift from monolithic container images to optimized, production-ready deployments. They embody the principle of "build fat, ship thin" - using all necessary tools during build time while delivering minimal, secure runtime images[3].

Start implementing multi-stage builds in your projects today to unlock significant performance gains and security improvements in your containerized applications! 🚀

[1] https://docs.docker.com/build/building/multi-stage/
[2] https://docs.docker.com/get-started/docker-concepts/building-images/multi-stage-builds/
[3] https://docs.docker.com/build/building/best-practices/
[4] https://docs.docker.com/guides/cpp/multistage/
[5] https://dev.to/raunakgurud09/mastering-docker-multistage-builds-1e0m
[6] https://dev.to/abhay_yt_52a8e72b213be229/streamline-your-docker-images-with-multi-stage-builds-340c
[7] https://depot.dev/blog/docker-multi-stage-builds
[8] https://dev.to/citrux-digital/understanding-docker-multistage-builds-3fm7
[9] https://labs.iximiuz.com/tutorials/docker-multi-stage-builds
[10] https://dev.to/kalkwst/multi-stage-dockerfiles-3e90
[11] https://ruan.dev/blog/2022/07/31/docker-multistage-builds-for-hugo
[12] https://www.cherryservers.com/blog/docker-multistage-build
[13] https://github.com/patrickhoefler/dockerfilegraph
[14] https://docs.docker.com/build/building/multi-platform/
[15] https://overcast.blog/building-efficient-multi-stage-dockerfiles-for-production-055f34c4baed
[16] https://learn.microsoft.com/en-us/dotnet/architecture/microservices/docker-application-development-process/docker-app-development-workflow
[17] https://docs.docker.com/get-started/docker-concepts/building-images/writing-a-dockerfile/
[18] https://earthly.dev/blog/docker-multistage/
[19] https://blog.devgenius.io/docker-multi-stage-build-in-detail-3d7da2948797?gi=265baab36942
[20] https://www.youtube.com/watch?v=ajetvJmBvFo

Notes & Cheatsheet: “Dockerizing an App – CKA Series #2”

Darshan Vasani — Sat, 26 Jul 2025 07:58:11 +0000

Notes & Cheatsheet: “Dockerizing an App – CKA Series #2”

🎯 Goal of the video

Show, from scratch, how to containerise (“dockerise”) a sample Node.js app, push it to Docker Hub and run it anywhere.
Lays foundation for later CKA topics (pods, images, registries, best-practices).

1 Prerequisites & Sandbox options

Local install: Docker Desktop (Mac, Windows, Linux).
Zero-install alternative: Play-with-Docker (Docker-provided 4-hour sandbox).

  labs.play-with-docker.com  →  Start  →  Add new instance

2 Clone sample project

# create workspace
mkdir day02_code && cd day02_code      

# pull sample “getting-started” todo app
git clone https://github.com/docker/getting-started.git
cd getting-started/app        # contains package.json, src/, etc.

3 Write the Dockerfile

# ─── 1. Base image ───────────────────────────────────────────────
FROM node:18-alpine                              # tiny Linux + Node.js

# ─── 2. Set workdir inside container ─────────────────────────────
WORKDIR /app

# ─── 3. Copy source code into image ──────────────────────────────
COPY . .

# ─── 4. Install prod deps only ───────────────────────────────────
RUN yarn install --production

# ─── 5. Expose port app listens on ───────────────────────────────
EXPOSE 3000

# ─── 6. Container start-up command ───────────────────────────────
CMD ["node","src/index.js"]

🧩 Flow diagram

Host FS ─► docker build
          │
          ▼
      Dockerfile  ──►  Layered Image (base → code → deps) ──► Registry

4 Build image locally

docker build -t day02-todo:latest .

Image is now visible via: docker images or Docker-Desktop > Images.

5 Push to Docker Hub

# 1. create repo on hub:  /test-repo  (public is fine)

# 2. tag local image for hub
docker tag day02-todo:latest  /test-repo:latest

# 3. login and push
docker login
docker push /test-repo:latest

6 Run the container anywhere

# pull (if on fresh machine)
docker pull /test-repo:latest

# run detached, publish host:3000 → container:3000
docker run -d -p 3000:3000 --name todoapp /test-repo:latest

Open http://localhost:3000 📝✅.

7 Troubleshoot inside the container

# open an interactive shell (Alpine uses sh)
docker exec -it todoapp sh
# inspect files, logs, env, etc.

8 Key Docker CLI quick-ref

Task	Command
List images	`docker images`
List running containers	`docker ps`
Stop & remove	`docker stop && docker rm`
Remove image	`docker rmi`
Clean dangling layers	`docker system prune`

9 Next-step teaser 🐣

Image weighs 217 MB—too big for a tiny todo app!

Hint for next video: multi-stage builds and other best practices shrink it dramatically.

💡 Takeaways

Dockerfile = deterministic recipe → identical runtime everywhere.
Layered cache speeds builds & transfers.
Registry push/pull workflow mirrors real CI/CD pipelines.
Knowing basic docker build/run/exec is a must before jumping into Kubernetes objects (pods, deployments, etc.). 🎓

🚀 Step-by-Step: Setup + GET Method in FastAPI

Darshan Vasani — Thu, 24 Jul 2025 16:51:42 +0000

🚀 Step-by-Step: Setup + GET Method in FastAPI

🧪 1. Set up Virtual Environment (Windows/Linux/Mac)

# Install virtualenv (if not already installed)
pip install virtualenv

# Create virtual environment
virtualenv venv

# Activate virtual environment
# Windows
venv\Scripts\activate

# macOS/Linux
source venv/bin/activate

📦 2. Install FastAPI and Uvicorn

pip install fastapi uvicorn

📁 3. Create Project Structure

fastapi_project/
│
├── main.py
├── requirements.txt
└── venv/

Save dependencies for later:

pip freeze > requirements.txt

🧠 4. Create `main.py` with GET Methods

✅ Using Path Parameters, Predefined Values, and Query Parameters

from fastapi import FastAPI, Query
from typing import Optional
from enum import Enum

app = FastAPI()

# 🧩 Predefined Path Parameter using Enum
class ModelName(str, Enum):
    alexnet = "alexnet"
    resnet = "resnet"
    lenet = "lenet"

# ✅ Basic GET Method with Path Parameter
@app.get("/items/{item_id}")
def read_item(item_id: int):
    return {"item_id": item_id}

# ✅ Path Parameter with Predefined Values
@app.get("/models/{model_name}")
def get_model(model_name: ModelName):
    if model_name == ModelName.alexnet:
        return {"model_name": model_name, "message": "Deep Learning FTW!"}
    elif model_name == ModelName.lenet:
        return {"model_name": model_name, "message": "LeNet is classic!"}
    return {"model_name": model_name, "message": "ResNet Rocks!"}

# ✅ With Query Parameters (Optional + Defaults)
@app.get("/products/")
def get_products(skip: int = 0, limit: int = 10, category: Optional[str] = Query(None, min_length=3)):
    return {
        "message": "Fetching products",
        "skip": skip,
        "limit": limit,
        "category": category or "all"
    }

▶️ 5. Run the Server

uvicorn main:app --reload

--reload allows auto-reload during development
Visit: http://127.0.0.1:8000

📘 6. Explore Interactive Docs (Swagger)

Open in browser:

http://127.0.0.1:8000/docs  # Swagger UI
http://127.0.0.1:8000/redoc # ReDoc

🔍 Usage Examples

GET /items/42 → {"item_id": 42}
GET /models/alexnet → Returns custom message
GET /products/?skip=5&limit=2&category=shoes

🧾 Summary Notes

Feature	Example	Description
Path Param	`/items/{id}`	URL-based dynamic value
Enum Param	`/models/{model_name}`	Restrict value to predefined set
Query Param	`/products/?skip=0&limit=10`	Additional filters, optional/default values
Swagger UI	`/docs`	Interactive API documentation
Virtualenv	`venv\Scripts\activate` / `source venv`	Isolate project dependencies

Absolutely! Here's a well-structured and beginner-to-advanced note on FastAPI Routers including why routers are important, how validation works behind the scenes using Pydantic, and how all of it comes together.

✅ 1. Why Use Routers in FastAPI?

Routers are used to modularize your FastAPI app — just like how controllers work in other frameworks (like Express.js, Django, Laravel).

✅ Benefits:

🔗 Separation of concerns: Separate logic by features (e.g., /products, /users)
📦 Scalability: Easier to manage larger codebases
♻️ Reusability: Routers can be reused and nested
🧪 Testability: Each router can be tested independently
🧼 Clean Code: Keeps your main.py small and readable

🧩 2. What Is a Router in FastAPI?

A router is an instance of APIRouter, where you can define endpoints just like @app.get() but independently.

💡 Example:

# routers/products.py

from fastapi import APIRouter

router = APIRouter()

@router.get("/products")
def get_products():
    return {"msg": "List of products"}

⚙️ 3. How to Register a Router

You register routers in your main app like this:

# app/main.py

from fastapi import FastAPI
from app.routers import products

app = FastAPI()
app.include_router(products.router, prefix="/api/v1", tags=["Products"])

🧠 This will expose the route: /api/v1/products

🔍 4. Behind the Scenes: Validation Using Pydantic

FastAPI uses Pydantic to validate:

✅ Path Parameters

✅ Query Parameters
✅ Request Body (JSON/Post Data)

Example:

from pydantic import BaseModel, Field

class Product(BaseModel):
    name: str = Field(..., min_length=3)
    price: float

Behind the scenes:

When a request hits the endpoint, FastAPI automatically parses and validates the incoming JSON into a Product object.
If data is invalid, FastAPI auto-generates a 422 error response with all validation issues.
You never have to manually try/except invalid input. Pydantic does it for you.

📦 5. Using Pydantic with Routers

# routers/products.py

from fastapi import APIRouter
from app.models.product import Product

router = APIRouter()

@router.post("/products")
def create_product(product: Product):
    return {"product": product}

🧠 When someone sends a POST request with JSON body, it’s automatically parsed and validated.
FastAPI converts it into a Python object (Product).
If fields are missing or invalid, it returns a structured error.

🧾 6. Validation Error Response Format (Auto-generated):

{
  "detail": [
    {
      "loc": ["body", "product", "name"],
      "msg": "field required",
      "type": "value_error.missing"
    }
  ]
}

🔍 7. FastAPI Dependency Injection with Routers (Advanced)

You can pass dependencies into routers for:

Auth middleware
DB connection injection
Role-based access

router = APIRouter(
    prefix="/products",
    tags=["Products"],
    dependencies=[Depends(auth_dependency)],
)

🧠 Summary: Why Routers + Pydantic Matter

Concept	Description
Routers	Help organize routes modularly, cleanly
Pydantic Models	Define data schemas + validation rules for request/response
Auto Validation	FastAPI + Pydantic auto-validate data before your logic runs
Error Handling	Validation errors are returned with proper HTTP status + reason
Docs Integration	Models are reflected in Swagger Docs for each route automatically

🔍 FastAPI Path Parameters

🔐 Predefined Values (Enum)

🔎 Query Parameters

Each section includes:

What it is
How it works
Code examples
Behavior behind the scenes (Pydantic + FastAPI)

📌 1. Path Parameters in FastAPI

✅ What are Path Parameters?

Path parameters are dynamic parts of the URL path that act as variables.

🧠 FastAPI treats them as required parameters and maps them using Python function parameters.

📘 Syntax Example:

from fastapi import FastAPI

app = FastAPI()

@app.get("/users/{user_id}")
def read_user(user_id: int):
    return {"user_id": user_id}

🛠 How it Works:

URL: /users/42
FastAPI converts "42" to an int (based on type hint)
If type mismatch (e.g., /users/abc), FastAPI returns a 422 error automatically.

⚙️ Behind the Scenes:

FastAPI uses:

Python type hints to enforce type
Pydantic to validate the value before function execution
Generates OpenAPI docs automatically with correct parameter types

✅ Optional Parameters?

No. Path parameters must be required.
If optional, you must redesign using query parameters instead.

🎯 2. Predefined Values with Enums (Validation)

✅ What are Predefined Values?

Sometimes, you want to restrict a path parameter to specific allowed values — not anything.

Use Python Enums for this.

🧱 Example:

from enum import Enum
from fastapi import FastAPI

app = FastAPI()

class ModelName(str, Enum):
    alexnet = "alexnet"
    resnet = "resnet"
    lenet = "lenet"

@app.get("/models/{model_name}")
def get_model(model_name: ModelName):
    return {"model_name": model_name}

🌐 Usage:

/models/alexnet ✅
/models/invalid_model ❌ → 422 Validation Error

🔍 Why Use Enum?

Ensures clients use only valid values
Auto-validates input (via Pydantic)
Automatically shows options in Swagger Docs dropdown
Prevents typos and unexpected inputs

⚙️ How it Works:

FastAPI internally maps the path to the Enum.
If not part of the Enum, it returns 422 with a message like:

{
  "detail": [
    {
      "loc": ["path", "model_name"],
      "msg": "value is not a valid enumeration member",
      "type": "type_error.enum"
    }
  ]
}

📥 3. Query Parameters in FastAPI

✅ What are Query Parameters?

Parameters that appear after the ? in a URL.
Used for filtering, sorting, pagination, searching, etc.

📘 Example:

@app.get("/products")
def get_products(skip: int = 0, limit: int = 10, q: str = None):
    return {"skip": skip, "limit": limit, "q": q}

📌 URL: /products?skip=5&limit=20&q=shoes

💡 Key Points:

Not positional like path parameters
Can be optional or have default values
Auto-validated using type hints

🧰 Advanced: Use `Query` for extra validation

from fastapi import Query

@app.get("/search")
def search_items(q: str = Query(..., min_length=3, max_length=50)):
    return {"query": q}

... = required
min_length, max_length, regex supported
Shows up in API docs with proper validation

🤯 Pydantic + Query Magic:

Pydantic validates:

Required/Optional values
Data types (e.g., int, bool, float, etc.)
Value constraints (min/max length, regex)

And FastAPI auto-generates:

Swagger documentation
Interactive query parameter fields
Error responses for invalid queries

📥 Optional Query Parameters

@app.get("/filter")
def filter_items(category: str = Query(None), in_stock: bool = Query(False)):
    return {"category": category, "in_stock": in_stock}

🧠 Summary Table

Feature	Path Parameters	Predefined (Enum)	Query Parameters
URL Format	`/items/{id}`	`/models/{model_name}`	`/products?skip=0&limit=10`
Required	✅ Always	✅ Always	❌ Can be optional
Type Validation	✅ (int, str, etc.)	✅ Only allowed values via Enum	✅ With type hints & Query()
Auto Swagger Docs	✅	✅ Dropdown with values	✅ Shows default and constraints
Common Use Case	Dynamic routes (ID, slug)	Model names, categories, user roles	Filtering, searching, pagination
Validation Method	Pydantic via type hints	Pydantic Enum	Pydantic + fastapi.Query

✅ Default Values

❓ Optional Parameters

📌 Using Both Together in Two Ways

🔍 1. Using Plain Function Parameters

FastAPI automatically treats query parameters with default values as optional.

🧪 Example:

@app.get("/items/")
def read_items(q: str = None, page: int = 1, limit: int = 10):
    return {"q": q, "page": page, "limit": limit}

Parameter	Type	Required?	Default
`q`	`str`	❌ Optional	`None`
`page`	`int`	❌ Optional	`1`
`limit`	`int`	❌ Optional	`10`

🧠 Explanation:

No Query() is used.
All parameters have defaults, so FastAPI treats them as optional.
Validation is still applied via Python types.

🧠 2. Using `Query()` for More Control

Use fastapi.Query() when you want:

To explicitly make a param optional or required
Add default value
Add validation (min, max, regex)

✅ Optional with Default (Best Practice):

from fastapi import Query

@app.get("/search")
def search_products(
    q: str = Query(None, min_length=2, description="Search term"),
    page: int = Query(1, ge=1, description="Page number"),
    limit: int = Query(10, le=100, description="Max results per page")
):
    return {"q": q, "page": page, "limit": limit}

Parameter	Required?	Default	Extras
`q`	❌	`None`	`min_length=2`
`page`	❌	`1`	`ge=1`
`limit`	❌	`10`	`le=100`

❗ Required Query Parameters

Use ... in Query(...) to make it required.

@app.get("/required")
def must_pass(q: str = Query(..., min_length=3)):
    return {"q": q}

⛔ /required → Error
✅ /required?q=abc → OK

✅ Optional with Pydantic `Optional[]` (Alternative)

from typing import Optional
from fastapi import Query

@app.get("/products")
def get_products(q: Optional[str] = Query(None)):
    return {"q": q}

🧾 Final Summary Table

Use Case	Code Example	Required?	Default	Notes
Basic optional param	`q: str = None`	❌	None	Simplest optional param
Required param	`q: str = Query(...)`	✅	None	Must be provided
Optional with default	`page: int = Query(1)`	❌	1	Custom default + validation
Optional with constraints	`q: str = Query(None, min_length=2)`	❌	None	Adds validation
Optional with `Optional[]`	`q: Optional[str] = Query(None)`	❌	None	More explicit for dev clarity

Docker Orchestration with AWS ECS – Full Guide (with Node.js Production Testing & Tips

Darshan Vasani — Sun, 20 Jul 2025 08:18:10 +0000

🚢 Docker Orchestration with AWS ECS – Full Guide (with Node.js Production Testing & Tips)

🧠 What is Docker Orchestration?

Docker Orchestration means managing the lifecycle of containers:

📦 Deploying
🔁 Scaling (up/down)
🚥 Load balancing
🛠️ Updating
💥 Handling failures

Popular Orchestration Tools:

Docker Swarm 🐝
Kubernetes (K8s) ☸️
Amazon ECS 🚀
Amazon EKS (for Kubernetes) ☁️

We'll focus on AWS ECS using Fargate (serverless) and EC2 launch type.

🛠️ Step 1: Setting Up AWS Account

Go to 👉 https://aws.amazon.com/
Sign up for a free tier account (needs credit/debit card 💳).
Enable MFA for security 🔐.
Set region (e.g., us-east-1, ap-south-1) 🌍.
Create an IAM user with AdministratorAccess if not using root.

🧴 Step 2: Setting up Amazon ECR (Elastic Container Registry)

ECR is AWS's private Docker registry.

🪜 Steps:

AWS Console → Search ECR → Create repository 📂
Configure:

Name: my-app
Visibility: Private 🔒
Tag immutability: Enabled ✅
1. Push Docker Image to ECR:

# Authenticate Docker to ECR
aws ecr get-login-password --region us-east-1 | docker login --username AWS --password-stdin <your-account-id>.dkr.ecr.us-east-1.amazonaws.com

# Build and Tag image
docker build -t my-app .

# Tag with ECR repo URI
docker tag my-app:latest <your-account-id>.dkr.ecr.us-east-1.amazonaws.com/my-app:latest

# Push image
docker push <your-account-id>.dkr.ecr.us-east-1.amazonaws.com/my-app:latest

🧰 Step 3: Setting up ECS Cluster

An ECS Cluster is where your containers run.

🪜 Steps:

AWS Console → ECS → Create Cluster
Choose:

Networking only → Fargate
EC2 + Networking → EC2
1. Cluster Name: my-app-cluster
2. Proceed → ECS will create VPC and subnets 🛰️

📝 Step 4: ECS Task Definition Setup

A Task Definition = Docker container blueprint.

👇 Key Components:

Task Role (IAM)
Docker Image from ECR
Port mappings (e.g., 80:3000)
CPU & Memory: 256 CPU, 512 MiB RAM
Log configuration: AWS CloudWatch
Environment variables, secrets 🔐
Health checks: /health or /

🪜 Steps:

ECS → Task Definitions → Create new
Launch Type: Fargate
Add container:

Name: my-app
Image: <ECR Image URL>
Port: 3000
Logging: awslogs, with group: /ecs/my-app

⚖️ Step 5: ECS Service Setup with Load Balancer

The Service handles:

Keeping tasks running
Restarting failed containers
Auto-scaling

🪜 Steps:

ECS → Your Cluster → Create Service
Launch Type: Fargate
Choose Task Definition
Desired tasks: 1 or more
Attach Load Balancer:

ALB → New or existing
Create Target Group → port 3000
Health Check path: /health
Listener on port 80 → forward to Target Group

Enable Auto Scaling (optional)

🧪 Step 6: Testing Our Service (💥 with Node.js Tips!)

✅ Basic Test

Go to EC2 > Load Balancers → Copy DNS URL 🌐
Visit in browser → You should see your app 🚀
Confirm task is running: ECS > Cluster > Tasks
Logs: CloudWatch > /ecs/my-app 📜

✅ Deep Testing (For Node.js Applications)

🔹 1. Check container logs:

aws logs get-log-events \
  --log-group-name "/ecs/my-app" \
  --log-stream-name "<your-log-stream>"

Or via CloudWatch Console.

🔹 2. CURL/HTTP test:

curl http://<load-balancer-dns>/health

✅ Ensure response is 200 OK. If not, ECS will kill and restart your task.

🔹 3. Test environment variables:

Add this in your Node.js app:

console.log('ENV:', process.env.NODE_ENV);

Set "NODE_ENV": "production" in Task Definition.

🔹 4. Debug failing deployments:

Check:

Task status (Stopped?)
View Reason (StoppedReason)
Logs (CloudWatch)
Health Check (endpoint must return 2xx)

🔹 5. Enable ECS Exec:

Run shell commands inside the running container:

aws ecs execute-command \
  --cluster my-app-cluster \
  --task <task-id> \
  --container my-app \
  --interactive \
  --command "/bin/sh"

Requires enabling ECS Exec & permissions.

🧹 Step 7: Clean up Resources (🧽 Avoid Charges!)

✅ Checklist:

Stop ECS Service
Delete Tasks
Delete Load Balancer
Delete Target Groups
Delete ECR Repository (optional)
Delete Cluster
Delete VPC (if created)
Delete CloudWatch logs

aws ecr delete-repository --repository-name my-app --force

⚙️ Manual vs Automatic Orchestration

Feature	Manual	Automatic
Deploy new container 🚀	CLI or Console	CI/CD + ECS Service Updates
Scale app 🔄	You change task count	Auto Scaling based on CPU/Memory/Requests
Monitor and Heal 🩺	Manual restart	ECS restarts crashed tasks
Load Balancing 🌐	Manually configure ELB	ECS auto-registers tasks to Target Groups
Image Updates 🔄	Push new tag and update task def	Use CodePipeline / GitHub Actions + Blue/Green

✅ Real-World Production Tips (💡 Especially for Node.js)

Use .env.production with dotenv and pass via Task Definition
Reverse proxy with NGINX (optional for advanced setups)
Health Check Endpoint (/health): return 200 OK JSON, no DB calls
Use pm2 inside container for better process management (optional)
Avoid console.log in production → Use winston or pino
Enable Structured Logging → Send logs to CloudWatch
Monitor memory & CPU metrics via CloudWatch
Enable ECS Exec to debug running container
Use HTTPS via ACM with Load Balancer
Auto-Deploy via GitHub Actions + AWS CLI or CodePipeline
Set container limits (soft/hard memory) to avoid OOM crashes
Use Secrets Manager for DB/API credentials 🔐
Test locally with docker run -p 3000:3000 my-app before push
Set NODE_ENV=production for optimized performance
Use a lightweight base image like node:18-alpine

🧾 Summary Cheatsheet

Step	Task	Tool/Service
1️⃣	Create AWS Account	AWS Console
2️⃣	Push Image to ECR	ECR, Docker CLI
3️⃣	Setup ECS Cluster	ECS
4️⃣	Define Task Definition	ECS
5️⃣	Create Service + Load Balancer	ECS + ALB
6️⃣	Test Your Application	Load Balancer URL
7️⃣	Cleanup	Console / CLI

🧩 Docker Compose – Full DevOps Power

Darshan Vasani — Sun, 20 Jul 2025 08:14:42 +0000

🧩 Docker Compose – Full DevOps Power

📘 What is Docker Compose?

Docker Compose is a tool for defining and managing multi-container Docker applications using a simple YAML file.

Instead of running docker run commands multiple times, Compose lets you:

✅ Define containers, networks, volumes, and environment variables
✅ Start everything with docker compose up
✅ Manage dependencies, ports, and data easily

⚙️ Basic Syntax of `docker-compose.yml`

version: "3.9"         # Compose file version
services:              # Define all containers here
  service1:            # A named service (container)
    image: nginx       # Use this image
    ports:
      - "8080:80"      # host:container

🧠 Why Use Docker Compose?

Feature 💡	Why It Helps 🚀
🧱 Declarative Setup	All infra defined in one YAML file
🔗 Built-in Networking	Services can talk by name (like DNS)
🧳 Volume Integration	Persistent data made easy
🔄 Auto-Dependency Mgmt	Start db before app, cache, etc.
🔥 Dev & Prod Configs	Easy environment switching

🌐 Networking in Docker Compose

✅ Auto Network Creation:

All services in a Compose file automatically share a custom bridge network with internal DNS!

Service Name	Container DNS Name
`db`	`db`
`redis`	`redis`
`backend`	`backend`

🔧 You can ping other containers by service name.

🔍 Example:

services:
  web:
    build: .
    ports:
      - "3000:3000"
  api:
    image: node:alpine
    depends_on:
      - web

In api, you can make requests like:

fetch("http://web:3000")

🗃️ Volumes in Docker Compose

✅ Define persistent data storage:

volumes:
  mydata:

Then attach to a service:

services:
  db:
    image: postgres
    volumes:
      - mydata:/var/lib/postgresql/data

🔥 Compose creates and manages these volumes for you!

🧠 Understanding Docker Compose Networking + Port Mapping 🔌

🏗️ Sample `docker-compose.yml` Setup:

Your project folder is:

📂 myapp/
  └── docker-compose.yml

🔧 Compose File:

services:
  web:
    build: .
    ports:
      - "8000:8000"

  db:
    image: postgres
    ports:
      - "8001:5432"

🚀 What Happens When You Run:

docker compose up

✅ Docker Compose Automatically Does:

🔧 Action	💬 What Happens
🧱 Creates a network	Named `myapp_default` (based on folder name)
📦 Launches web container	Joins `myapp_default` as `web`
📦 Launches db container	Joins `myapp_default` as `db`
🧠 Enables DNS lookup	`web` can reach `db` by hostname `db`

🌐 Internal Networking (Container ↔ Container)

✅ Inside the web container, your app can connect to the database like this:

postgres://db:5432

🧠 Why? Because:

Docker provides internal DNS to resolve service names
The port 5432 is the internal (container) port, exposed by the Postgres container

🌍 External Networking (Host ↔ Container)

You’ve mapped container ports to host ports like this:

  web:
    ports:
      - "8000:8000"     # Host: 8000 → Container: 8000

  db:
    ports:
      - "8001:5432"     # Host: 8001 → Container: 5432

So from your host machine, you can:

Access web on 👉 http://localhost:8000
Connect to Postgres on 👉 postgres://localhost:8001

🧠 Important Concept: `HOST_PORT:CONTAINER_PORT`

Concept	Example	Meaning
`HOST_PORT`	`8001`	Port on your machine
`CONTAINER_PORT`	`5432`	Port inside the container
Mapping	`8001:5432`	Requests to `localhost:8001` go to Postgres in container on port `5432`

🧩 Real World Analogy

🧳 Think of containers as hotel rooms.

Each has its own room number (container port)
The front desk (your host machine) assigns a guest-access number (host port)

So:

5432 = actual DB server inside room
8001 = external phone number to reach that room from outside

🔒 Internal vs External Communication Recap

Context	URL Format	Who uses it?
🔁 Container-to-container	`postgres://db:5432`	Inside Docker network
🌍 Host-to-container	`postgres://localhost:8001`	From your laptop / browser

✅ Internal comms use service names + container port
✅ External comms use localhost + host port

📌 Final Notes

Docker Compose auto-creates a network (unless you override it)
You don’t need to expose ports unless you want outside access
Use internal ports (CONTAINER_PORT) when services talk to each other
Expose only the ports you need to keep things secure 🔐

🧪 Bonus Tip: Inspect the Compose Network

docker network inspect myapp_default

This will show:

Containers in the network
Their IPs
Connection metadata

🌐 Docker Compose Advanced Networking – A Complete Guide

🚀 1. Multi-Host Networking via Overlay (Swarm Mode)

💡 Overlay networking allows containers on different Docker hosts to communicate — as if they were on the same network!

🔌 Use Case:

✅ Deploying a multi-host microservice system
✅ Need backend containers on host A to talk to database on host B

🛠️ How It Works:

Requires Swarm mode (docker swarm init)
Docker uses overlay driver to create a virtual network across machines
Compose uses this with no special setup, if Swarm mode is active

🧪 Example:

networks:
  my_overlay:
    driver: overlay

Then attach to services:

services:
  web:
    networks:
      - my_overlay
  db:
    networks:
      - my_overlay

✅ Compose will automatically connect services to the multi-host overlay network

🧠 Overlay vs Bridge (Single Host Only)

Feature	`bridge` (default)	`overlay` (Swarm)
Host Limit	1 machine	Multiple machines
Use case	Local dev	Distributed apps
DNS-based discovery	✅ Yes	✅ Yes
Needs Swarm mode?	❌ No	✅ Yes

🧱 2. Custom Networks in Compose

You’re not limited to just the default network. You can define and connect containers to specific networks using the networks: key.

🧪 Example Topology

services:
  proxy:
    build: ./proxy
    networks:
      - frontend

  app:
    build: ./app
    networks:
      - frontend
      - backend

  db:
    image: postgres
    networks:
      - backend

🧭 Network Layout

frontend network:
  - proxy
  - app

backend network:
  - app
  - db

proxy 🔁 app 🔁 db

✅ proxy can't see db, but app can talk to both
✅ Great for enforcing security boundaries 🔐

🔌 3. Custom Network Drivers & Options

networks:
  frontend:
    driver: bridge
    driver_opts:
      com.docker.network.bridge.host_binding_ipv4: "127.0.0.1"

  backend:
    driver: custom-driver

bridge: Standard Docker single-host network
custom-driver: Plug in advanced/third-party networking solutions (e.g., macvlan, overlay, weave)

📛 4. Rename Docker Network (Custom Name)

networks:
  frontend:
    name: custom_frontend
    driver: bridge

🔁 Instead of projectname_frontend, Docker will create custom_frontend.

✅ Useful in CI/CD or pre-defined environments.

🧮 5. Assigning Static IPs in Compose

Use ipv4_address under the service's network attachment.

services:
  web:
    networks:
      app_net:
        ipv4_address: 172.28.0.4

networks:
  app_net:
    driver: bridge
    ipam:
      config:
        - subnet: 172.28.0.0/16

✅ Be careful — misconfiguring subnets or overlapping IPs can break your network.

⚙️ 6. Customize the `default` Network

You can override the default Compose-generated network like this:

services:
  web:
    build: .
    ports:
      - "8000:8000"

networks:
  default:
    driver: custom-driver-1

✅ Still lets you use default behavior, but with custom driver/settings.

🌉 7. Use a Pre-Existing Docker Network

Want to connect to a network created outside Compose? Use external: true

networks:
  network1:
    name: my-pre-existing-network
    external: true

🎯 Compose will connect to the existing network, not recreate it.

🔁 Helpful when:

Using shared infra (like reverse proxies)
Reusing CI/CD networking
Linking across Compose projects

🧾 Final Cheatsheet

Feature	Keyword	Description
Multi-host networking	`overlay` driver	Works with Swarm
Isolated networks	`networks:` per service	Scoped communication
Custom drivers	`driver: bridge`	Control behavior
Rename network	`name:`	Use meaningful names
Static IPs	`ipv4_address`	Predictable networking
Pre-existing networks	`external: true`	Connect to outside network
Customize default	`networks: default`	Override auto network

🧠 Final Takeaways

✅ Compose makes networking declarative, secure, and powerful
✅ Use custom networks to enforce boundaries and structure
✅ Use overlay for multi-host magic
✅ Use external to plug into existing infra
✅ DNS-based discovery simplifies microservice URLs

🏗️ Custom Docker Builds in Compose

Use your own `Dockerfile` with build context:

backend:
  build:
    context: .                 # current folder
    dockerfile: Dockerfile     # name of your Dockerfile
  ports:
    - "8000:8000"

🧠 Docker Compose will build the image before running the service.

🧪 Compose vs Manual Docker Commands

Task	Docker CLI	Docker Compose
Start container	`docker run`	`docker compose up`
Stop container	`docker stop`	`docker compose down`
Rebuild image	`docker build`	`docker compose build`
View logs	`docker logs`	`docker compose logs`

🧱 Full Project Example: E-Commerce Stack

# 📦 Project Name
name: e-commerce

services:
  # 🔧 Backend Service
  backend:
    build:
      context: .                 # Use current directory as build context
      dockerfile: Dockerfile    # Dockerfile to build the backend image
    container_name: backend     # Name of the backend container
    ports:
      - "8000:8000"             # Map host:container port
    depends_on:
      - db                      # Start db first
      - redis                   # Start redis first

  # 🗃️ PostgreSQL Database Service
  db:
    image: postgres:16          # Latest stable PostgreSQL
    container_name: postgres
    environment:
      POSTGRES_USER: postgres
      POSTGRES_PASSWORD: postgres
      POSTGRES_DB: postgres
    volumes:
      - postgres_data:/var/lib/postgresql/data

  # ⚡ Redis In-Memory Data Store
  redis:
    image: redis:7-alpine       # Lightweight Redis image
    container_name: redis
    volumes:
      - redis_data:/data

# 🗃️ Named Volumes for Persistent Storage
volumes:
  postgres_data:
  redis_data:

🔥 How It Works:

backend is built from the local Dockerfile
db & redis use official images
Volumes persist data between restarts
All services can talk to each other via names (backend, db, redis)

✅ Start Everything:

docker compose up --build

⛔ Stop & Clean Everything:

docker compose down -v

🧰 Pro Tips

Tip 💡	Description
`depends_on`	Control boot order of containers (not readiness!)
`volumes:`	Use named volumes for clean reuse & backups
`env_file:`	Load `.env` for cleaner configuration
`profiles:`	Enable or disable services dynamically
`networks:`	Customize default networks if needed

🎓 Final Summary

Feature	Compose Benefit 🚀
Networking	🔗 Name-based access between containers
Volumes	📦 Persistent storage, managed cleanly
Builds	🛠️ Custom image building from source
Automation	🧠 Multi-container orchestration made easy
Reusability	♻️ YAML can be reused across environments

🧩 Example: Internal-only Docker Compose Network (No Exposed Ports) 🐳

✅ Scenario:

You’re building a simple backend + database + Redis stack that:

Does not need to be accessed from the host/browser
Only needs container-to-container communication
Should remain internal and secure (no ports: exposed)

📦 `docker-compose.yml` (No Ports Exposed)

version: "3.9"

services:
  # 🔧 Backend API
  backend:
    build:
      context: .
    container_name: backend
    depends_on:
      - db
      - redis
    environment:
      DB_HOST: db
      REDIS_HOST: redis

  # 🗃️ PostgreSQL Database
  db:
    image: postgres:16
    container_name: postgres
    environment:
      POSTGRES_USER: postgres
      POSTGRES_PASSWORD: postgres
      POSTGRES_DB: postgres
    volumes:
      - postgres_data:/var/lib/postgresql/data

  # ⚡ Redis Cache
  redis:
    image: redis:7-alpine
    container_name: redis
    volumes:
      - redis_data:/data

volumes:
  postgres_data:
  redis_data:

🧠 Key Points

💡 Concept	✅ Benefit
❌ No `ports:` field	Nothing is exposed to host machine
✅ Internal network	Docker Compose auto-creates a shared bridge
🔗 DNS by service name	`backend` can talk to `db`, `redis` via name
🔐 Security	Fully isolated, no public entry points

💬 How Services Communicate

Inside backend container, you can do:

// Node.js example (env used above)
const pg = require('pg');
const redis = require('redis');

const db = new pg.Client({
  host: process.env.DB_HOST, // "db"
  user: 'postgres',
  password: 'postgres',
  database: 'postgres'
});

const redisClient = redis.createClient({
  url: 'redis://redis:6379'
});

✅ Works seamlessly, because Docker Compose provides built-in DNS resolution for service names.

🚀 Run the Stack:

docker compose up --build

👀 Nothing exposed outside, but all services talk inside.
Want to debug? Use:

docker exec -it backend sh

🧪 When to Use This Pattern

Use Case	Why it Fits
🧱 Internal APIs/microservices	No external access needed
👷 Workers/CRON jobs	Runs in background only
🔐 Security-sensitive apps	Reduce attack surface
🧪 Local-only testing	Don't expose unnecessary ports

🔐 Final Tip

If you later need external access (e.g., for testing):
Just add a single port to the backend:

ports:
  - "8000:8000"

But for private, secure, container-to-container apps — no ports is cleanest. ✅

🛠️ Docker Compose – Custom Docker Builds (Full Guide) 🐳

🧩 Overview

Docker Compose allows two primary ways to set up containers:

Method	Keyword	Use Case
🛠️ Build locally from Dockerfile	`build:`	During development
☁️ Pull prebuilt image	`image:`	CI/CD & production

✅ 1. Local Dockerfile Build – Using `build:`

When you want to build your Docker image directly from your source code:

services:
  backend:
    build:
      context: .                  # 📁 Location of source code (root of app)
      dockerfile: Dockerfile      # 📝 File to use (default is 'Dockerfile')
    ports:
      - "8000:8000"
    container_name: backend

📁 Folder Structure:

myapp/
├── backend/
│   ├── Dockerfile
│   ├── server.js
│   └── package.json
└── docker-compose.yml

🧠 Explanation:

Field	Meaning
`context:`	Folder Docker will send to the daemon for the build
`dockerfile:`	Custom name or path to Dockerfile
`build:`	Triggers a local build when you run `docker compose up --build`

🔁 You don’t push/pull — just edit → build → run locally:

docker compose up --build

☁️ 2. Pull Image from Registry – Using `image:`

In production or CI/CD, it’s better to pull prebuilt, versioned images from Docker Hub or GitHub Container Registry.

services:
  backend:
    image: dpvasani56/myapp-backend:v1.0.3
    ports:
      - "8000:8000"

✅ Works only if you've pushed this image earlier:

docker build -t dpvasani56/myapp-backend:v1.0.3 .
docker push dpvasani56/myapp-backend:v1.0.3

🧠 This is great when:

You want reproducible builds ✅
You deploy to servers that don’t have your source code ✅
You want fast CI/CD ✅

🔁 Both Build & Image (Hybrid)

services:
  backend:
    image: dpvasani56/myapp-backend:latest
    build:
      context: ./backend
      dockerfile: Dockerfile

💡 In this case:

Local build happens first
Image is tagged and stored locally as dpvasani56/myapp-backend:latest
Useful for building locally but keeping consistent image tags

🧪 Real Example: Backend + Frontend + DB

services:
  backend:
    build:
      context: ./backend
      dockerfile: Dockerfile
    image: dpvasani56/app-backend:dev
    ports:
      - "8000:8000"

  frontend:
    image: dpvasani56/app-frontend:latest   # Pulled from Docker Hub

  db:
    image: postgres:16
    volumes:
      - pgdata:/var/lib/postgresql/data

volumes:
  pgdata:

📤 Push/Deploy Workflow

Step	Action
1️⃣	Build locally: `docker build -t dpvasani56/app-backend:dev .`
2️⃣	Push to registry: `docker push dpvasani56/app-backend:dev`
3️⃣	On server: `docker compose pull && docker compose up -d`

✅ Easy deployment
✅ No source code leak
✅ Fast start time

🧱 Custom Dockerfile Path or Name

build:
  context: ./src
  dockerfile: Dockerfile.prod

👉 You can place your Dockerfile anywhere and name it however you want.

🧠 Best Practices

Stage	Use `build:`	Use `image:`
Development 👨‍💻	✅ Yes	❌ Optional
Production 🚀	❌ Avoid	✅ Required
CI/CD 🧪	✅ Build & Push	✅ Pull
Collaboration 🧑‍🤝‍🧑	✅ Share Compose file	✅ Share tagged image

🗃️ Full Compose File with Both Options

version: "3.9"

services:
  backend:
    build:
      context: ./backend
      dockerfile: Dockerfile
    image: dpvasani56/my-backend:latest
    ports:
      - "8000:8000"

  frontend:
    image: dpvasani56/my-frontend:latest
    ports:
      - "3000:3000"

  postgres:
    image: postgres:16
    environment:
      POSTGRES_USER: postgres
      POSTGRES_PASSWORD: password
    volumes:
      - pgdata:/var/lib/postgresql/data

volumes:
  pgdata:

🧾 Summary Cheatsheet

Field	Use When?	Notes
`build:`	You have Dockerfile locally	Great for dev
`image:`	You push/pull from registry	Ideal for prod
`build` + `image`	Build locally with version tag	Best of both worlds
`context`	Define source code directory	Defaults to `.`
`dockerfile`	Use custom name/location	Optional

📦 Docker Volumes – A2Z Storage Guide 🐳

Darshan Vasani — Sun, 20 Jul 2025 08:13:39 +0000

📦 Docker Volumes – A2Z Storage Guide 🐳

🔍 What is a Docker Volume?

A Docker volume is a persistent storage mechanism managed by Docker outside the container filesystem.

✅ Volumes survive container restarts
✅ Volumes are managed by Docker
✅ They’re perfect for storing databases, logs, user uploads, and config files

📁 Real-World Analogy:

🧳 Think of a volume as a USB stick plugged into your container.

You can eject the container 💣
The USB (volume) still has all your files 💾

🧠 Why Use Volumes?

✅ Benefit	💬 Why It’s Awesome
Persistent Storage	Data stays even after container dies
Decoupled	Separate from container logic
Shared Access	Mount same volume into multiple containers
Backup Friendly	Easy to archive/export
Safe from image rebuilds	Won't be deleted accidentally

🔌 Types of Docker Volume Mounts

Type	Syntax Example	Use Case
🐳 Named Volume	`-v my-volume:/app/data`	Default, managed by Docker
🗂️ Host Bind	`-v /host/folder:/container/folder`	Use host machine's file system
🧪 Anonymous	`-v /app/data`	Randomly named, temporary

🔧 1. Using a Named Volume

docker volume create mydata

docker run -d \
  --name db \
  -v mydata:/var/lib/mysql \
  mysql

✅ The data is stored in:

/var/lib/docker/volumes/mydata/_data

🗂️ 2. Attaching Host Folders (Bind Mounts)

docker run -d \
  --name webapp \
  -v /home/user/project:/usr/src/app \
  node:alpine

✅ Mounts a host folder directly inside the container.

⚠️ Bind mounts are powerful but risk exposing sensitive host files if misused.

🔥 Differences: Volume vs Bind Mount

Feature	Volume (Docker-managed)	Bind Mount (Host folder)
Managed by Docker	✅ Yes	❌ No
Host portability	✅ Portable	❌ Host-specific
Data safety	✅ Isolated	❌ Depends on host path
Security	✅ Better	⚠️ Potentially risky

🔁 3. Share Volume Between Multiple Containers

🧪 Example:

docker volume create shared-data

docker run -d --name writer \
  -v shared-data:/data \
  busybox sh -c "echo hello > /data/file.txt && sleep 9999"

docker run --rm --name reader \
  -v shared-data:/data \
  busybox cat /data/file.txt

📦 Both writer and reader share the same volume.

🔂 Read-only Volume Mount

Prevent writing:

-v mydata:/app/data:ro

✅ Makes volume read-only inside the container!

📦 Volume Lifecycle Commands

🔧 Command	💬 What It Does
`docker volume create <name>`	Create a volume
`docker volume ls`	List all volumes
`docker volume inspect <name>`	View volume details
`docker volume rm <name>`	Delete volume
`docker volume prune`	Delete all unused volumes

🧪 Using Volumes in `docker-compose.yml`

version: "3.9"
services:
  db:
    image: postgres
    volumes:
      - pgdata:/var/lib/postgresql/data

volumes:
  pgdata:

✅ Docker creates & manages the pgdata volume

🧠 Volume Naming Tip

Named volumes persist and can be reused by name
Anonymous volumes are created automatically and can be hard to track

docker run -v /data nginx    # Anonymous volume
docker run -v myvol:/data nginx  # Named volume ✅

📤 Backing Up & Restoring Volumes

🧳 Backup:

docker run --rm \
  -v myvolume:/data \
  -v $(pwd):/backup \
  busybox tar czvf /backup/backup.tar.gz /data

♻️ Restore:

docker run --rm \
  -v myvolume:/data \
  -v $(pwd):/backup \
  busybox tar xzvf /backup/backup.tar.gz -C /

🚨 Volume Gotchas to Avoid

⚠️ Mistake	💥 Problem
Not naming volumes	Hard to manage & reuse
Mixing bind mount with sensitive paths	Potential host damage
Forgetting to prune	Unused volumes pile up
Overwriting app folder with empty volume	App might not start!

🧾 Summary Table

Type	Managed	Persistent	Use Case
🐳 Named Volume	Docker	✅	Safe default, backups
🗂️ Bind Mount	Host	✅	Mount local code
🧪 Anonymous Volume	Docker	⚠️ Temporary	Quick test, not tracked

✅ Final Takeaways

📦 Volumes are best practice for persistent, portable storage.
💡 Use named volumes for databases, uploads, logs.
⚙️ Use bind mounts for local dev.
🧠 Share volumes to enable inter-container communication via files.
🧽 Clean up with docker volume prune.

🌐 Docker Networking A2Z – Masterclass for Developers & DevOps

Darshan Vasani — Sun, 20 Jul 2025 08:12:11 +0000

🌐 Docker Networking A2Z – Masterclass for Developers & DevOps

📦 What is Docker Networking?

🧠 Docker networking allows containers to communicate with:

Each other 📞
The host machine 🖥️
The external internet 🌍

Docker automatically creates networks and connects containers based on mode.

🧠 Key Terms

Term	Meaning	Emoji
Network	Virtual connection b/w containers	🛣️
Bridge	Default, isolated internal network	🌉
Host	Shares host’s network stack	🏠
None	No network access	🚫
Overlay	Cross-host communication (Swarm)	🕸️

🌉 Bridge Mode (Default)

🧱 Bridge network is like a private switch where containers talk to each other.

🧵 Created Automatically:

docker network ls

Look for: bridge

🛠 How it works:

Containers get private IPs (like 172.17.0.x)
They can access the internet via NAT
But cannot be accessed from outside without -p port mapping

🧪 Try it:

docker run -d --name container1 nginx
docker run -d --name container2 busybox sleep 9999

# Ping container1 from container2 by IP
docker exec -it container2 ping 172.17.0.x

❌ By default, they can’t talk by name unless in custom network

🧱 Custom Bridge Network (Recommended)

🎯 Custom networks support container name resolution (DNS)!

📦 Create a custom bridge:

docker network create my-network

🚀 Launch containers into it:

docker run -d --name app1 --network my-network nginx
docker run -it --name app2 --network my-network busybox sh

Now, inside app2:

ping app1

✅ Works! 🎉 Containers can ping by name!

✅ Why Use Custom Bridge?

Feature	Benefit
🧠 DNS	Resolve container names
🔐 Isolation	Only containers in same network can talk
🔄 Flexibility	Use multiple networks
⚙️ Control	Inspect with `docker network inspect`

🏠 Host Network Mode

Shares the host’s network stack directly.

🚀 Use:

docker run --network host nginx

✅ Pros:

🔥 Super fast — no NAT or port mapping
🧪 Useful for monitoring tools (Prometheus, Grafana)

❌ Cons:

⚠️ No isolation
🧱 Cannot run 2 containers on same port!

🚫 None Network Mode

Container has no networking at all.

docker run --network none busybox

🔒 Fully isolated
Useful for security testing or offline compute jobs

🌐 Overlay Network (Advanced – Docker Swarm)

🕸️ Enables containers on different hosts to communicate

Use Case:

Docker Swarm
Distributed Microservices

docker network create --driver overlay my-overlay

Requires Swarm mode:

docker swarm init

🔌 Connect Containers to Multiple Networks

docker network create frontend
docker network create backend

docker run -d --name api \
  --network frontend \
  --network-alias api \
  nginx

Then attach to another network:

docker network connect backend api

🧠 Inspect a Network

docker network inspect my-network

Shows:

Container list
IPs
Aliases
Subnets

🧰 CLI Recap

Command	Purpose
`docker network ls`	List networks
`docker network create <name>`	Create a custom network
`docker run --network <name>`	Connect to specific network
`docker network inspect <name>`	Inspect config and members
`docker network rm <name>`	Delete a network
`docker network connect`	Connect running container
`docker network disconnect`	Disconnect container

🎯 Best Practices for Docker Networking

Practice	Why It’s Great
✅ Use custom bridge networks	Enable name resolution + isolation
🚫 Avoid host mode unless needed	Can expose host stack
🧱 Use none mode for compute-only jobs	Maximum isolation
🔐 Limit network access	Avoid connecting everything together
🔍 Use `inspect` to debug IPs	Know who talks to whom
🧪 Use `busybox` or `alpine` to test ping	Lightweight network testing tools

🧾 Summary Table

Mode	Isolated?	Can Use DNS?	Host Access?	Notes
`bridge`	✅ Yes	❌ No (unless custom)	✅ via `-p`	Default
`custom bridge`	✅ Yes	✅ Yes	✅ via `-p`	Best for local
`host`	❌ No	✅ Yes	Direct	No port mapping
`none`	✅ Yes	❌ No	❌ No	For isolation
`overlay`	✅ Cross-host	✅ Yes	Swarm only	For multi-node

🧠 Final Analogy

Bridge network = Private Wi-Fi router 🌐
Custom bridge = Guest Wi-Fi with name tags 🏷️
Host mode = Ethernet cable directly into host 💻
None mode = Airplane mode ✈️
Overlay = Corporate VPN connecting multiple offices 🏢🏢

🌐 Docker Networking Overview

📦 Container networking is the foundation of container communication. Every container is equipped with a network interface, IP address, routing table, DNS config, etc.

✅ By default, containers can:

Make outbound connections (internet)
Be connected to default or custom networks
Be isolated or exposed, depending on the configuration

🚦 Types of Docker Network Drivers

Network Driver	Description	Use Case
`bridge`	🧱 Default isolated network for single-host	Local development
`host`	🏠 Shares host’s network namespace	Low-latency, host-level apps
`none`	🚫 No networking at all	Offline or compute-only tasks
`overlay`	🌍 Enables multi-host (Swarm) communication	Distributed systems
`macvlan`	🪪 Assigns MAC + IP from host’s network	IoT, legacy systems
`ipvlan`	Similar to macvlan, IP only	More control over routing
`custom plugins`	🛠 Extendable third-party solutions	SDN, advanced use

🧱 Default Bridge Network vs 🧩 User-Defined Bridge Network

Both are based on the bridge driver, but they differ significantly in features & behavior.

🧱 Default `bridge` Network

Created automatically by Docker when installed.

docker network ls
# OUTPUT will contain:
# bridge    bridge    local

Example:

docker run -d --name app1 nginx
docker run -d --name app2 busybox sleep 999

📛 Limitations:

❌ Limitation	Explanation
🚫 No DNS	Can't resolve container names
📵 Not isolated	All containers using default bridge are technically on the same LAN
🔄 Poor discoverability	Need to link manually or use IPs
🔓 Less secure	Containers can talk across networks by default if not isolated

🧩 User-Defined Bridge Network

Created with:

docker network create my-custom-net

✅ Advantages:

✅ Feature	Benefit
🧠 Built-in DNS	Containers can resolve each other by name
🔒 Isolated environment	Containers only talk to others on the same network
🔄 Auto service discovery	Works like microservices
🔧 Fine-grained control	Inspect, attach, detach easily
📂 Easier multi-container orchestration	Compose, Swarm, or manually

⚖️ Comparison: Default vs User-Defined Bridge

Feature	Default `bridge` 🌉	User-Defined Bridge 🧩
DNS support	❌ No	✅ Yes
Service name resolution	❌ No	✅ Yes
Isolation	❌ Less secure	✅ Scoped per network
Compose support	🚫 Not recommended	✅ Fully supported
Security	Basic	Scoped & controlled
Container-to-container name access	❌ Only via IP	✅ Via name (`ping app1`)
Preferred for production/dev	❌ No	✅ Yes

🔍 Inspecting Networks

docker network inspect my-custom-net

📊 Output includes:

Subnet
Gateway
Connected containers
DNS aliases

🧪 Example Test

1. Default Bridge (no DNS):

docker run -d --name alpha nginx
docker run -it --rm busybox
# ping alpha – ❌ fails

2. User-defined Bridge:

docker network create testnet
docker run -d --name alpha --network testnet nginx
docker run -it --rm --network testnet busybox
# ping alpha – ✅ works

🧱 Other Drivers – Quick Overview

Driver	Emoji	Key Use
`host`	🏠	High-perf apps (no NAT), not isolated
`none`	🚫	Secure offline or processing containers
`overlay`	🌍	Multi-host Swarm networking
`macvlan`	🪪	Assign physical IPs from host’s LAN
`ipvlan`	🧭	Fine-grained routing control
`custom plugin`	🧰	CNI integrations, SDNs (like Calico)

🔐 When to Use What?

Use Case	Best Network Driver
Local dev, isolated apps	🧩 User-defined bridge
Multi-container orchestration	🧩 Custom bridge + Docker Compose
High-speed, low-latency app (e.g. Prometheus)	🏠 Host network
No internet access container	🚫 None
Containers across multiple hosts	🌍 Overlay (Swarm mode)
Assign IPs from LAN for legacy systems	🪪 Macvlan

🎯 Key Docker Networking Commands Cheat Sheet 🐳

🔧 Command	💬 Description
`docker network ls`	📜 List all available networks
`docker network create <name>`	🛠️ Create a custom bridge network
`docker network rm <name>`	🗑️ Remove a network
`docker network inspect <name>`	🔍 View detailed info about a network
`docker run --network <name> <image>`	🚀 Create container in a specific network (unnamed)
`docker run -d --name <container> --network <network> <image>`	✅ Create named container in a custom network
`docker network connect <network> <container>`	🔗 Attach an existing container to a network
`docker network disconnect <network> <container>`	❌ Detach a container from a network

✅ Real-World Example: Create and Use a Custom Bridge Network

# 1️⃣ Create a custom bridge network
docker network create my-bridge

# 2️⃣ Run a container (nginx) attached to that network
docker run -d --name my-app --network my-bridge nginx

# 3️⃣ Run another container (busybox) in the same network
docker run -it --name client --network my-bridge busybox

# 4️⃣ Inside 'client', you can ping 'my-app' by name
ping my-app

🎉 Result: client can resolve and communicate with my-app using container name thanks to Docker’s internal DNS in custom bridge networks.

🧠 Bonus Tip: Disconnect & Reconnect

docker network disconnect my-bridge my-app    # Disconnect from network
docker network connect my-bridge my-app       # Reconnect to same or new network

🧠 Final Takeaways

🧱 Default bridge is basic → no name resolution, low security
🧩 User-defined bridge is preferred for real-world apps
🌍 Use overlay for distributed microservices
🧰 Know when to use each driver to optimize performance & security
🧪 Always test communication with tools like ping, curl, netcat

🌐 Docker Networking Logic – Container Communication

✅ 1. User-Defined Bridge Network = Container DNS Heaven 🧠

🧩 When you create a user-defined bridge network, Docker automatically enables an internal DNS service.

📦 That means containers can talk to each other by name! 🎉

Example:

docker network create my-net

docker run -d --name db --network my-net mongo
docker run -d --name web --network my-net node-app

🎯 Now, inside web, you can:

ping db

✅ Boom! It works because Docker auto-resolves db → container's IP inside the same network.

❌ 2. Default Bridge Network = Only IP-Based Access 🔢

Containers in the default bridge cannot resolve each other by name.

Example:

docker run -d --name app1 nginx       # default bridge
docker run -it --name app2 busybox    # default bridge

Inside app2:

ping app1    # ❌ FAILS

Why? Because DNS resolution doesn’t work in the default bridge without using the old --link option (now deprecated 🚫).

📛 Legacy `--link` (Avoid Using It)

docker run -d --name db mongo
docker run -d --name web --link db node-app

⚠️ Works — but deprecated & removed in newer Docker versions.

🔄 3. Containers in Different Networks = 🚫 No Communication

Example:

docker network create net-a
docker network create net-b

docker run -d --name app1 --network net-a nginx
docker run -d --name app2 --network net-b busybox

Inside app2:

ping app1    # ❌ FAILS – isolated networks

🧱 Networks are isolated by default. Containers on different bridge networks cannot talk to each other unless you connect one container to both using:

docker network connect net-a app2

Now, app2 belongs to both networks!

🧠 Real-World Analogy

Concept	Analogy
🧱 Default Bridge	Talking in a crowd with no names, only IPs (👤192.168.x.x)
🧩 User-defined Bridge	Talking in a chatroom where everyone has a username (📛 @web, @db)
🔌 Multi-Network	Having one foot in two rooms 🦶🏽🚪

🧪 Recap – Container Communication Matrix

Scenario	Communicate by name?	Communicate by IP?	Notes
Same user-defined bridge	✅ Yes	✅ Yes	Best practice
Same default bridge	❌ No	✅ Yes	No name resolution
Different networks	❌ No	❌ No	Unless manually connected
With `--link` (legacy)	⚠️ Yes	✅ Yes	Deprecated, avoid

✅ Summary

🧩 User-defined networks allow name-based communication via Docker's built-in DNS.
🧱 Default bridge networks don't support DNS — only IPs.
🔐 Each network is isolated — containers inside a network can talk, but can't reach containers in other networks unless manually connected.
💡 Use docker network connect to join a container to multiple networks if needed.

🌐 Overview of Advanced Docker Network Drivers

Driver	Purpose	Host-to-Container	Container-to-Host	Cross-Host Support
`overlay`	Cross-host container communication (Swarm)	✅ Yes	✅ Yes	✅ Yes
`macvlan`	Assign real MAC & IP from LAN to container	❌ No (by default)	✅ Yes	❌ No
`ipvlan`	IP-level control without creating MAC addresses	✅ Yes	✅ Yes	❌ No

1️⃣ 🌍 Overlay Network

🔍 What It Is:

Allows containers on different Docker hosts to securely communicate as if they were on the same LAN.

💡 Requires Docker Swarm (or other orchestrators).
Creates an encrypted VXLAN tunnel between hosts.

🧠 Real-world Analogy:

Like a VPN that connects branch offices (containers) across cities (hosts).

✅ Use Cases:

Multi-host microservices
Docker Swarm services
HA + distributed architecture

🚀 How to Use (with Swarm):

# 1. Initialize Swarm
docker swarm init

# 2. Create overlay network
docker network create --driver overlay my-overlay

# 3. Deploy service to use overlay
docker service create \
  --name webapp \
  --replicas 3 \
  --network my-overlay \
  nginx

🔐 Key Features:

Feature	Benefit
🔒 Encrypted traffic	Uses IPSEC tunneling
🔄 Auto service discovery	Works across nodes
📦 Built-in load balancing	Container-to-service routing
🔧 Works with Docker Compose (v3+)	Great for multi-host orchestration

2️⃣ 🪪 Macvlan Network

🔍 What It Is:

Allows containers to appear as physical devices on the host’s network, each with their own IP and MAC.

💡 The container bypasses Docker’s NAT, appearing directly on your LAN.

🧠 Real-world Analogy:

Like plugging a new computer (container) directly into your office switch with its own IP.

✅ Use Cases:

Legacy apps that require static IPs or MACs
IoT, embedded, or bare metal simulation
When containers must be reachable from the LAN directly

🚀 How to Use:

# 1. Create macvlan network
docker network create -d macvlan \
  --subnet=192.168.1.0/24 \
  --gateway=192.168.1.1 \
  -o parent=eth0 \
  macvlan-net

# 2. Run a container on that network
docker run -d --name myrouter \
  --network macvlan-net \
  busybox sleep 3600

📌 parent=eth0: your physical host’s network interface

⚠️ Limitations:

❌ Limitation	Description
🚫 No container-to-host	Can't ping container from host by default
🔒 Can bypass Docker security stack	Be cautious in shared infra
🌍 Requires IP address planning	Must avoid IP conflicts

🛠 Tip to Enable Host ↔ Container Communication (Workaround):

Create a dummy interface on the host:

ip link add macvlan-shim link eth0 type macvlan mode bridge
ip addr add 192.168.1.200/24 dev macvlan-shim
ip link set macvlan-shim up

3️⃣ 🧬 IPvlan Network

🔍 What It Is:

Similar to macvlan, but no extra MACs per container.
All containers share host’s MAC, just get different IPs.

⚙️ More compatible with cloud and DHCP setups where duplicate MACs are not allowed.

🧠 Analogy:

Multiple workers using one ID card (MAC) but different phone numbers (IP).

✅ Use Cases:

Performance-sensitive systems
Cloud infra with MAC restrictions
Advanced network routing with minimal overhead

🚀 How to Use:

docker network create -d ipvlan \
  --subnet=192.168.1.0/24 \
  --gateway=192.168.1.1 \
  -o parent=eth0 \
  ipvlan-net

docker run -it --rm --network ipvlan-net alpine

🧪 IPvlan supports two modes:

l2: Same subnet, like macvlan
l3: Different subnet, routing via host

✅ Benefits:

Feature	Benefit
🧠 Efficient	No MAC duplication
🔒 More predictable IP rules	Good for secured environments
🧰 Custom routing support	Great for advanced network setups

📊 Advanced Network Drivers Comparison Table

Feature/Driver	`overlay` 🌍	`macvlan` 🪪	`ipvlan` 🧬
Cross-host support	✅	❌	❌
Requires Swarm?	✅	❌	❌
Uses physical IP/MAC	❌	✅	✅ (IP only)
Host ↔ container communication	✅	❌ (manual fix)	✅ (limited)
Best for	Microservices on multiple hosts	LAN-level communication	Custom IP routing or cloud infra
Security model	Swarm-controlled	Exposes real IP on LAN	More controlled than macvlan
Complexity	🟡 Medium	🔴 High	🟠 Medium-High

🔐 Security Considerations

Driver	Security Tip
Overlay	Isolated per service; enable encryption
Macvlan	Bypasses Docker’s firewall — isolate via VLAN
IPvlan	Good firewall compatibility; still isolate with subnet rules

🔁 When to Use What?

Situation	Use Driver
🔀 Multi-host Swarm deployments	`overlay`
📡 Direct LAN access required	`macvlan`
🧬 Controlled IP mapping, no MAC exposure	`ipvlan`

🧠 Summary

overlay: Best for Swarm, cross-node services, scalable infra
macvlan: Best for LAN visibility, legacy hardware, IP-bound apps
ipvlan: Best for performance and controlled environments (e.g., cloud)

🚫 Docker `none` Network Driver – Ultimate Guide

❓ What is the `none` Network?

The none network is a special Docker network driver that completely disables networking for a container.

🚫 No IP address
🚫 No routing
🚫 No DNS
🚫 No internet access
🚫 No communication with host or other containers

🔐 Use Case:

When you want your container to run in complete isolation, especially for:

✅ CPU-intensive or file-only tasks
✅ Secure environments with no external communication
✅ Containers that interact only via volume sharing or IPC
✅ Avoiding network-related attacks (like SSRF, port scanning, etc.)

🧠 Real-World Analogy

It’s like putting a person in a soundproof, windowless room 🧍‍♂️🔇
They can compute, read, or write files – but cannot talk or hear the outside world.

🧪 How to Use It

docker run -it --rm --network none alpine

Then inside the container:

ping google.com   # ❌ Fails
ip addr            # Shows no IP

✅ The container runs, but it’s completely cut off from any kind of networking.

🔍 Check from the Host

docker inspect <container-id> | grep -i "NetworkMode"
# Output: "NetworkMode": "none"

🧰 Useful Scenarios

Use Case	Why `none` Works
🧮 Data processing / math computation	Doesn't need the internet
🔒 Security sandboxing	No attack vector via networking
🧪 Testing firewall rules	Simulate “no internet” condition
🧊 Build-only stages	Avoid leaking credentials over network
🚀 DevOps CI/CD	Run isolated build/test tasks with no exposure

📛 Warning

You cannot ping, curl, apt update, or download anything inside containers using --network none
Any tools that require internet or inter-container access will fail
It's not usable for most microservices or web APIs

🧠 Tip: Combine with Volumes or IPC

If you want to exchange data without a network:

# Create a shared volume
docker volume create shared-data

# Use it with the isolated container
docker run -it --rm --network none -v shared-data:/data alpine

✅ This lets you read/write to shared storage without needing any network access.

🧾 Summary Table

Feature	`none` Driver
IP address	❌ None
DNS	❌ None
Host access	❌ None
Container-to-container	❌ None
Internet	❌ None
Use case	Security, sandboxing, isolated compute

🔐 Final Verdict

If you need absolute network isolation, --network none is your zero-trust go-to option.
It’s perfect for:

🔐 Security-first workloads
🧪 Testing internal-only logic
🚫 Disabling all remote calls

🔒 Why Secure User Management in Docker Matters?

Darshan Vasani — Sun, 20 Jul 2025 08:10:57 +0000

🔒 Why Secure User Management in Docker Matters?

🧠 By default, Docker containers run processes as root, which is:

A huge security risk 🧨
Can lead to host exploitation
Bad for CI/CD and prod environments

⚠️ NEVER ship containers that run as root in production!

🔍 Real-World Analogy

🏡 Giving root access is like giving a guest 🔓 the master key to your house, including bank vaults, server room, and more.
🧑‍💻 Instead, give them only what they need – just one room!

✅ How to Add a Secure User in Docker

📦 Example (Linux-based):

# Create a group & user with no login shell
RUN addgroup --system --gid 1001 appgroup \
 && adduser --system --uid 1001 --ingroup appgroup --disabled-password appuser

# Switch to non-root user
USER appuser

🔑 Command	Purpose
`--system`	Marks as a system-level user/group
`--disabled-password`	Prevents password login
`USER appuser`	Runs all next steps as a non-root user

🔁 Typical Secure Dockerfile Flow

FROM node:20-alpine

WORKDIR /app

# Copy and build with root privileges
COPY . .
RUN npm install && npm run build

# 🔒 Create a secure user
RUN addgroup -S appgroup && adduser -S appuser -G appgroup

# ✅ Drop privileges
USER appuser

CMD ["node", "dist/index.js"]

🧠 Best Practices for Secure User Management

✅ Best Practice	💬 Why It’s Important
🧑‍💻 Avoid root in final image	Reduces attack surface
🔐 Use `USER` instruction	Ensures all commands run as non-root
📂 Set correct permissions (`chown`)	Ensure new user can access copied files
🔍 Audit with `docker scan` or `trivy`	Catch misconfigurations
👁️ Keep image minimal	Less packages = fewer CVEs
📜 Use `.dockerignore`	Prevent leaking `.env`, `keys`, `.git`

🛡️ Preventing Permission Issues with Files

COPY --chown=appuser:appgroup . .

# OR fix it manually
RUN chown -R appuser:appgroup /app

✅ Ensures the appuser has access to source files
⛔ Otherwise you might get EACCES or permission denied errors.

🔐 Dockerfile Security Summary Table

Feature	Good Practice	Why?
`USER`	Use non-root user	🧱 Least privilege
`COPY`	Use `--chown` flag	🧽 File ownership fix
`RUN`	Avoid `sudo`, limit shell access	🔒 Prevent privilege escalation
`ENTRYPOINT`/`CMD`	Should not run as root	✅ Always run app as secure user

🧪 Check Current User in Container

You can debug by checking UID:

docker run -it your-image whoami
docker run -it your-image id

🧰 Bonus Tip: Use Docker Compose Securely

services:
  api:
    image: dpvasani56/secure-api
    user: "1001:1001"

📌 You can enforce user ID even if Dockerfile doesn’t specify it.

✅ Final Checklist for Secure User Management

✅ Task	Status
Create system user & group	✔️
Assign proper UID:GID	✔️
Switch user with `USER`	✔️
Set file ownership (`--chown`)	✔️
Remove unnecessary packages	✔️
Test permissions inside container	✔️

🧱 What is a Multi-Stage Build in Docker?

Darshan Vasani — Sun, 20 Jul 2025 08:09:36 +0000

🧱 What is a Multi-Stage Build in Docker?

Multi-stage build allows you to use multiple FROM statements in a single Dockerfile to:

Build the app in one stage 🏗️

Copy only what's needed to a smaller final image 📦

❓ Why Do We Need It?

✅ Main Goals:

🚀 Benefit	💬 Why it Matters
⚡ Smaller Images	Only copy what's needed into final image
🔐 More Secure	No dev tools or secrets in production image
🛠️ Cleaner CI/CD	Separate build & runtime environment
📚 Better Layer Caching	Speeds up builds
🌍 Environment Separation	One image builds everything!

🧠 Real-World Analogy

Imagine:

🏗️ Stage 1 = Construction site (messy, heavy tools)
🏠 Stage 2 = Finished house (clean, cozy)

You build in the messy environment, but only move the furniture into the clean house. 🧹

🧪 Multi-Stage Build Syntax

# 🔨 Stage 1: Build Stage
FROM node:20-alpine AS builder
WORKDIR /app
COPY package*.json ./
RUN npm ci
COPY . .
RUN npm run build

# 📦 Stage 2: Final Production Image
FROM node:20-alpine
WORKDIR /app

# Copy only final build artifacts (no source or node_modules)
COPY --from=builder /app/dist ./dist
COPY --from=builder /app/package.json ./
RUN npm ci --omit=dev

# Set env vars, port and run
ENV PORT=3000
EXPOSE 3000
CMD ["node", "dist/index.js"]

🔍 Key Concepts Explained

Keyword	Meaning
`AS builder`	Give a name to this stage
`--from=builder`	Copy files from previous stage
`npm ci --omit=dev`	Install only production deps
`COPY . .`	Used only in build stage to avoid code bloat in final image

📦 Before vs After: Image Size

Approach	Image Size	Contents
😵‍💫 Traditional Single Build	~900MB	Full source code + dev dependencies
🤩 Multi-Stage Build	~200MB	Just built app + runtime dependencies

💥 Real Project Example: React App

# Step 1: Build React App
FROM node:20-alpine AS builder
WORKDIR /app
COPY package*.json ./
RUN npm install
COPY . .
RUN npm run build

# Step 2: Serve using NGINX
FROM nginx:alpine
COPY --from=builder /app/build /usr/share/nginx/html
EXPOSE 80
CMD ["nginx", "-g", "daemon off;"]

✅ This builds the app with Node.js, and serves the static files via NGINX (no Node.js in final image!)

🎯 Common Multi-Stage Use Cases

Use Case	Description
💻 Frontend builds	Use `node` + `nginx` combo
🔧 Backend builds	Build with TS/Go/Rust, then copy binaries only
🧪 Testing stage	Add test/linting in one stage, skip in final
📦 CI/CD pipelines	Clean, reproducible builds across stages

🧰 Pro Tips & Best Practices

💡 Tip	✅ Recommendation
Use `--omit=dev`	Strip dev-only packages in final stage
Use `.dockerignore`	Exclude `node_modules`, `.git`, `tests/`, etc
Use labels	Add metadata like version, author, etc
Don’t copy everything	Use exact `COPY` paths for size control
Use named stages	Easier to copy from (`--from=builder`)
Keep final image minimal	Just enough to run your app (no tools!)

🔄 Combine with Docker Compose

You can define multi-stage builds in your Dockerfile and just run:

docker-compose build
docker-compose up

Your services will use the optimized final image automatically 🧠✅

🛠️ Example Multi-Stage for TypeScript API

# Stage 1: Compile TS
FROM node:20-alpine AS builder
WORKDIR /app
COPY . .
RUN npm install
RUN npm run build

# Stage 2: Run with only JS output
FROM node:20-alpine
WORKDIR /app
COPY --from=builder /app/dist ./dist
COPY --from=builder /app/package*.json ./
RUN npm ci --omit=dev
CMD ["node", "dist/server.js"]

✅ Summary: When to Use Multi-Stage Builds?

✅ Always use if:

You're using build tools like tsc, webpack, vite
You want minimal production images
You want to separate testing/staging/building
You want faster CI builds & smaller attack surface

🧾 Final TL;DR Cheatsheet

Stage	Purpose	Base Image	Output
Stage 1 (builder)	Build, compile, test	`node`, `golang`, `rust`, etc.	`/dist`, `/build`, etc.
Stage 2 (prod)	Serve/run app only	`node:alpine`, `nginx`, etc.	Final slim image

📦 Full Dockerfile (Context Recap)

FROM node:20-alpine3.19 as base

# Stage 1: Build Stuff
FROM base as builder

WORKDIR /home/build

COPY package*.json .
COPY tsconfig.json .

RUN npm install

COPY src/ src/

RUN npm run build

# Stage 2: Runner
FROM base as runner

WORKDIR /home/app

COPY --from=builder /home/build/dist dist/
COPY --from=builder /home/build/package*.json .

RUN npm install --omit=dev

RUN addgroup --system --gid 1001 nodejs
RUN adduser --system --uid 1001 nodejs

USER nodejs

EXPOSE 8000
ENV PORT=8000

CMD [ "npm", "start" ]

🎯 A2Z Breakdown of Each Section

🧱 `FROM node:20-alpine3.19 as base`

🧠 What it does:

Starts from a minimal Node.js 20 Alpine image
Alpine is lightweight (~5MB), good for small, fast images
as base names this stage for reuse

🧩 Think of base like a shared template that both stages use.

🔨 Stage 1: Builder

FROM base as builder
WORKDIR /home/build

🔧 What happens here:

We switch to a new build stage, using base image
WORKDIR /home/build sets a directory for our build process

COPY package*.json .
COPY tsconfig.json .
RUN npm install

📦 Install dependencies:

package*.json copied to install dependencies
tsconfig.json is required for TypeScript compilation
npm install installs all dependencies (dev + prod)

COPY src/ src/
RUN npm run build

🛠️ Build your app:

Copies your app's TypeScript code
npm run build compiles TS into JS → typically inside /dist

✅ End Result of Stage 1:

A folder /home/build/dist with compiled production-ready JS output.

🚀 Stage 2: Runner

FROM base as runner
WORKDIR /home/app

📁 What it does:

We now create a fresh container just for running the app.
WORKDIR /home/app is where your app will run from.

COPY --from=builder /home/build/dist dist/
COPY --from=builder /home/build/package*.json .

📂 Copy built artifacts only:

Only copy the dist/ folder and package files (no source, no tsconfig)
Ensures the final image is slim & clean

RUN npm install --omit=dev

🔐 Production-only install:

Installs only prod dependencies (no dev tools like eslint, tsc, etc.)
Keeps final image light and secure ✅

👮 Add Secure Non-Root User

RUN addgroup --system --gid 1001 nodejs
RUN adduser --system --uid 1001 nodejs
USER nodejs

🔐 Why?

Running as root is dangerous in containers ❌
We create a user nodejs with limited permissions for safety
UID/GID 1001 is just an arbitrary non-root system user

🌐 Port & Env Setup

EXPOSE 8000
ENV PORT=8000

EXPOSE 8000: Documents that the app uses port 8000
ENV PORT=8000: Sets the default port for app to use internally

You still need to use -p to map it to host:
docker run -p 8000:8000 <image>

🚦 Start the App

CMD [ "npm", "start" ]

🟢 Default entrypoint when container runs

This triggers your "start" script from package.json:

  "start": "node dist/index.js"

✅ Summary Table

🔹 Section	🔍 Purpose
`FROM base`	Reuse image to reduce duplication
`builder`	Compiles TypeScript into JS
`runner`	Runs a minimal production image
`npm install` in builder	Installs full deps for building
`npm install --omit=dev` in runner	Installs only what's needed to run
`COPY --from=builder`	Efficient file copy without rebuild
`USER nodejs`	Enhances container security

📊 Resulting Benefits

🚀 Benefit	✅ Achieved
Small Image	✅ Only runtime code in final image
Secure	✅ Non-root user, no dev tools
Faster Builds	✅ Reuses build layers
Clean Code Separation	✅ No TypeScript or build files inside final container
Portable	✅ Can run on any platform with Node 20

🧠 Bonus Tip: View Image Sizes

docker images

Compare the multi-stage image (~100MB) vs a single-stage image (~400–600MB) 🤯

🔚 Final Thoughts

This approach follows Docker best practices:

Multi-stage

Production-ready

Secure by default

Reproducible builds

🧩 Docker Layer Caching

Darshan Vasani — Sun, 20 Jul 2025 08:06:51 +0000

🧩 Docker Layer Caching: What & Why?

When you build a Docker image, each instruction (like COPY, RUN, etc.) creates a layer. Docker caches these layers 🔄 so it can reuse them in future builds, making things faster!

🔥 Why Layer Order Matters

Docker reads your Dockerfile top to bottom 📜⬇️
The first changed line invalidates the cache for all lines after it ❌🚫

📊 Example: Bad vs Good Sequence

🚫 BAD Dockerfile (Unoptimized Layer Order)

COPY . .        # ❌ Copies everything first (even changing README breaks cache)
RUN npm install # Cache busts often!

✅ GOOD Dockerfile (Optimized Layer Order)

COPY package*.json ./  # ✅ Only changes when dependencies change
RUN npm install        # ✅ Reused most of the time
COPY . .               # ✅ Source code comes after

🧠 Why?

If you copy the whole source before installing deps, any code change breaks the cache for dependencies!
By copying just package.json first, Docker only re-installs when dependencies change.

✅ Recommended Layer Order Cheat Sheet 📝

Layer	Why It Comes Here
`FROM`	Base image, foundation layer 🏗️
`WORKDIR`	Set working directory 📁
`COPY package*.json ./`	Dependency file copied first for caching 🧃
`RUN npm ci` or `RUN npm install`	Install deps (caches as long as package.json doesn’t change) 📦
`COPY . .`	Now copy the actual app code 🧑‍💻
`EXPOSE` & `ENV`	Doesn’t affect cache much, but goes here 🔌
`CMD`	Entrypoint, doesn't affect caching 🟢

🧠 Pro Caching Tips

💡 Tip	🛠️ Description
🧩 Use `.dockerignore`	Prevent unnecessary files (e.g. `.git`, `node_modules`) from breaking cache.
🧪 Use `npm ci`	Faster and more reproducible in CI/CD than `npm install`.
🧑‍🏭 Split dev & prod builds	Use multi-stage builds to keep production images small and cache efficient.
🔐 Use exact base versions	Use `node:20-alpine` instead of `node:alpine` to avoid unexpected cache busts.

🚀 Visual Analogy

Think of Docker caching like making layered sandwiches 🥪:

🥖 If you change the base bread (early layers), the whole sandwich needs to be rebuilt.

🧀 But if you change just the top slice of tomato 🍅 (code), you don’t need to rebuild the whole thing.

🔁 Final Thought

💬 Always structure your Dockerfile to keep slow-changing layers at the top and fast-changing layers (like source code) at the bottom — this will save build time ⏱️ and make CI/CD faster ⚡.

📘 Complete Docker Image Publishing CheatSheet 🐳🚀

Darshan Vasani — Fri, 11 Jul 2025 15:12:09 +0000

📘 Complete Docker Image Publishing CheatSheet 🐳🚀

🌐 What is a Docker Registry?

A Docker registry is a storage for Docker images 🗃️. You can:

✅ Push your custom images to it
📥 Pull images when needed
🔐 Optionally set them private/public

🏠 Popular Registries:

🐳 Docker Hub (hub.docker.com)
🔐 GitHub Container Registry (ghcr.io)
☁️ Google Artifact Registry / Amazon ECR / GitLab / Azure ACR

✨ Structure of Docker Image Name

<registry>/<username>/<repo>:<tag>

Part	Example	Meaning
Registry	`docker.io` (default)	Where image is stored 🌍
Username	`dpvasani56`	Your DockerHub or GitHub ID 👤
Repo	`node-application`	Your app/project name 📦
Tag	`v1`, `latest`	Version tag 🏷️

🚀 Two Ways to Publish Docker Image to Docker Hub

📦 Step 0: Build the image

docker build -t dpvasani56/node-application:v1 .

🧭 Method 1: Manual Push to Docker Hub

✅ Step-by-step:

1️⃣ Login to Docker Hub

docker login

🧠 Enter your Docker Hub username and password.

2️⃣ Push your image

docker push dpvasani56/node-application:v1

✅ Image will now appear on your Docker Hub at:
📍 https://hub.docker.com/r/dpvasani56/node-application

3️⃣ Pull from any system

docker pull dpvasani56/node-application:v1

🎯 Then run:

docker run -p 3000:3000 dpvasani56/node-application:v1

🧠 Method 2: Push from GitHub via GitHub Container Registry (ghcr.io)

🔧 Step-by-step:

1️⃣ Create a GitHub repo

Name it like: node-application

2️⃣ Login to GitHub Container Registry

echo <GH_TOKEN> | docker login ghcr.io -u USERNAME --password-stdin

💡 Use a Personal Access Token (PAT) from GitHub with write:packages permission.

3️⃣ Tag your image

docker tag node-application ghcr.io/dpvasani56/node-application:v1

4️⃣ Push to GitHub Container Registry

docker push ghcr.io/dpvasani56/node-application:v1

✅ Image is now available at:
🔗 https://github.com/dpvasani56/packages

📌 Tagging Summary

# Tag for DockerHub
docker tag node-application dpvasani56/node-application:v1

# Tag for GitHub Registry
docker tag node-application ghcr.io/dpvasani56/node-application:v1

🧼 Optional Cleanup

docker image rm <image-name>

Use this to save space once pushed.

🧠 Bonus: Automate with GitHub Actions

Use this snippet in .github/workflows/docker.yml:

jobs:
  push_to_registry:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - name: Log in to GitHub Container Registry
        run: echo "${{ secrets.GH_PAT }}" | docker login ghcr.io -u dpvasani56 --password-stdin

      - name: Build and Push
        run: |
          docker build -t ghcr.io/dpvasani56/node-application:v1 .
          docker push ghcr.io/dpvasani56/node-application:v1

✅ Set GH_PAT as a GitHub secret with correct permissions.

🔥 Quick Recap Table

Action	Docker Hub	GitHub Container Registry
Login	`docker login`	`docker login ghcr.io`
Tag	`dpvasani56/app:v1`	`ghcr.io/dpvasani56/app:v1`
Push	`docker push dpvasani56/app:v1`	`docker push ghcr.io/dpvasani56/app:v1`
Pull	`docker pull dpvasani56/app:v1`	`docker pull ghcr.io/dpvasani56/app:v1`

🧾 Sample Push Command (Your Request)

docker push dpvasani56/node-application:v1

✅ This pushes your image to Docker Hub under your account.

🏷️ Step-by-Step: Tag & Push Docker Image to Docker Hub (`dpvasani56/node-application`)

Assume your image is locally named:

my-app

And you want to push it as:
📦 dpvasani56/node-application

🔧 Step 1: Tag the Image

🏷️ Think of this as giving your image a Docker Hub label 🎫

docker tag my-app dpvasani56/node-application

✅ This tags your image for Docker Hub upload.

✅ Optional: Add a Version Tag

docker tag my-app dpvasani56/node-application:v1

🎯 This is better for version control in CI/CD and releases.

🔐 Step 2: Login to Docker Hub

docker login

➡️ Enter Docker Hub credentials for dpvasani56.

📤 Step 3: Push the Image

👉 With version tag:

docker push dpvasani56/node-application:v1

👉 Or default (`latest` tag):

docker push dpvasani56/node-application

✅ Image now available at:
🔗 https://hub.docker.com/r/dpvasani56/node-application

📥 Step 4: Pull & Use It Anywhere

docker pull dpvasani56/node-application:v1

Run the app:

docker run -p 3000:3000 dpvasani56/node-application:v1

📦 All Commands Recap

# 🔨 Build your image
docker build -t my-app .

# 🏷️ Tag for Docker Hub
docker tag my-app dpvasani56/node-application:v1

# 🔐 Login to Docker Hub
docker login

# 📤 Push to Docker Hub
docker push dpvasani56/node-application:v1

# 📥 Pull from Docker Hub (anywhere)
docker pull dpvasani56/node-application:v1

# ▶️ Run it
docker run -p 3000:3000 dpvasani56/node-application:v1

🧠 Helpful Tips

Command	Use
`docker images`	View tagged images locally 🗂️
`docker rmi <image>`	Remove an image locally 🧹
`docker ps`	View running containers 🚀
`docker stop <id>`	Stop a container manually 🛑

🧭 Docker Port Mapping & CLI Flag CheatSheet

Darshan Vasani — Fri, 11 Jul 2025 15:11:02 +0000

🧭 Docker Port Mapping & CLI Flag CheatSheet

🚪 Port Mapping 101: `-p` vs `-P`

Flag	Meaning	Analogy	Example
`-p <host>:<container>`	Map host port to container port	Custom Door Mapping 🚪	`-p 8080:3000`
`-P` (uppercase)	Auto-map all exposed ports to random host ports	🔀 Auto Door Mapping	`-P`

🎯 Example 1: Manual Mapping with `-p`

docker run -p 8080:3000 my-app

🔧 Host port 8080 mapped to container's 3000
Access via localhost:8080

🎲 Example 2: Auto Mapping with `-P`

docker run -P my-app

Maps ALL EXPOSEd ports to random available host ports
View with docker ps

🌍 Multiple Port Mappings

docker run -p 3000:3000 -p 5000:5000 my-multi-app

Maps multiple services/APIs or frontend/backend apps
Great for full-stack containers!

📡 Exposing Ports in Dockerfile

EXPOSE 3000 5000 7000

🧠 Note:
`EXPOSE` is just documentation for which ports the container listens to.
It does not publish or map ports. Use `-p` or `-P` in `docker run` to do that.

📦 Exposing a Range of Ports

docker run -p 8000-8010:8000-8010 my-app

Useful for services like WebRTC, game servers, or load balancers needing multiple ports.
🔄 Keep range symmetrical between host and container.

🧪 Combine with `--rm`, `-it`, `-d`

Flag	Meaning	Emoji	Use-case
`--rm`	Auto-remove container when it exits	🧹	Clean test runs
`-i`	Interactive (stdin open)	🎤	Needed for terminal apps
`-t`	TTY (format output)	🖥️	Pretty output formatting
`-it`	Combo: interactive + tty	🧑‍💻	Needed for shell, REPLs
`-d`	Detached mode (run in background)	🛸	Long-running servers

🔥 Full Command Example

docker run -it --rm -p 4000:3000 my-app

Runs interactively
Auto deletes after exit
Maps port 3000 → 4000

🚀 Run in Detached Mode + Multiple Ports + Clean Exit

docker run -d --rm -p 3000:3000 -p 5000:5000 my-app

Runs in background
Auto-cleans after stopping
Maps frontend + backend

🕵️‍♂️ How to Check Mapped Ports?

docker ps

Column	What it Shows
`PORTS`	Host:Container mapping (e.g., `0.0.0.0:8080->3000/tcp`)

🛑 Stop a Detached Container

docker stop <container_id>

🧠 Real-World Analogies

Concept	Analogy
Container port	📦 Internal phone extension
Host port	☎️ Public phone number
`-p`	Assigning specific number to extension
`-P`	Letting system pick a number for you randomly

✅ Summary Cheatsheet Table

Option	Purpose	Usage
`-p 8080:80`	Manual port map	`host:container`
`-p 8000-8010:8000-8010`	Port range mapping	Bulk apps
`-P`	Auto-map all exposed ports	Quick tests
`EXPOSE 3000`	Doc port in Dockerfile	Doesn't publish
`--rm`	Auto-remove after exit	Clean containers
`-it`	Interactive terminal	Needed for shell
`-d`	Run in background	Daemonized apps

🧾 Real Full Example: Production-style Run

docker run -d --rm \
  -p 8080:80 \
  -p 443:443 \
  --name my-nginx \
  nginx:alpine

✅ Starts Nginx with HTTP & HTTPS
✅ Cleans itself after docker stop
✅ Runs in background with custom name

DEV Community: Darshan Vasani

🐳 Docker Multi-Stage Build - Complete Documentation

🐳 Docker Multi-Stage Build - Complete Documentation

📋 Table of Contents

🎯 What is Docker Multi-Stage Build?

🔤 Key Concepts

🚀 Why Use Multi-Stage Builds?

🏗️ How Multi-Stage Builds Work

🔍 Single-Stage vs Multi-Stage Comparison

🔄 Build Process Flow

📦 Step-by-Step Tutorial

🛠️ Project Setup

📝 Creating Multi-Stage Dockerfile

🔧 Build Process

📊 Build Output Analysis

🔄 Multi-Stage Build Workflow

🎯 Complete Workflow Diagram

🔍 Stage Dependency Graph

🛠️ Commands and Best Practices

🔧 Essential Docker Commands

Build Commands

Image Management

Container Operations

🔍 Debugging and Inspection Commands

📊 Performance Comparison

📏 Size Comparison

📈 Benefits Breakdown

🔧 Debugging and Troubleshooting

🔍 Container Investigation Commands

🕵️ Inside Container Exploration

🚨 Common Issues and Solutions

✨ Best Practices

🎯 Multi-Stage Build Best Practices

1. 🏷️ Use Named Stages

2. 📦 Choose Optimal Base Images

3. 🎯 Copy Only What's Needed

4. 📋 Optimize Layer Caching

🔒 Security Best Practices

5. 👤 Use Non-Root User

6. 🗑️ Remove Unnecessary Packages

🚀 Performance Best Practices

7. 🎯 Use Specific Targets

8. 📦 Multi-Architecture Support

🔄 Advanced Multi-Stage Patterns

9. 🧪 Testing Stage

10. 🔀 Parallel Builds

📊 Additional Best Practices

🎓 Summary

🔑 Key Achievements

🚀 Implementation Steps

Notes & Cheatsheet: “Dockerizing an App – CKA Series #2”

Notes & Cheatsheet: “Dockerizing an App – CKA Series #2”

1 Prerequisites & Sandbox options

2 Clone sample project

3 Write the Dockerfile

4 Build image locally

5 Push to Docker Hub

6 Run the container anywhere

7 Troubleshoot inside the container

8 Key Docker CLI quick-ref

9 Next-step teaser 🐣

💡 Takeaways

🚀 Step-by-Step: Setup + GET Method in FastAPI

🚀 Step-by-Step: Setup + GET Method in FastAPI

🧪 1. Set up Virtual Environment (Windows/Linux/Mac)

📦 2. Install FastAPI and Uvicorn

📁 3. Create Project Structure

🧠 4. Create main.py with GET Methods

✅ Using Path Parameters, Predefined Values, and Query Parameters

▶️ 5. Run the Server

📘 6. Explore Interactive Docs (Swagger)

🔍 Usage Examples

🧾 Summary Notes

✅ 1. Why Use Routers in FastAPI?

✅ Benefits:

🧩 2. What Is a Router in FastAPI?

💡 Example:

⚙️ 3. How to Register a Router

🔍 4. Behind the Scenes: Validation Using Pydantic

✅ Path Parameters

🧠 4. Create `main.py` with GET Methods

🧰 Advanced: Use `Query` for extra validation

🧠 2. Using `Query()` for More Control

✅ Optional with Pydantic `Optional[]` (Alternative)

⚙️ Basic Syntax of `docker-compose.yml`