Comprehensive Docker Guide

Table of Contents

1. Introduction to Docker
2. Docker Architecture
3. Installation
4. Docker Images
5. Docker Containers
6. Dockerfile
7. Docker Volumes
8. Docker Networks
9. Docker Compose
10. Docker Registry
11. Best Practices
12. Troubleshooting
13. Advanced Topics

Introduction to Docker

Docker is a containerization platform that packages applications and their dependencies into lightweight, portable containers. Containers ensure applications run consistently across different environments.

Key Benefits

• Consistency: "It works on my machine" → "It works everywhere"
• Portability: Run anywhere Docker is installed
• Efficiency: Share OS kernel, lighter than VMs
• Scalability: Easy to scale up/down
• Isolation: Applications run in isolated environments

Containers vs Virtual Machines

Containers	Virtual Machines
Share host OS kernel	Each VM has full OS
Lightweight (MBs)	Heavy (GBs)
Fast startup	Slow startup
More efficient resource usage	Resource overhead

Docker Architecture

Core Components

Docker Client: Command-line interface (docker command)
Docker Daemon: Background service managing containers
Docker Images: Read-only templates for containers
Docker Containers: Running instances of images
Docker Registry: Storage for Docker images (Docker Hub, etc.)

Docker Engine Architecture

┌─────────────────┐    ┌─────────────────┐
│   Docker CLI    │───▶│  Docker Daemon  │
└─────────────────┘    │   (dockerd)     │
                       └─────────┬───────┘
                                 │
                    ┌────────────┼────────────┐
                    │            │            │
            ┌───────▼───┐ ┌──────▼───┐ ┌─────▼─────┐
            │  Images   │ │Containers│ │  Networks │
            └───────────┘ └──────────┘ └───────────┘

---

Installation

Linux (Ubuntu/Debian)

# Update package index
sudo apt-get update

# Install prerequisites
sudo apt-get install apt-transport-https ca-certificates curl gnupg lsb-release

# Add Docker's GPG key
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /usr/share/keyrings/docker-archive-keyring.gpg

# Add Docker repository
echo "deb [arch=amd64 signed-by=/usr/share/keyrings/docker-archive-keyring.gpg] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null

# Install Docker
sudo apt-get update
sudo apt-get install docker-ce docker-ce-cli containerd.io

# Add user to docker group (avoid using sudo)
sudo usermod -aG docker $USER

macOS

# Install using Homebrew
brew install --cask docker

Windows

Download Docker Desktop from docker.com and follow the installation wizard.

Verify Installation

docker --version
docker run hello-world

Docker Images

Images are read-only templates used to create containers. They're built in layers using a Dockerfile.

Basic Image Commands

# List images
docker images
docker image ls

# Pull an image
docker pull ubuntu:20.04
docker pull nginx:alpine

# Remove an image
docker rmi ubuntu:20.04
docker image rm nginx:alpine

# Build an image
docker build -t myapp:1.0 .

# Image history
docker history ubuntu:20.04

# Image inspection
docker inspect ubuntu:20.04

# Search for images
docker search nginx

Image Naming Convention

[registry-host[:port]/]username/repository:tag

Examples:
- nginx:latest
- ubuntu:20.04
- gcr.io/my-project/my-app:v1.0
- localhost:5000/myapp:dev

Docker Containers

Containers are running instances of Docker images.

Container Lifecycle Commands

# Run a container
docker run ubuntu:20.04
docker run -d nginx:alpine  # Detached mode
docker run -it ubuntu:20.04 /bin/bash  # Interactive

# List containers
docker ps          # Running containers
docker ps -a       # All containers

# Stop a container
docker stop <container-id>
docker stop <container-name>

# Start a stopped container
docker start <container-id>

# Restart a container
docker restart <container-id>

# Remove a container
docker rm <container-id>
docker rm $(docker ps -aq)  # Remove all containers

# Execute commands in running container
docker exec -it <container-id> /bin/bash

Container Options

# Port mapping
docker run -p 8080:80 nginx

# Volume mounting
docker run -v /host/path:/container/path ubuntu

# Environment variables
docker run -e MY_VAR=value ubuntu

# Container name
docker run --name my-container ubuntu

# Memory and CPU limits
docker run -m 512m --cpus="1.5" ubuntu

# Complete example
docker run -d \
  --name web-server \
  -p 8080:80 \
  -v /host/html:/usr/share/nginx/html \
  -e NGINX_HOST=example.com \
  nginx:alpine

Container Management

# View container logs
docker logs <container-id>
docker logs -f <container-id>  # Follow logs

# Container statistics
docker stats
docker stats <container-id>

# Container processes
docker top <container-id>

# Container inspection
docker inspect <container-id>

# Copy files to/from container
docker cp file.txt <container-id>:/path/to/file.txt
docker cp <container-id>:/path/to/file.txt ./file.txt

Dockerfile

A Dockerfile is a script containing instructions to build a Docker image.

Basic Dockerfile Structure

# Use official base image
FROM ubuntu:20.04

# Set metadata
LABEL maintainer="[e-mail]"
LABEL version="1.0"

# Set working directory
WORKDIR /app

# Install packages
RUN apt-get update && apt-get install -y \
    python3 \
    python3-pip \
    && rm -rf /var/lib/apt/lists/*

# Copy files
COPY requirements.txt .
COPY . .

# Install Python dependencies
RUN pip3 install -r requirements.txt

# Expose port
EXPOSE 8000

# Set environment variables
ENV PYTHONPATH=/app
ENV FLASK_ENV=production

# Create user for security
RUN useradd -m appuser
USER appuser

# Command to run when container starts
CMD ["python3", "app.py"]

Dockerfile Instructions

Instruction	Purpose	Example
FROM	Base image	FROM ubuntu:20.04
RUN	Execute commands	RUN apt-get update
COPY	Copy files	COPY . /app
ADD	Copy + extract	ADD archive.tar.gz /app
WORKDIR	Set working directory	WORKDIR /app
EXPOSE	Expose ports	EXPOSE 8000
ENV	Environment variables	ENV NODE_ENV=production
CMD	Default command	CMD ["python", "app.py"]
ENTRYPOINT	Entry point	ENTRYPOINT ["./start.sh"]
VOLUME	Mount point	VOLUME ["/data"]
USER	Switch user	USER appuser

Multi-stage Builds

# Build stage
FROM node:16 AS builder
WORKDIR /app
COPY package*.json ./
RUN npm ci --only=production

# Production stage
FROM node:16-alpine
WORKDIR /app
COPY --from=builder /app/node_modules ./node_modules
COPY . .
EXPOSE 3000
CMD ["npm", "start"]

Build Image

# Build from current directory
docker build -t myapp:1.0 .

# Build with different Dockerfile
docker build -f Dockerfile.prod -t myapp:prod .

# Build with build args
docker build --build-arg VERSION=1.0 -t myapp .

# No cache
docker build --no-cache -t myapp .

Docker Volumes

Volumes provide persistent data storage for containers.

Types of Volumes

1. Named Volumes (Managed by Docker)
2. Bind Mounts (Host filesystem)
3. tmpfs Mounts (In memory)

Volume Commands

# Create a volume
docker volume create my-volume

# List volumes
docker volume ls

# Inspect volume
docker volume inspect my-volume

# Remove volume
docker volume rm my-volume

# Prune unused volumes
docker volume prune

Using Volumes

# Named volume
docker run -v my-volume:/data ubuntu

# Bind mount
docker run -v /host/path:/container/path ubuntu
docker run -v $(pwd):/app ubuntu  # Current directory

# Read-only mount
docker run -v /host/path:/container/path:ro ubuntu

# tmpfs mount
docker run --tmpfs /tmp ubuntu

Volume Examples

# Database with persistent storage
docker run -d \
  --name postgres-db \
  -e POSTGRES_PASSWORD=password \
  -v postgres-data:/var/lib/postgresql/data \
  postgres:13

# Development environment
docker run -it \
  -v $(pwd):/workspace \
  -w /workspace \
  node:16 /bin/bash

Docker Networks

Networks enable communication between containers.

Network Types

• bridge: Default network (containers can communicate)
• host: Use host's network directly
• none: No networking
• overlay: Multi-host networking (Swarm)

Network Commands

# List networks
docker network ls

# Create network
docker network create my-network
docker network create --driver bridge my-bridge

# Inspect network
docker network inspect my-network

# Connect container to network
docker network connect my-network my-container

# Disconnect container from network
docker network disconnect my-network my-container

# Remove network
docker network rm my-network

Using Networks

# Run container on specific network
docker run -d --network my-network --name web nginx
docker run -d --network my-network --name db postgres

# Containers can communicate by name
# web container can reach db container at hostname "db"

Network Example

# Create custom network
docker network create --driver bridge app-network

# Run database
docker run -d \
  --name database \
  --network app-network \
  -e POSTGRES_PASSWORD=password \
  postgres:13

# Run web application
docker run -d \
  --name webapp \
  --network app-network \
  -p 8080:80 \
  my-web-app

Docker Compose

Docker Compose manages multi-container applications using YAML files.

Installation

# Linux
sudo curl -L "https://github.com/docker/compose/releases/latest/download/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
sudo chmod +x /usr/local/bin/docker-compose

# Or install via pip
pip install docker-compose

# Verify installation
docker-compose --version

docker-compose.yml Structure

version: '3.8'

services:
  web:
    build: .
    ports:
      - "8000:8000"
    volumes:
      - .:/app
    environment:
      - DEBUG=1
    depends_on:
      - db
      - redis
    networks:
      - app-network

  db:
    image: postgres:13
    environment:
      POSTGRES_PASSWORD: password
      POSTGRES_DB: myapp
    volumes:
      - postgres-data:/var/lib/postgresql/data
    networks:
      - app-network

  redis:
    image: redis:alpine
    networks:
      - app-network

volumes:
  postgres-data:

networks:
  app-network:
    driver: bridge

Docker Compose Commands

# Start services
docker-compose up
docker-compose up -d  # Detached mode

# Stop services
docker-compose down
docker-compose down -v  # Remove volumes

# Build services
docker-compose build
docker-compose build --no-cache

# Scale services
docker-compose up -d --scale web=3

# View logs
docker-compose logs
docker-compose logs web

# Execute commands
docker-compose exec web /bin/bash

# List services
docker-compose ps

# Restart services
docker-compose restart

Advanced Compose Example

version: '3.8'

services:
  nginx:
    image: nginx:alpine
    ports:
      - "80:80"
    volumes:
      - ./nginx.conf:/etc/nginx/nginx.conf
    depends_on:
      - web
    networks:
      - frontend

  web:
    build:
      context: .
      dockerfile: Dockerfile.prod
    environment:
      - DATABASE_URL=postgresql://user:pass@db:5432/app
      - REDIS_URL=redis://redis:6379
    depends_on:
      - db
      - redis
    networks:
      - frontend
      - backend
    deploy:
      replicas: 2
      restart_policy:
        condition: on-failure

  db:
    image: postgres:13
    environment:
      POSTGRES_USER: user
      POSTGRES_PASSWORD: pass
      POSTGRES_DB: app
    volumes:
      - postgres-data:/var/lib/postgresql/data
    networks:
      - backend

  redis:
    image: redis:alpine
    networks:
      - backend

volumes:
  postgres-data:

networks:
  frontend:
  backend:

Docker Registry

Docker Hub (Public Registry)

# Login to Docker Hub
docker login

# Tag image for Docker Hub
docker tag myapp:latest username/myapp:latest

# Push to Docker Hub
docker push username/myapp:latest

# Pull from Docker Hub
docker pull username/myapp:latest

Private Registry

# Run local registry
docker run -d -p 5000:5000 --name registry registry:2

# Tag for local registry
docker tag myapp:latest localhost:5000/myapp:latest

# Push to local registry
docker push localhost:5000/myapp:latest

# Pull from local registry
docker pull localhost:5000/myapp:latest

Registry Authentication

# Login to private registry
docker login myregistry.com

# Or specify credentials
echo "password" | docker login --username myuser --password-stdin myregistry.com

Best Practices

Dockerfile Best Practices

1. Use official base images

FROM node:16-alpine  # Official, minimal

1. Minimize layers

# Bad - multiple layers
RUN apt-get update
RUN apt-get install -y python3
RUN apt-get install -y python3-pip

# Good - single layer
RUN apt-get update && apt-get install -y \
    python3 \
    python3-pip \
    && rm -rf /var/lib/apt/lists/*

1. Use .dockerignore

node_modules
*.log
.git
.DS_Store

1. Don't run as root

RUN useradd -m appuser
USER appuser

1. Use multi-stage builds

FROM node:16 AS builder
# Build stage

FROM node:16-alpine
COPY --from=builder /app/dist ./dist
# Production stage

Security Best Practices

1. Scan images for vulnerabilities

docker scan myapp:latest

1. Use minimal base images

FROM alpine:latest  # Minimal
FROM scratch        # Empty base

1. Keep images updated

docker pull ubuntu:latest  # Get latest security patches

1. Use secrets management

# Don't put secrets in Dockerfile
# Use environment variables or secret management tools

Performance Best Practices

1. Optimize layer caching

# Copy dependencies first (changes less frequently)
COPY package.json .
RUN npm install

# Copy source code last (changes frequently)
COPY . .

1. Use .dockerignore
2. Remove unnecessary files

RUN apt-get update && apt-get install -y package \
    && rm -rf /var/lib/apt/lists/*

1. Use appropriate image sizes

docker images --format "table {{.Repository}}\t{{.Tag}}\t{{.Size}}"

Troubleshooting

Common Issues and Solutions

Container won't start

# Check logs
docker logs <container-id>

# Check container configuration
docker inspect <container-id>

Port already in use

# Find process using port
lsof -i :8080

# Use different port
docker run -p 8081:80 nginx

Permission denied

# Add user to docker group
sudo usermod -aG docker $USER

# Restart shell or logout/login

Out of disk space

# Clean up containers
docker container prune

# Clean up images
docker image prune

# Clean up volumes
docker volume prune

# Clean up everything
docker system prune -a

Debugging Commands

# System information
docker system info
docker system df  # Disk usage

# Container debugging
docker exec -it <container> /bin/bash
docker logs -f <container>
docker stats <container>

# Network debugging
docker network ls
docker network inspect bridge

Advanced Topics

Docker Swarm (Orchestration)

# Initialize swarm
docker swarm init

# Join swarm
docker swarm join --token <token> <manager-ip>:2377

# Deploy stack
docker stack deploy -c docker-compose.yml myapp

# List services
docker service ls

# Scale service
docker service scale myapp_web=5

Health Checks

HEALTHCHECK --interval=30s --timeout=3s --start-period=60s \
  CMD curl -f http://localhost:8000/health || exit 1

Multi-platform Builds

# Create builder
docker buildx create --name mybuilder --use

# Build for multiple platforms
docker buildx build --platform linux/amd64,linux/arm64 -t myapp:latest --push .

Docker Secrets (Swarm)

# Create secret
echo "mysecretpassword" | docker secret create db_password -

# Use in service
docker service create --secret db_password myapp

Container Resource Limits

# Memory and CPU limits
docker run -m 512m --cpus="1.5" myapp

# In docker-compose.yml
services:
  web:
    image: myapp
    deploy:
      resources:
        limits:
          memory: 512M
          cpus: '1.5'

Docker BuildKit

# syntax=docker/dockerfile:1
FROM ubuntu:20.04

# BuildKit features
RUN --mount=type=cache,target=/var/cache/apt \
    apt-get update && apt-get install -y python3

Useful Commands Reference

Cleanup Commands

# Remove all stopped containers
docker container prune

# Remove all unused images
docker image prune -a

# Remove all unused volumes
docker volume prune

# Remove all unused networks
docker network prune

# Remove everything unused
docker system prune -a --volumes

Monitoring Commands

# System-wide information
docker system info
docker system df

# Container statistics
docker stats
docker top <container>

# Event monitoring
docker events

Backup and Restore

# Export container
docker export <container> > backup.tar

# Import container
docker import backup.tar myapp:backup

# Save image
docker save myapp:latest > myapp.tar

# Load image
docker load < myapp.tar

This comprehensive guide covers Docker from basics to advanced usage. Practice with hands-on examples to solidify your understanding of containerization concepts and Docker workflows.