DEV Community

Cover image for Here's Everything I Know About Docker
Odipo Otieno (KwargDevs)
Odipo Otieno (KwargDevs)

Posted on

Here's Everything I Know About Docker

#webdev #programming #javascript #docker

Docker: The Missing Manual (Notes)

Motto: "Build, Ship, and Run Any App, Anywhere"

Table of Contents

  1. Chapter 1: What is Docker and Why Do We Need It?

    • 1.1 The "Works on My Machine" Problem
    • 1.2 Virtual Machines vs. Containers
    • 1.3 Core Benefits of Docker
    • 1.4 Key Docker Terminology (Brief Overview)

  2. Chapter 2: Docker's Core Components

    • 2.1 Docker Engine (Daemon, REST API, CLI)
    • 2.2 Images: The Blueprints
    • 2.3 Containers: The Running Instances
    • 2.4 Registries: Storing and Sharing Images (e.g., Docker Hub)

  3. Chapter 3: Getting Started - Installation & First Steps

    • 3.1 Installing Docker (Desktop & Server)
    • 3.2 Verifying Installation
    • 3.3 Running Your First Container: hello-world
    • 3.4 Basic Docker Commands: run, ps, images, stop, rm

  4. Chapter 4: Working with Docker Images

    • 4.1 Pulling Images from a Registry (docker pull)
    • 4.2 Listing Images (docker images)
    • 4.3 Running Containers from Images (docker run in detail)
    • 4.4 Removing Images (docker rmi)
    • 4.5 Understanding Image Layers

  5. Chapter 5: Building Your Own Images with Dockerfile

    • 5.1 Introduction to Dockerfile
    • 5.2 Common Dockerfile Instructions (FROM, RUN, COPY, CMD, ENTRYPOINT, WORKDIR, EXPOSE, ENV)
    • 5.3 Building an Image (docker build)
    • 5.4 Tagging Images
    • 5.5 .dockerignore File
    • 5.6 Best Practices for Writing Dockerfiles

  6. Chapter 6: Managing Containers

    • 6.1 Listing Containers (docker ps, docker ps -a)
    • 6.2 Starting, Stopping, and Restarting Containers (start, stop, restart)
    • 6.3 Removing Containers (docker rm)
    • 6.4 Viewing Container Logs (docker logs)
    • 6.5 Executing Commands in a Running Container (docker exec)
    • 6.6 Inspecting Containers (docker inspect)

  7. Chapter 7: Data Persistence with Volumes

    • 7.1 The Ephemeral Nature of Container File Systems
    • 7.2 What are Volumes?
    • 7.3 Types of Mounts: Volumes vs. Bind Mounts
    • 7.4 Creating and Managing Volumes (docker volume create, ls, inspect, rm)
    • 7.5 Using Volumes with Containers (-v or --mount flag)

  8. Chapter 8: Docker Networking

    • 8.1 Introduction to Docker Networking
    • 8.2 Default Networks (Bridge, Host, None)
    • 8.3 Port Mapping (-p flag)
    • 8.4 User-Defined Bridge Networks for Container Communication
    • 8.5 Linking Containers (Legacy, good to know)
    • 8.6 Inspecting Networks (docker network inspect)

  9. Chapter 9: Multi-Container Applications with Docker Compose

    • 9.1 What is Docker Compose?
    • 9.2 The docker-compose.yml File
    • 9.3 Key Compose Directives (version, services, image, build, ports, volumes, environment, depends_on)
    • 9.4 Common Compose Commands (up, down, ps, logs, exec, build)
    • 9.5 Example: Web App + Database

  10. Chapter 10: Cleaning Up Docker Resources

    • 10.1 Removing Dangling Images and Volumes
    • 10.2 docker system prune
    • 10.3 Selective Pruning

  11. Chapter 11: Advanced Topics & Next Steps

    • 11.1 Multi-Stage Builds
    • 11.2 Private Registries
    • 11.3 Security Considerations
    • 11.4 Introduction to Orchestration (Kubernetes, Docker Swarm)

  12. Appendix A: Glossary of Docker Terms

  13. Appendix B: Common Docker Commands Cheat Sheet

Chapter 1: What is Docker and Why Do We Need It?

  • 1.1 The "Works on My Machine" Problem

    • Developers build apps in specific environments (OS, libraries, dependencies).
    • When deploying to staging/production, differences in these environments cause bugs.
    • Docker aims to package an application with ALL its dependencies, ensuring consistency across environments.

  • 1.2 Virtual Machines (VMs) vs. Containers

    • VMs: Emulate an entire hardware system, including a guest OS. Heavyweight, slow to boot, resource-intensive.

      +-----------------+  +-----------------+
|      App A      |  |      App B      |
+-----------------+  +-----------------+
|   Bins/Libs A   |  |   Bins/Libs B   |
+-----------------+  +-----------------+
|    Guest OS A   |  |    Guest OS B   |
+-----------------+  +-----------------+
|    Hypervisor   |
+-----------------+
|    Host OS      |
+-----------------+
|    Hardware     |
+-----------------+

    • Containers: Share the host OS kernel. Lightweight, fast boot, less overhead. Isolate processes.

      +-----------------+  +-----------------+
|      App A      |  |      App B      |
+-----------------+  +-----------------+
|   Bins/Libs A   |  |   Bins/Libs B   |
+-----------------+  +-----------------+
|  Docker Engine  |
+-----------------+
|    Host OS      |
+-----------------+
|    Hardware     |
+-----------------+

  • 1.3 Core Benefits of Docker

    • Consistency: Same environment from dev to production.
    • Portability: Run anywhere Docker is installed.
    • Isolation: Apps run in isolated environments, preventing conflicts.
    • Scalability: Easy to scale up/down instances of an application.
    • Speed: Fast deployment and startup compared to VMs.
    • Efficiency: Better resource utilization.

  • 1.4 Key Docker Terminology (Brief Overview)

    • Image: A read-only template with instructions for creating a Docker container. (Blueprint)
    • Container: A runnable instance of an image. (The actual house built from the blueprint)
    • Dockerfile: A text file with commands to assemble an image.
    • Registry: A storage system for Docker images (e.g., Docker Hub, private registries).
    • Docker Engine: The underlying client-server technology that builds and runs containers.

Chapter 2: Docker's Core Components

  • 2.1 Docker Engine

    • Docker Daemon (dockerd): A persistent background process that manages Docker objects (images, containers, networks, volumes). Listens for Docker API requests.
    • REST API: Specifies interfaces that programs can use to talk to the daemon and instruct it.
    • Docker CLI (docker): The command-line tool that users interact with. It uses the Docker API to control or interact with the Docker daemon.

  • 2.2 Images: The Blueprints

    • Read-only templates.
    • Composed of multiple layers (e.g., base OS, then a library, then app code). Each instruction in a Dockerfile creates a layer.
    • Layers are cached and reused, making builds and sharing efficient.
    • Stored in a Docker registry.

  • 2.3 Containers: The Running Instances

    • A live, runnable instance of an image.
    • When a container is created, Docker adds a thin writable layer on top of the image layers.
    • Changes made inside a container (e.g., creating/modifying files) are stored in this writable layer.
    • Containers are isolated from each other and the host machine, but can be configured to communicate.
    • Can be started, stopped, moved, and deleted.

  • 2.4 Registries: Storing and Sharing Images

    • A repository for Docker images.
    • Docker Hub: The default public registry. Contains thousands of official and community images.
    • Private Registries: Can be hosted on-premises or by cloud providers (e.g., AWS ECR, Google GCR, Azure ACR) for private image storage.
    • Commands: docker login, docker pull, docker push.

Chapter 3: Getting Started - Installation & First Steps

  • 3.1 Installing Docker

    • Docker Desktop (Windows/macOS): Easiest way for local development. Includes Docker Engine, CLI, Docker Compose, Kubernetes. Download from Docker's official website.

    • Linux: Usually installed via package manager (e.g., apt for Debian/Ubuntu, yum for CentOS). Follow official Docker installation guides.

      • Often requires adding Docker's official GPG key and repository.

      • Post-installation: Add your user to the docker group to run Docker commands without sudo.

        sudo groupadd docker # if it doesn't exist
sudo usermod -aG docker $USER
newgrp docker # or log out and log back in

  • 3.2 Verifying Installation

    docker --version  # Shows Docker CLI version
docker version    # Shows Client and Server (Engine) version details
docker info       # Detailed system-wide information

  • 3.3 Running Your First Container: hello-world

    • This image is designed to test that your installation is working correctly. 
    docker run hello-world
    • What happens:
      1. CLI sends command to Docker Daemon.
      2. Daemon checks if hello-world image is local.
      3. If not, it pulls it from Docker Hub.
      4. Daemon creates a new container from the image.
      5. Daemon runs the executable within the container.
      6. The container prints a message and exits.

  • 3.4 Basic Docker Commands

    • docker run [OPTIONS] IMAGE [COMMAND] [ARG...]: Create and run a new container from an image.
      • docker run -d nginx: Run Nginx web server in detached mode (background).
      • docker run -it ubuntu bash: Run an interactive terminal in an Ubuntu container.
    • docker ps: List running containers.
      • docker ps -a: List all containers (running and stopped).
    • docker images: List locally available images.
    • docker stop <CONTAINER_ID_OR_NAME>: Stop one or more running containers.
    • docker rm <CONTAINER_ID_OR_NAME>: Remove one or more stopped containers.

Chapter 4: Working with Docker Images

  • 4.1 Pulling Images from a Registry (docker pull)

    • Downloads an image or a repository from a registry. 
    docker pull ubuntu            # Pulls the latest Ubuntu image
docker pull ubuntu:20.04      # Pulls Ubuntu with tag 20.04
docker pull redis:alpine      # Pulls a specific Redis variant (alpine is small)
docker pull myregistry.com/myimage:tag # Pull from a private registry

  • 4.2 Listing Images (docker images)

    • Displays all top-level images, their repository, tags, and size. 
    docker images
# REPOSITORY   TAG       IMAGE ID       CREATED        SIZE
# ubuntu       latest    xxxxxxxxxxxx   2 weeks ago    72.9MB
# nginx        latest    yyyyyyyyyyyy   3 weeks ago    133MB

  • 4.3 Running Containers from Images (docker run in detail)

    • docker run [OPTIONS] IMAGE [COMMAND] [ARG...]
    • Common Options:
      • -d or --detach: Run container in background (detached mode).
      • -it: Interactive + TTY. Combines -i (keep STDIN open) and -t (allocate a pseudo-TTY). Used for interactive shells.
      • -p <host_port>:<container_port>: Publish a container's port(s) to the host.
        • docker run -d -p 8080:80 nginx (Access Nginx on host's port 8080)
      • --name <name>: Assign a name to the container.
        • docker run -d --name my_web_server nginx
      • -v <host_path>:<container_path> or <volume_name>:<container_path>: Mount a volume. (More in Ch 7)
      • -e <KEY>=<VALUE> or --env <KEY>=<VALUE>: Set environment variables.
      • --rm: Automatically remove the container when it exits. Useful for short-lived tasks.
        • docker run --rm ubuntu echo "Hello"

  • 4.4 Removing Images (docker rmi)

    • Removes one or more images.
    • You cannot remove an image if it's used by any container (even stopped). Remove containers first. 
    docker rmi ubuntu:20.04
docker rmi <IMAGE_ID>
docker rmi $(docker images -q -f "dangling=true") # Remove dangling (untagged) images

  • 4.5 Understanding Image Layers

    • Each instruction in a Dockerfile (RUN, COPY, ADD) creates a new layer.
    • Layers are stacked on top of each other.
    • Layers are read-only.
    • When an image is pulled, Docker pulls only the layers it doesn't already have.
    • When a container is started, a thin writable layer is added on top.
    • docker history <IMAGE_NAME_OR_ID> shows the layers of an image.

Chapter 5: Building Your Own Images with Dockerfile

  • 5.1 Introduction to Dockerfile

    • A text document that contains all the commands a user could call on the command line to assemble an image.
    • Automates the image creation process.
    • Default name: Dockerfile (case sensitive).

  • 5.2 Common Dockerfile Instructions

    • FROM <image>[:<tag>]: Specifies the base image. Must be the first instruction.
      • FROM ubuntu:20.04
    • RUN <command>: Executes any commands in a new layer on top of the current image. Often used for installing packages.
      • RUN apt-get update && apt-get install -y python3
    • COPY <src> <dest>: Copies new files or directories from <src> (host build context) and adds them to the filesystem of the container at path <dest>.
      • COPY ./app /app
    • ADD <src> <dest>: Similar to COPY, but can also handle URLs and auto-extract tar files. COPY is generally preferred for its explicitness.
    • WORKDIR /path/to/workdir: Sets the working directory for subsequent RUN, CMD, ENTRYPOINT, COPY, ADD instructions.
      • WORKDIR /app
    • CMD ["executable","param1","param2"] (exec form, preferred) or CMD command param1 param2 (shell form)
      • Provides defaults for an executing container. These defaults can be overridden when starting a container with docker run.
      • Only the last CMD instruction in a Dockerfile takes effect.
      • CMD ["python3", "app.py"]
    • ENTRYPOINT ["executable","param1","param2"] (exec form, preferred)
      • Configures a container that will run as an executable.
      • Arguments to docker run <image> will be appended after ENTRYPOINT parameters.
      • Harder to override than CMD. CMD can be used to provide default arguments to ENTRYPOINT.
      • ENTRYPOINT ["nginx", "-g", "daemon off;"]
    • EXPOSE <port> [<port>/<protocol>...]: Informs Docker that the container listens on the specified network ports at runtime. Does NOT actually publish the port. It's documentation. Use -p in docker run to publish.
      • EXPOSE 80
    • ENV <key>=<value>: Sets environment variables.
      • ENV APP_VERSION=1.0
    • USER <username_or_UID>: Sets the user name (or UID) and optionally the user group (or GID) to use when running the image and for any RUN, CMD and ENTRYPOINT instructions that follow it in the Dockerfile.
    • ARG <name>[=<default value>]: Defines a build-time variable. Can be set using --build-arg with docker build. Not available in running containers unless also set with ENV.

  • 5.3 Building an Image (docker build)

    • docker build [OPTIONS] PATH | URL | -
    • The PATH is the build context – files at this location are sent to the Docker daemon.
    • Common Options:
      • -t <name>[:<tag>] or --tag <name>[:<tag>]: Name and optionally a tag in the 'name:tag' format.
        • docker build -t my-app:1.0 . (Builds from Dockerfile in current directory .)
      • -f <path/to/Dockerfile> or --file <path/to/Dockerfile>: Specify path to Dockerfile.
        • docker build -f Dockerfiles/dev.Dockerfile -t my-dev-app .
      • --build-arg <varname>=<value>: Set build-time variables.

  • 5.4 Tagging Images

    • Tags are aliases to image IDs, making them human-readable.
    • Format: repository:tag (e.g., myapp:latest, myapp:v1.2, username/myapp:v1.2).
    • Can be applied during build (-t) or after with docker tag. 
    docker tag <SOURCE_IMAGE_ID_OR_NAME:TAG> <TARGET_IMAGE_NAME:TAG>
docker tag my-app:1.0 myusername/my-app:1.0

  • 5.5 .dockerignore File

    • Similar to .gitignore.
    • A file named .dockerignore in the root of the build context.
    • Specifies files and directories to exclude from the build context sent to the Docker daemon.
    • Helps avoid sending large or sensitive files/directories, speeding up builds and improving security.

    • Example .dockerignore:

      .git
node_modules
*.log
secrets/

  • 5.6 Best Practices for Writing Dockerfiles

    • Use a specific base image tag: Avoid latest for production (e.g., python:3.9-slim instead of python:latest).
    • Keep images small:
      • Use small base images (e.g., alpine).
      • Clean up unnecessary files after RUN (e.g., apt-get clean).
      • Combine RUN instructions where logical to reduce layers (e.g., apt-get update && apt-get install -y ... && rm -rf /var/lib/apt/lists/*).
      • Use multi-stage builds (see Ch 11).
    • Order matters for caching: Place instructions that change less frequently (like FROM, installing dependencies) before instructions that change often (like COPY . /app).
    • Use .dockerignore.
    • Minimize layers: Each instruction creates a layer. Group related commands.
    • Prefer COPY over ADD unless you specifically need ADD's tar extraction or URL features.
    • Run as non-root user: Use USER instruction.
    • Use exec form for CMD and ENTRYPOINT: CMD ["executable", "param1"] is better than CMD executable param1.

Chapter 6: Managing Containers

  • 6.1 Listing Containers (docker ps, docker ps -a)

    • docker ps: Shows only running containers.
    • docker ps -a or docker ps --all: Shows all containers (running and stopped).
    • docker ps -q: Shows only container IDs (quiet). Useful for scripting.

  • 6.2 Starting, Stopping, and Restarting Containers

    • docker start <CONTAINER_ID_OR_NAME>: Starts one or more stopped containers.
    • docker stop <CONTAINER_ID_OR_NAME>: Stops one or more running containers gracefully (sends SIGTERM, then SIGKILL after timeout).
    • docker kill <CONTAINER_ID_OR_NAME>: Kills one or more running containers immediately (sends SIGKILL).
    • docker restart <CONTAINER_ID_OR_NAME>: Restarts one or more containers.

  • 6.3 Removing Containers (docker rm)

    • Removes one or more stopped containers.
    • docker rm <CONTAINER_ID_OR_NAME>
    • docker rm -f <CONTAINER_ID_OR_NAME> or docker rm --force <CONTAINER_ID_OR_NAME>: Force remove a running container.
    • docker rm $(docker ps -aq -f "status=exited"): Remove all exited containers.

  • 6.4 Viewing Container Logs (docker logs)

    • Fetches the logs of a container.
    • docker logs <CONTAINER_ID_OR_NAME>
    • Options:
      • -f or --follow: Follow log output.
      • --tail <N>: Show the last N lines.
      • --since <TIMESTAMP>: Show logs since a timestamp (e.g., "2023-10-26T10:00:00").
      • --until <TIMESTAMP>: Show logs before a timestamp.
      • -t or --timestamps: Show timestamps.

  • 6.5 Executing Commands in a Running Container (docker exec)

    • Runs a new command in a running container.
    • docker exec [OPTIONS] <CONTAINER_ID_OR_NAME> <COMMAND>

    • Common options:

      • -i: Keep STDIN open.
      • -t: Allocate a pseudo-TTY.

      • -it: Often used together for interactive shells:

        docker exec -it my_nginx_container bash  # Opens a bash shell in my_nginx_container
docker exec my_postgres_container psql -U user -d dbname # Run psql non-interactively

      • -d: Detached mode.

  • 6.6 Inspecting Containers (docker inspect)

    • Returns low-level information on Docker objects (containers, images, volumes, networks).
    • Output is in JSON format.
    • docker inspect <CONTAINER_ID_OR_NAME>
    • docker inspect <IMAGE_ID_OR_NAME>

    • Can use --format to extract specific information using Go template syntax.

      docker inspect --format='{{.NetworkSettings.IPAddress}}' my_container

Chapter 7: Data Persistence with Volumes

  • 7.1 The Ephemeral Nature of Container File Systems

    • When a container is removed (docker rm), any data written to its writable layer is lost.
    • This is problematic for applications that need to store persistent data (databases, user uploads, logs).

  • 7.2 What are Volumes?

    • The preferred mechanism for persisting data generated by and used by Docker containers.
    • Managed by Docker (/var/lib/docker/volumes/ on Linux hosts by default).
    • Decoupled from the container lifecycle: volumes persist even if containers are removed.
    • Can be shared among multiple containers.

  • 7.3 Types of Mounts

    • Volumes:
      • Stored in a part of the host filesystem managed by Docker.
      • Non-Docker processes on the host should not modify this part of the filesystem.
      • Best choice for most use cases.
      • Platform-agnostic (work the same on Linux, Windows, macOS).
    • Bind Mounts:
      • Mount a file or directory from the host machine's filesystem into a container.
      • The path on the host machine can be anywhere.
      • Useful for development (e.g., mounting source code into a container for live reloading).
      • Host-dependent paths, potential permission issues.
    • tmpfs Mounts (Linux only):
      • Stored in the host's memory only, never written to the host filesystem. For temporary, sensitive data.

  • 7.4 Creating and Managing Volumes

    • docker volume create <VOLUME_NAME>: Create a named volume.

      docker volume create my_data_volume

    • docker volume ls: List volumes.

    • docker volume inspect <VOLUME_NAME>: Display detailed information on one or more volumes.

    • docker volume rm <VOLUME_NAME>: Remove one or more volumes.

      • A volume cannot be removed if it is currently used by a container.

    • docker volume prune: Remove all unused local volumes.

  • 7.5 Using Volumes with Containers (-v or --mount flag)

    • Using -v (or --volume):

      • Named Volume: -v <VOLUME_NAME>:<CONTAINER_PATH>

        docker run -d --name my_db -v pg_data:/var/lib/postgresql/data postgres:13
# 'pg_data' will be created by Docker if it doesn't exist.

      • Bind Mount: -v <HOST_PATH>:<CONTAINER_PATH>

        docker run -d --name my_app -v /path/on/host/app_code:/usr/src/app my_app_image
# Note: For relative paths on host, use $(pwd)/path
# docker run -d --name my_app -v $(pwd)/app_code:/usr/src/app my_app_image

      • Anonymous Volume: -v <CONTAINER_PATH> (Docker creates a volume with a random ID)

        • Generally not recommended as they are harder to reference later.

    • Using --mount: More explicit and verbose. Preferred for clarity, especially with more options.

      • Syntax: --mount type=<TYPE>,source=<SOURCE>,target=<TARGET>[,<OPTIONS>]

      • Named Volume:

        docker run -d --name my_db --mount type=volume,source=pg_data,target=/var/lib/postgresql/data postgres:13

      • Bind Mount:

        docker run -d --name my_app --mount type=bind,source=/path/on/host/app_code,target=/usr/src/app my_app_image

      • Can add options like readonly: --mount type=volume,source=my_config,target=/etc/config,readonly

Chapter 8: Docker Networking

  • 8.1 Introduction to Docker Networking

    • Containers need to communicate with each other and the outside world.
    • Docker provides networking capabilities to achieve this.

  • 8.2 Default Networks

    • When Docker is installed, it creates three networks automatically:
      • bridge: The default network for containers if none is specified. Containers on the same bridge network can communicate by IP address. For external access, ports must be mapped.
      • host: Removes network isolation between the container and the Docker host. The container shares the host's networking namespace. If you run a container that binds to port 80 using host networking, the container's application is available on port 80 on the host's IP address.
      • none: Disables all networking for the container. It gets a loopback interface but no external network interface.

  • 8.3 Port Mapping (-p or --publish flag)

    • Allows access to a container's application from the host machine or external network.
    • docker run -p <HOST_PORT>:<CONTAINER_PORT> <IMAGE>
      • docker run -d -p 8080:80 nginx (Host's port 8080 maps to container's port 80)
    • docker run -p <IP_ADDRESS>:<HOST_PORT>:<CONTAINER_PORT> <IMAGE> (Bind to a specific IP on the host)
    • docker run -p <CONTAINER_PORT> <IMAGE> (Assign a random available host port)

  • 8.4 User-Defined Bridge Networks for Container Communication

    • Recommended over the default bridge network for production.
    • Provide better isolation and automatic DNS resolution between containers on the same network.
    • Containers can refer to each other by their names.

    • Create a network:

      docker network create my_app_network

    • Run containers on the network:

      docker run -d --name my_db --network my_app_network postgres_image
docker run -d --name my_web_app --network my_app_network -p 80:8080 my_webapp_image
      • Now my_web_app can connect to my_db using hostname my_db (e.g., postgres://my_db:5432).

  • 8.5 Linking Containers (Legacy, good to know)

    • --link <CONTAINER_NAME_OR_ID>:<ALIAS>
    • Used on the default bridge network before user-defined networks became prevalent.
    • Injected environment variables and /etc/hosts entries into the receiving container.
    • Considered legacy; user-defined bridge networks are preferred.

  • 8.6 Inspecting Networks

    • docker network ls: List networks.
    • docker network inspect <NETWORK_NAME_OR_ID>: Show detailed information about a network, including connected containers.
    • docker network connect <NETWORK_NAME> <CONTAINER_NAME>: Connect a running container to a network.
    • docker network disconnect <NETWORK_NAME> <CONTAINER_NAME>: Disconnect a container from a network.

Chapter 9: Multi-Container Applications with Docker Compose

  • 9.1 What is Docker Compose?

    • A tool for defining and running multi-container Docker applications.
    • Uses a YAML file (docker-compose.yml) to configure application services, networks, and volumes.
    • Simplifies management of complex applications with a single command.

  • 9.2 The docker-compose.yml File

    • Default filename: docker-compose.yml or docker-compose.yaml.
    • Defines services, networks, and volumes for an application.

  • 9.3 Key Compose Directives

    • version: Specifies the Compose file format version (e.g., '3.8'). (Less critical now, but good practice).

    • services: Defines the different containers (services) that make up your application.

      • Each service has a name (e.g., web, db, api).
      • image: <image_name>:<tag>: Specifies the image to start the container from.

      • build: <path_to_dockerfile_context>: Specifies the build context (directory containing Dockerfile) or detailed build options.

        # Simple build context
build: ./my-app
# Detailed build options
build:
  context: ./my-app
  dockerfile: Dockerfile-dev
  args:
    APP_VERSION: 1.2

      • ports: ["<HOST_PORT>:<CONTAINER_PORT>"]: Exposes ports.

      • volumes: ["<HOST_PATH_OR_NAMED_VOLUME>:<CONTAINER_PATH>"]: Mounts volumes.

        volumes:
  - ./my-app-code:/code
  - db_data:/var/lib/postgresql/data # Named volume

      • environment: ["<KEY>=<VALUE>"] or environment: {KEY: VALUE}: Sets environment variables.

      • depends_on: [<service_name>]: Specifies service dependencies. Compose starts dependencies before the service. Note: This only waits for the container to start, not for the application inside to be ready.

      • networks: [<network_name>]: Connects services to specified networks.

      • command: <command_to_run>: Overrides the default command for the image.

      • restart: "always" | "on-failure" | "unless-stopped": Restart policy.

    • networks: Defines custom networks.

      networks:
  app_net:
    driver: bridge

    • volumes: Defines named volumes.

      volumes:
  db_data: # This defines a named volume 'db_data'
    driver: local # Optional, default is local

  • 9.4 Common Compose Commands

    • (Assumes you are in the directory with docker-compose.yml)
    • docker-compose up: Builds (if necessary), creates, starts, and attaches to containers for a service.
      • docker-compose up -d: Start services in detached mode.
    • docker-compose down: Stops and removes containers, networks, and (optionally) volumes created by up.
      • docker-compose down -v: Remove named volumes declared in the volumes section.
    • docker-compose ps: Lists containers for the services.
    • docker-compose logs [SERVICE_NAME]: Displays log output from services.
      • docker-compose logs -f web: Follow logs for the web service.
    • docker-compose exec <SERVICE_NAME> <COMMAND>: Execute a command in a running service container.
      • docker-compose exec web bash
    • docker-compose build [SERVICE_NAME]: Builds or rebuilds services.
    • docker-compose pull [SERVICE_NAME]: Pulls service images.
    • docker-compose stop [SERVICE_NAME]: Stops running containers without removing them.
    • docker-compose start [SERVICE_NAME]: Starts existing containers.
    • docker-compose rm [SERVICE_NAME]: Removes stopped service containers.

  • 9.5 Example: Web App + Database (docker-compose.yml)

    version: '3.8'

services:
  web:
    build: ./webapp  # Assumes Dockerfile is in ./webapp directory
    ports:
      - "5000:5000" # Host port 5000 maps to container port 5000
    volumes:
      - ./webapp:/usr/src/app # Mount webapp code for development
    networks:
      - app-network
    depends_on:
      - db
    environment:
      - DATABASE_URL=postgresql://user:password@db:5432/mydatabase
      - FLASK_ENV=development

  db:
    image: postgres:13-alpine
    volumes:
      - db_data:/var/lib/postgresql/data # Persist database data
    networks:
      - app-network
    environment:
      - POSTGRES_USER=user
      - POSTGRES_PASSWORD=password
      - POSTGRES_DB=mydatabase
    ports: # Optional: expose db port to host for direct access/debugging
      - "5432:5432"

networks:
  app-network:
    driver: bridge

volumes:
  db_data: # Defines the named volume 'db_data'

Chapter 10: Cleaning Up Docker Resources

  • Docker objects (images, containers, volumes, networks) can consume significant disk space. Regular cleanup is important.

  • 10.1 Removing Dangling Resources

    • Dangling images are layers that have no relationship to any tagged images. They often result from rebuilding images.
    • docker image prune: Remove dangling images.
      • docker image prune -a: Remove all unused images (not just dangling).
    • docker volume prune: Remove all unused local volumes (volumes not used by any container).
    • docker network prune: Remove all unused networks.
    • docker container prune: Remove all stopped containers.

  • 10.2 docker system prune

    • A shortcut command to clean up multiple resource types at once.
    • docker system prune: Removes:
      • All stopped containers
      • All unused networks
      • All dangling images
      • All build cache
    • docker system prune -a: Additionally removes all unused images (not just dangling ones).
    • docker system prune --volumes: Also removes all unused volumes (use with caution!).

  • 10.3 Selective Pruning

    • You can use filters with list commands and then pipe to rm / rmi.

    • Remove all containers created more than 24 hours ago:

      docker ps -a --filter "status=exited" --filter "created_before=$(date -d '24 hours ago' -u +%Y-%m-%dT%H:%M:%SZ)" -q | xargs --no-run-if-empty docker rm

    • Remove images with a specific label:

      docker images --filter "label=stage=test" -q | xargs --no-run-if-empty docker rmi

Chapter 11: Advanced Topics & Next Steps

  • 11.1 Multi-Stage Builds

    • A technique to create smaller, more secure production images.
    • Uses multiple FROM statements in a Dockerfile. Each FROM starts a new build stage.
    • You can selectively COPY artifacts (like compiled binaries, static assets) from an earlier stage (e.g., a build stage with SDKs and tools) to a later, leaner stage (e.g., a runtime stage with only the necessary runtime).

    • Example:

      # Stage 1: Build stage (with build tools)
FROM golang:1.19-alpine AS builder
WORKDIR /app
COPY . .
RUN go build -o myapp .

# Stage 2: Production stage (lean runtime)
FROM alpine:latest
WORKDIR /root/
COPY --from=builder /app/myapp .
# Copy other necessary files like configs if needed
# COPY --from=builder /app/config.json .
CMD ["./myapp"]

    • Result: Final image only contains the myapp binary and alpine base, not the Go SDK.

  • 11.2 Private Registries

    • For storing proprietary images you don't want on Docker Hub.

    • Options:

      • Docker Hub Private Repositories: Paid feature of Docker Hub.

      • Self-hosted Registry: Run Docker's registry image.

        docker run -d -p 5000:5000 --restart=always --name registry registry:2

        Then tag images: docker tag my_image localhost:5000/my_image and push: docker push localhost:5000/my_image.
        (Needs TLS for non-localhost access in production).

      • Cloud Provider Registries: AWS ECR, Google GCR, Azure ACR, GitLab Container Registry, GitHub Packages. Offer IAM integration, scalability, etc.

  • 11.3 Security Considerations

    • Use official/verified base images.
    • Keep images updated: Regularly rebuild images to get security patches.
    • Run as non-root user: Use USER instruction in Dockerfile.
    • Minimize attack surface: Only include necessary packages/files. Use multi-stage builds.
    • Scan images for vulnerabilities: Tools like Trivy, Clair, Docker Scout.
    • Limit container capabilities: Use --cap-drop and --cap-add.
    • Use read-only root filesystem: --read-only flag.
    • Manage secrets securely: Use Docker secrets, HashiCorp Vault, or cloud provider secret managers (not environment variables for sensitive data in production).

  • 11.4 Introduction to Orchestration

    • Manages the lifecycle of containers at scale, especially in distributed environments.
    • Handles deployment, scaling, load balancing, service discovery, self-healing.
    • Kubernetes (K8s): The de-facto standard. Powerful, complex, large ecosystem.
    • Docker Swarm: Docker's native orchestration. Simpler than Kubernetes, good for smaller deployments.
    • Cloud provider services (AWS ECS, Azure Container Instances, Google Cloud Run) often build on or integrate with these.

Appendix A: Glossary of Docker Terms

  • Image: A read-only template used to create containers. Contains the application and all its dependencies.
  • Container: A runnable instance of an image. It's an isolated environment for your application.
  • Dockerfile: A script containing instructions to build a Docker image.
  • Registry: A storage system for Docker images (e.g., Docker Hub).
  • Repository: A collection of related Docker images, usually different versions of the same application (e.g., ubuntu repository, with tags 20.04, latest).
  • Tag: A label applied to an image to distinguish versions or variants (e.g., myapp:1.0, myapp:latest).
  • Layer: Images are built in layers. Each instruction in a Dockerfile creates a new layer.
  • Volume: A mechanism for persisting data generated by and used by Docker containers, managed by Docker.
  • Bind Mount: Mounts a file or directory from the host system into a container.
  • Network: Enables communication between containers and between containers and the host/external world.
  • Docker Engine: The core Docker software, a client-server application with a daemon process, a REST API, and a CLI.
  • Docker Compose: A tool for defining and running multi-container Docker applications using a YAML file.
  • Orchestration: Managing the lifecycle of containers at scale (e.g., Kubernetes, Docker Swarm).
  • Dangling Image: An image layer that is not associated with any tagged image.

Appendix B: Common Docker Commands Cheat Sheet

Images:

  • docker images: List images
  • docker pull <image>:<tag>: Download image
  • docker rmi <image_id_or_name>: Remove image
  • docker build -t <name>:<tag> .: Build image from Dockerfile in current dir
  • docker history <image>: Show image layers
  • docker tag <source_image> <target_image>: Tag an image

Containers:

  • docker run [options] <image> [command]: Create and run a new container
    • -d: Detached (background)
    • -it: Interactive TTY
    • -p <host_port>:<container_port>: Port mapping
    • --name <name>: Assign name
    • -v <volume_or_host_path>:<container_path>: Mount volume/bind mount
    • -e <KEY>=<VALUE>: Environment variable
    • --rm: Auto remove on exit
  • docker ps: List running containers
  • docker ps -a: List all containers
  • docker stop <container_id_or_name>: Stop container
  • docker start <container_id_or_name>: Start stopped container
  • docker restart <container_id_or_name>: Restart container
  • docker rm <container_id_or_name>: Remove container
  • docker logs <container_id_or_name>: View container logs
    • -f: Follow logs
  • docker exec -it <container_id_or_name> <command>: Execute command in running container (e.g., bash)
  • docker inspect <container_id_or_name>: Low-level info

Volumes:

  • docker volume create <name>: Create volume
  • docker volume ls: List volumes
  • docker volume inspect <name>: Inspect volume
  • docker volume rm <name>: Remove volume
  • docker volume prune: Remove unused volumes

Networks:

  • docker network ls: List networks
  • docker network create <name>: Create network
  • docker network inspect <name>: Inspect network
  • docker network rm <name>: Remove network
  • docker network connect <network> <container>: Connect container
  • docker network disconnect <network> <container>: Disconnect container

Docker Compose (in dir with docker-compose.yml):

  • docker-compose up -d: Start services in background
  • docker-compose down: Stop and remove services, networks
  • docker-compose down -v: Also remove volumes
  • docker-compose ps: List services' containers
  • docker-compose logs [service_name]: View logs
  • docker-compose exec <service_name> <command>: Exec command in service
  • docker-compose build [service_name]: Build/rebuild services
  • docker-compose pull [service_name]: Pull service images

System / Cleanup:

  • docker version: Show Docker version info
  • docker info: Display system-wide info
  • docker system df: Show Docker disk usage
  • docker system prune: Remove stopped containers, unused networks, dangling images, build cache
    • -a: Also remove unused images
    • --volumes: Also remove unused volumes

Top comments (1)

Collapse
 
victorpy4 profile image
v-py4 •

Thanks to this drop of info! It's very helpful as a doc to consult each time to time.