Docker Architecture, Life Cycle of Docker Containers and Data Management

One common challenge for DevOps teams is managing an application’s dependencies and technology stack across various cloud and development environments. As part of their routine tasks, they must keep the application operational and stable — regardless of the underlying platform that it runs on.

Docker enables efficiency and reduces operational overheads so that any developer, in any development environment, can build stable and reliable applications. Docker provides the ability to package and run an application in a loosely isolated environment called a container. Isolation and security allow you to run many containers simultaneously on a given host. You can easily share containers while you work, and be sure that everyone you share with gets the same container that works in the same way.

Docker Architecture

The Core components of the Docker consists of

1. Docker daemon
2. Docker client
3. Docker Desktop
4. Docker registry
5. Docker images
6. Docker containers

The Docker daemon
The Docker daemon (dockerd) listens for Docker API requests and manages Docker objects such as images, containers, networks, and volumes. A daemon can also communicate with other daemons to manage Docker services.

The Docker client
The Docker client (docker) is the primary way that many Docker users interact with Docker. When you use commands such as docker run, the client sends these commands to dockerd, which carries them out. The docker command uses the Docker API. The Docker client can communicate with more than one daemon.

Docker Desktop
Docker Desktop is an easy-to-install application for your Mac or Windows environment that enables you to build and share containerised applications and Microservices. Docker Desktop includes the Docker daemon (dockerd), the Docker client (docker), Docker Compose, Docker Content Trust, Kubernetes, and Credential Helper. For more information, see Docker Desktop.

Docker registries
A Docker registry stores Docker images. Docker Hub is a public registry that anyone can use, and Docker is configured to look for images on Docker Hub by default. You can even run your own private registry.
When you use the docker pull or docker run commands, the required images are pulled from your configured registry. When you use the docker push command, your image is pushed to your configured registry.

Docker objects
When you use Docker, you are creating and using images, containers, networks, volumes, plugins, and other objects. This section is a brief overview of some of those objects.

Images
An image is a read-only template with instructions for creating a Docker container. Often, an image is based on another image, with some additional customisation. For example, you may build an image which is based on the ubuntu image, but installs the Apache web server and your application, as well as the configuration details needed to make your application run.

You might create your own images or you might only use those created by others and published in a registry. To build your own image, you create a Dockerfile with a simple syntax for defining the steps needed to create the image and run it. Each instruction in a Dockerfile creates a layer in the image. When you change the Dockerfile and rebuild the image, only those layers which have changed are rebuilt. This is part of what makes images so lightweight, small, and fast, when compared to other virtualisation technologies.

Containers
A container is a runnable instance of an image. You can create, start, stop, move, or delete a container using the Docker API or CLI. You can connect a container to one or more networks, attach storage to it, or even create a new image based on its current state.

By default, a container is relatively well isolated from other containers and its host machine. You can control how isolated a container’s network, storage, or other underlying subsystems are from other containers or from the host machine.

A container is defined by its image as well as any configuration options you provide to it when you create or start it. When a container is removed, any changes to its state that are not stored in persistent storage disappear.

Docker Container Lifecycle Management

The lifecycle of a docker container consists of five states:

1. Created state
2. Running state
3. Paused state/unpaused state
4. Stopped state
5. Killed/Deleted state

Created state
In the created state, a docker container is created from a docker image.

docker create --name <name-of-container> <docker-image-name>

Running state
In the running state, the docker container starts executing the commands mentioned in the image. To run a docker container, use the docker run command.

docker run <container-id>
or
docker run <container-name>

The docker run command creates a container if it is not present. In this case, the creation of the container can be skipped.

Paused state
In the paused state, the current executing command in the docker container is paused. Use the docker pause command to pause a running container.

docker pause container <container-id or container-name>

Note: docker pause pauses all the processes in the container. It sends the SIGSTOP signal to pause the processes in the container.

Unpaused state
In the unpaused state, the paused container resumes executing the commands once it is unpaused.

Use the docker unpause command to resume a paused container.

Then, Docker sends the SIGCONT signal to resume the process.

docker unpause <container-id or container-name>

Stopped state
In the stopped state, the container’s main process is shutdown gracefully. Docker sends SIGTERM for graceful shutdown, and if needed, SIGKILL, to kill the container’s main process.
Use the docker stop command to stop a container.

docker stop <container-id or container-name>

Restarting a docker container would translate to docker stop, then docker run, i.e., stop and run phases.

Killed/Deleted state
In the killed state, the container’s main processes are shutdown abruptly. Docker sends a SIGKILL signal to kill the container’s main process.

docker kill <container-id or container-name>

Complete lifecycle of a Docker container

Data Management in Docker Containers

Docker helps developers to develop faster and to release faster. Now we need to move applications to the new world of Microservices and Containers, where meaningful and persistent data exists. How are we going to achieve this with Docker?

By default all files created inside a container are stored on a writable container layer. This means that:

• The data doesn’t persist when that container no longer exists, and it can be difficult to get the data out of the container if another process needs it.
• A container’s writable layer is tightly coupled to the host machine where the container is running. You can’t easily move the data somewhere else.
• Writing into a container’s writable layer requires a storage driver to manage the filesystem. The storage driver provides a union filesystem, using the Linux kernel. This extra abstraction reduces performance as compared to using data volumes, which write directly to the host filesystem.

Docker Solution for Data Management

Docker has two options for containers to store files on the host machine, so that the files are persisted even after the container stops: volumes, and bind mounts.

Choose the right type of mount
No matter which type of mount you choose to use, the data looks the same from within the container. It is exposed as either a directory or an individual file in the container’s filesystem.

An easy way to visualise the difference among volumes, bind mounts, and tmpfs mounts is to think about where the data lives on the Docker host.

Volumes are stored in a part of the host filesystem which is managed by Docker (/var/lib/docker/volumes/ on Linux).

• Non-Docker processes should not modify this part of the filesystem. Volumes are the best way to persist data in Docker. Volumes are created and managed by Docker.
• You can create a volume explicitly using the docker volume create command, or Docker can create a volume during container or service creation.
• A given volume can be mounted into multiple containers simultaneously.
• You can remove unused volumes using docker volume prune

Bind mounts may be stored anywhere on the host system.
They may even be important system files or directories.

• Non-Docker processes on the Docker host or a Docker container can modify them at any time.
• Bind mounts have limited functionality compared to volumes. When you use a bind mount, a file or directory on the host machine is mounted into a container.
• The file or directory does not need to exist on the Docker host already. It is created on demand if it does not yet exist.
• If you are developing new Docker applications, consider using named volumes instead.
• You can’t use Docker CLI commands to directly manage bind mounts.

tmpfs mount is not persisted on disk, either on the Docker host or within a container.

• tmpfs mounts are stored in the host system’s memory only, and are never written to the host system’s filesystem.
• It can be used by a container during the lifetime of the container, to store non-persistent state or sensitive information.

Conclusion

Docker is an open platform for developing, shipping, and running applications. Docker enables you to separate your applications from your infrastructure so you can deliver software quickly. With Docker, you can manage your infrastructure in the same ways you manage your applications.

Volumes are the preferred mechanism for persisting data generated by and used by Docker containers. While bind mounts are dependent on the directory structure and OS of the host machine, volumes are completely managed by Docker. Volumes have several advantages over bind mounts:

• Volumes are easier to back up or migrate than bind mounts.
• You can manage volumes using Docker CLI commands or the Docker API.
• Volumes work on both Linux and Windows containers.
• Volumes can be more safely shared among multiple containers.

References

https://docs.docker.com/get-started/overview
https://docs.docker.com/storage
https://docs.docker.com/storage/volumes
https://docs.docker.com/storage/bind-mounts