How to Handle Storage in Docker?

Why should i need to store data?

Lets have a scenario here, we have our Mysql containers running and we try to fetch data in a application. While fetching the data, container stopped abbruptly. if you start container again and request for data. NOW ALL YOUR DATA IS GONE !

1. The life of the data depends on the container, once container is gone, data too is gone, even if another process needs it , bring back could be herculean task

2.Container Writeable layer is tightly coupled with host ,
where you can't move the data.

3.If need to store data , we can use storage driver to manage
file system, then again this extra abstraction will reduce
the performence

Which data should I back-up?

We need to back up data to a permanent storage. We have 2 layers of data. Read only layer(permanently stores data) and Read/Write data(Volatile). Obivous we need to back up R/W data.

Where Should to Backup ?

Docker provides storage objects:

1. Volume
2. Bind mount
3. temp fs

Volume:

1. Managed by Docker,Dedicated Directory in host's file system which are mounted on containers.
2. We can use Volume for multiple containers simulatneously, No automatic deletion of volumes, we need to delete if its not required.

3. It may be named or anonymous. Anonymous volumes are not given an explicit name when they are first mounted, so Docker provides unique random name withing the Docker host.Named volumes can persist data after we restart or remove a container. Also, it's accessible by other containers.

4. Volumes supports volume drivers, which allow you to store your data on remote hosts or cloud providers.

Volume:

Container Provides data and User provides commands to store/manage data to docker engine. But what container knows about is just name of the volume but not the path of the volume in the host. Even the external application having access to the container, wont be able to access Data stored in volume. Providing isolation and security for both host and containers

where can we use ?
1. Sharing data among multiple running containers.
2. When the Docker host is not guaranteed to have a
given directory or file structure.
3. When you want to store your container’s data on a
remote host or a cloud provider, rather than locally.
4. We can use for backup,restore or migration from one
host to another
5. Application requires high-performance I/O on Docker
Desktop. and also fully native files system.

Bind mount:
Its is very similar to Volume mounts, but with limited benefits. A file or Directory in host system is mounted on container. It is referenced by absolute path on host machine.

It is created on demand if not existed. Bind mounts useful in this case but it expects the host system to contain the sprcific directory structure. hence developer himself sometimes might not have that structure in his host.

since it exposes the storage location of the container, which can make dents on the overall security of application or host.

consider using named volumes. we can't use CLI commands to directly manage bind mounts.

where can we use ?

1. Sharing configuration file from host to containers. By this docker provides DNS resolution to containers, by mounting /etc/resolve.conf into each container.

2. When we share source code/build artifacts between a development environment host and a container. we may mount a app project/directory on a host, when u create same project everytime , it uses the built artifacts.

If you use Docker for development this way, your production Dockerfile would copy the production-ready artifacts directly into the image, rather than relying on a bind mount.

1. When the file or directory structure of the Docker host is guaranteed to be consistent.

tempfs
tmpfs a temporary file system.

Volumes and bind-mount allows you to share files between host and container, and data is persisted even container is stopped.On the other hand,tmpfs mount, only persists in the host's memory, not in storage. When the container stops, the tmpfs mount removed.

Only in Linux we have tmpfs mounts. When you create a container with tmpfs mount, the container can create files outside the container's writable layer. They can be created not shipped

example we can use this type of storage is like user session, browser history in incognito.

where to use ?

1. When we don't want to persist data, if we want data until the container is running.

Named pipes:
An npipe mount used for communication between the host and container.

where to use ?
This is used to run a third-party tool inside a container and connect to the Docker Engine API using a named pipe.