Docker compose, orchestrating and automating services

Docker compose is a tool that allows us to replace most of the commands and configurations that we normally run in the terminal to orchestrate and automate Docker services: such as creating several containers based on different images, connected to each other by networks and using volumes to persist their data, for example.

Even working on a simple dockerized project, we often run several different Docker commands in the terminal. And this entire process must be memorized following a specific order to execute the same commands as many times as necessary during the development life cycle. I think you can imagine how complicated it can be especially on large projects, right?

What’s Docker compose?

If you are the kind of developer who saves Docker commands in a text file along with other instructions that you need to memorize, I would like to introduce Docker compose to you. :)

“Compose simplifies the control of your entire application stack, making it easy to manage services, networks, and volumes in a single, comprehensible YAML configuration file. Then, with a single command, you create and start all the services from your configuration file.” - Docker documentation

Docker commands to be converted into Docker compose

In the next section you will find a Docker compose file example containing commands to create 3 different containers used in a same application: "database" container using Mongo, "backend" using Node and "frontend" using React. The database container will be based on a Docker hub image, while the backend and frontend will have specific directories in the local project with their own custom images and files. The database and backend containers must be connected as we need to run CRUD methods on the database and we do this using Docker networking. In addition to these settings, we will also have other settings such as ports to provide a way to make requests to the backend from the frontend, and volumes and bind mounts to persist data.

Let's imagine the following basic directories and files structure

├── backend/
│ ├── Dockerfile
│ └── app.js
├── env/
│ ├── backend.env
│ └── mongo.env
├── frontend/
│ ├── Dockerfile
│ └── src
├── compose.yaml

You can check out all the Docker commands from the above scenario - and their full explanations -(which we could run in the terminal, but which will be translated into configurations in Docker compose file) in two articles I wrote before called How to use Docker Images, Containers, Volumes and Bind Mounts and How to connect different containers with Docker networking.

Creating a Docker compose file

Now it’s time to create the compose file based on the scenario described in the previous section.

Docker compose file

First of all, create a file in the root project directories called compose.yaml. YAML is a text format that uses indentation to specify dependencies between configuration options. Be aware that incorrect indentation will cause problems with executing commands properly.

services:
  mongodb:
    image: 'mongo'
    volumes:
      - data:/data/db
    env_file:
      - ./env/mongo.env
  backend:
    build: ./backend
    ports:
      - '80:80'
    volumes:
      - ./backend:/app
      - /app/node_modules
    env_file:
      - ./env/backend.env
    depends_on:
      - mongodb
  frontend:
    build: ./frontend
    ports:
      - '3000:3000'
    volumes:
      - ./frontend/src:/app/src
    stdin_open: true
    tty: true
    depends_on:
      - backend

volumes:
  data:

Docker compose file explanations:

• services: we can understand Docker service as a container. Therefore, this top-level element called “services” is a key map where each key represents an individual container, which will be created following its configurations: such as image, volumes and ports, for example. So here we see three different containers named "mongodb", "frontend" and "backend".
• image: this key specifies the image this container is based on to be created. It can be a local image or an image from the Docker hub.
• build: is used to define the dockerfile path to build an image to be used by this service. It's useful when the image isn't already created.
• volumes (inside services): when inside the service, specifies the volumes this container will create (or use if already created).
• volumes (same level of services): At the top level, this key specifies all named volumes created in each service that should be shared among all services. Each named volume must be on a line followed by a comma. This syntax is a little strange because there is nothing after the comma, but this is how it should be declared.
• env_file: defines the path of a file containing the environment variables to be used by the service.
• depends_on: when working with multiple containers, it is used to specify when a container depends on another container to run. In the above example, the frontend container will be created only after the backend which will be created after mongodb.
• ports: exposes a container port to a port on the host machine (local_port:container_port). In our case, the backend can be accessed with port 80 while the frontend with port 3000 (localhost:80 and localhost:3000 respectively).
• stdin_open and tty: allow we send input to the Docker container, which is important when working with React, for example.

Some important extra notes:

We can also create a key called networks inside each service to connect all containers which have the same network names. However, as Docker already do it automatically, most of the times it's not necessary. Like volumes, networks must be declared at the top-level to be shared across different services.

If you want to declare environment vars within this compose file, you can use the environment key instead of env_file. Be aware that if you need to prevent others from seeing this sensitive data you should use a file and add this file in .gitignore, for example.

The services keys are the container names. So you can use these names to create connection between containers. For example, you can connect to the mongodb container from the backend using a solution like mongodb://mongodb:27017/my_database_name where the second "mongodb" is the name of the mongodb container.

While we need to specify the absolute system path to use bind mounts to map our project's folders and files with those inside the Docker container when running the command in the terminal, in the docker compose file we only need the relative path.

Commands to run and manage Docker compose

• docker compose --help: see all options you can use with docker compose command
• docker compose up: to build, create and start all images and containers
• docker compose up -d: same as above but run containers in detach mode (in background)
• docker compose down: stops and removes containers and networks created by up
• docker compose down -v: same as above but also delete volumes
• docker compose down -v -rmi: same as above but also delete images used by services

Check all docker compose CLI options in the Docker docs.

Conclusion

Docker compose helps to quickly set up a development environment for our dockerized projects, especially when using multiple containers. And once this file is created and our services are configured correctly within this file, we can start the containers by running just one command in the terminal.

I also showed how to orchestrate containers so that one starts exactly when another is already created, and also how to configure ports and manage volumes using Docker Compose.

I hope you learned a lot from this article.
See you next time! 😁