Start Local Services Fast in Zero Lines of Code with Docker Compose

...unless YAML counts as code.

Many companies have home-grown scripts and tools to start services locally. In my experience, the scripts start as well-intentioned wrappers simplifying common docker and docker-compose commands. They tend to age poorly, however. They get complex, are a maintenance burden, and force developers to learn the wrapper commands instead of the underlying tool commands they may already be familiar with. They don't wrap each flag or option, making them less flexible. When the tools they wrap update, the tool must change to take advantage of new features. This, in turn, can lead to using deprecated, slow, or less secure commands, impacting the engineering team's ability to deliver value to customers.

Twice I have converted internal tools to use docker-compose directly. Using this approach enables,

• YAML's merge syntax for sharing common configuration between services
• Showing all services and their dependencies in once place (or multiple, depending on your use-case)
• Quickly iterating on configuration changes
• Relying on robust, community-supported CLI with no maintenance requirements
• Using new features on day one instead having to add support
• Exporting the COMPOSE_FILE variable so you can start services from any directory.

We'll walk through an example docker-compose.yml file using this approach.

version: "2.4"

x-depends-on-postgres: &depends-on-postgres
  postgres: { condition: service_healthy }

x-interval5-retries45-timeout5
  interval: 5s
  retries: 45
  timeout: 5s

First, note we use v2.4 syntax instead of v3.x. Think of docker-compose v2 and v3 more like a fork than an upgrade, with v2 perfectly capable provided you don't need Docker Swarm support. Ideally, your services are resilient to dependencies being down, allowing you to start every service in parallel. If they aren't, docker-compose v2.4 syntax allows using depends_on conditions and healthcheck keys together, allowing you to start services in parallel if they're not dependent on each other and in order if they are. If you're using v3.x, this page gives options to accomplish something similar, they're just more work and, sadly, there's no plan to add the v2 functionality back.

Next, we add common configuration using keys that start with x- via YAML merge key types, which docker-compose supports. This example only has two services so not much is shared, but this can lead to a large reduction in duplication for more realistic scenarios.

services:
  my-service:
    depends_on:
      <<: *depends-on-postgres
    environment:
      SERVICE_NAME: my-service
    image: ${MY_SERVICE_REPO:-MY_DEFAULT_REGISTRY}/my-service:latest

  postgres:
    environment:
      PGDATA: /var/lib/postgresql/data/pgdata
      POSTGRES_DATABASE: my-service-data
      POSTGRES_PASSWORD: xxx
      POSTGRES_USER: user
    healthcheck:
      test:
        "res=$$(echo 'SELECT 1' | psql --host localhost --username $$POSTGRES_USER
        --dbname $$POSTGRES_DATABASE --quiet --no-align --tuples-only) &&
        [$$res = '1'] && exit 0 || exit 1"
      <<: *interval5-retries45-timeout5
    image: postgres:12
    ports: [5432:5432]
    volumes:
      - ./data/postgres:/var/lib/postgresql/data/pgdata:delegated
    command: postgres

In the services section, add each as you normally would. Then, specify the depends_on key for any service with dependencies and the healthcheck key for any service depended on by another service. To add a healthcheck,

• Many services may share a healthcheck, in that case, pull it into a YAML merge key type to reduce duplication
• For new or one-off services, start by looking here to see if one exists or for inspiration. Once you have a command you think works, exec into the container with docker-compose exec SERVICE sh and test it. Some containers won't have curl or other tools, so you may need to get creative. Once it's working, add the command to docker-compose.yml, start the service, and run docker-compose ps to ensure it becomes healthy. If it doesn't, run docker inspect $(docker-compose ps -q SERVICE) | jq '.[].State.Health' to see what's being returned, edit the command, and try again.

You'll notice the image for my-service's image is ${MY_SERVICE_REPO:-MY_DEFAULT_REGISTRY}/my-service:latest. This allows quickly swapping between local and hosted images using a .env file. By default, it uses MY_DEFAULT_REGISTRY. To use a local image, add MY_SERVICE_REPO=local to .env.

services:
  # ...as shown above

  flow-backend:
    depends_on:
      my-service: { condition: service_healthy }
      # ...other services
    image: rwgrim/docker-noop

You'll likely have services you tend to start together. To make this easier, optionally add "flows" at the bottom of services. These only exist to start their dependents, so they use the tiny rwgrim/docker-noop image, which simply exits successfully. In a real application, maybe you'll have flow-backend, flow-frontend, and then a flow-all that depends on both of the previous flows.

Here's the full example,

version: "2.4" # v3 doesn't support using depends_on conditions with healthcheck

#############################################################################
# Shared config via merge key types, https://yaml.org/type/merge.html
#############################################################################
x-depends-on-postgres: &depends-on-postgres
  postgres: { condition: service_healthy }

x-interval5-retries45-timeout5
  interval: 5s
  retries: 45
  timeout: 5s

services:
  #############################################################################
  # Services
  #############################################################################
  my-service:
    depends_on:
      <<: *depends-on-postgres
    environment:
      SERVICE_NAME: my-service
    image: ${MY_SERVICE_REPO:-MY_DEFAULT_REGISTRY}/my-service:latest

  postgres:
    environment:
      PGDATA: /var/lib/postgresql/data/pgdata
      POSTGRES_DATABASE: my-service-data
      POSTGRES_PASSWORD: xxx
      POSTGRES_USER: user
    healthcheck:
      test:
        "res=$$(echo 'SELECT 1' | psql --host localhost --username $$POSTGRES_USER
        --dbname $$POSTGRES_DATABASE --quiet --no-align --tuples-only) &&
        [$$res = '1'] && exit 0 || exit 1"
      <<: *interval5-retries45-timeout5
    image: postgres:12
    ports: [5432:5432]
    volumes:
      - ./data/postgres:/var/lib/postgresql/data/pgdata:delegated
    command: postgres

  #############################################################################
  # Flows
  #############################################################################
  flow-backend:
    depends_on:
      my-service: { condition: service_healthy }
      # ...other services
    image: rwgrim/docker-noop

For even higher productivity, export COMPOSE_FILE='path/to/docker-compose.yml' and create the alias alias dcp=docker-compose, allowing you to start services from any directory!

Day-to-day, you'll use docker-compose commands directly. Common commands: run dcp up -d SERVICE_1 SERVICE_2 ... to create and start the provided services and their dependencies (the -d or --detach flag runs them in the background), dcp ps to list their statuses, dcp stop to stop them, dcp start to start them back up, and dcp down to remove containers, images, and volumes. Run dcp pull to download the latest images, dcp logs SERVICE_1 SERVICE_2 to view logs, dcp config --services to validate the docker-compose.yml file and list all services defined, and dcp config --services | grep flow to list only flows. Run dcp -h for all commands.

And that's it! All services configured in one file with minimal duplication that you can run anywhere using the newest features of a community-supported CLI with no code to maintain!