Why you should be using Docker, even if you're new(ish) to software development or have no idea what Docker is.

Have you found yourself spending too much time setting up and running services like databases on your local machine so you can build your project? Is that setup fragile, and requires a lot of effort to fix when something goes wrong? Does it take forever because you only do it once every six months have to Google everything again? Do you want a simple, predictable way to get your database (or Redis cache, etc.) up and running so you never have to worry about those things again? If so, then read on!

What's in this post

In this post, we'll walk through how you can use Docker by writing a single, <15-line configuration file and running one command from your terminal that will let you download and start a database server, which you can then connect to from your project's code. You can add this to any project in any language and immediately have a predictable way to run your database server without the hassle of managing it yourself. The possibilities extend way beyond databases, but this post will walk through a PostgreSQL setup as a common, practical example.

Who this post is for

• You are comfortable or even somewhat familiar with working in a terminal and running commands there.
• You write code that connects to a database or any other external service.
• You want to learn about and use other external services like caches, message brokers, or other, specialized databases.
• You have no idea what Docker is, or you've never used it yourself.

⚠️ note: Docker is a bit of a memory hog. It can easily use 2-3Gb of memory even when it's not doing anything. If you try running it on your machine and it's consuming too many resources, this solution may not be suitable for your current setup.

What this post isn't

This post is not an introduction to Docker itself and does not go into any real detail about how Docker works or what it does. The goal here is not to understand Docker; it's to improve and simplify your development environment. Maybe, as a bonus, this will be a gentle introduction to Docker itself, but if you find that you're still confused about what's going on under the hood, that's OK.

The Problem

As stated at the beginning of this article, installing, configuring, running, and maintaining services like a database on your local machine can be a pain, and often the time spent getting things to work is time not spent learning about how to use these tools or working on your project.

Just like you can npm install your dependencies and (for the most part) it Just Works™, you should be able to do the same with your database, right? Well, with Docker, you can!

The Solution

Let's dive right in. Here are the complete steps to get up and running. I will explain exactly what we're doing in each step in the next section. The only real prerequisite here is that you have some project that needs to connect to a database. If you're not using Postgres, I'll leave it as a challenge for you to try swapping in MySQL, MongoDB, etc.

note: Before you begin, if you have Postgres installed and it's running on port 5432, please make sure you stop it, e.g. brew services stop postgresql. You can test this by just running psql. You should get an error that says something like error: could not connect to server. Alternatively, if you don't want to stop your local Postgres server, you can continue on and just use a different port from what I'm using here.

1. Download and install Docker Engine.
2. From the root directory of your project, run touch docker-compose.yml in your terminal to create our config file. Copy and paste the config from the code block below into this file.
3. From the root directory, run docker-compose up to start Postgres. This will download the Docker image and run it as a container (keep reading for what this means).
4. Either run your application that connects to your database, or connect with psql postgres://dev:dev_password@localhost:5432/my_project.

When you're done, run docker-compose down to stop the containers.

If you're using Git, you'll want to add .pgdata to your .gitignore file.

note: Go ahead and replace the database name, user, password, etc. with whatever you need for your project.

# docker-compose.yml
version: 3.8
services:
  postgres:
    image: postgres:13.1
    ports:
      - 5432:5432
    volumes:
      - .pgdata:/var/lib/postgresql/data
    environment:
      POSTGRES_HOST: localhost
      POSTGRES_PORT: 5432
      POSTGRES_DB: my_project
      POSTGRES_PASSWORD: dev_password
      POSTGRES_USER: dev

That's it. Once Docker Engine is installed, this config is all you need to get Postgres running.

A Step-by-Step Explanation

If you've read about Docker elsewhere or things are running and you're happy to get back to work, you can stop here. Below is a further explanation of what we've configured.

Definitions

OK, I lied a little when I said we weren't going to talk about Docker. I have to briefly introduce two really important terms in Docker: image and container.

A Docker image is a set of instructions for creating a container. It's often described as being a template or blueprint for building the container. It's a static asset that can be downloaded (often written as "pulling an image"). A container is like a computer that runs inside your computer. There is a ton of stuff I'm skipping over here, but the container is the active part of this. The container is going to start up and run some software in an isolated environment (more on that later).

OK, hopefully that wasn't too bad. Again, we're glossing over a lot, but just like you start or run a server, you run Docker containers. The special thing about them is they are isolated and can come preinstalled with a bunch of handy software (Postgres!) so that we don't have to worry about it ourselves.

Install Docker Engine

Docker Engine, which we installed in step one, is an application that will run on our machine and give us command line tools to use Docker. Think of it like the npm for Docker. And similarly, just like you need to have Node installed to run Node JS code, you need to have Docker Engine installed to run Docker containers.

After you download it, you can run docker -v in your terminal. If it's installed, you should see something like this print out.

Docker version 19.03.13, build 4484c46d9d

Note that we also have docker-compose available to us. That came with Docker Engine.

Create the Docker Compose config

In step two we created the configuration file that tells Docker what we want to run and how to wire it up.

Here's the same config as above, with some comments about what each line is doing. The really important stuff will be explained in more detail below.

# docker-compose.yml
# specify the version of Docker Compose we're using
version: 3.8
# Here is where we can list all the different containers we want to run. We could run Postgres, Redis, Rabbit MQ, even local AWS services like S3, all through a single config!
services:
  # We can name each service whatever we'd like
  postgres:
    # Here is where we tell Docker the source code of what we want to run. More on this below.
    image: postgres:13.1
    # We can also bind a local (left) port to the port in the container (right). This is very important. More on this below.
    ports:
      - 5432:5432
    # This is optional, but works similarly to the ports config (local left, in-container right).
    # Volumes lets us share directories and files between our local machine and the Docker container. More on this below.
    volumes:
      - .pgdata:/var/lib/postgresql/data
    # When Docker runs the container, it will set these as environment variables. For the Postgres image, if we set these, it will create the database, user, etc. for us automatically.
    environment:
      POSTGRES_HOST: localhost
      POSTGRES_PORT: 5432
      POSTGRES_DB: my_project
      POSTGRES_PASSWORD: dev_password
      POSTGRES_USER: dev

image

For our purposes here, setting the image is like naming an npm package, or Ruby Gem, for example. You can put the name and version of any image available in Docker Hub, a public registry of Docker images. We can pull images not just for Postgres, but for MongoDB, MySQL, Redis, and many more.

ports

This is one of the most important lines in the config, and it's where we need to talk a little about what Docker is and what it does.

Running Docker containers is like running a computer inside your computer. It has an OS, and since we built this "computer" from a Postgres image, it has Postgres installed on it. For many reasons, these containers are run in isolation. This means that by default there's no connection between our local machine and what's going on inside these containers.

But if the container can't talk to anything outside itself, it wouldn't be very useful. So we can tell Docker what we want to open up, like the network (the ports, in our case). By binding our local port 5432 to the Docker container's port 5432, we can use psql from our local machine to connect to our database when it's running. Docker has exposed this port in the network for us, and we can connect to it like it was running right there on our local machine.

Still confused? That's OK. Like I said, this post isn't meant to teach you about Docker. If you can tolerate a little magic, you can continue and dive into Docker later.

volumes

Just like ports, volumes lets us tell Docker that we want to bind something inside of the container to something outside of the container (your local machine). Remember, all that stuff happening inside the Docker container is inaccessible to you by default.

Strictly speaking, we don't need this, but it can be useful if you want to easily delete your entire database and start over. Just delete the .pgdata directory and restart with docker-compose.

environment

This is also important, and is specific to Postgres. By adding these environment variables, the Postgres image will automatically create the database, user, and set the password for us.

Running Docker containers with Docker Compose

When we run docker-compose up, Docker will pull (download) the images specified in the config from Docker Hub if it isn't available locally and run it as a container on our machine. This is really powerful. We now have Postgres running in an isolated environment, all set up the way we need, ready to be used.

Handy commands

To manage these Docker containers with Docker Compose, here are some handy commands, some repeated from the steps above.

• docker-compose up (starts Docker containers, i.e. starts your database server)
• docker-compose up -d (starts Docker containers in background)
• docker-compose down (stops Docker containers, i.e. stops the database server)
• docker-compose up --force-recreate (recreates Docker containers, probably you won't need this)
• docker-compose down --rmi all --volume (stops Docker containers and removes all images and volumes, this one isn't super handy but it can be useful to delete things if you need to start over)

Beyond Docker Compose, don't forget you can delete the .pgdata directory to wipe away all Postgres data (the whole database!) and start over if you need to.

Wrapping up

You've just made a huge step forward in how you can set up your development environment. Whether you want to bring someone else onto your project and give them an easy way to get it running or you have multiple projects going on your local machine, Docker Compose can help you run external services with a simple, declarative configuration. I hope you found this useful, or at the very least learned something new. If you have questions or feedback, or you get stuck and need help, feel free to contact me on twitter, where my handle is @deanboop.

What's next?

You've got a template here you can drop into any project, but there's a lot of magic happening. How do you create your own Docker image? What is a container, really? How do they work? What about other configurations for Docker Compose? If you want answers to those questions, then it's time to head on over to Docker Curriculum and the Docker Compose documentation and start reading a bit more!