How to use multi-stage Docker builds

Tom Withers — Mon, 19 Aug 2019 13:21:21 +0000

This post was first published on the Ministry of Justice Digital & Technology technical blog on July 26th 2019.

Docker has revolutionized the way software engineers build applications, small maintainable pieces of software that can be easily scaled to meet demand. But with new technology, there are now more things to take into consideration. One such issue that arose with Docker is image size. Traditionally to get an efficient docker image you would use some complex shell commands within the Dockerfile to keep the layers as small as possible.

In Docker 17.05 multi-stage builds were introduced and this means we can now make docker images even smaller, below I'm going to go over an example of how to do this with a Symfony app and composer.

With multi-stage builds, you can use multiple FROM commands within a single Dockerfile. Each FROM command can use a different base and symbolises a new stage in the build. The most useful thing is that you can selectively copy artifacts from one stage to another leaving everything that you don't need behind.

So let's create a new Dockerfile and add our first FROM command. This will be using composer as a base image so we can get all of our composer dependencies. We can also name the different stages within the build by adding AS <NAME> to the FROM command.

FROM composer AS composer

Now let's copy over our composer.json, composer.lock. We're also going to add a RUN command which will run a composer install.

FROM composer AS composer

COPY composer.json /app
COPY composer.lock /app

RUN composer install

So the first part of our multi-stage build is done, this will allow us to install our composer dependencies into the first image and then copy them across into the final image.

Next, we need to build our final stage:

FROM php:7-fpm-alpine

WORKDIR /var/www

COPY ./ /app
COPY --from=composer /app/vendor /app/vendor

By using FROM php:7-fpm-alpine we start a new build using php:7-fpm-alpine as the base image, we then use COPY --from=composer to copy over the vendor folder into our new image. We leave all the composer gubbins behind, and it's not saved into our final Image.

Full Dockerfile:

FROM composer AS composer

COPY composer.json /app
COPY composer.lock /app

RUN composer install

FROM php:7-fpm-alpine

WORKDIR /var/www

COPY ./ /app
COPY --from=composer /app/vendor /app/vendor

You only need a single Dockerfile. Just run docker build and docker the build process will start.

The end result of using multi-stage builds is a slim production-ready image without any complexity. I hope you can see the benefits of using multi-stage builds and I would definitely encourage you to try them out.

Python config file with YAML

Tom Withers — Mon, 06 Aug 2018 08:23:21 +0000

So I was googling for this myself about 4 weeks ago and couldn't find a straight answer, so I'm about to share an easy step by step solution with you.

First off I'm going to assume that you already know some python and have a project setup ready to go inside a venv.

First off lets import yaml,

import yaml

so now lets create a method called load_config

import yaml

def load_config(config_file):

As you can see we are also passing in config_file, Next, we are going to open our config file as a stream using open which is built into python.

import yaml

def load_config(config_file):
    with open(config_file, 'r') as stream:

Using open, we can now open our config file and return a stream. So here we pass in our config_file variable which we defined earlier, we also pass in another parameter which is a string of r this basically tells open which mode we want to open the file in. Now we can move on to loading our YAML file.

def load_config(config_file):
    with open(config_file, 'r') as stream:
        try:
            return yaml.safe_load(stream)
        except yaml.YAMLError as exc:
            print(exc)

As you can see above, we now load our YAML file using safe_load. It's extremely important to use safe_load here, safe_load only resolves basic YAML tags, whereas load does not. This is also wrapped in a try except block so we can handle any errors.

All that's left to do now is call our method to load the file, make sure you pass in the path to your config.yml file here.

config = load_config('config.yml')

This is my first ever blog post any feedback would be greatly appreciated! :)

DEV Community: Tom Withers

How to use multi-stage Docker builds

Python config file with YAML