Hot Reloading Go Programsâ€Š–â€ŠWith Docker, Glide & Fresh

#go #programming #docker #development

Introduction

Developing Go applications can be a slog, especially if you’re not using an IDE (out of preference) and you’re incorporating the Go program as a service in a larger ecosystem, that must be run locally during development. For example, a queue worker which runs alongside a large web application, running as Docker containers.
Every time we make a change to our code we’d have to kill our Docker container, re-build our Go binary and re-run the Docker container. This is inefficient and could easily be automated. Let’s get cracking shall we?

Prerequisites

I’m making the assumption you know the basics of how Go programs work and how to develop them. I’m also assuming you know what Docker & Containers are, if not I’d recommend you read up on them, or try to pick out the bits of this post which are relevant to your needs.

Setting Up Fresh &Â Glide

The reality of the problem we’re facing here is that we shouldn’t be left to manually re-boot the container or even make the effort of doing something when we change a line of code. This would be tedious and waste so much time.
We need something that can just re-build the binary on the fly when files are changed. It would’ve been possible to develop a simple watcher ourselves but again, that’s time, and someone else has already done that for us. So I welcome to you, Fresh.

Fresh is a Go program which you can include and run outside and inside the Docker container, it’ll watch for files, re-build your binary and execute it. It has a simple configuration file which you can include in your project folder so you can ignore directories etc.

By default it’ll look for a file called runner.conf which would look something like this:

root:              .
tmp_path:          ./tmp
build_name:        runner-build
build_log:         runner-build-errors.log
valid_ext:         .go, .tpl, .tmpl, .html
no_rebuild_ext:    .tpl, .tmpl, .html
ignored:           assets, tmp
build_delay:       600
colors:            1
log_color_main:    cyan
log_color_build:   yellow
log_color_runner:  green
log_color_watcher: magenta
log_color_app:

Fresh is really easy to get started with, just go get github.com/pilu/fresh and run fresh. It provides a nice step-by-step print out in your terminal of what it’s doing and what files it’s seen changes on.

It works great and certainly does what we need itÂ to.

My only gripe with it, is that it doesn’t seem to pick up new files; though simply saving or “touching” an existing file will trigger the re-build. Though this could be due to my configuration, I’ve not spent enough time looking into this to find the cause.

Now in comes Glide, go get on steroids. This is my goto package manager for all of my Go programs. This tool allows us to specify our programs dependencies and pull them in with ease. This makes it easier to collaborate on a project and to lock down versions of packages. I highly recommend this tool to anyone who’s looking to be serious about this.

In essence it’s a beautiful wrapper around “go get”, with a configuration file, version locking and caching. Why wouldn’t you useÂ it?

Due to Fresh’s limitations, it can only watch so many files at once, so we’ll have to ignore our Glide vendor folder. Booooooo. Unfortunately this means that any time we update a dependency we’ll have to manually restart Fresh. Though a simple workaround for this though would be to run glide update && touch main.go this way we’ll trigger Fresh’s watchers to re-build once we’ve updated our Glide dependencies, cool right?

Just add vendor to the “ignored key” in your config, like so (This will keep Fresh from complaining about too many files to watch):

ignored: assets, tmp, vendor

This setup so far would work brilliantly if you’re working outside of the Docker ecosystem, so next we’ll look at integrating this into a simple Dockerfile solution that you can run alongside other Docker containers.

Setting UpÂ Docker

If you haven’t already installed & configured Docker, then please do.

We don’t have to change much about what we’re doing here, but instead of running the commands directly in the terminal; we’ll build a Docker image based off the official golang:alpine image that will handle these for us. Leaving the only commands we’ll need to run manually being glide update && touch main.go and the typical one off “Docker run” when we add/remove/update Glide dependencies, with no need to rebuild binaries or containers ourselves.

Create a Dockerfile with the following contents:

# create image from the official Go image
FROM golang:alpine

RUN apk add --update tzdata \
    bash wget curl git;

# Create binary directory, install glide and fresh
RUN mkdir -p $$GOPATH/bin && \
    curl https://glide.sh/get | sh && \
    go get github.com/pilu/fresh

# define work directory
ADD . /go/src/project_folder
WORKDIR /go/src/project_folder

# serve the app
CMD glide update && fresh -c runner.conf main.go

Remember to replace “project_folder” with the folder name of the project you’re working on. And in order for Glide to function properly outside of the Docker image, your project must be in your $GOPATH like it is in the Dockerfile, so $GOPATH/src/project_folder

In order to boot up this container we can run the following command in our project root (wherever your Dockerfile & main.go files are):

docker run -it --volume=$(PWD):/go/src/project_folder --name=my_project_name image_name

And if we’re running it alongside other containers in a Docker Network we’ll need to add the network flag to this command, so if you had a network called my_app it’ll be:

docker run -it --volume=$(PWD):/go/src/project_folder --name=my_project_name --network=my_app image_name

Note, we need to create a volume from our working directory to the folder within our containers folder, this will allow our watcher running inside the container to pick up changes in our local filesystem.

This should be everything you need to get started writing awesome hot-reloaded Go programs with managed dependencies running on a Docker container. High Five

Note: I do not recommend you use this setup for production applications, this is merely a neat way to remove the time required to re-build your code when you’re developing.

Good luck, have fun!

This is a re-post of https://medium.com/@craigchilds94/hot-reloading-go-programs-with-docker-glide-fresh-d5f1acb63f72

Top comments (6)

൱αϯr๏ • Jun 29 '17 • Edited

Would a more flexible but general-purpose file-watcher like Modd be a replacement for Fresh? Or maybe an additional layer of watching/automation?

On one hand, this would add a system dependency rather than letting you rely on Go packages... but on the other hand, this lets you watch any/all files and directories in a project, so the "watch Glide's vendor folder" problem is solvable. In addition triggering tasks from watch patterns, Modd can also run and manage processes; so you could automatically run/restart/shutdown your Docker container here, plus run Fresh's watcher as well. Or if the project also has, say, some MkDocs content, its trivial to watch/build/serve your docs' HTML or just run mkdocs serve.

A heavier option for this kind of "watch the world and manage child-watchers" thing is Watchman, but I like the like Modd's simpler and more ad-hoc way of doing things.

Either way, I like leaving domain-specific watching to focused tools like Fresh (or available built-in functionality like what sass/compass and most static site generators ship with). Then something like Modd or Watchman lets me fill in the gaps, tie the domain-specific bits together, and watch a project as a whole. One command, and the whole dev. environment (and all it's supporting/peripheral bits) is up and running, live-reloaded wherever possible, and I'm one ctrl-c away from shutting it all down when my attention gets sucked away to something else.