01 - Setup Go Server with Reload in Docker

#go #docker #tutorial #traefik

Project Overview

This article is the beginning of what I hope to be a "several"-part tutorial on setting up a memorization app. The client application for managing memorization objects (we'll call them "memthings") will be built with React/Typescript. The API for the backend application managing these "memthings" will be built with Node/Typescript. User accounts and authentication will be managed by a separate application with its UI built in Vue and its backend built in Go. All of these applications will be setup to run inside of docker containers and will be "hosted" on the same test domain with routing provided by Traefik. Congrats if you made it through this paragraph. Phew!

Here's an overview of the application, along with a video demo of what we'll be building! Today, we'll be working at scaffolding out the application in the upper-left corner, along with properly routing requests to this container with Traefik.

And for those of you that prefer a video version of this tutorial.

If anything is unclear or I have made any errors, please find the functioning code on Github.

Vamos!

Creating Directories

For reference, your directory structure will be as follows by the end of this tutorial (minus the readme content).

To begin begin scaffolding out the application, we'll need to create a folder which will contain all of our sub-projects along with application-wide configuration files (.env, docker-compose.yml). We do this by creating a root memrizr folder (it's VERRRRRY important to do cutesy re-spellings when creating an app). We also create an account directory where we'll write our Golang application.

mkdir memrizr

cd memrizr

git init

mkdir account

Next, we'll create a workspace file in the root folder called memrizr.code-workspace. This file allows us to create a "multi-root" workspace in VS Code. While this step is not required, it is helpful when working with Golang projects which use GO Modules (with a go.mod file to list dependencies). This is because the Go Developer tools (language server) will only function if the go.mod file is at the project's root. There are requests to update this (which would eliminate the problem below), but it would appear to be a difficult task.

We add the following content to the memrizr.code-workspace file. This adds the account folder and the root folder as workspaces. While I would prefer to merely add root application files to the workspace, VS Code doesn't currently support this. And from this Github issue, it doesn't look like they're keen on making this work any time soon. Alas, maybe you're a fortunate soul reading this two years down the line when neither of the above challenges exists!

{
  "folders": [
    {
      "path": "./account"
    },
    {
      "path": "."
    }
  ]
}

Adding main.go

We'll now move into our account folder, initialize go modules, and add gin, a web framework and http router written in, and for, Golang. When running the commands below, make sure to replace the name of my Github repository with your own when initializing the module.

cd account

go mod init github.com/jacobsngoodwin/memrizr

go get -u github.com/gin-gonic/gin

We'll now create the main go file. In this file we:

Create a router for handling http requests to different paths. For now, we create a single GET route to "/api/account", along with its accompanying handler function.
Next, we create a standard Golang http server server struct, applying the gin router we just created.
Finally, we run the server in a go routine. You can learn more about graceful shutdown of servers in the link in the code. In essence, the code creates a channel called quit which listens for termination signals (like ctrl-c). The <- quit line blocks subsequent code until such a termination signal is received, after which the server is shut down.

package main

import (
    "context"
    "log"
    "net/http"
    "os"
    "os/signal"
    "syscall"
    "time"

    "github.com/gin-gonic/gin"
)

func main() {
    // you could insert your favorite logger here for structured or leveled logging
    log.Println("Starting server...")

    router := gin.Default()

    router.GET("/api/account", func(c *gin.Context) {
        c.JSON(http.StatusOK, gin.H{
            "hello": "world",
        })
    })

    srv := &http.Server{
        Addr:    ":8080",
        Handler: router,
  }

    // Graceful server shutdown - https://github.com/gin-gonic/examples/blob/master/graceful-shutdown/graceful-shutdown/server.go
    go func() {
        if err := srv.ListenAndServe(); err != nil && err != http.ErrServerClosed {
            log.Fatalf("Failed to initialize server: %v\n", err)
        }
    }()

    log.Printf("Listening on port %v\n", srv.Addr)

    // Wait for kill signal of channel
    quit := make(chan os.Signal)

    signal.Notify(quit, syscall.SIGINT, syscall.SIGTERM)

    // This blocks until a signal is passed into the quit channel
    <-quit

    // The context is used to inform the server it has 5 seconds to finish
    // the request it is currently handling
    ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
    defer cancel()

    // Shutdown server
    log.Println("Shutting down server...")
    if err := srv.Shutdown(ctx); err != nil {
        log.Fatalf("Server forced to shutdown: %v\n", err)
}

You can no test this by running go run main.go inside of the account folder.

Setup Docker

Let's create a Dockerfile (as written) inside of our account directory. This Dockerfile will make use of stages. The first stage, builder, is where we download and install all dependencies and then build ths application binary. We'll also use this builder stage in our docker-compose development environment later for auto-reloading our application. We'll do this by means of the application called reflex, which we download in the builder stage.

In the second half of the Dockerfile, we extract the built application from builder, and run it in a separate container. This eliminates all of the unnecessary code from the builder stage and creates a leaner final application container which is ready for deployment.

FROM golang:alpine as builder

WORKDIR /go/src/app

# Get Reflex for live reload in dev
ENV GO111MODULE=on
RUN go get github.com/cespare/reflex

COPY go.mod .
COPY go.sum .

RUN go mod download

COPY . .

RUN go build -o ./run .

FROM alpine:latest
RUN apk --no-cache add ca-certificates
WORKDIR /root/

#Copy executable from builder
COPY --from=builder /go/src/app/run .

EXPOSE 8080
CMD ["./run"]

Creating docker-compose file

Now we'll create a docker-compose.yml file. This file can be used to spin-up several docker-containers for development by simply running the command docker-compose up in the same folder as this file (by default, though command-line args can be used to define a filepath, as well). We'll save this file in the root folder of the project.

In this file we create two services.

reverse-proxy - Traefik
- This service will be running the Traefik docker image, the reverse proxy we'll be using to route http requests to the 4 applications we'll be creating.
- We configure Traefik with an array of commands which correspond to command line arguments. For this course, we'll not be setting up TLS certificates, and therefor set --api.insecure=true. The next two arguments tell Traefik to look for configurations in docker containers. Normally, however, Traefik will automatically expose Docker containers with some defaults. We will disable this by setting with --providers.docker.exposedByDefault=false.
- This service's port mappings are configured to let us access the standard http port (80) from the docker host (our machine), and also to access a nice dashboard provided by Traefik on port 8080.
account - Go application in account folder
- First off, take note of the build key in this file. We tell docker-compose that we want to create a container found in the account folder. However, remember how the docker file used a "multi-stage" build? For development, we don't need to run the final, lean build. Therefore, we set target: builder to tell docker-compose to create a container only using the first build stage labeled as builder in the Dockerfile.
- We add some labels, which is how we setup Docker containers to be automatically detected by Traefik. Since we are not exposing all containers by default, we must explicitly set "traefik.enable=true". The next label is also very important. This sets routing rules, telling Traefik to route any http requests to malcorp.test/api/account to our account application.
- Finally, recall that we installed a tool called reflex in our Dockerfile. This tool watches for any changes to *.go files (with a regular expression in this case), and then reruns the main.go file. We actually use go run ./ instead of go run main.go as we will have multiple *.go files in the root directory.

version: "3.8"
services:
  reverse-proxy:
    # The official v2 Traefik docker image
    image: traefik:v2.2
    # Enables the web UI and tells Traefik to listen to docker
    command:
      - "--api.insecure=true"
      - "--providers.docker"
      - "--providers.docker.exposedByDefault=false"
    ports:
      # The HTTP port
      - "80:80"
      # The Web UI (enabled by --api.insecure=true)
      - "8080:8080"
    volumes:
      # So that Traefik can listen to the Docker events
      - /var/run/docker.sock:/var/run/docker.sock
  account:
    build:
      context: ./account
      target: builder
    image: account
    expose:
      - "8080"
    labels:
      - "traefik.enable=true"
      - "traefik.http.routers.account.rule=Host(`malcorp.test`) && PathPrefix(`/api/account`)"
    environment:
      - ENV=dev
    volumes:
      - ./account:/go/src/app
    # have to use $$ (double-dollar) so docker doesn't try to substitute a variable
    command: reflex -r "\.go$$" -s -- sh -c "go run ./"

Add test domain to hosts file

In order to use the malcorp.test domain for development, we must add it to our hosts file so that when we enter it into our browser, curl, postman, etc., it resolves to our local machine. You can do this by adding the following line to /etc/hosts on Mac/Linux or c:\windows\system32\drivers\etc\hosts (or possibly both if you're using Windows Subsystem for Linux). In either case, you will need to save the file with administrative privileges (sudo on Mac/Linux)

127.0.0.1       malcorp.test

Running the application 🤞🏼

Welp... here goes nothin'! Let's give it a try. In the root folder, run docker-compose up (or docker-compose up --build if you ever need a fresh rebuild of containers).

Now you can open your browser and type http://malcorp.test/api/account and you should receive the JSON response