Setup Go with VSCode in Docker and Air for debugging

I have to say that Visual Studio Code, for all its pros and cons for various language, it's a totally awesome IDE for Go. Sure, Golang is great in itself but there's little there to justify the EUR 200 price for a year. I'd rather pay a more moderate amount for the Go VSCode extension.

Between the Docker extension, the Remote Docker (which allows you the use of a container as if it were a local environment) and official Go extension (that also brings you all the Go tooling if you don't already have that) the development experience is top notch.

The debugging though is a mixed bag. If you're working locally, using Go's top debug tool (Delve) is straightforward. If you're developing in/for containers though, it gets a bit more complicated. Should you also want hot reloading (which you do when using Docker, otherwise you need to stop/restart containers and the debug session manually on every change).

Before getting hands-on, I'll go over my use case and what makes it a bit different.

Requirements

I need to be able to develop with VSCode on a local codebase (to reuse Git credentials mainly) but the code should run on a Docker Compose development stack with lots of moving parts. So many, in fact, that the Docker Compose for my Go project is but a small part of a project spanning over 18 sub-projects, each with a Compose stack where all stacks connect on the same Compose network so that you can start locally just the services you need so they can communicate.

For Apple M1 / ARM, the only change is highlighted for the Dockerfile.

Limitations

This means I can't use Remote Containers in the intended way, as I can't port all my tooling inside the container / git credentials / etc. With Go is easy, but Go is just one of the tools used - I wanted a way that's in line with everyone else (Typescript / PHP and others)

The closest working solution I found was here.

But for hot reloading I was already using Air, so why should I script my solution like a barbarian? However, on Air github there's a discussion around using Delve with Air with no definitive solution (also because some people use it locally while others remotely and yet more people use it in Docker).

What we'll need

• VSCode with the official Go extension
• Go installed (duh) - since modules went GA I've only been using it with modules so no more GOPATH headaches
• Delve - nowadays it comes as part of the tooling which the Go extension asks you to install on first run. If that's not the case, you can install it manually
• Docker - well, duh, it's right there in the title
• Air - there are many reloading modules, but Air allows you to customize the commands for building and executing, which we'll need
• a Go project - otherwise what are we here for, right?

Step 1

A Dockerfile (./Dockerfile). It needs Air & Delve installed as well, so that will be part of the solution. You can also use Air like a barbarian and reload you whole Compose stack on changes but I prefer to just rebuild inside the container.

FROM golang:1.16-alpine AS base
WORKDIR /app

ENV GO111MODULE="on"
ENV GOOS="linux"
ENV CGO_ENABLED=0

# System dependencies
RUN apk update \
    && apk add --no-cache \
    ca-certificates \
    git \
    && update-ca-certificates

### Development with hot reload and debugger
FROM base AS dev
WORKDIR /app

# Hot reloading mod
RUN go get -u github.com/cosmtrek/air && go install github.com/go-delve/delve/cmd/dlv@latest
EXPOSE 8080
EXPOSE 2345

ENTRYPOINT ["air"]

### Executable builder
FROM base AS builder
WORKDIR /app

# Application dependencies
COPY . /app
RUN go mod download \
    && go mod verify

RUN go build -o my-great-program -a .

### Production
FROM alpine:latest

RUN apk update \
    && apk add --no-cache \
    ca-certificates \
    curl \
    tzdata \
    && update-ca-certificates

# Copy executable
COPY --from=builder /app/my-great-program /usr/local/bin/my-great-program
EXPOSE 8080

ENTRYPOINT ["/usr/local/bin/my-great-program"]

It's nothing too fancy. We setup a base Docker step with shared dependencies. There are two steps that depend on base: one development step that also needs Air & Delve and a generic builder for the prod-ready version. The last step is the production one which clean-copies the binary into a minimalistic Alpine image so that we don't bring all the dev dependencies along.

• On ARM architectures (darwin arm or linux), Delve will work the same way but for Air you may need to either build it locally or download the appropriate binary from Github.

Step 2

A compose stack (./docker-compose.yaml). You may or may not need a compose stack. This works just as well with a plain docker run command which exposes the ports relevant for your application + port 2345 for Delve. While the example only has the Go app in it, I am using it as my case was based on compose.

version: "3.8"
services:
  my-service:
    container_name: my-service
    build:
      context: .
      target: dev
    volumes:
    - .:/app:rw,delegated
    networks:
    - my_network
    ports:
    - 8181:8080
    - 2345:2345

networks:
  my_network:
    name: my_network

The main things here are:

• I am naming my container to make it easier on the eyes and not allow compose to generate it by naming convention
• I am targeting the dev step. This means the container build will build the base and the dev steps, without the builder and the final production
• I am exposing ports 8080 to my localhost:8181 and port 2345 to the same port on my local. This is dealer's choice.
• I am mapping my local (project) folder entirely as /app with "delegated" sync. This is so that changes are reflected ASAP to be detected by Air.

Step 3

Air configuration TOML file (./.air.toml).

# Config file for [Air](https://github.com/cosmtrek/air) in TOML format

# Working directory
# . or absolute path, please note that the directories following must be under root.
root = "."
tmp_dir = "tmp"

[build]
# Just plain old shell command. You could use `make` as well.
cmd = "go build -gcflags='all=-N -l' -o ./tmp/main ."
# Binary file yields from `cmd`.
bin = "tmp/main"
# Customize binary.
full_bin = "dlv exec --accept-multiclient --log --headless --continue --listen :2345 --api-version 2 ./tmp/main"
# Watch these filename extensions.
include_ext = ["go", "tpl", "tmpl", "html"]

I am just focusing on what's needed for customization.

• "root" - Air will watch the current folder recursively
• "cmd" the initial command to execute on refresh/build. We set gcflags as required by debugger to prevent all the Go optimizations and save the binary to ./tmp/main
• "full_bin" - This is what Air will execute on refresh. Normally it's just the compiled binary but in this case we set it to execute Delve. Since the binary is there, we ask Delve to execute it, in a headless mode (and not block the execution of the binary - without it we need to start the session for the program to actually run), listen for debug instructions on port 2345 from all hosts and accept multiple clients.

Step 4

VSCode launch configuration

{
    // Use IntelliSense to learn about possible attributes.
    // Hover to view descriptions of existing attributes.
    // For more information, visit: https://go.microsoft.com/fwlink/?linkid=830387
    "version": "0.2.0",
    "configurations": [
        {
            "name": "Delve into Docker",
            "type": "go",
            "request": "attach",
            "mode": "remote",
            "substitutePath": [
                {
                    "from": "<full absolute path to project>",
                    "to": "/app/",
                },
            ],
            "port": 2345,
            "host": "127.0.0.1",
            "showLog": true,
            "apiVersion": 2,
            "trace": "verbose"
        }
    ]
}

Since the project will be running in Docker, we ask the debugger to attach to a running instance to be found locally on port 2345 (which will point to Docker). We must (I had lots of trouble to discover this) provide a list of substitute paths (local to remote), which need to be absolute. You can also use the predefined variable "${workspaceFolder}/".

Execution

Once your Compose stack is up, you can do a ( on Mac) to launch the debugger. You can switch to debug console in VSCode to check messages exchanged with the Debugger. Beyond that, you can setup breakpoints by clicking on the left side of line numbers (as a red dot appears).

You can:

• check in scope variables (top left screen by default)
• add watches (below variables)
• check the call stack and errors
• list breakpoints (bottom left)
• use the debug console to execute instructions in the current break scope

Caveats

Editing a file will automatically rebuild and restart your program inside the container, but the debugger doesn't take kindly to the procedure. Despite the "continue" flag, it can happen that you may need to refresh the debug session from the debug minibar (the curled green arrow).

That should be it to get you started! Now on the the hard part ...

Good luck!

Comments

[Christian]

Hey! Thanks for nice guide! Question though: when I use the exact setup above, my changes does not seem to be reflected when air reloads. It is reloading, but the changes are not there. I checked the docker-container and the files indeed were changed. If I removed delve from the equation, it works like a charm. Thoughts?

[Andrei Dascalu]

Hm, I can't say I have this behaviour. I've updated my setup to Go 1.17 and running under Mac with latest air and delve I do have another issue, which is that while the reload works ok, the debug session needs to be manually stopped and restarted while clearing the breakpoints - which is another issue at the moment.
What is your setup like (OS, Docker version?)

[Ronny Arvelo]

i also have this problem, am using version 1.18, i thought it could be the that i wasn't following graceful shutdown procedure but i did whats on gin-gonic examples and that did not fixed it. I am getting this warning

2022/03/29 19:33:38 listen: listen tcp :8080: bind: address already in use

I changed 8181 port for 8080. I haven't tested breakpoints for Goland IDE.

[Emmanuel Bourmault]

That almost worked for me, but then I get server failed to start: listen tcp :8080: bind: address already in use when the hot reload happens. There is some sort of problem with Air and I can't figure out what it is.
There has to be something that kills the previous running server somehow.

[jshnaidman]

You're missing the -c .air.toml to run using the config you created in the CMD (or in a command: field in the docker-compose).

Other than that, the guide worked perfectly. Fantastic guide. Before I knew about air, I had tried to setup remote debugging myself and I also wasted a lot of time until I discovered the substitutePath trick. Thanks a bunch.

[Ivan Sabido]

This is not working on Chip Apple M1 Pro.
I have been looking for more information but almost every configuration is similar to this but when I set a breakpoint or several breakpoints, those are not working I request using postman and the server responds as if there were no breakpoints.
Any ideas that can help me?

[Andrei Dascalu]

strange, this still works for me. Although in the time since my original post, I've noticed that after rebuild on changes I need to restart the debugger.

[vuong ⛈️]

Really nice setup. I've tried it even with another hot loading library (github.com/githubnemo/CompileDaemon) and it still can work by the general way. Thanks so much!!

[asdf]

In my case, this is working but with 1 issue: every time I changed a file, the debugger "Delve into Docker" will get terminated due to its target port being reset by air.
Does anyone know any setting that can prevent the remote debugger from stopping when change occurs?