An Introduction to Go Tooling

This article was originally written by Ayooluwa Isaiah on the Honeybadger Developer Blog.

An overview of Go tooling

Tooling is generally considered one of the stronger aspects of the Go ecosystem. The go command is the gateway to many of the tools that will be discussed in this post.

By learning about each of the tools discussed here, you'll become much more efficient when working on Go projects and perform common tasks quickly and reliably.

Viewing environmental variables

The go env command is used to display information about the current Go environment.
Here's a sample of what this command outputs:

GO111MODULE="on"
GOARCH="amd64"
GOBIN="/home/ayo/go/bin"
GOCACHE="/home/ayo/.cache/go-build"
GOENV="/home/ayo/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/home/ayo/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/lib/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/lib/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/dev/null"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build145204762=/tmp/go-build -gno-record-gcc-switches"

If you want to view the value of a specific variable, you can pass them as arguments to the go env command:

$ go env GOPATH GOROOT GOBIN
/home/ayo/go
/usr/lib/go
/home/ayo/go/bin

The documentation for each of the variables can be accessed using the command below:

go help environmental

Running code with go run

Assuming you have a main.go file with the following code,

package main
import "fmt"
func main() { fmt.Println("Welcome to Go!") }

You can run it using the go run command, as we've already seen several times in this series:

$ go run main.go
Welcome to Go!

The go run command compiles the program, creates an executable in your /tmp directory, and executes this binary in one step. If you want to execute several files at once, you can pass them all as arguments to go run:

$ go run main.go time.go input.go

Or you can use a wildcard:

$ go run *.go

You can also run an entire package at once, as of Go v1.11:

$ go run ./foo # Run the package in the `foo` directory
$ go run .     # Run the package in the current directory

Formatting code with gofmt

If you've been writing Go code for any length of time, you will know that there are strict conventions for how code should be formatted. The gofmt command is what enforces these conventions for all Go code in existence.

The code snippet shown in the previous section was not formatted properly, so let's format it with gofmt, as demonstrated below:

$ gofmt main.go
package main

import "fmt"

func main() { fmt.Println("Welcome to Go!") }

This formats the code in the source file and prints the result to the standard output. If you want to overwrite the source file with the formatted output, you need to add the -w flag.

$ gofmt -w main.go

To format Go source files recursively (current directory and subdirectories),
specify a . as the argument to gofmt.

gofmt .

Fixing import statements

Before you can use a package in your code, you need to import it. If you fail to do so, the code will not compile, and an error will be displayed. Given the following code in your main.go file,

package main

func main() {
    fmt.Println("Welcome to Go!")
}

You should see the following error if you attempt to execute the program:

$ go run main.go
# command-line-arguments
./main.go:4:2: undefined: fmt

Before the code can compile, the fmt package must be imported. You can add the necessary code manually or use the goimports command, which adds the necessary import statements for you.

$ goimports main.go
package main

import "fmt"

func main() {
    fmt.Println("Welcome to Go!")
}

The command also removes any imported packages that are no longer referenced and formats the code in the same style as gofmt. So, you can also think of goimports as a replacement for gofmt.

The value of goimports becomes apparent if you set your editor to run it on saving a Go source file. That way, you won't need to worry about importing a package before using it or importing statements that are no longer needed. It'll be done automatically for you as soon as you save the file. Most code editors have some sort of plugin or setting that should help with this.

Building your project

To build an executable binary for your program, use the go build command. This will output a single binary in the current directory:

$ go build
$ ./demo
Welcome to Go!

The binary produced with go build is specific to your operating system architecture, and it contains everything you need to run the program. Therefore, you can transfer it to another machine with the same architecture, and it'll run in the same manner even if Go is not installed.

If you want to cross-compile a binary for an architecture other than your own, all you need to do is change the values of the GOOS and GOARCH environmental variables before running the go build command.

For example, the following command can be used to produce a binary for a 64-bit Windows machine:

$ GOOS=windows GOARCH=amd64 go build

To compile for Linux, macOS, ARM, Web Assembly, or other targets, please refer to the Go docs to see the combinations of GOOS and GOARCH that are available to you.

Installing Go binaries

The go install command is an alternative to go build if you want to be able to run the program from outside its source directory.

Assuming your main.go file is in a directory called demo, the following command will create a demo binary in your $GOPATH/bin directory.

$ go install

The $GOPATH should be $HOME/go on most computers. You can check it with the
go env command:

$ go env GOPATH
/home/ayo/go

If you list the contents of $GOPATH/bin, you should see a demo binary:

$ ls $GOPATH/bin
demo

This binary can be executed by running the demo command from any location on your filesystem. This only works as long as the $GOPATH/bin directory has been added to your $PATH.

$ demo
Welcome to Go!

Listing package information

The default invocation of go list returns the name of the import path for the directory you are currently in or the provided package path:

$ go list
github.com/ayoisaiah/demo
$ go list github.com/joho/godotenv
github.com/joho/godotenv

We can customize the output of the go list command using the -f flag, which allows you to execute a Go template that has access to the internal data structures of the go tool. For example, you can list the name of the fmt using the command below:

$ go list -f "{{ .Name }}" fmt
fmt

That's not very interesting on its own, but there's more. You can print all the dependencies of a package using the {{ .Imports }} template. Here's the output for the fmt package:

$ go list -f "{{ .Imports }}" fmt
[errors internal/fmtsort io math os reflect strconv sync unicode/utf8]

Or, you can list the complete set of transitive dependencies for a package:

$ go list -f "{{ .Deps  }}" fmt
[errors internal/bytealg internal/cpu internal/fmtsort internal/oserror internal/poll internal/race internal/reflectlite internal/syscall/execenv internal/syscall/unix internal/testlog io math math/bits os reflect runtime runtime/internal/atomic runtime/internal/math runtime/internal/sys sort strconv sync sync/atomic syscall time unicode unicode/utf8 unsafe]

You can also use the go list command to check for updates to dependencies and subdependencies:

$ go list -m -u all

Or, check for updates to a specific dependency:

$ go list -m -u go.mongodb.org/mongo-driver
go.mongodb.org/mongo-driver v1.1.1 [v1.4.0]

There's a lot more you can do with the go list command. Make sure to check the documentation for flags and other template variables that may be used with the command.

Displaying documentation for a package or symbol

The go doc command prints the documentation comments associated with the item
identified by its arguments. It accepts zero, one, or two arguments.

To display the package documentation for the package in the current directory, use the command without any arguments:

$ go doc

If the package is a command (the main package), documentation for exported symbols are omitted in the output, except when the -cmd flag is provided.

We can use the go doc command to view the docs for any package by passing the import path for the package as an argument to the command:

$ go doc encoding/json
package json // import "encoding/json"

Package json implements encoding and decoding of JSON as defined in RFC 7159. The mapping between JSON and Go values is described in the documentation for the Marshal and Unmarshal functions.
[truncated for brevity]

If you want to view the documentation for a specific method in a package, simply pass it as a second argument to go doc:

$ go doc encoding/json Marshal

A second command, godoc, presents the documentation for all Go packages (including any third-party dependencies you have downloaded) as a webpage. Running the command will start a web server on port 6060 by default, but you can change the address with the -http flag.

$ godoc -http=:6060

Performing static analysis

The go vet command is one that helps with detecting suspicious constructs in your code that may not be caught by the compiler. These are things that may not necessarily prevent your code from compiling but will affect code quality, such as unreachable code, unnecessary assignments, and malformed format string arguments.

$ go vet main.go # Run go vet on the `main.go` file
$ go vet .       # Run go vet on all the files in the current directory
$ go vet ./...   # Run go vet recursively on all the files in the current directory

This command is composed of several analyzer tools, which are listed here, with each one performing a different check on the
file.

All checks are performed by default when the go vet command is executed. If you only want to perform specific checks (ignoring all the others), include the name of the analyzer as a flag and set it to true. Here's an example that runs the printf check alone:

$ go vet -printf=true ./...

However, passing the -printf=false flag to go vet will run all checks except printf.

$ go vet -printf=false ./...

Adding dependencies to your project

Assuming you have Go modules enabled, go run, go build, or go install will download any external dependencies needed to fulfill the import statements in your program. By default, the latest tagged release will be downloaded or the latest commit if no tagged releases are available.

If you need to download a specific version of a dependency other than the one Go fetches by default, you can use the go get command. You can target a specific version or commit hash:

$ go get github.com/joho/godotenv@v1.2.0
$ go get github.com/joho/godotenv@d6ee687

This method may be used to upgrade or downgrade dependencies as needed. Any downloaded dependencies are stored in the module cache located at $GOPATH/pkg/mod. You can use the go clean command to clear the module cache for all projects in one go:

$ go clean -modcache

Working with Go modules

We covered Go modules in detail in part 2 of this series. Here's a summary of the commands you need to know to work with modules effectively:

• go mod init will initialize modules in your project.
• go mod tidy cleans up unused dependencies or adds missing ones. Make sure to run this command before committing to any changes to your code.
• go mod download will download all modules to the local cache.
• go mod vendor copies all third-party dependencies to a vendor folder in your project root.
• go mod edit can be used to replace a dependency in your go.mod file with a local or forked version. For example, if you need to use a fork until a patch is merged upstream, use the following code:

go mod edit -replace=github.com/gizak/termui=github.com/ayoisaiah/termui

Testing and benchmarking your code

Go has a built-in testing command called go test and a testing package, which can be combined to provide a simple but complete unit testing experience. The test tool also includes benchmarking and code coverage options to help you profile your code even more.

Let's write a simple test to demonstrate some of the capabilities of the test tool. Modify the code in your main.go file, as shown below:

package main

import "fmt"

func welcome() string {
    return "Welcome!"
}

func main() {
    fmt.Println(welcome())
}

Then, add the test in a separate main_test.go file in the same directory:

package main

import "testing"

func TestWelcome(t *testing.T) {
    expected := "Welcome to Go!"
    str := welcome()
    if str != expected {
        t.Errorf("String was incorrect, got: %s, want: %s.", str, expected)
    }
}

If you run go test in the terminal, it should fail:

$ go test
--- FAIL: TestSum (0.00s)
    main_test.go:9: String was incorrect, got: Welcome!, want: Welcome to Go!.
FAIL
exit status 1
FAIL    github.com/ayoisaiah/demo   0.002s

We can make the test pass by modifying the return value of the welcome function in the main.go file.

func welcome() string {
    return "Welcome to Go!"
}

Now, the test should pass successfully:

$ go test
PASS
ok      github.com/ayoisaiah/demo   0.002s

If you have lots of test files with test functions but only want to selectively run a few of them, you can use the -run flag. It accepts a regular expression string to match the test functions that you want to run:

$ go test -run=^TestWelcome$ . # Run top-level tests matching "TestWelcome"
$ go test -run= String .       # Run top-level tests matching "String" such as "TestStringConcatenation"

You should also know the following test flags, which often come in handy when
testing Go programs:

• The -v flag enables the verbose mode so that the names of the tests are printed in the output.
• The -short flag skips long-running tests.
• The -failfast flag stops testing after the first failed test.
• The -count flag runs a test multiple times in succession, which is useful if you want to check for intermittent failures.

Code coverage

To view your code coverage status, use the -cover flag, as shown below:

$ go test -cover
PASS
coverage: 50.0% of statements
ok      github.com/ayoisaiah/demo   0.002s

You can also generate a coverage profile using the -coverprofile flag. This allows you to study the code coverage results in more detail:

$ go test -coverprofile=coverage.out

You will find a coverage.out file in the current directory after running the above command. The information contained in this file can be used to output an HTML file containing the exact lines of code that have been covered by existing tests.

$ go tool cover -html=coverage.out

When this command is run, a browser window pops up showing the covered lines in green and uncovered lines in red.

Benchmarking

The benchmarking tool in Go is widely accepted as a reliable way to measure the performance of Go code. Benchmarks are placed inside _test.go files, just like tests. Here's an example that compares the performance of different string concatenation methods in Go:

// main_test.go
package main

import (
    "bytes"
    "strings"
    "testing"
)

var s1 = "random"

const LIMIT = 1000

func BenchmarkConcatenationOperator(b *testing.B) {
    var q string
    for i := 0; i < b.N; i++ {
        for j := 0; j < LIMIT; j++ {
            q = q + s1
        }
        q = ""
    }
    b.ReportAllocs()
}

func BenchmarkStringBuilder(b *testing.B) {
    var q strings.Builder
    for i := 0; i < b.N; i++ {
        for j := 0; j < LIMIT; j++ {
            q.WriteString(s1)
        }
        q.Reset()
    }
    b.ReportAllocs()
}

func BenchmarkBytesBuffer(b *testing.B) {
    var q bytes.Buffer
    for i := 0; i < b.N; i++ {
        for j := 0; j < LIMIT; j++ {
            q.WriteString(s1)
        }
        q.Reset()
    }
    b.ReportAllocs()
}

func BenchmarkByteSlice(b *testing.B) {
    var q []byte
    for i := 0; i < b.N; i++ {
        for j := 0; j < LIMIT; j++ {
            q = append(q, s1...)
        }
        q = nil
    }
    b.ReportAllocs()
}

This benchmark can be invoked using the go test -bench=. command:

$ go test -bench=.
goos: linux
goarch: amd64
pkg: github.com/ayoisaiah/demo
BenchmarkConcatenationOperator-4        1718        655509 ns/op     3212609 B/op        999 allocs/op
BenchmarkStringBuilder-4              105122         11625 ns/op       21240 B/op         13 allocs/op
BenchmarkBytesBuffer-4                121896          9230 ns/op           0 B/op          0 allocs/op
BenchmarkByteSlice-4                  131947          9903 ns/op       21240 B/op         13 allocs/op
PASS
ok      github.com/ayoisaiah/demo   5.166s

As you can see, the concatenation operator was the slowest of the bunch at 655509 nanoseconds per operation, while bytes.Buffer was the fastest at 9230 nanoseconds per operation. Writing benchmarks in this manner is the best way to determine performance improvements or regressions in a reproducible way.

Detecting race conditions

A race detector is included with the go tool and can be activated with the -race option. This is useful for finding problems in concurrent systems, which may lead to crashes and memory corruption.

$ go test -race ./...
$ go run -race main.go
$ go build -race ./...
$ go install -race ./...

Wrapping up

In this article, we covered several tools included in each Go environment and provided
short descriptions of how they can be used. This is only the tip of the iceberg, so be sure to check out the full documentation for a deeper explanation of the capabilities of each command.

Thanks for reading, and happy coding!

Comments

[Johan PIQUET]

Oh yes so useful! Thank you! ... and the "go clean -modcache" was what I needed to resolve some of my GIT troubles. Thank you a lot!