Why GOPROXY Matters and Which to Pick

Starting with Go 1.13, Go modules are the standard package manager in Golang, automatically enabled on installation along with a default GOPROXY.

But with other GOPROXY options like JFrog GoCenter, as well as your own Go module packages you need to keep secure from public view, what kind of configuration should you choose? How can you keep your public and private Golang resources from becoming a tangled knot?

Let’s take a look at what a GOPROXY is for, and some of the ways you can set one up for a system that is fast, reliable, and secure.

What Is a GOPROXY?

A GOPROXY controls the source of your Go module downloads and can help assure builds are deterministic and secure.

When developing in Golang before the GOPROXY era, module dependencies were downloaded directly from their source repositories in VCS systems such as GitHub, Bitbucket, Bazaar, Mercurial or SVN. Dependencies from a third party are typically downloaded from public source repos. Private dependencies must authenticate with the VCS system where they are stored to download the module source files.

While the above workflow was popularly used, it lacked the two fundamental requirements of a deterministic and secure build and development process: immutability & availability. Modules can be wiped out by the author or versions can be edited. While these scenarios are considered to be bad practice, they do occur frequently.

Using a GOPROXY

Setting a GOPROXY for your Golang development or CI environment redirects Go module download requests to a cache repository.

Using a GOPROXY for module dependencies helps enforce the immutability requirement. By returning the module from the GOPROXY’s cache, it always provides the same code for a requested version, even if the module has been improperly modified more recently in the VCS repo.

The GOPROXY’s cache also helps ensure the module is always available, even if the original in the VCS repo is destroyed.

There are different ways to use GOPROXY, depending on the source of go modules dependencies you expect to use.

Public GOPROXY

A public GOPROXY is a centralized repository available to Golang devs across the globe. It hosts open-source Go modules that have been made available from third parties in publicly accessible VCS project repositories. Most, like JFrog GoCenter, are provided to the Golang developer community for free.

To use a public GOPROXY, set the Golang environment variable to its URL:

$ export GOPROXY=https://gocenter.io

The above setting redirects all module download requests to GoCenter. Downloads from a public GOPROXY can be much faster than directly from the VCS, by downloading a module archive file.

In addition to fulfilling downloads, a public GOPROXY can also provide GoLang developers with more detailed information about the modules it holds. JFrog GoCenter offers a rich UI with the ability to search and access security information such as CVEs, non-security metadata such as adoption statistics, and gosumdb support. This metadata helps users make better decisions when selecting a public Go module.

Private Go Modules

Typically, GoLang projects make use of both open-source and private module dependencies. Some users use the GOPRIVATE environment variable to specify a list of paths that must bypass GOPROXY and GOSUMDB and download private modules directly from those VCS repos. For example, you may want to use GoCenter to retrieve all open-source modules but request private modules only from your company’s servers.

To use the GoCenter public GOPROXY along with private modules, set the Golang environment variables:

$ export GOPROXY=https://gocenter.io,direct
$ export GOPRIVATE=*.internal.mycompany.com

This use of GOPRIVATE also ensures that your use of these private modules isn’t “leaked” through requests to a public GOPROXY & checksum database server on an open network. Another alternative is to use GONOSUMDB variable that includes references to private go modules. While this configuration enables the Go client to resolve both public and private module dependencies, it doesn’t enforce immutability or availability requirements for private modules.

Private GOPROXY

A private GOPROXY is one you install to store both public and private Go modules on your own infrastructure.

Public modules are cached locally by proxying a public GOPROXY in a binary repository manager like JFrog Artifactory. Private modules are also cached in a repository from their VCS repos. In this way, immutability and availability can be guaranteed for both public and private Go modules.

In Artifactory, a combination of a remote repository for GoCenter, a remote Go module repository that points to private GitHub repos (for private modules) and a local Go module repository can be combined into a single virtual repository, to access as a single unit.

To set your GOPROXY for a virtual repository in Artifactory named “go”:

$ export GOPROXY="https://:@my.artifactory.server/artifactory/api/go/go
$ export GONOSUMDB="github.com/mycompany/*,github.com/mypersonal/*"

Since the modules in your private VCS repos will not have entries in the public checksum database at sum.golang.org, they must be excluded from this oversight check by the go client. Setting GONOSUMDB to your private VCS repos accomplishes this, and will prevent your go get commands for these private modules from failing due to checksum mismatch.

In this configuration, you are assured that none of your references to private modules are “leaked,” while also enforcing immutability and availability of both public and private modules.

Cutting Through Knots

As you can see, using a private GOPROXY provides the most certainty, reliability, and security.

You can also speed the resolution of module dependencies through network proximity of your private GOPROXY to your build tools. JFrog Artifactory is one of the option that's available and it can be installed where you most need it: on-prem or in the cloud, or as a SaaS subscription on all three major public cloud providers.

Those benefits aren’t just limited to Go development, either. Most technology companies use more than one language and multiple package managers. For example, if code is written in Golang, then npm might be used for UI, Docker might be used to distribute bits and Helm might be used to deploy applications on K8s.

https://jfrog.com/blog/why-goproxy-matters-and-which-to-pick/