Buildpacks with Go

Joel Holmes — Wed, 03 Nov 2021 19:03:00 +0000

Take 35% off Continuous Delivery in Go
by entering fccholmes4 into the discount code at checkout at manning.com.

Today there are so many ways to deploy something. You can go down the Function as a Service (FaaS), Platform as a Service (PaaS), Container as
a Service (CaaS), or Infrastructure as a Service (IaaS). All of these have one thing in common: they are virtualized. Each layer in our As a Service Parfait abstracts something different. A Function is an abstraction on a Platform which is an abstraction on a Container which and abstraction on a Virtualized Server. Under it all are pieces of specialized hardware to help manage these abstractions.

The cloud is just someone else's computer.

When you look at various cloud providers it all makes sense. Pay for hardware and charge for software. Functions as a Service and Platforms as a Service run in specialized containers for their virtualized container management. They have an understanding of their hardware and optimize their containers for it. This can all be done by using a special base images or allowing them to help build the images for you. These images are known as Buildpacks.

Buildpacks are inextricably linked with how many PaaS work under the hood. In fact, this technology was first developed by Heroku in 2011 and has been used by various other companies such as Pivotal and Google to help run their PaaS. The concepts are simple: you provide the code and we'll build the image. Under the hood, PaaS are building their own custom images based on the libraries and dependencies that their platforms need to make the code run as efficiently as possible and deploy them as containers in their hosting environment. This gives you resilience and substantial up time while they can get the most out of their hardware by running isolated, secure, and maintainable applications.

If you are building an application, Buildpacks will give you a lot of features that will make your application more robust, like advanced caching, multiprocessing, language detection, and much more. The big game changer for Buildpacks recently is this notion of Cloud Native Buildpacks that allow you, as the developer, to take advantage of the PaaS-like ecosystem of building an application with the portability of using containers.

So what goes on inside of a Buildpack? When triggering a Buildpack, it goes through two stages: detection and build. When triggering a build, the Buildpack will analyze your source code to first determine if it can recognize the source code and build the container, this is known as the detection stage. In our case it will look for Go files or a .mod file. If we were building a JavaScript application it would look for a package.json file, or a pom.xml file for a Java application.

When entering the building stage, the Buildpack will determine what the runtime should be, how the library should be built, installation of dependencies, and compilation and running of the application itself. It does this through the use of a builder which is an image specifically for creating the application based on the detection done in the previous step. The build and running of an image is done through a stack which combines the build and run environments.

Docker is just one container runtime. There are many other container runtimes out there that aren't as popular.

All of this can allow different groups to create a process for identifying and building applications specific for their runtimes and environments. This means that Google, Amazon, Heroku, and Microsoft can build their own container runtimes that are optimized for their hardware and you can tap into that performance by using their Buildpack. Let's try it out with Google.

Quick Hello API

To demonstrate the power of this we are going to create quick Go project that just returns "Hello World" when you call an HTTP endpoint. Run the following commands:

mkdir hello-api
go mod init hello-api
touch main.go

Then open up the source file and add the following.

package main

import (
    "encoding/json"
    "log"
    "net/http"
)

func main() {

    addr := ":8080"

    mux := http.NewServeMux()

    mux.HandleFunc("/hello",
        func(w http.ResponseWriter, r *http.Request) {
            enc := json.NewEncoder(w)
            w.Header().
                Set("Content-Type",
                    "application/json; charset=utf-8")
            resp := map[string]interface{}{
                "hello": "world",
            }
            if err := enc.Encode(resp); err != nil {
                panic("unable to encode response")
            }
        })

    log.Printf("listening on %s\n", addr)

    log.Fatal(http.ListenAndServe(addr, mux))
}

Let's build a container

To start, we are going to use a Buildpack to build and run our container locally. Then, we will use the same process to deploy the container to production. Following that, we will work on building our own container using our own definition and deploy that as well. This way you will have the knowledge of how to build and maintain your own containerized deployment and local development. First, we must first install our container runtime; in this case Docker.

Docker has three different installation types depending on your operating system so it will be best if you follow the directions for the one that best suits you. This will give us our container runtime. Now, we need to create a container. To do this we will use a Buildpack. Let's install pack first, which is a tool built and maintained by Cloud Native buildpacks.

Pack will help us choose and build our application into a container using a defined Buildpack. To demonstrate this, let's see what pack suggests we use to build our application. Type pack builder suggest and see what options come up.

Google:                gcr.io/buildpacks/builder:v1      Ubuntu 18 base image with buildpacks for .NET, Go, Java, Node.js, and Python                                                      
Heroku:                heroku/buildpacks:18              Base builder for Heroku-18 stack, based on ubuntu:18.04 base image                                                                
Heroku:                heroku/buildpacks:20              Base builder for Heroku-20 stack, based on ubuntu:20.04 base image                                                                
Paketo Buildpacks:     paketobuildpacks/builder:base     Ubuntu bionic base image with buildpacks for Java, .NET Core, NodeJS, Go, Python, Ruby, NGINX and Procfile                        
Paketo Buildpacks:     paketobuildpacks/builder:full     Ubuntu bionic base image with buildpacks for Java, .NET Core, NodeJS, Go, Python, PHP, Ruby, Apache HTTPD, NGINX and Procfile     
Paketo Buildpacks:     paketobuildpacks/builder:tiny     Tiny base image (bionic build image, distroless-like run image) with buildpacks for Java Native Image and Go

Notice that these packs are not focused on a specific language, rather they provide a broad foundation for multiple languages. You may also notice that these languages are the ones that are supported for FaaS and PaaS offerings on Google. This is because underneath the covers our FaaS and PaaS are actually running within a container using a Buildpack. Now let's build our application by typing the following:

pack build hello-api --builder gcr.io/buildpacks/builder:v1

Let's take a look at what happened. If you look closely you can see that
the builder identifies that our application is a Go project from our
module file and will look for a main package to run. There are
configurations that can be done if you have more than one main function.
Each Buildpack will have its own configuration. To see how our container
runs just type in:

docker run hello-api

You should see your server run. Call your hello endpoint and see that your application is running in a nice, neat portable package. Now you can use it however you want! With this process you can explore and work with Buildpacks and build new and greater things.

If you want to learn more about the book, check it out on Manning's
liveBook platform here.

If you want to learn more about the motivations behind the book please read my other post on my personal site.

DEV Community: Joel Holmes

Buildpacks with Go

Quick Hello API

Let's build a container