DEV Community: Khalil Drissi

Building and releasing Go binaries

Khalil Drissi — Sun, 01 Jan 2023 19:41:45 +0000

Introduction

Whether you’ve written a CLI or a HTTP server, most of the time Go code can be compiled down to a static binary which is self-contained and can run on the target system you specify. In an earlier chapter we covered cross-compilation, at this point you should consider whether you need to target one environment such as Windows, Linux or MacOS, or multiple.

The release-it example

Release-it repo is an example of a static Go binary that can be built and released through GitHub Actions. It uses a Makefile, so it can also be run in other CI pipelines or manually as and when you create a new release. The Makefile has the following properties, all of which were covered in previous chapters already:
• It builds binaries for multiple platforms and Operating Systems
• It injects the Commit and Version from git into the final binary

Version := $(shell git describe --tags --dirty)
# Version := "dev"
GitCommit := $(shell git rev-parse HEAD)
LDFLAGS := "-s -w -X main.Version=$(Version) -X main.GitCommit=$(GitCommit)"
.PHONY: all
all: gofmt dist
.PHONY: gofmt
gofmt:
@test -z $(shell gofmt -l ./ | tee /dev/stderr) || (echo "[WARN] Fix formatting issues
↪
with 'make fmt'" && exit 1)
.PHONY: dist
dist:
mkdir -p bin/
CGO_ENABLED=0 GOOS=linux go build -mod=vendor -a -ldflags $(LDFLAGS) -installsuffix cgo
↪
-o bin/release-it-amd64
CGO_ENABLED=0 GOOS=darwin go build -mod=vendor -a -ldflags $(LDFLAGS) -installsuffix cgo
↪
-o bin/release-it-darwin
GOARM=7 GOARCH=arm CGO_ENABLED=0 GOOS=linux go build -mod=vendor -a -ldflags $(LDFLAGS)
↪
-installsuffix cgo -o bin/release-it-arm
GOARCH=arm64 CGO_ENABLED=0 GOOS=linux go build -mod=vendor -a -ldflags $(LDFLAGS)
↪
-installsuffix cgo -o bin/release-it-arm64
GOOS=windows CGO_ENABLED=0 go build -mod=vendor -a -ldflags $(LDFLAGS) -installsuffix cgo
↪
-o bin/release-it.exe

In addition, it runs gofmt for good measure.
If there were any unit tests in the project, then we could add an additional target for them, such as:

.PHONY: gotest
gotest:
    go test ./...

Then update the all line to gofmt gotest dist which would run each of the tasks for us.
Clone the example and build it:

$ mkdir $GOPATH/src/github.com/alexellis/
$ cd $GOPATH/src/github.com/alexellis/
$ git clone https://github.com/alexellis/release-it
$ cd release-it
$ make

By default make will try to run the all target, but you can also run make gofmt if you want to test formatting before committing to a build. You’ll find each binary in the bin/ folder and can run them on your system if it matches the OS/architecture pair, or copy it to another system to run it there. When you release binaries to users, GitHub’s release functionality can come in handy because it allows you to upload the binaries manually or via automation.

Releasing your binaries with a GitHub Action

GitHub Actions is free to use for open source repositories (i.e public ones) and comes with a number of free minutes for private ones also. This example should be adaptable to other platforms like Jenkins or GitLab CI, since the concepts are largely the same.
In our example, we’ll first checkout the code from GitHub for the release tag, then run make all, followed by uploading the release artifacts to the GitHub release.

name: publish
on:
  push:
    tags:
      - '*'
jobs:
  publish:
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@master
      with:
        fetch-depth: 1
    - name: Make all
      run: make all
    - name: Upload release binaries
      uses: alexellis/upload-assets@0.2.2
      env:
        GITHUB_TOKEN: ${{ github.token }}
      with:
        asset_paths: '["./bin/release-it*"]'

Each step in the workflow file can use a specific GitHub Action like alexellis/upload-assets@0.2.2 or actions/checkout@master, but when you leave that field blank, it uses the base system such as ubuntu-latest. On the base Ubuntu runner you can execute standard shell and build commands like make all. The alexellis/upload-assets action is one that I built to use on all the various OSS projects I maintain and allows a number of files to be uploaded.
Users can then go ahead and download your binary and run it on their system.

For instance:

$ cd /tmp/
$ curl -sLSf \
-o ./release-it \
https://github.com/alexellis/release-it/releases/download/0.2.11/release-it-darwin \
    && chmod +x ./release-it \
    && ./release-it
release-it: 0.2.11, commit: 4379cbf694c55bd98f5080d76ac5972fd89dcc65

Containerization your Go application with Docker

Khalil Drissi — Sun, 01 Jan 2023 19:08:05 +0000

Introduction

Docker is a container technology for packaging and shipping applications. a container will run the same way regardless of the host OS. It provides a lot of the advantages of a virtual machine, but in a more lightweight container. It's possible to limit the resource consumption of individual containers and sandbox your environment.This recipe will set up a Docker container for a basic Go application, store some version information about the container, and demonstrate hitting a handler from a Docker endpoint.

Requirements

You must have Docker and Dockercompose installed on your system. You can install Docker from:
https://docs.docker.com/install
*This will also include Docker Compose.

Let's begin

create a new directory for your project and navigate to it.
Run the following command:

$ go mod init myapp

*You should see a file called go.mod appear in your folder.

Create a file called dockerfile with the following content:

FROM alpine

ADD ./example/example /example
EXPOSE 8000
ENTRYPOINT /example

Create a file called setup.sh with the following content:

#!/usr/bin/env bash
pushd example
env GOOS=linux go build -ldflags "-X main.version=1.0 -X
main.builddate=$(date +%s)"
popd
docker build . -t example
docker run -d -p 8000:8000 example

Create a file called version.go with the following content:

package docker
import (
"encoding/json"
"net/http"
"time"
)
// VersionInfo holds artifacts passed in
// at build time
type VersionInfo struct {
Version string
BuildDate time.Time
Uptime time.Duration
}
// VersionHandler writes the latest version info
func VersionHandler(v *VersionInfo) http.HandlerFunc {
t := time.Now()
return func(w http.ResponseWriter, r *http.Request) {
v.Uptime = time.Since(t)
vers, err := json.Marshal(v)
if err != nil {
w.WriteHeader
(http.StatusInternalServerError)
return
}
w.WriteHeader(http.StatusOK)
w.Write(vers)
}
}

6.Create a new directory named example and navigate to it.

Create a main.go file with the following content:

package main
import (
"fmt"
"net/http"
"strconv"
"time"
"github.com/PacktPublishing/
Go-Programming-Cookbook-Second-Edition/
chapter11/docker"
)
// these are set at build time
var (
version string
builddate string
)
var versioninfo docker.VersionInfo
func init() {
// parse buildtime variables
versioninfo.Version = version
i, err := strconv.ParseInt(builddate, 10, 64)
if err != nil {
panic(err)
}
tm := time.Unix(i, 0)
versioninfo.BuildDate = tm
}
func main() {
http.HandleFunc("/version",
docker.VersionHandler(&versioninfo))
fmt.Printf("version %s listening on :8000\n",
versioninfo.Version)
panic(http.ListenAndServe(":8000", nil))
}

Navigate back to the starting directory.
Run the following command:

$ bash setup.sh

Run the following commands:

$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
b390ef429fbd example "/bin/sh -c /example" 22 seconds ago Up 23 seconds 0.0.0.0:8000->8000/tcp optimistic_wescoff

$ curl localhost:8000/version
{"Version":"1.0","BuildDate":"2017-04-
30T21:55:56Z","Uptime":48132111264}
$docker kill optimistic_wescoff # grab from first output
optimistic_wescoff

The go.mod file may be updated and the go.sum file should now be present in the top-level recipe directory.

Conclusion:

we created a script that compiles the Go binary for the Linux architecture and sets private variables in main.go. These variables are used to return version information on a version endpoint. Once the binary is compiled, a Docker container is created that contains the binary. This allows us to use very small container images since the Go runtime is self-contained in the binary. We then run the container while exposing the port on which the container is listening for HTTP traffic. Lastly, we curl the port on localhost and see our version information returned.

Quick guide for easy postgres database replication

Khalil Drissi — Mon, 19 Dec 2022 20:58:30 +0000

Replication is one of the most important aspects of achieving high availability. Any unexpected failures on a database server could cause downtime for an application or a business. Configuring replication is thus the right practice to ensure that there is an option to perform fail-over in the event of disasters.

There are two types of replication methods that are built into the community PostgreSQL source, namely streaming replication and logical replication. in this article we will focus on the streaming replication because it is by far the method that is aimed at solving high availability concerns in production.

Setting up streaming replication in PostgreSQL

We can set up streaming replication within two instances running on different ports on the same server. But that doesn't solve the purpose of high availability. So, it is recommended to build replication between servers that are geographically distributed.

In order to get started with the steps being discussed, we need two servers (a master and a standby)

We will set up the replication using the following steps:

Step 1. Create a replication user on the master:

$ psql -c "CREATE USER replicator WITH REPLICATION ENCRYPTED
PASSWORD 'secret'

Step 2. Add necessary entries to the pg_hba.conf file of the master:

$ echo "host replication replicator <slave_ip_address>/32 md5" >>
$PGDATA/pg_hba.conf
$ psql -c "select pg_reload_conf()"

Step 3. On the master, validate the parameters required to set up replication. The best way is as follows:

$ psql -c "select name, setting from pg_settings where name IN
('listen_addresses','archive_mode','archive_command','wal_keep_segm
ents','restore_command')"

Step 4. Modify parameters that require modification on the master:

$ psql -c "ALTER SYSTEM SET listen_addresses TO '*'";
$ psql -c "ALTER SYSTEM SET archive_mode TO 'ON'";
$ psql -c "ALTER SYSTEM SET archive_command TO 'cp %p
/archives/%f'";
$ psql -c "ALTER SYSTEM SET restore_command TO 'cp /archives/%f
%p'";
$ psql -c "ALTER SYSTEM SET wal_keep_segments TO '100'";

Step 5. Restart the master if required or reload it, depending on the parameters modified:

$ pg_ctl -D $PGDATA restart -mf

$ pg_ctl -d $PGDATA reload

Step 6. Run pg_basebackup from the standby to take a backup of the master:

$ pg_basebackup -h <master_ip> -U replicator -p 5432 -D $PGDATA -Fp
-Xs -P -R

Step 7. Validate the mandatory parameters needed for replication on the standby:

$ cat $PGDATA/postgresql.auto.conf

Step 8. Add the primary_conninfo setting and standby.signal if they do not exist:

$ echo "primary_conninfo = 'user=replicator password=secret
host=<master_IP> port=5432 sslmode=prefer sslcompression=0
gssencmode=prefer krbsrvname=postgres target_session_attrs=any'" >>
$PGDATA/postgresql.auto.conf

$ echo "standby_mode = 'ON'" >> $PGDATA/postgresql.auto.conf

$ touch $PGDATA/standby.signal

Step 9. Start the standby server or reload the configuration, depending on the parameter changes made:

$ pg_ctl -D $PGDATA start

$ sudo systemctl start postgresql-14