DEV Community: Geoffrey Sagini Mogambi

Helm Charts: An Organised Way to Install Apps on a Kubernetes Cluster

Geoffrey Sagini Mogambi — Mon, 27 Nov 2023 12:59:16 +0000

Helm is a package manager for Kubernetes that helps in automating the deployment of applications in the Kubernetes cluster. Helm packages are called charts. A chart is a collection of related resources inside a directory that can deploy a variety of applications like databases, web applications, monitoring applications, machine learning applications and other related solutions to a Kubernetes cluster.

The charts can be downloaded from a repository and shared. A repository is a hub where Kubernetes packages are stored in the form of Helm charts. You can access these charts from ArtifactHub which is the repository containing the Helm packages. Once a chart is deployed in a Kubernetes cluster a release is created. A release is an instance of a chart running in the cluster and every time you install a chart a release is created.

In this article, you will set up Helm and learn how to search, install, upgrade, do rollbacks, uninstall and manage chart releases. You will also create and package your own charts and set up your chart repositories that host charts that you want to install.

Prerequisites

Minikube installed which will be used to run the Kubernetes cluster locally on your machine. To start Minikube run the following command:

minikube start --driver=virtualbox

Here Virtualbox is used as the driver of choice to start the minikube cluster. To use virtualbox ensure that it is installed. You can also use docker as a driver instead of VirtualBox. The Minikube command to use with the docker driver is minikube start --driver=docker. Note that the drivers which you can use with minikube start are not just restricted to these two drivers demonstrated above.

Kubectl should also be installed. Luckily kubectl comes pre-installed with Minikube.
Ensure you have docker installed on your machine. Docker is a virtualization platform for shipping applications as containers.
Basic knowledge of docker and kubernetes will be beneficial to follow this tutorial. To confirm if your local Kubernetes cluster is running use the following command.

minikube status

An output similar to the following will be displayed.

minikube
type: Control Plane
host: Running
kubelet: Running
apiserver: Running
kubeconfig: Configured

You can test the connectivity of the cluster with the following command.

kubectl cluster-info

If everything works fine there will be no errors and you are now connected to the local Kubernetes cluster. With the basic connection setup in place, it's time to install Helm.

Helm Installation

In this section you will see how to install helm on various os platforms.

Installing Helm On Linux

To install Helm in Linux use the following commands.

curl https://baltocdn.com/helm/signing.asc | gpg --dearmor | sudo tee /usr/share/keyrings/helm.gpg > /dev/null
sudo apt-get install apt-transport-https --yes
echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/helm.gpg] https://baltocdn.com/helm/stable/debian/ all main" | sudo tee /etc/apt/sources.list.d/helm-stable-debian.list
sudo apt-get update
sudo apt-get install helm

Installing Helm on macOS

Homebrew package manager for macOS will be used to install Helm.

brew install helm

Installing Helm on Windows

Type the following commands on the Windows terminal to install helm. You can use either Scoop a command-line installer for Windows or Chocolatey which is a Package Manager for Windows to install helm.

Installing with Scoop

scoop install helm

Installing with Chocolately

choco install kubernetes-helm

To confirm if Helm is installed type helm on the terminal and an output displaying various helper commands from helm will be displayed.

Searching Helm Charts from a Chart Repository

Helm charts are stored in a chart repository. This repository analogy is similar to docker hub where you can pull or push images. In a chart repository, you can also pull your charts from the repository and update the repository locally for you to use the chart.
The repository that hosts Helm charts is ArtifactHub. ArtifactHub is a web based application that enables finding, installing and publishing Kubernetes packages and configurations.
Searching for the helm charts can be done in two ways:

Through the terminal.
Through the artifacthub graphical user interface.

Let's search for one of the most commonly used charts in DevOps which is Grafana. Grafana is an interactive data visualization platform that provides charts that are put into dashboards from which insights from complex infrastructure can be analyzed. You can also query and set alerts based on the set criteria for metrics on the supported integrated data sources with grafana. Searching grafana helm chart via the terminal can be done with the following command.

helm search hub grafana

The output will display a list of grafana charts from which you can choose from.

output

You can also search the Grafana chart via the artifacthub homepage and all the indexed charts related to grafana will be displayed.

A list of available grafana charts will be displayed from which you can choose the one you want to use.

The Helm charts in ArtifactHub have instructions on how to install the charts on the cluster. Select the first grafana chart displayed on the results list and click the install button shown on the right side of the page. A pop-up with the installation instructions will be displayed as shown below.

You can copy the instructions at the terminal to add the grafana repository of the grafana helm chart to the cluster:

helm repo add grafana https://grafana.github.io/helm-charts

output

"grafana" has been added to your repositories

Once the repo has been added let Helm know it exists by doing an update:

helm repo update

output

Hang tight while we grab the latest from your chart repositories...
...Successfully got an update from the "grafana" chart repository
Update Complete. ⎈Happy Helming!⎈

Having done the above, the next step is to install the Helm chart.

Helm Chart Installation

The grafana chart repo added in the previous section contains configuration variables that can be modified if you want to upgrade or do a rollback of a release. The values are stored in values.yaml file and can be viewed with the command helm show values chartname. The chartname in your case is (repository/chartname) which is grafana/grafana as shown in the following command:

 helm show values grafana/grafana

A long output of the values stored in the values.yaml file for grafana chart will be displayed. To install the grafana helm chart, helm install command is used with the following format:

 helm install releasename repository/chartname

In your instance replace the above command with the following values:

helm install grafana grafana/grafana

The displayed output will be similar to the following:

output

NAME: grafana
LAST DEPLOYED: Tue Nov 21 23:53:13 2023
NAMESPACE: default
STATUS: deployed
REVISION: 1
NOTES:
1. Get your 'admin' user password by running:
   kubectl get secret --namespace default grafana -o 
   jsonpath="{.data.admin-password}" | base64 --decode ; echo
...

The output displayed has a NAME which is the name of your release grafana. Other metadata information include namespace, revision, status and deployment date of when the helm chart was installed. The NOTES section contains instructions of how to implement the installed chart and access to it.
To review the deployed charts use helm list command:

helm list

output

NAME    NAMESPACE   REVISION    UPDATED                                 STATUS      CHART           APP VERSION
grafana default     1           2023-11-21 23:53:13.531164235 +0300 EAT deployed    grafana-7.0.8   10.1.5

To view if the installed grafana release is working, run the command below to see the pods running:

kubectl get pods

output

NAME                                   READY   STATUS      RESTARTS   AGE
grafana-f4bf7955b-6lzbn                1/1     Running     0          63m

You can also view all Kubernetes resource components by using the command kubectl get all which will show resources like pods, deployments, services and replicasets.

Upgrading a Helm Release

With the deployed release now in place, you may want to modify the release to handle more traffic by increasing the number of pods running in the cluster. For this to be done you will have to do an upgrade of the deployed release. At this point, you may ponder whether you have to uninstall the release and redeploy it afresh again with the new changes. The good thing for you is that is not necessary. You can use helm upgrade command to upgrade the release and a new version of the chart will be created.
The grafana chart has a configuration known as replicas that control the number of deployed pods. Currently, one pod is running in the cluster. To confirm the pod running, use the following command:

kubectl get pods

output

NAME                                   READY   STATUS      RESTARTS   AGE
grafana-f4bf7955b-6lzbn                1/1     Running     0          63m

From the above, you can see the pod that is running is one as shown in the READY column with 1/1. What if you want to implement redundancy in your system to prevent failure in case one of the pods is not running properly? Redundancy in such a scenario can be achieved by adding more pods. Set the configuration variable replicas of the grafana chart to use 3 pods with the following command:

helm upgrade grafana  grafana/grafana --set replicas=3 --reuse-values

The --reuse-values flag instructs helm to merge the new value set with the old values. This means the old values are added on top of the previous release. The updated release will look like the one below:

output

Release "grafana" has been upgraded. Happy Helming!
NAME: grafana
LAST DEPLOYED: Wed Nov 22 01:03:28 2023
NAMESPACE: default
STATUS: deployed
REVISION: 2
NOTES:
1. Get your 'admin' user password by running:

   kubectl get secret --namespace default grafana -o jsonpath="{.data.admin-password}" | base64 --decode ; echo
...

To view whether the pods running are three in number after upgrading, run the command below:

kubectl get pods

output

NAME                                   READY   STATUS      RESTARTS   AGE
grafana-564c5b7d4b-b7fjh               1/1     Running     0          116s
grafana-564c5b7d4b-p9dgq               1/1     Running     0          2m7s
grafana-564c5b7d4b-z4h5j               1/1     Running     0          2m18s

From the above results, you can see three grafana pods are now running after upgrading the release. You can now test the grafana dashboard chart in the browser to see if it is running on the Kubernetes cluster. Start by getting the password you will use to login in with the following command:

kubectl get secret --namespace default grafana -o jsonpath="{.data.admin-password}" | base64 --decode ; echo

A lengthy string containing the password will be shown. Keep it somewhere safe as you will need it for login. Next is to get the Grafana URL by running the following commands at the terminal:

export POD_NAME=$(kubectl get pods --namespace default -l "app.kubernetes.io/name=grafana,app.kubernetes.io/instance=grafana" -o jsonpath="{.items[0].metadata.name}")

Proceed to use the POD_NAME variable with the following command:

kubectl --namespace default port-forward $POD_NAME 3000

The output at the terminal will display results similar to the one below after doing a port-forward:

output

Forwarding from 127.0.0.1:3000 -> 3000
Forwarding from [::1]:3000 -> 3000

Go to the browser of your choice and paste 127.0.0.1:3000 and a graphical user interface to login to grafana will be displayed as shown below:

To login use admin as the username and for the password field copy the string of password you earlier got when you ran the kubectl get secret ... command. When logged in, the dashboard will appear like the one below and you can now explore the grafana dashboard features.

With this in place, the next step is to do rollback changes and delete the release.

Implementing a Rollback and Deleting a Release

When you do a Helm Upgrade on a release, a new version is created in an incremental manner. Helm stores all the revisions of a release internally which allows you to return to a previous revision if needed. In your scenario, you want to revert back to one pod from the current three pods running. The helm rollback release_name release_version command will be used for the revision. To do a rollback to revision 1 which is the state where you had one pod run the following command:

helm rollback grafana 1

The output you get will show the rollback is successful as shown below.

output

Rollback was a success! Happy Helming!

You can check the revised revision by listing the releases.

helm list

The new revision is 3 and not 1.

output

NAME    NAMESPACE   REVISION    UPDATED                                 STATUS      CHART           APP VERSION
grafana default     3           2023-11-22 01:15:42.462861105 +0300 EAT deployed    grafana-7.0.8   10.1.5

In every change whether an upgrade or rollback of a release, helm will create a new revision. In your case, the revision will be 3 but still will be similar to revision 1 which has one pod. You can confirm this by running:

kubectl get pods

The output below will show one pod running in your grafana release.

output

NAME                                   READY   STATUS      RESTARTS   AGE
grafana-f4bf7955b-bl5rw                1/1     Running     0          2m30s

To delete a release and associated revisions use helm delete release_name command. The following command will delete the grafana release in the cluster:

helm delete grafana

The output will display results stating that grafana has been uninstalled:

output

release "grafana" uninstalled

You can now confirm if the release has been deleted:

helm list

The output displayed will show no release list available:

output

NAME    NAMESPACE       REVISION        UPDATED STATUS  CHART   APP VERSION

Deploying Custom Charts

In this section you are going to deploy your own custom helm charts. Use the following command to create a custom chart named kubechart:

helm create kubechart

A new directory with the name of the chart will be created. To view the contents of the chart created use the tree command at the root of the project where the chart has been created:

tree

output

.
└── kubechart
    ├── charts
    ├── Chart.yaml
    ├── templates
    │   ├── deployment.yaml
    │   ├── _helpers.tpl
    │   ├── hpa.yaml
    │   ├── ingress.yaml
    │   ├── NOTES.txt
    │   ├── serviceaccount.yaml
    │   ├── service.yaml
    │   └── tests
    │       └── test-connection.yaml
    └── values.yaml

4 directories, 10 files

Below is a brief overview of the above folder structure:

charts: charts is a directory that contains other charts called subcharts. At the moment it's empty but as your business needs grow subcharts can be placed inside this folder.
Chart.yaml: This is where the metadata information related to your chart will reside. This includes information like API version of the chart, name and description of the project.
templates: This directory contains all resource definitions that your chart will install on the cluster.
values.yaml: This file contains the default values for the chart. These values can be overridden when using helm install or helm upgrade.

You can now install the created custom chart. The created custom chart you are using contains the default nginx image for demonstration purposes but you can follow these exact steps when using your own images. In the helm install command pass flags --dry-run and --debug while pointing to the current chart directory to see what helm would deploy to the cluster.

helm install kubechart --dry-run --debug ./kubechart

A lengthy output will be displayed and will contain all the resource definitions to be applied to your cluster. You can now go ahead to package the chart for distribution.

helm package ./kubechart

The generated output will be similar to the following:

Successfully packaged chart and saved it to: .../kubechart-0.1.0.tgz

You can now go ahead to install the packaged chart. The installation process is similar to how you install charts from the artifacthub repository.

helm install kubechart kubechart-0.1.0.tgz

The output will contain a set of instructions that you can use to run your chart.

output

NAME: kubechart
LAST DEPLOYED: Fri Nov 24 16:25:08 2023
NAMESPACE: default
STATUS: deployed
REVISION: 1
NOTES:
1. Get the application URL by running these commands:
  export POD_NAME=$(kubectl get pods --namespace default -l "app.kubernetes.io/name=kubechart,app.kubernetes.io/instance=kubechart" -o jsonpath="{.items[0].metadata.name}")
  export CONTAINER_PORT=$(kubectl get pod --namespace default $POD_NAME -o jsonpath="{.spec.containers[0].ports[0].containerPort}")
  echo "Visit http://127.0.0.1:8080 to use your application"
  kubectl --namespace default port-forward $POD_NAME 8080:$CONTAINER_PORT

You have now successfully deployed your custom chart running nginx.

Conclusion

You now know how to use helm charts to install, upgrade, do rollbacks and uninstall helm releases on your Kubernetes cluster. You have also seen how to use the ArtifactHub chart repository to get helm charts. You have also seen how to create your own custom charts by installing and packaging the charts.
For further guidance regarding helm charts visit official helm page.

How to Deploy PostgreSQL with Docker and Docker Compose

Geoffrey Sagini Mogambi — Wed, 15 Nov 2023 08:57:49 +0000

In this article you will learn how to deploy PostgreSQL with Docker and Docker Compose. Postgres is a free, open source and advanced database management system that supports both SQL(relational) and JSON(non-relational) querying. Docker on the other hand is an open platform used for developing, shipping and running applications as containers. Docker applications can be run on any platform once the image has been build.

Prerequisites
Deploying PostgreSQL with Docker
Deploying PostgreSQL with Docker Compose
Conclusion

Prerequisites

Before delving further into this article the following requirements will be beneficial to have:

Basic knowledge of docker and its commands.
Familiarity with docker compose will come in handy.
An Understanding of how relational databases like PostgreSQL work.

Deploying PostgreSQL with Docker

The first approach you are going to use in deploying Postgres is searching the Postgres image from docker hub. You will then launch a Postgres container from the image pulled from docker hub.
Begin by searching the Postgres image from the Docker Registry with the following command at the terminal.

docker search postgres

The output of the displayed available Postgres images from Docker hub should be similar to the one below.

Having done the above you can go ahead to pull the latest PostgreSQL image with the following command.

docker pull postgres:latest

An output similar to the following pulling the latest Postgres image from the Docker hub will be observed.

You can verify the downloaded image with the following command

docker images

Now it's time to run your container from the downloaded Postgres image. To run the container execute the following command.

docker run --name postgres-container -e POSTGRES_USER=root -e POSTGRES_PASSWORD=secret -p 5432:5432 -d postgres

Let's go through the above command that is used to create your container.

You start by stating the name of the container which is postgres-container from the --name tag.
The Postgres user will be root from the environment variable POSTGRES_USER and password will be secret from the environment variable POSTGRES_PASSWORD.
Next you create a port mapping by binding the host network followed by a colon and then the corresponding container network with the -p tag. The port value 5432 used here is the same for both our host and container network but a different port number can still be used for both host and container network.
Finally you state Postgres as the name of the image which will use the latest image tag pulled from the docker hub. You can go ahead to press enter and Docker will start the Postgres container returning a long unique ID. To view all running containers use the docker ps command. The output of the above steps will be displayed as below.

The Postgres server is now ready to accept incoming connections. To execute it, use the docker exec command.

docker exec -it postgres-container psql -U root

The above command instructions are explained below:

The -it flag instructs docker to run the command as an interactive TTY session. Afterward, you specify the name of the container which is postgres-container.
You can now run psql command to access the Postgres console. Use the -U flag to tell psql that you want to connect as root user.
And just like that you are inside the Postgres server and you can now create a database, table or do any administrative task required of Postgres server. A display similar to the following will be shown.

To exit from the Postgres server just type \q in the shell inside the Postgres server as shown below.

root=# \q

With that, you have seen how to use Postgres with docker in the terminal. In the next section let's see how to deploy Postgres with an even better approach that is neater which is docker compose.

Deploying PostgreSQL with Docker Compose

To deploy Postgres with docker compose you will have to write a docker compose yaml file. All the steps will be in a single YAML file making it neat and clear to follow the steps of pulling the Postgres image and running it as a container.
To create the yaml file just type the following at the terminal in a folder of your choice to create the docker compose yaml file.

Create a folder where your yaml file will reside. At the terminal type the following

mkdir ~/docker-compose-pg && cd ~/docker-compose-pg

You can then create your docker-compose file as shown below.

touch docker-compose-pg.yaml

After creating the docker-compose-pg.yaml file, type the following in the file.

version: "3.8"
services:
  postgres:
    image: postgres:14-alpine
    restart: always
    environment:
      - POSTGRES_USER=root
      - POSTGRES_PASSWORD=secret
    ports:
      - "5432:5432"
    volumes:
     - db:/var/lib/postgresql/data
volumes:
  db:

The created docker compose file has the following contents:

It uses the docker-compose file of version 3.8
Next you create a service named Postgres. A service is analogous to docker run when you want to run a container from an image.
The docker-compose file will use the postgres:14-alpine image from docker hub and will restart automatically if the container stops.
The next step is to define the environment variables. In your case, you define the Postgres user and password to be used when executing the Postgres database. To learn more about environment variables you can refer to docker-compose environment guide
The next step is to map the host port 5432 to the container port 5432 as Postgres will run in that port in the container.
The last part of your docker-compose is to manage the volume which is named db. Volumes help in persisting the data during the lifetime of the container or should the container restart. To show volumes attached to your Postgres container just type docker volume ls and docker volume inspect to get the volume information. You are now ready to start your Postgres container. Type the following command on the terminal.

docker-compose -f docker-compose-pg.yaml up -d

This command will start the Postgres container in the background and leave it running. Since your file has a different name from the default (docker-compose.yaml) file you will need to use the -f flag and then specify the name of your docker compose file which is docker-compose-pg.yaml.
The output will be similar to the following after running the command.

To confirm that our container is running use the command below.

docker ps

The output should show you the name of the container as below which is postgresyaml-postgres-1. One thing to note is if you don't state the container_name inside the docker compose file, after the creation of the container, docker compose will assign the container name with the name of the location where your docker compose file is. In your case since the docker compose file docker-compose-pg.yaml lies in folder postgresyaml then the container name will contain the name of the folder combined with the service name and 1 appended to the name at the end. The name of the container will become postgresyaml-postgres-1.

Now it's time to execute your container in the terminal and use the Postgres server. The docker command to use is shown below.

docker exec -it postgresyaml-postgres-1 psql -U root

And voila you are inside the Postgres server. You can type the following SQL commands in psql terminal to see the current timestamp and date. For the current timestamp type select now(); and for the date select current_date;. The display will be as follows.

The setup is now complete and you can exit the Postgres server by using the q command.

Conclusion

In this post, you learned how to deploy Postgres with docker and docker-compose. You first began by pulling the Postgres image from the docker hub. From the pulled image you deployed the Postgres container. A similar process was done with docker compose but in a cleaner approach which put all the container configurations in a yaml file. You can now deploy Postgres with both docker and docker compose.

Metaprogramming in Golang: Reflection Package guide

Geoffrey Sagini Mogambi — Wed, 15 Nov 2023 08:41:59 +0000

Reflection package is a very important feature in Golang because it allows metaprogramming, which is the capability of an application to examine its own structure. It works with types and it's used to analyze types at runtime. Types, Values and Kinds are the foundation of the reflect package. The reflection package allows you to extract the type and value from any interface{} variable.

Let's implement the reflection package with examples.

reflect.TypeOf()

The TypeOf function is used to describe the reflect.type value that is passed into the TypeOf method.

package main

import (
    "fmt"
    "reflect"
)

func main() {
    var1 := "Reflection Package"
    fmt.Println(reflect.TypeOf(var1))

    var2 := 100
    fmt.Println(reflect.TypeOf(var2))

    var3 := true
    fmt.Println(reflect.TypeOf(var3))

    var4 := []int{1, 2, 3, 4, 5, 6, 7, 8, 9, 10}
    fmt.Println(reflect.TypeOf(var4))

    var5 := map[string]interface{}{"Name": "Paul", "Number": 1000}
    fmt.Println(reflect.TypeOf(var5))
}

output

string
int
bool
[]int
map[string]interface {}

The output will show the various data types representation assigned by the variable. If the variable assigned is a string, the typeOf will be a string. If it is a Map variable the typeOf will show type Map and so on.

reflect.Value()

The reflect.value is used to show the underlying value of the respective variable passed to reflect.ValueOf method.

package main

import (
    "fmt"
    "reflect"
)

func main() {
    var1 := "Reflection Package"
    fmt.Println(reflect.ValueOf(var1))

    var2 := 100
    fmt.Println(reflect.ValueOf(var2))

    var3 := true
    fmt.Println(reflect.ValueOf(var3))
    fmt.Println(reflect.ValueOf(&var3))

    var4 := []int{1, 2, 3, 4, 5, 6, 7, 8, 9, 10}
    fmt.Println(reflect.ValueOf(var4))

    var5 := map[string]interface{}{"Name": "Paul", "Number": 1000}
    fmt.Println(reflect.ValueOf(var5))
}

output

Reflection Package
100
true
0xc00001a0c0
[1 2 3 4 5 6 7 8 9 10]
map[Name:Paul Number:1000]

reflect.Kind()

Kind is a property of reflect.Type whose results display basic types and generic complex types. When using kind, for built in types Kind and Type will be the same but for the custom types they will differ.

package main

import (
    "fmt"
    "reflect"
)

func main() {
    var1 := "Reflection Package"
    fmt.Println(reflect.TypeOf(var1).Kind())

    var2 := 100
    fmt.Println(reflect.TypeOf(var2).Kind())

    var3 := true
    fmt.Println(reflect.TypeOf(var3).Kind())

    var4 := []int{1, 2, 3, 4, 5, 6, 7, 8, 9, 10}
    fmt.Println(reflect.TypeOf(var4).Kind())

    var5 := map[string]interface{}{"Name": "Paul", "Number": 1000}
    fmt.Println(reflect.TypeOf(var5).Kind())
}

output

string
int
bool
slice
map

reflect.Copy()

The copy method will copy contents from the source to the destination until either the source has been exhausted or the destination has been filled. Both Source and Destination must have a kind of slice or array and the elements in the slice or array must be of the same type. If the source has elements of string type the equivalent type of string should also be used in the destination. The same scenario will be applied to other types like boolean, integers, floats, and any other type.

package main

import (
    "fmt"
    "reflect"
)

func main() {
    source := reflect.ValueOf([]int{1, 2, 3, 4})
    destination := reflect.ValueOf([]int{5, 6, 7, 8})

    // Copy() function retuns number of elements copied
    counter := reflect.Copy(destination, source)

    fmt.Println(counter)
    fmt.Println(source)
    fmt.Println(destination)
}

output

4
[1 2 3 4]
[1 2 3 4]

reflect.DeepEqual()

The DeepEqual method will return either True or False if the value in your case X and Y types follow the following rules.

Array values are deeply equal if their corresponding elements are deeply equal.
Struct values are deeply equal if their corresponding fields both exported and unexported are deeply equal.
Func values are deeply equal if both are nil otherwise they are not deeply equal.
Interface values will be said to be deeply equal if they hold equal concrete deep values.
For Map values deep equal applies when all of the following is true; they are both nil or both are non nil, they have the same length and either of them has the same Map object or corresponding Keys (matched using GO equality) map to deeply equal values.
Pointer values are deeply equal if using == operator shows equality or if they point to deeply equal values.
Slice values are deep equal when they have the same length and both are nil or non nil.
For other type values like strings, bools, numbers and channels they are deeply equal if using == operator shows equality.

package main

import (
    "fmt"
    "reflect"
)

type Employee struct {
    name    string
    address int
    email   string
}

func main() {
    // DeepEqual checks whether slices are are equal
    slice1 := []string{"a", "b", "c", "d"}
    slice2 := []string{"g", "h", "i", "j"}
    result := reflect.DeepEqual(slice1, slice2)
    fmt.Println(result)

    slice3 := []string{"a", "b", "c", "d"}
    slice4 := []string{"a", "b", "c", "d"}
    result2 := reflect.DeepEqual(slice3, slice4)
    fmt.Println(result2)

    // DeepEqual will check whether the arrays are equal
    arr1 := [3]int{10, 20, 30}
    arr2 := [3]int{10, 20, 30}
    result = reflect.DeepEqual(arr1, arr2)
    fmt.Println(result)

    // DeepEqual will check whether the structs are equal
    emp1 := Employee{"Sagini", 10259, "sagini@gmail"}
    emp2 := Employee{"Kamau", 6789, "kamau@gmail"}
    result = reflect.DeepEqual(emp1, emp2)
    fmt.Println(result)
}

output

false
true
true
false

reflect.Swapper()

The Swapper method is used to swap elements in a given slice and will throw a panic should the underlying interface{} provided not be a slice. Indexes of the respective values are used to do the swap.

package main

import (
    "fmt"
    "reflect"
)

func main() {
    swapSlice := []int{1, 2, 3, 4, 5, 6, 7}
    swap := reflect.Swapper(swapSlice)
    fmt.Printf("Original slice is :%v\n", swapSlice)

    // Swapper() function swaps elements in the provided slice using their index
    swap(1, 6)
    fmt.Printf("After the swap slice is :%v\n", swapSlice)
}

output

Original slice is :[1 2 3 4 5 6 7]
After the swap slice is :[1 7 3 4 5 6 2]

reflect.NumField()

The NumField method returns the number of fields in a given struct.

package main

import (
    "fmt"
    "reflect"
)

type Employee struct {
    Name    string
    Address int
    Email   string
}

func main() {
    emp := Employee{"Geoffrey", 675757, "geoff@yahoo"}
    empType := reflect.TypeOf(emp)
    fmt.Println(empType.NumField())
}

output

reflect.Field()

The Field method returns the reflect.value of the field that is name of the field accessed and it's type at the specified index.

package main

import (
    "fmt"
    "reflect"
)

type Employee struct {
    Name    string
    Address int
    Email   string
}

func main() {
    emp := Employee{"Geoffrey", 675757, "geoff@yahoo"}
    empType := reflect.TypeOf(emp)

    // create a loop to loop over the struct
    for i := 0; i < empType.NumField(); i++ {
        field := empType.Field(i)
        fmt.Println(field.Index, field.Name, field.Type)
    }

}

output

[0] Name string
[1] Address int
[2] Email string

reflect.FieldByName()

The FieldByName method is used to either get or set a struct field value by using name of a given field.

package main

import (
    "fmt"
    "reflect"
)

type Address struct {
    Latitude  int
    Longitude int
    Places    string
}

func main() {
    addr := Address{10, 30, "HeavensPark"}
    fmt.Println(reflect.ValueOf(&addr).Elem().FieldByName("Latitude"))
    fmt.Println(reflect.ValueOf(&addr).Elem().FieldByName("Longitude"))
    fmt.Println(reflect.ValueOf(&addr).Elem().FieldByName("Places"))

    reflect.ValueOf(&addr).Elem().FieldByName("Latitude").SetInt(20)
    reflect.ValueOf(&addr).Elem().FieldByName("Longitude").SetInt(40)
    reflect.ValueOf(&addr).Elem().FieldByName("Places").SetString("TreeTops")

    fmt.Println(reflect.ValueOf(&addr).Elem().FieldByName("Latitude"))
    fmt.Println(reflect.ValueOf(&addr).Elem().FieldByName("Longitude"))
    fmt.Println(reflect.ValueOf(&addr).Elem().FieldByName("Places"))
}

output

10
30
HeavensPark
20
40
TreeTops

reflect.FieldByIndex()

The FieldByIndex method will return a nested field corresponding to an index. A panic will occur if during the evaluation of struct, it finds a nil pointer or a field that is not a struct.

package main

import (
    "fmt"
    "reflect"
)

type Person struct {
    firstName string
    lastName  string
}

type Account struct {
    Person
    accountName string
}

func main() {
    acc := Account{
        Person:      Person{"Sagini", "Geoffrey"},
        accountName: "Stanchart",
    }
    t := reflect.TypeOf(acc)

    fmt.Printf("%#v\n", t.FieldByIndex([]int{0}))
    fmt.Printf("%#v\n", t.FieldByIndex([]int{0, 0}))
    fmt.Printf("%#v\n", t.FieldByIndex([]int{0, 1}))
    fmt.Printf("%#v\n", t.FieldByIndex([]int{1}))
}

output

reflect.StructField{Name:"Person", PkgPath:"", Type:(*reflect.rtype)(0x490ba0), Tag:"", Offset:0x0, Index:[]int{0}, Anonymous:true}
reflect.StructField{Name:"firstName", PkgPath:"main", Type:(*reflect.rtype)(0x48b4c0), Tag:"", Offset:0x0, Index:[]int{0}, Anonymous:false}
reflect.StructField{Name:"lastName", PkgPath:"main", Type:(*reflect.rtype)(0x48b4c0), Tag:"", Offset:0x10, Index:[]int{1}, Anonymous:false}
reflect.StructField{Name:"accountName", PkgPath:"main", Type:(*reflect.rtype)(0x48b4c0), Tag:"", Offset:0x20, Index:[]int{1}, Anonymous:false}

reflect.MakeSlice()

The MakeSlice method creates a new slice value. From the created slice you can find its type, length and capacity.

package main

import (
    "fmt"
    "reflect"
)

func main() {
    var intSlice []int
    var intType reflect.Value = reflect.ValueOf(&intSlice)
    createdSlice := reflect.MakeSlice(reflect.Indirect(intType).Type(), 5, 8)

    fmt.Println("Kind of slice is:", createdSlice.Kind())
    fmt.Println("Length of slice is:", createdSlice.Len())
    fmt.Println("Capacity of slice is:", createdSlice.Cap())
}

output

Kind of slice is: slice
Length of slice is: 5
Capacity of slice is: 8

reflect.MakeMap()

The MakeMap method will create a new Map of a specified type.

package main

import (
    "fmt"
    "reflect"
)

func main() {
    var newMap map[string]interface{}
    var mapType reflect.Value = reflect.ValueOf(&newMap)
    mapCreated := reflect.MakeMap(reflect.Indirect(mapType).Type())

    fmt.Println("Kind of map created is", mapCreated.Kind())
}

output

Kind of map created is map

reflect.MakeChan()

The MakeChan method will create a new channel with a specified type and buffer-size. Since the created channel has a buffer size it is of type buffer channel. Unbuffered channels don't have a buffer size specified.

package main

import (
    "fmt"
    "reflect"
)

func main() {
    var newChan chan string
    var chanType reflect.Value = reflect.ValueOf(&newChan)
    chanCreated := reflect.MakeChan(reflect.Indirect(chanType).Type(), 100)

    fmt.Println("Kind of channel created is:", chanCreated.Kind())
    fmt.Println("Capacity of channel create is:", chanCreated.Cap())
}

output

The kind of channel created is: chan
The capacity of channel created is: 100

reflect.MakeFunc()

The MakeFunc method is used to create a new function value.
It gets the new function of the given Type that wraps the function fn.

package main

import (
    "fmt"
    "reflect"
)

type Addition func(int64, int64) int64

func main() {
    typ := reflect.TypeOf(Addition(nil))

    // verify if function. If not function return an error
    if typ.Kind() != reflect.Func {
        panic("A function is expected")
    }

    add := reflect.MakeFunc(typ, func(args []reflect.Value) (results []reflect.Value) {
        a := args[0].Int()
        b := args[1].Int()
        return []reflect.Value{reflect.ValueOf(a + b)}
    })

    fn, ok := add.Interface().(Addition)
    if !ok {
        return
    }
    fmt.Println(fn(10, 5))
}

output

Conclusion

You have gone through how to use the reflection package in Golang. This is just the surface of its usage as it can be used in many areas of Golang. The advantage of Reflection is that it allows code to be flexible and handles unknown data types by analyzing their memory representation. However, this is not cost-free as it introduces complexity in code and slows down performance. You can learn more on reflection guidance here.

Deploying a Golang RESTful API with Gin, SQLC and PostgreSQL

Geoffrey Sagini Mogambi — Wed, 15 Nov 2023 08:36:22 +0000

REST which stands for Representative State Transfer is an architectural style for designing APIs(Application Programming Interface). An API is a set of rules through which programs communicate with each other. The API is normally created on the server side and allows the client to talk to it.

REST has a set of rules on how APIs should be designed and most of these rules follow the CRUD(Create, Read, Update, and Delete) pattern where you request a resource and data is sent back which is called a Response. The request anatomy comprises four sections namely the endpoint which is the request URL, the method, the headers, and the data/body.

In this article, you will implement a CRUD RESTful API in Golang with Gin HTTP web framework, SQLC for creating CRUD database functions and PostgreSQL as the database engine of choice for this project. You will also learn to use the Golang migrate library for database schema migrations.

Prerequisites
Setting up the Golang project
Setting up PostgreSQL with Docker Compose
Performing database migration with Golang migrate
Using SQLC to generate Go code from SQL queries
Implementing CRUD in Golang Rest API
Route Creation for the Contact Handlers
Testing the Golang RESTful API with REST Client
Conclusion

Prerequisites

Before getting started ensure you have Go installed installed on your machine. At the time of writing this guide 1.21.3 is the latest stable version of Go.
You should also have Vscode installed as it is the IDE of choice for this exercise but feel free to use any other IDE of your liking. Install Go extension for vscode to make it easy to get features like intellisense, code navigation, testing, debugging and many more features that will guide you in your Golang development journey.
Ensure Docker is also installed on your machine.
Some basic knowledge of Golang and SQL(Structured Query Language) will also come in handy to follow along this article.

Setting up the Golang project

To start this project navigate to your preferred directory on your machine and open the terminal. From the terminal create a new folder with the project name, navigate to the project folder, initialize the Golang project and open VsCode editor. Use the following commands to set up the project.

mkdir golang-crudsqlc-rest
cd golang-crudsqlc-rest/
go mod init golang-crudsqlc-rest
code .

Once the above setup is done, install the relevant packages for the project by running the following commands.

go install github.com/sqlc-dev/sqlc/cmd/sqlc@latest
go install github.com/cosmtrek/air@latest
go get github.com/lib/pq
go get github.com/spf13/viper
go get -u github.com/gin-gonic/gin

The migrate package is installed slightly differently from the rest of the packages. Since I am using Ubuntu the below commands will be run at the terminal. If you are using macOS or Windows operating system refer to migration installation page to see how to install migrate CLI.

curl -L https://packagecloud.io/golang-migrate/migrate/gpgkey | apt-key add -
echo "deb https://packagecloud.io/golang-migrate/migrate/ubuntu/ $(lsb_release -sc) main" > /etc/apt/sources.list.d/migrate.list
apt-get update
apt-get install -y migrate

Let's have a brief overview of what each package does.

sqlc: Is a library that is used to generate Golang code that has type safe interfaces from SQL queries.
air: Is a live reloading utility that is used to observe changes in Go applications. Run the air command in the root of your project and it will take care of listening to changing events in your code without the need to restart the application.
pq: It's GO Postgres driver used for GO database/sql. You have to import this driver for you to use the database/sql if you are using the Postgresql engine.
viper: Is a configuration package solution used to read configurations of all types ranging from environment variables, buffer files, command line flags to files written in JSON, YAML and TOML to name just but a few.
gin: Is a web framework written in Go and it comes with a lot of prebuilt in features ranging from routes grouping, JSON validation, middleware support and error management.
migrate: This library helps with database migration either through incremental changes in the schema or reversal changes made in the database schema.

Setting up PostgreSQL with Docker Compose

The database of choice used for this project is PostgreSQL. The database engine will be run as a container. To pull the image from docker ensure docker is installed on your machine. Remember this is one of the prerequisites stated earlier on.
At the root of the project create a docker-compose.yaml file by typing touch docker-compose.yaml. This file will have the following configurations.

version: "3.9"
services:
  postgres:
    image: postgres:latest
    container_name: postgres
    ports:
      - "5432:5432"
    volumes:
      - db:/var/lib/postgresql/data
    env_file:
      - ./app.env

volumes:
  db:

When the above file is run using the required docker-compose command it will pull the latest Postgres image from docker hub and run it as a Postgres container. The container name is postgres which will be mapped to port 5432 from where the postgres server will listen to. The env_file property will be mapped to an external file named app.env which will store sensitive information related to your postgres database.
At the root of your project create an app.env file with the following credentials. Feel free to use credentials of your choice.

app.env

DB_DRIVER=postgres
DB_SOURCE=postgresql://root:secret@localhost:5432/contact_db?sslmode=disable
POSTGRES_USER=root
POSTGRES_PASSWORD=secret
POSTGRES_DB=contact_db
SERVER_ADDRESS=8000

After setting the above run the PostgreSQL container with the following command.

docker-compose up -d

The container will be run in a detached mode and will pull the image from docker hub. Once the image has been downloaded and container is up and running you can type docker ps command to see if postgres name of the container will be displayed at the terminal. The result will be something similar to the one below.

To stop the container type the following command.

docker-compose down

Performing database migration with Golang migrate

Database migration is one of the most important tasks when it comes to version tracking of data. It involves tracking increments and decrements of version changes in the populated database and this helps to ensure that there are no dirty schema migrations.
In this section you will go through how to write database migration using golang-migrate library. Remember you have already installed golang migrate library in the project setup. If not sure you can always refer the project setup section. To confirm that migrate library is installed type migrate at the terminal and an output similar to the following will be displayed.

Let's describe the commonly used commands for the above output:

create - Is used to create new migration files.
up - For migrating files of the schema to a newer version based on the prefix sequence.
down - Used to migrate files of the schema in a reverse manner based on the prefix sequence.

With migration package in place You can now create a folder db/migration where the migration files will reside. Run the following command to create the folder.

mkdir -p db/migration

You can now create the up/down migration files where your SQL code will be placed with the following command.

migrate create -ext sql -dir db/migration -seq init_schema

The -ext flag refers to the extension of the file which is SQL and the -dir flag refers to the directory where SQL files will reside.
The -seq flag will generate a sequential version number for the migration files.

If you cd to the db/migration folder you will notice two migration files have been generated and both have version 1 in the file names prefix. The suffix of the files however is different with one showing up.sql and another showing down.sql.
Before any database migration operation is done you need to have a SQL table in the up migration file and a reverse operation for removing this table in the down migration file. Add the following SQL code in the folder with the file name db/migration/000001_init_schema.up.sql.

CREATE TABLE "contacts"(
    "contact_id" UUID NOT NULL DEFAULT (uuid_generate_v4()),
    "first_name" VARCHAR NOT NULL,
    "last_name" VARCHAR NOT NULL,
    "phone_number" VARCHAR NOT NULL,
    "street" VARCHAR NOT NULL,
    "created_at" TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
    "updated_at" TIMESTAMP NOT NULL,
    CONSTRAINT "contacts_pkey" PRIMARY KEY("contact_id")
);

-- CREATE UNIQUE INDEX ON "contacts" ("phone_number");
CREATE UNIQUE INDEX "contacts_phone_number_key" ON "contacts"("phone_number");

The SQL code above does the following:

Creates a table with the name contacts and the primary key column is contact_id. The primary key column uses uuid data type which helps in ensuring singularity of values with no duplicates.
A unique constraint on thephone_number column is also created which ensures the values of this column will be unique when keyed in the database.

PostgreSQL enables you to store and compare UUID values but it doesn't have default functions for creating UUID values by default. It depends on third-party modules that offer algorithms to generate UUIDS. You will install uuid-ossp module which has some functions that contain algorithms to generate UUIDS.
Before going further let's do one operation in the migration down file. In the db/migration/000001_init_schema.down.sql file add the following code.

DROP TABLE IF EXISTS contacts;

The above SQL statement will drop the contacts table .
Now you are ready to install Postgres uuid-ossp third-party module. Run the Postgres container with the following command at the terminal.

docker exec -it postgres bash

Proceed to log in as a user and populate the database name you earlier set on the environment variables. Command similar to this one is used psql -U <user> <database name>

psql -U root contact_db

You can now view all the available extensions in Postgres by running the following SQL command.

select * from pg_available_extensions;

An output similar to the following will be displayed. Scroll down repeatedly in the list until you reach the uuid-ossp extension as highlighted below in white.

Having located the uuid-ossp extension on the list, you can now install the extension as shown below.

CREATE EXTENSION IF NOT EXISTS  "uuid-ossp";

With Postgres uuid-ossp extension installed you can exit the Postgres shell and Docker shell using the exit command.
It's time to run the migration script and push this table to the database.
Run the following command to execute the migration up script

migrate -path db/migration -database  "postgresql://root:secret@localhost:5432/contact_db?sslmode=disable" -verbose up

If the migration is successful, proceed to open any Postgres graphical user interface(GUI) client and login with the database credentials specified in app.env file. The GUI client used in this article is TablePlus but there are other GUI clients you can still use to connect to the database like PgAdmin and Beekeper Studio. Connect to the Postgres server container by typing the credentials similar to the following.

If the connection is successful the contacts table created by Golang migrate will be created with a schema definition similar to the following.

The schema migrations table stores the latest migration version which in this scenario is version 1 since there is a single migration that has been migrated. To revert back to the changes made by the up migration script run the following command.

migrate -path db/migration -database "postgresql://root:secret@localhost:5432/contact_db?sslmode=disable" -verbose down

Using SQLC to generate Go code from SQL queries

Having successfully migrated scripts to the database schema you can go ahead to generate Golang CRUD code with sqlc. Ensure you have sqlc installed. Refer to the project setup section to see how to install sqlc.
Run sqlc at the terminal and if the output is similar to the following you have successfully installed sqlc in your machine.

With sqlc in place run the following command to generate sqlc.yaml at the root of your project.

sqlc init

A sqlc.yaml file will be generated at the root of the project. Replace the generated content with the following.

sqlc.yaml

version: "1"
packages:
  - name: "db"
    path: "./db/sqlc"
    queries: "./db/query/"
    schema: "./db/migration/"
    engine: "postgresql"
    emit_json_tags: true
    emit_prepared_queries: true
    emit_interface: false
    emit_exact_table_names: false
    emit_empty_slices: true

The name option is the name of the Go package generated. In this case db is the package name.
Next, you specify the folder path where the generated Golang files will be generated. Create a new folder sqlc inside the db so that the path references the path with string ./db/sqlc.
queries tells sqlc where to look for SQL query files. Create a new folder query inside the db folder so that queries references the path with string ./db/query.
The schema points to where our migration files are stored. In your case, it's ./db/migration.
engine section tells sqlc which database engine to use. In this scenario, the database engine is PostgreSQL.
emit_json_tags if set to true directs sqlc to add JSON tags on the generated structs.
emit_prepared_queries tells sqlc to generate code that supports prepared queries. This is useful in the performance optimization of code.
emit_interface if set to true, outputs a Querier interface in the generated package. The querier interface generated is useful if you want to mock the db higher- level functions.
emit_exact_table_names instructs sqlc to create struct names that will be the same as table names. The generated name of the struct will mirror the table name in singularity i.e. contact table will have a Contact struct.

Before generating queries with sqlc generate populate the SQL queries in folder db/query/contact.sql with the following code.

db/query/contact.sql

-- name: CreateContact :one
INSERT INTO contacts(
    first_name,
    last_name,
    phone_number,
    street,
    created_at,
    updated_at
) VALUES (
    $1, $2, $3, $4, $5, $6
) RETURNING *;

-- name: GetContactById :one
SELECT * FROM contacts
WHERE contact_id = $1 LIMIT 1;

-- name: ListContacts :many
SELECT * FROM contacts
ORDER BY contact_id
LIMIT $1
OFFSET $2;

-- name: UpdateContact :one
UPDATE contacts
SET
first_name = coalesce(sqlc.narg('first_name'), first_name),
last_name = coalesce(sqlc.narg('last_name'), last_name),
phone_number = coalesce(sqlc.narg('phone_number'), phone_number),
street = coalesce(sqlc.narg('street'), street),
updated_at = coalesce(sqlc.narg('updated_at'), updated_at)
WHERE contact_id = sqlc.arg('contact_id')
RETURNING *;

-- name: DeleteContact :exec
DELETE FROM contacts
WHERE contact_id = $1;

Below is a breakdown of what the above code does. Five SQL queries that represent CRUD operations have been created for create, insert, update, get and delete operations.
Each query has some annotations namely --name, :one, :many and :exec. The --name annotation will generate the function name to be used by the Golang function. For instance -- name: CreateContact :one annotation tells sqlc to generate Golang function that will have one record.
:many annotation instructs sqlc generated query to generate multiple row records during the database-specific CRUD operation. The :exec annotation label instructs postgres not to return the deleted record.
For nullable parameters sqlc.narg() is used. It tells sqlc to ignore the nullability it has inferred and replace it with a generated nullable parameter instead. Simply put you can update each field independently without affecting the nullability of fields.
With the SQL queries above explained you can now generate Golang code with sqlc generate. Once the Golang crud has been generated by sqlc generate, open the db/sqlc folder and three new Golang files will be generated.

contact.sql.go - contains Golang crud functions and struct definitions.
db.tx - contains the DBTX interface with the four common methods of ExecContext, PrepareContext, QueryContext and QueryRowContext that SQL db and SQL tx object both have.
models.go contains the struct definition for the Contact table.

If you see red line in visual studio code just run go mod tidy so that dependencies added by SQLC are installed.

Loading Environment Variables with Viper

Viper is a Golang package for configuration management of various file formats. Viper finds, loads, and unmarshal configuration files in JSON, TOML, YAML, HCL, INI, envfile or Java properties formats. In this project, viper is used to find values stored in the environment file app.env which stores sensitive information like database passwords and other secret information relevant to the project.
Ensure you have viper installed and this was done earlier in the project setup section. Create util/config.go file in the root of the project with the following code.

util/config.go

package util

import "github.com/spf13/viper"

type Config struct {
    DbDriver         string `mapstructure:"DB_DRIVER"`
    DbSource         string `mapstructure:"DB_SOURCE"`
    PostgresUser     string `mapstructure:"POSTGRES_USER"`
    PostgresPassword string `mapstructure:"POSTGRES_PASSWORD"`
    PostgresDb       string `mapstructure:"POSTGRES_DB"`
    ServerAddress    string `mapstructure:"SERVER_ADDRESS"`
}

func LoadConfig(path string) (config Config, err error) {
    viper.AddConfigPath(path)
    viper.SetConfigName("app")
    viper.SetConfigType("env")

    viper.AutomaticEnv()

    err = viper.ReadInConfig()
    if err != nil {
        return
    }

    err = viper.Unmarshal(&config)
    return
}

With this in place Viper will now load the environment variables from the util/config.go.

Creating Request Validation Structs

SQLC has already generated structs from SQl queries which can be passed to Gin web framework for validation of the request body during request of the respective resource endpoints. However since more validation bindings will be required, you can go ahead and create your custom structs.

Create a schemas/contacts.schema.go file and add the following Golang structs.

schemas/contacts.schema.go

package schemas

type CreateContact struct {
    FirstName   string `json:"first_name" binding:"required"`
    LastName    string `json:"last_name" binding:"required"`
    PhoneNumber string `json:"phone_number" binding:"required"`
    Street      string `json:"street" binding:"required"`
}

type UpdateContact struct {
    FirstName   string `json:"first_name"`
    LastName    string `json:"last_name"`
    PhoneNumber string `json:"phone_number"`
    Street      string `json:"street"`
}

CreateContact struct will be validated by Gin Gonic when creating contacts request payload records in the database.
UpdateContact struct will be validated by Gin Gonic when updating the contacts payload records in the database.

Route Controller Creation

In this section, you will perform the CRUD operations by creating five functions. These functions will perform the CRUD operations by using the generated sqlc CRUD methods that were generated from the SQL queries.

The five functions to be used are defined below:

CreateContact - Adds new contact records to the database.
UpdateContact - Will edit the contact records already stored in the database.
GetContactById - Retrieve a single record in the database.
GetAllContacts - Returns all the selected contacts from the database.
DeleteContactById - Removes an instance of a record from the database.

Ensure you have Gin Gonic Http library installed and this has already been covered in the project setup section.
Create controllers/contact.controllers.go file where all your CRUD methods will be stored and add the following code to begin with.

controllers/contact.controllers.go

package controllers

import (
    "context"
    "database/sql"
    "net/http"
    "strconv"
    "time"

    db "github.com/Geoff89/sqlccrud/db/sqlc"
    "github.com/Geoff89/sqlccrud/schemas"

    "github.com/gin-gonic/gin"
    "github.com/google/uuid"
)

type ContactController struct {
    db  *db.Queries
    ctx context.Context
}

func NewContactController(db *db.Queries, ctx context.Context) *ContactController {
    return &ContactController{db, ctx}
}

Implementing CRUD in Golang Rest API

Creating a post route handler for contact creation.

This route handler will be called when a POST request is made to the endpoint http://localhost:8000/api/contacts. In the file controllers/contact.controller.go add the following code.

// Create contact  handler
func (cc *ContactController) CreateContact(ctx *gin.Context) {
    var payload *schemas.CreateContact

    if err := ctx.ShouldBindJSON(&payload); err != nil {
        ctx.JSON(http.StatusBadRequest, gin.H{"status": "Failed payload", "error": err.Error()})
        return
    }

    now := time.Now()
    args := &db.CreateContactParams{
        FirstName:   payload.FirstName,
        LastName:    payload.LastName,
        PhoneNumber: payload.PhoneNumber,
        Street:      payload.Street,
        CreatedAt:   now,
        UpdatedAt:   now,
    }

    contact, err := cc.db.CreateContact(ctx, *args)

    if err != nil {
        ctx.JSON(http.StatusBadGateway, gin.H{"status": "Failed retrieving contact", "error": err.Error()})
        return
    }

    ctx.JSON(http.StatusOK, gin.H{"status": "successfully created contact", "contact": contact})
}

schemas.CreateContact struct is bound to ctx.ShouldBindJSON() method to validate the request body against the json tags validation rules listed in the struct. Afterwards, initialize the &db.CreateContactParams{} struct with the required fields provided in the request payload. db.CreateContact() CRUD method is called afterward to insert a new record in the database and if successful http.StatusOK will be shown.

Adding an update route handler for updating contacts.

This is the second route endpoint created to edit contact records in the database at endpoint http://localhost:8000/api/contacts/:contactId when PATCH request is used.

controllers/contact.controller.go

// Update contact handler
func (cc *ContactController) UpdateContact(ctx *gin.Context) {
    var payload *schemas.UpdateContact
    contactId := ctx.Param("contactId")

    if err := ctx.ShouldBindJSON(&payload); err != nil {
        ctx.JSON(http.StatusBadRequest, gin.H{"status": "Failed payload", "error": err.Error()})
        return
    }

    now := time.Now()
    args := &db.UpdateContactParams{
        ContactID:   uuid.MustParse(contactId),
        FirstName:   sql.NullString{String: payload.FirstName, Valid: payload.FirstName != ""},
        LastName:    sql.NullString{String: payload.LastName, Valid: payload.LastName != ""},
        PhoneNumber: sql.NullString{String: payload.PhoneNumber, Valid: payload.PhoneNumber != ""},
        Street:      sql.NullString{String: payload.PhoneNumber, Valid: payload.Street != ""},
        UpdatedAt:   sql.NullTime{Time: now, Valid: true},
    }

    contact, err := cc.db.UpdateContact(ctx, *args)

    if err != nil {
        if err == sql.ErrNoRows {
            ctx.JSON(http.StatusNotFound, gin.H{"status": "failed", "message": "Failed to retrieve contact with this ID"})
            return
        }
        ctx.JSON(http.StatusBadGateway, gin.H{"status": "Failed retrieving contact", "error": err.Error()})
        return
    }

    ctx.JSON(http.StatusOK, gin.H{"status": "successfully updated contact", "contact": contact})
}

schemas.UpdateContact struct is bound to ctx.ShouldBindJSON() method to validate the request body against validation rules listed in the struct. Next, the ID of the record to be updated is retrieved from the request parameter and db.UpdateContactParams{} struct initialized with the required updated fields. db.UpdateContact() CRUD method is called afterwards to update the specified record in the database.

Retrieving a single contact record handler

This endpoint http://localhost:8000/api/contacts/:contactId will get a specific contact record when GET request is called . The specified ID parameter will get the specific record in the database and call db.GetContactById() method wich will retrieve the record that matches the parameter ID.

controllers/contact.controller.go

// Get a single handler
func (cc *ContactController) GetContactById(ctx *gin.Context) {
    contactId := ctx.Param("contactId")

    contact, err := cc.db.GetContactById(ctx, uuid.MustParse(contactId))
    if err != nil {
        if err == sql.ErrNoRows {
            ctx.JSON(http.StatusNotFound, gin.H{"status": "failed", "message": "Failed to retrieve contact with this ID"})
            return
        }
        ctx.JSON(http.StatusBadGateway, gin.H{"status": "Failed retrieving contact", "error": err.Error()})
        return
    }

    ctx.JSON(http.StatusOK, gin.H{"status": "Successfully retrived id", "contact": contact})
}

Getting all records handler

This handler will retrieve a specific number of records from the database when a request is made from endpoint http://localhost:8000/api/contacts?page=1&limit=10.

controllers/contact.controllers.go

// Retrieve all records handlers
func (cc *ContactController) GetAllContacts(ctx *gin.Context) {
    var page = ctx.DefaultQuery("page", "1")
    var limit = ctx.DefaultQuery("limit", "10")

    reqPageID, _ := strconv.Atoi(page)
    reqLimit, _ := strconv.Atoi(limit)
    offset := (reqPageID - 1) * reqLimit

    args := &db.ListContactsParams{
        Limit:  int32(reqLimit),
        Offset: int32(offset),
    }

    contacts, err := cc.db.ListContacts(ctx, *args)
    if err != nil {
        ctx.JSON(http.StatusBadGateway, gin.H{"status": "Failed to retrieve contacts", "error": err.Error()})
        return
    }

    if contacts == nil {
        contacts = []db.Contact{}
    }

    ctx.JSON(http.StatusOK, gin.H{"status": "Successfully retrieved all contacts", "size": len(contacts), "contacts": contacts})
}

The .DefaultQuery() is used to specify query parameters page and limit. These query parameters are useful for pagination. The limit query sets the number of rows to be retrieved from the database and the page query parameter is useful for pagination of your records. Next, the &db.ListContactsParams{} is initialized with the specific query parameters before calling db.ListContacts() method to get all records from the database.

Deleting record route handler

This endpoint will delete a specific record from the database as specified by the contactID parameter when
http://localhost:8000/api/contacts/:contactId endpoint DELETE request is made. The db.GetContactById() is used to confirm whether the specific record to be deleted exists in the database after which db.DeleteContact() deletes the specific record from the database.

controllers/contact.controllers.go

// Deleting contact handlers
func (cc *ContactController) DeleteContactById(ctx *gin.Context) {
    contactId := ctx.Param("contactId")

    _, err := cc.db.GetContactById(ctx, uuid.MustParse(contactId))
    if err != nil {
        if err == sql.ErrNoRows {
            ctx.JSON(http.StatusNotFound, gin.H{"status": "failed", "message": "Failed to retrieve contact with this ID"})
            return
        }
        ctx.JSON(http.StatusBadGateway, gin.H{"status": "Failed retrieving contact", "error": err.Error()})
        return
    }

    err = cc.db.DeleteContact(ctx, uuid.MustParse(contactId))
    if err != nil {
        ctx.JSON(http.StatusBadGateway, gin.H{"status": "failed", "error": err.Error()})
        return
    }

    ctx.JSON(http.StatusNoContent, gin.H{"status": "successfuly deleted"})

}

The complete source code for the route handlers will be as follows:

controllers/contact.controllers.go

package controllers

import (
    "context"
    "database/sql"
    "net/http"
    "strconv"
    "time"

    db "github.com/Geoff89/sqlccrud/db/sqlc"
    "github.com/Geoff89/sqlccrud/schemas"

    "github.com/gin-gonic/gin"
    "github.com/google/uuid"
)

type ContactController struct {
    db  *db.Queries
    ctx context.Context
}

func NewContactController(db *db.Queries, ctx context.Context) *ContactController {
    return &ContactController{db, ctx}
}

// Create contact  handler
func (cc *ContactController) CreateContact(ctx *gin.Context) {
    var payload *schemas.CreateContact

    if err := ctx.ShouldBindJSON(&payload); err != nil {
        ctx.JSON(http.StatusBadRequest, gin.H{"status": "Failed payload", "error": err.Error()})
        return
    }

    now := time.Now()
    args := &db.CreateContactParams{
        FirstName:   payload.FirstName,
        LastName:    payload.LastName,
        PhoneNumber: payload.PhoneNumber,
        Street:      payload.Street,
        CreatedAt:   now,
        UpdatedAt:   now,
    }

    contact, err := cc.db.CreateContact(ctx, *args)

    if err != nil {
        ctx.JSON(http.StatusBadGateway, gin.H{"status": "Failed retrieving contact", "error": err.Error()})
        return
    }

    ctx.JSON(http.StatusOK, gin.H{"status": "successfully created contact", "contact": contact})
}

// Update contact handler
func (cc *ContactController) UpdateContact(ctx *gin.Context) {
    var payload *schemas.UpdateContact
    contactId := ctx.Param("contactId")

    if err := ctx.ShouldBindJSON(&payload); err != nil {
        ctx.JSON(http.StatusBadRequest, gin.H{"status": "Failed payload", "error": err.Error()})
        return
    }

    now := time.Now()
    args := &db.UpdateContactParams{
        ContactID:   uuid.MustParse(contactId),
        FirstName:   sql.NullString{String: payload.FirstName, Valid: payload.FirstName != ""},
        LastName:    sql.NullString{String: payload.LastName, Valid: payload.LastName != ""},
        PhoneNumber: sql.NullString{String: payload.PhoneNumber, Valid: payload.PhoneNumber != ""},
        Street:      sql.NullString{String: payload.PhoneNumber, Valid: payload.Street != ""},
        UpdatedAt:   sql.NullTime{Time: now, Valid: true},
    }

    contact, err := cc.db.UpdateContact(ctx, *args)

    if err != nil {
        if err == sql.ErrNoRows {
            ctx.JSON(http.StatusNotFound, gin.H{"status": "failed", "message": "Failed to retrieve contact with this ID"})
            return
        }
        ctx.JSON(http.StatusBadGateway, gin.H{"status": "Failed retrieving contact", "error": err.Error()})
        return
    }

    ctx.JSON(http.StatusOK, gin.H{"status": "successfully updated contact", "contact": contact})
}

// Get a single handler
func (cc *ContactController) GetContactById(ctx *gin.Context) {
    contactId := ctx.Param("contactId")

    contact, err := cc.db.GetContactById(ctx, uuid.MustParse(contactId))
    if err != nil {
        if err == sql.ErrNoRows {
            ctx.JSON(http.StatusNotFound, gin.H{"status": "failed", "message": "Failed to retrieve contact with this ID"})
            return
        }
        ctx.JSON(http.StatusBadGateway, gin.H{"status": "Failed retrieving contact", "error": err.Error()})
        return
    }

    ctx.JSON(http.StatusOK, gin.H{"status": "Successfully retrived id", "contact": contact})
}

// Retrieve all records handlers
func (cc *ContactController) GetAllContacts(ctx *gin.Context) {
    var page = ctx.DefaultQuery("page", "1")
    var limit = ctx.DefaultQuery("limit", "10")

    reqPageID, _ := strconv.Atoi(page)
    reqLimit, _ := strconv.Atoi(limit)
    offset := (reqPageID - 1) * reqLimit

    args := &db.ListContactsParams{
        Limit:  int32(reqLimit),
        Offset: int32(offset),
    }

    contacts, err := cc.db.ListContacts(ctx, *args)
    if err != nil {
        ctx.JSON(http.StatusBadGateway, gin.H{"status": "Failed to retrieve contacts", "error": err.Error()})
        return
    }

    if contacts == nil {
        contacts = []db.Contact{}
    }

    ctx.JSON(http.StatusOK, gin.H{"status": "Successfully retrieved all contacts", "size": len(contacts), "contacts": contacts})
}

// Deleting contact handlers
func (cc *ContactController) DeleteContactById(ctx *gin.Context) {
    contactId := ctx.Param("contactId")

    _, err := cc.db.GetContactById(ctx, uuid.MustParse(contactId))
    if err != nil {
        if err == sql.ErrNoRows {
            ctx.JSON(http.StatusNotFound, gin.H{"status": "failed", "message": "Failed to retrieve contact with this ID"})
            return
        }
        ctx.JSON(http.StatusBadGateway, gin.H{"status": "Failed retrieving contact", "error": err.Error()})
        return
    }

    err = cc.db.DeleteContact(ctx, uuid.MustParse(contactId))
    if err != nil {
        ctx.JSON(http.StatusBadGateway, gin.H{"status": "failed", "error": err.Error()})
        return
    }

    ctx.JSON(http.StatusNoContent, gin.H{"status": "successfuly deleted"})

}

Route Creation for the Contact Handlers

With the specific route controllers in place, it's time to create Gin Gonic router where CRUD endpoints for the controller handlers will be invoked.
Create a route/contact.route.go file with the following code

routes/contact.route.go

package routes

import (
    "github.com/Geoff89/sqlccrud/controllers"
    "github.com/gin-gonic/gin"
)

type ContactRoutes struct {
    contactController controllers.ContactController
}

func NewRouteContact(contactController controllers.ContactController) ContactRoutes {
    return ContactRoutes{contactController}
}

func (cr *ContactRoutes) ContactRoute(rg *gin.RouterGroup) {

    router := rg.Group("contacts")
    router.POST("/", cr.contactController.CreateContact)
    router.GET("/", cr.contactController.GetAllContacts)
    router.PATCH("/:contactId", cr.contactController.UpdateContact)
    router.GET("/:contactId", cr.contactController.GetContactById)
    router.DELETE("/:contactId", cr.contactController.DeleteContactById)
}

The final step to do before testing your endpoints is to create the main.go file which is the main entry point to your API project when the GIN server is started and connection to the database is made. Add the following code to the main.go file.

package main

import (
    "context"
    "database/sql"
    "fmt"
    "log"
    "net/http"

    "github.com/Geoff89/sqlccrud/controllers"
    dbCon "github.com/Geoff89/sqlccrud/db/sqlc"
    "github.com/Geoff89/sqlccrud/routes"
    "github.com/Geoff89/sqlccrud/util"
    "github.com/gin-gonic/gin"
    _ "github.com/lib/pq"
)

var (
    server *gin.Engine
    db     *dbCon.Queries
    ctx    context.Context

    ContactController controllers.ContactController
    ContactRoutes     routes.ContactRoutes
)

func init() {
    ctx = context.TODO()
    config, err := util.LoadConfig(".")

    if err != nil {
        log.Fatalf("could not loadconfig: %v", err)
    }

    conn, err := sql.Open(config.DbDriver, config.DbSource)
    if err != nil {
        log.Fatalf("Could not connect to database: %v", err)
    }

    db = dbCon.New(conn)

    fmt.Println("PostgreSql connected successfully...")

    ContactController = *controllers.NewContactController(db, ctx)
    ContactRoutes = routes.NewRouteContact(ContactController)

    server = gin.Default()
}

func main() {
    config, err := util.LoadConfig(".")

    if err != nil {
        log.Fatalf("failed to load config: %v", err)
    }

    router := server.Group("/api")

    router.GET("/healthcheck", func(ctx *gin.Context) {
        ctx.JSON(http.StatusOK, gin.H{"message": "The contact APi is working fine"})
    })

    ContactRoutes.ContactRoute(router)

    server.NoRoute(func(ctx *gin.Context) {
        ctx.JSON(http.StatusNotFound, gin.H{"status": "failed", "message": fmt.Sprintf("The specified route %s not found", ctx.Request.URL)})
    })

    log.Fatal(server.Run(":" + config.ServerAddress))
}

Going through the above code an init function loads the environment variables from the app.env file, connects to the database and starts an instance of Gin Gonic engine. In Golang the init function is usually executed first before the main function. For tasks that you want to run before firing the server such as database connections use init function.

Finally, in the main function, you create the API routes and start the GIN server to listen to the port provided.
Now it is time to test the RESTful api endpoints. Type air at the terminal in the root of the project. It will automatically start the server for you, and show the endpoints you will make requests to. When run successfully at the terminal air will display output similar to the one below.

Testing the Golang RESTful API with REST Client

The Golang API server is now running together with SQLC for running database operations. Here you will use REST Client extension for Visual Studio to run the API tests. For advanced testing of APIs with more features you can use Postman.

HTTP POST Request: This will create records.

This request will create a new record in the database. A HTTP POST request will be made to the endpoint http://localhost:8000/api/contacts with a json object in the request body. Gin Gonic server will then validate the request body with the given Golang CreateContact struct instructions and finally invoke .CreateContact() method to insert a new contact record in the database. Add a few more records following the screenshot guide below.

HTTP GET Request: This will retrieve all records

To retrieve all records in the database make a GET request to the endpoint http://localhost:8000/api/contacts. For pagination purposes, this request will retrieve the first 10 results.

HTTP GET Request: Retrieving a single record.

To get a single record from the database, send a GET request to the http://localhost:8000/api/contacts/:contactId endpoint. The server will query the database and retrieve the record that matches the params contactId requested by client.

HTTP PATCH Request: will update a specified record

For updating a record in the database make a PATCH HTTP request to the http://localhost:8000/api/contacts/:contactId endpoint with a specific ID from the database record as the value of the URL parameter. In the body of the JSON object change values of the columns you want to update. Proceed to make a request and Gin Gonic Server will only update the fields requested to be updated and return an updated record to the client.

HTTP DELETE Request: This will remove a record

To remove a record from the database make a request at the
http://localhost:8000/api/contacts/:contactId endpoint by specifying the ID from the database as the URL parameter.

Conclusion

In this guide, you learned the principles of RESTful APIs. You started by generating Golang CRUD code with SQLC and later on ran database migration using the Golang migrate library. You also built a RESTful CRUD API that runs on Gin Gonic HTTP web framework and uses PostgreSQL as the database of choice.

You can find the code for this project in this Github Repo.

DEV Community: Geoffrey Sagini Mogambi

Helm Charts: An Organised Way to Install Apps on a Kubernetes Cluster

Prerequisites

Helm Installation

Installing Helm On Linux

Installing Helm on macOS

Installing Helm on Windows

Installing with Scoop

Installing with Chocolately

Searching Helm Charts from a Chart Repository

output

output

output

Helm Chart Installation

output

output

output

Upgrading a Helm Release

output

output

output

output

Implementing a Rollback and Deleting a Release

output

output

output

output

output

Deploying Custom Charts

output

output

Conclusion

How to Deploy PostgreSQL with Docker and Docker Compose

Table of Contents

Prerequisites

Deploying PostgreSQL with Docker

Deploying PostgreSQL with Docker Compose

Conclusion

Metaprogramming in Golang: Reflection Package guide

reflect.TypeOf()

output

reflect.Value()

output

reflect.Kind()

output

reflect.Copy()

output

reflect.DeepEqual()

output

reflect.Swapper()

output

reflect.NumField()

output

reflect.Field()

output

reflect.FieldByName()

output

reflect.FieldByIndex()

output

reflect.MakeSlice()

output

reflect.MakeMap()

output

reflect.MakeChan()

output

reflect.MakeFunc()

output

Conclusion

Deploying a Golang RESTful API with Gin, SQLC and PostgreSQL

Table of Contents

Prerequisites

Setting up the Golang project

Setting up PostgreSQL with Docker Compose

app.env

Performing database migration with Golang migrate

Using SQLC to generate Go code from SQL queries

db/query/contact.sql

Loading Environment Variables with Viper

util/config.go

Creating Request Validation Structs