Building a Go Mod CLI to generate dependency trees

Kapil Pau — Tue, 15 Jun 2021 21:49:22 +0000

So, picture the scene, you have a Go project and then get a security alert saying a vulnerability has been detected. You look at your go.mod to see if you're using it and it's nowhere to be seen, but then you see it in your go.sum. How do you find where it is coming from?

Sound familiar? Well that's the situation I found myself in last week. So obviously, I tried to find a tool that would do it but couldn't find one, so like any programmer would do, I decided to build one.

The tool can be found here, with amd64 binaries for Linux, Windows and Mac available in the releases.

Installation

To install, simply download the appropriate binary for your operating system from the latest release, at the time of writing this, it was v1.2.0. Once downloaded, make sure the binary is executable and moved to be in your PATH, with the name go-tree.

How to use it

The tool only works with go mod projects, and requires the environment variable GOPATH to be set (by default, it will be $HOME/go). You can either run it from within the root directory of your go project (where you go.mod is located), or you can use the -modulePath flag to pass in a relative or absolute path to the project you want to scan.

$ go-tree -modulePath $GOPATH/src/path/to/module/to/scan

You can also use the -maxDepth flag to set the maximum recursion level, i.e. how far down the tree to scan. The options are either -1 or an integer above 0, -1 is to indicate no limit and is the default value.

$ go-tree -maxDepth -1

The final flag you can use is the -find flag, which is the whole reason this tool exists. If you specify this flag with the module you would like to find, it will print the full tree for all of the instances of that package in the dependency tree. Note that if you use -find, the -maxDepth will be ignored.

$ go-tree -find github.com/kapilpau/go-mod-dependency-tree

None of these flags are required.

How it works

As this is a single function cli tool, the whole code is contained within a single file. The code has two main pathways, both of which work in a similar way.

The first is the straight tree dump, i.e. where you don't specify the -find flag. This route recursively searches each dependency's go.mod to find all of the dependencies for that module and prints out the name and version of the dependency. For this, we read in the go.mod file for your project, find all of the modules in the requires section and look for the module in the src or pkg folders, in your GOPATH.

If they have exist and have a go.mod file, we continue through the chains and look for their dependencies. If we can't find a dependency in either location, or it doesn't have a go.mod file, we end that branch there and move on.

The other pathway is for when a user is searching for a specific module in the chain. In this case, it is slightly more complicated as we need to decide what to print out later on. For this, we use a custom tree struct, named dependencyChain. This struct has two fields, module (the name of the module currently being scanned) and children (the dependencies of the current module).

We do a similar recursive search to the one detailed above, however, rather than just simply printing out the values as we find them, we have to perform head recursion, so we can look at the outputs of the later recurssions before deciding what to do. So, if we find the module we're looking for, we end the branch of the tree there, as it would be wasteful to carry on, and we populate the dependencyChain object to pass back. Then, when we have the list of dependencyChains for each module, we check the size of the children field and if it is not empty, we pass it upwards, otherwise we ignore it. The reason we do this check is because we only want to see the branches that end in the module we're looking for.

Once we have completed this head recursive search, we perform a tail recursive print, to loop through the children of each dependencyChain and display it as a tree.

If the module you are looking for does not exist in the chain, or it cannot be found (as it may be in a non-go mod enabled project), then a message is printed out at the end to say so.

Lessons learned

I learned a lot from this project, namely how easy it is to create and build cli binaries in Go, even being able to build for different operating systems and architectures, without having to natively use an instance of them. This is definitely just the first of many more to come.

I got to apply the principles of recursion, that we spent so long learning at uni, to a real-life scenario.

I gained a deeper understanding of how Go stores dependencies, and where to find them when I need them.

Docker vs Kubernetes, which should I use?

Kapil Pau — Tue, 27 Apr 2021 19:34:54 +0000

Ok before going any further with this blog, I have to confess something. The title is a bit catfishy but it's a question I see a lot online so hopefully I will be able to answer it for anyone who has thought about it.

Let me start by initially defining what they both are.

Docker

Docker is a type of containerisation technology which allows for standardisation of packaging and shipping of products. Containers, as with Virtual Machines, run a virtual computer within your computer, however, unlike VMs, which run in parallel with the host OS and isolate resources, containers run within the host OS and are therefore much smaller, lightweight and quicker to spin up.

As with a traditional VM, containers are spun up using an image and that image contains the OS and the applications and libraries required for the container to perform whatever task is needed. Images are created using Dockerfiles, which contain a number of steps for the Docker engine to complete in order to build the image.

Docker images are typically in the 100s of MB in size and are stored in registries. Registries can be public or private, depending on the sensitivity of the images, and can be self-hosted or 3rd party hosted.

Kubernetes

Kubernetes is the industry standard container orchestration system, this is where you'll start to work out why the title is catfishy. A container orchestration tool manages the deployment, networking, availability and scaling on containers and services.

Kubernetes, often referred to as kube or k8s, is a service every cloud provider provides, with some providing their own custom Kubernetes services, like Google Kubernetes Engine (GKE) or RedHat's OpenShift Container Platform (OCP).

Kubernetes allows users to decoratively deploy their applications by telling Kubernetes what they want and letting the engine handle the deployment, maintenance and upgrade. This is done through the creation of a YAML representation of resources. The resources can be for deployment of containers, network and DNS management or auto-scalers for their deployments.

Which should I use?

The key aspect at the heart of this question is that it's a false dilemma, it's not necessarily a question of "or", because one is fundamentally dependant upon the other. Kubernetes is a tool for augmenting containers, of which Docker is the industry standard. You can have Docker without Kubernetes, but not the other way round.

There are lots of use-cases where Docker will do the job needed, and Kubernetes is over-kill. A Kubernetes cluster requires as least two computers, or nodes, to run and, generally, requires more maintenance than a normal server. If you don't have multiple microservices which need inter-connectivity and only want one instance of your container running, then running containers natively in Docker will suffice. Using the --restart flag when starting the container will achieve the same results as a Kubernetes cluster.

On the flip side, if you are running a number of microservices, with a service mesh, and require high availability and firewalls, then Kubernetes will simplify this and require very little infrastructure knowledge. The number of required nodes for a Kubernetes cluster means that they will cost more than just using Docker natively, and if you want someone to manage the infrastructure for you, it will drive the cost up even more.

tl;dr

If you have a large, multi-microservice, production application, I'd advise Kubernetes.

If you are only using one or two containers, with limited communication between them, you should be fine with native Docker.

DEV Community: Kapil Pau