DEV Community: Johan Lejdung

A mini-guide - Go Modules and Private Repositories

Johan Lejdung — Sat, 06 Feb 2021 11:53:10 +0000

The official dependency management system Go Modules have been supported since Go 1.11, and has since exploded in use. As of 2020 of the adoption of Go Modules has risen to 82% among developers.

Brief Background

Before Go Modules were a thing, there was a variety of community-created dependency management systems such as dep, godep, and govendor. All of these tried to solve the issue of managing the package versions used in applications.

You might not even have heard about $GOPATH, but this used to be something every Go developer had to wrap their heads around. $GOPATH is the path to the location of your workspace, all of your code had to live within this space before Go Modules. That meant that you weren't able to put your code anywhere you wanted, Go Modules changed all of that and now allows developers to place their code wherever they please as long as they have a go.mod file in the directory.

Go Modules

One of the first things you want to do after creating a folder or repository for your new project is to name the module. You typically name it the same as the path to your repository, such as: github.com/johan-lejdung/my-repo but you are free to name it whatever you want.

You create the module with the following command

go mod init github.com/johan-lejdung/my-repo

After you've run the command there is a go.mod and go.sum file in your repository, in the interest to keep this article brief feel free to read more about these two files over at this post.

The beauty of Go Modules is that you won't have to update the dependencies yourself, most built-in Go commands will automatically update the dependencies in your code as you run it. So whenever you build, run or fetch a package it’s all updated automatically.

Private Repositories

If you have your code in a private repository you might have issues accessing it and are presented with errors such as 410 Gone. Go 1.13 introduced a new environmental variable $GOPRIVATE .

From: tip.golang.org

The GOPRIVATE environment variable controls which modules the go command considers to be private (not available publicly) and should therefore not use the proxy or checksum database. The variable is a comma-separated list of glob patterns (in the syntax of Go's path.Match) of module path prefixes. For example,

GOPRIVATE=*.corp.example.com,rsc.io/private

causes the go command to treat as private any module with a path prefix matching either pattern, including git.corp.example.com/xyzzy, rsc.io/private, and rsc.io/private/quux.

To be able to access your private repository you will need to set your $GOPRIVATE with the following command:

go env -w GOPRIVATE=github.com/repoURL/private-repo

You can also include all repositories under an organization or provide a comma-separated list of individual repositories.

go env -w GOPRIVATE=github.com/<OrgNameHere>/*

If you are using SSH to access your Git Repository you should consider adding the following to your ~/.ssh/config file to enable the Go command to access the git service:

Host github.com
 AddKeysToAgent yes
 UseKeychain yes
 IdentityFile ~/.ssh/id_github

Where ~/.ssh/id_github is the private key for your SSH authentication. The same setup should work with other repositories such as Gitlab and Bitbucket.

If you are using SSH to access a local Git Repository you should add the following to your ~/.gitconfig

[url "ssh://git@git.local.intranet/"]

       insteadOf = https://git.local.intranet/

I hope you have enjoyed this mini-guide. For updates on new articles and content from me, follow me on twitter @JohanLejdung.

Pitching your idea - Things I've learned

Johan Lejdung — Thu, 10 Dec 2020 21:25:48 +0000

Whether you are working hard on a side project, or just started building your company you are going to communicate your idea sometime. And most likely you are going to want someone to invest time or money in your idea.

I’ve been doing a lot of pitches together with my cofounder for our startup ChainTraced lately. As a practise I’ve written down a couple of reflections and things I’ve learned.

I’m going to keep it short and consice, so lets dig in! 📚

Presenting in person 🎤

Practise your on-stage energy, posture and tone of voice. Looking calm and secure is more imporant than you might think. Competition is fierce! You want to stand out in every positive way you can and you don’t want to bore your listeners.

With slides in person 🌟

Use the slides to enhance your pitch, but don't count on them to carry the entire pitch. Interact with your slides as a presenter, guide your audience through the presentation. Replace long text with images, graphs, titles and list/points. Reduce potential distractions in the slides, such as moving images and too much text.

More than one? 😄😅

If possible, avoid having too many people present. Synchronise your on-stage energies and tone of voice and make sure the hand-offs are woven into the presentation, don’t switch back and forth. Especially if there is a microphone or other physical item involved. The slight pause needed to transfer the item is enough to ruin the flow.

Online pitch 🧑‍💻

If you are pitching online, give a bit more attention to the presentation. Make sure it looks good, has a coherent flow and doesn’t have distractions in it. Guiding the listener through your presentation is more important now that you are online, you can’t physically point at the slides so make sure they are self explanatory or make use of animations.

Time it! ⏱

Make sure that your pitch doesn't go on for too long, or too short. Depending on the platform you are presenting on, you want to make sure you have time for questions or at the very least book a followup meeting if you are talking to an investor.

Know your content 📚

I’m not much for practicing, but that doesn’t stop me from studying. If you know your content you are prepared to go "off script" if needed. A little bit of practicing together with a deep knowledge of the content have served me well.

Build story 💭

Make your pitch engaging by building a story around the problem you are solving. Done correctly, this engages the audience to want to understand the problem and how you are solving it. Engagement obviously greatly increases your chances to stand out 👏

Keep it simple 🏕

Don’t underestimate the complexity of the problem and your solution. Replace jargon with common words, and don’t go overboard on explanations. Try the pitch on a family member or a friend to help identify the pain points, where they fall off.

I hope this has been of use, either way it was a good practice for me to writing down some reflections. I don’t do it nearly enough 🙃 I'd love to hear your thoughts and comments.

If you liked this micro-blog sharing it with friends on Twitter is greatly appreciated!

Johan Lejdung

@johanlejdung

I’ve been doing a lot of pitches together with my cofounder for our #startup ChainTraced lately. As a practise I’ve written down a couple of reflections and things I’ve learned 😊 Most of these should apply for other kinds of presentations too

Thread 🧵

#vc #presenting #pitch

19:28 PM - 09 Dec 2020

Images (in order):
Photo by Tool., Inc on Unsplash
Photo by William Iven on Unsplash
Photo by Andrew Neel on Unsplash

A mini-guide — Middleware, and how it works in Go

Johan Lejdung — Tue, 22 Oct 2019 18:28:13 +0000

In this article I will explain the concept of middleware and create a working example in Go. The code for the complete project will be linked at the end of the article.

The article is written as a companion piece to my previous article A mini-guide — Build a REST API as a Go microservice together with MySQL. Although this article is standalone and will be readable and understandable without having read the other one beforehand.

This code is using Go Modules, if you are unfamiliar with Go Modules read about it in this other article of mine.

Let’s dive in! <⌨>️

Middleware, what even are they?

If you are already familiar with Middleware and just want to get to the action, you can skip this section.

If you feel like Ryan Reynolds above when I mention Middleware, I got you covered 🤗

In the world of networking a middleware is code that executes on network-based requests made by a client, which will then chain the request to the next middleware and eventually the final executing function. This is useful when you want to separate operations shared across multiple endpoints, such as: authentication, performance logging and data collection.

As an example; let’s say we have an endpoint that returns a list of messages posted by a user, and another that returns sensitive personal information about that user.

Both endpoints are only accessible when the user is logged into the service. Additionally, we also want to log the execution time of each request made to our server. The code for the authentication and logging could be written together with the controller and copied over to the two endpoints. But the example is easily split into four different packages.

One that logs data about the request (LOG), one that verifies that the user is logged in (AUTH), one that returns a list of messages (RETURN MSG) and one that returns user data (RETURN USER).

By writing these functionalities as middleware we can easily chain the request and return prematurely at each middleware if needed, for example if the user is not logged in whilst making the request.

It might be chained as this:

[LOG] -> [AUTH] -> [RETURN MSG]
[LOG] -> [AUTH] -> [RETURN USER]

With the logic for LOG and AUTH separated, we only have to write the code and tests for it once. 🎉

Middleware in Go

In Go all of the network requests are fulfilling an interface called Handler in the net/http package. For those unfamiliar with Go, a handler responds to a HTTP request and handles both reads from the request and writes to the response.

Because Go comes with and expects this Handler interface in it’s most crucial functions, it’s very easy and reliable to build packages around it and extend the interface with wrappers, such as middleware.

A quick way to create a middleware would be to wrap the net/http.Handler.ServeHTTP method;

ServeHTTP(ResponseWriter, *Request)

by adding a outer function that takes a Handler as a parameter, as well as returning a Handler which in this case takes the form of a HandlerFunc.

The function above takes an argument of type Handler, which is used to chain the request at the end of the method. Because we also return a Handler type, we can chain the calls. It could look as such;

firstMiddleware(secondMiddleware(lastHandler))

From this simple method alone we can make a pretty good implementation of middleware. It might not be as robust as we would like, but it’s doable.

Going forward I will use a package called Negroni that makes this process a whole lot easier, although feel free to roll your own implementation. 👍

Negroni wraps around your router of choice and provides functionality for middleware management. Such as a simpler, and more dynamic way of chaining middleware.

Implementation — Wrapping the Handler

In order to keep it as simple as possible, I’m planning on implementing a middleware that will log the execution time of a network request. The middleware will be built and executed with Negroni.

Let’s start with creating our router and wrapping it in Negroni!

As seen in the code sample above; we start out, on line 12, with creating a new router with the gorilla/mux package, a popularly used package for router management. On the next few rows we register our endpoint, a GET endpoint on the root path /. So far, so good, and some of you will have seen this exact code before!

On line 18 something new happens, we create a new instance of a Negroni handler, on line 19 we tell the instance to use the router that we previously created.

Because Negroni fulfils the Handler interface, we can use it just as we would have used the router. Which we do on line 21, we pass in the negroni handler in place of where we would normally pass in the mux router. If you execute this code and call the endpoint you observe that it works as expected, eg. it logs “Endpoint handler called” when calling:

curl localhost:8080/

Implementation — The birth of a Middleware

Okay! So we have Negroni and an endpoint setup, let's continue with the middleware creation!

As previously explained, a middleware is really just a Handler. So all we need to do is make a struct that fulfils the interface of a Handler. The twist here is that since we are using Negroni we have to implement their version of the Handler interface.

As seen below it’s similar to the net/http.Handler interface, except that it also has support for built in chaining with the next parameter.

type Handler interface {
    ServeHTTP(rw http.ResponseWriter, r *http.Request, next http.HandlerFunc)
}

Let’s create a simple logger.

The code snippet above shows how you could create a really simple middleware by fulfilling the negroni.Handler interface. As seen, on line 12 we log some arbitrary information. On the next line we save the current timestamp to the variable t.

Finally; on line 14 we continue the execution of the next handler, which might be a another chained middleware or the final execution function. After that call has finished execution we log the total execution time.

Let’s go ahead and add it to the router to see our execution time! ⏱

As seen, the only change in the code here is on line 20.

n.Use(&mw.Logger{})

With that line of code we are telling Negroni to use the Logger middleware that we created, the middleware are executed in the order they are inserted.

If we execute this code and call the endpoint we end up with this log:

The logger middleware is executing!
Endpoint handler called
Execution time: 25.146µs

The first and third line is generated from the middleware, whilst the second line is generated in the function endpointHandler; the final handler function for the endpoint. We’ve successfully made a middleware! 🙌

Let’s play with the idea that we also had an Authentication middleware (AuthMW), the AuthMW would make sure that no unauthorised requests reaches the execution function. To add this to the chain we would simply do the following:

n.Use(&mw.Logger{})
n.Use(&mw.Auth{})

With that change, we have built the first parts of a middleware chain. 👏

The code above would execute the logger before continuing to the Auth middleware. In the same fashion you could build a chain of middleware for individual router paths, if we combine these two methods of chaining middleware we end up with some pretty powerful tools for building our routes and the code we want them to execute!

The final code is available below.

johan-lejdung / go-microservice-middleware-guide

Middleware example

Run the code:

go run main.go

Try calling the endpoint with

curl localhost:8080/ -v

View on GitHub

I hope that you found this interesting and that you learned something!

If you did, you might find my previous article interesting as well. In it I go through how to build a REST API with a MySQL server connection in Go 🚀

A mini-guide — Build a REST API as a Go microservice together with MySQL

Johan Lejdung ・ Aug 11 '19

#go #beginners #sql #webdev

Furthermore, If you liked this article sharing it with friends or on Twitter is greatly appreciated!

// Detect dark theme var iframe = document.getElementById('tweet-1186712393428209665-317'); if (document.body.className.includes('dark-theme')) { iframe.src = "https://platform.twitter.com/embed/Tweet.html?id=1186712393428209665&theme=dark" }

Although this article took way longer than anticipated to write. It is my ambition to continue writing short guides like this, if you have any suggestions please comment below.

Happy coding! </⌨>️

A mini-guide — Build a REST API as a Go microservice together with MySQL

Johan Lejdung — Sun, 11 Aug 2019 20:26:29 +0000

I have recently found myself coding and deploying a lot of Go microservices, both at my day-job at Storytel and for my side-project Wiseer. In this mini-tutorial, I will create a simple REST-API with a MySQL database. The code for the complete project will be linked at the end of the article.

If you haven’t already done so I recommend taking the Go Tour as a complementary to this article.

Let’s dive in!

Setup the API

One of the first things to do is to choose a routing package. Routing is what connects an URL to an executable function. The mux package has served me well for routing, but there are other alternatives as well, with no real difference in performance. Such as httprouter and chi. I will be using mux in this guide.

For the sake of simplicity, we will create a single endpoint that prints out a message.

The code above creates a Router, associates a URL with an handler function — postFunction in our case — and starts a server on port 8080 using that Router.

Easy enough, huh? 🤠

Database connection

Let's build on the code above by connecting it to a MySQL database. Go provides an interface for SQL databases but requires a driver. I’m using go-sql-driver for this example.

The code is placed in another package, called db, and assumes there is a database running on localhost:3306 with a database called demo. The returned database automatically handles a connection pool to the database.

Let us update the postFunction from the previous code-snippet, to use this database.

That’s really it! It’s rather simple, but there are a few issues with the code above and a few nice-to-haves missing. This is where it gets a bit trickier, but don't jump ship just yet! ⚓️

Structs & Dependencies

If you’ve examined the code above, you might have noticed that we are opening the database on each API-call; even though the opened database is safe for concurrent use. We need some dependency management to make sure we only open the database once, for this, we want to use a struct.

We start by creating a new package, called app, to host our struct and it’s methods. Our App struct has two fields; a Router and a Database accessed at ln 17 and ln 24. We also set the returned status code manually at the end of the method at ln 30.

The main package and function also needs a few changes to make use of the new App struct. We remove the postFunction and setupRouter functions from this package, since it’s now in the app package. We are left with:

To make use of our new struct, we open a database and a new Router. Then we insert both of them into the fields of our new App struct.

Congratulations! You now have a connection to your database that will be used concurrently across all of your incoming API-calls 🙌

As a final step, we will add a GET-Method to our router setup and return the data in JSON. We start by adding a struct to fill our data with, and map the fields to JSON.

We follow that up with an expansion of the app.go file, with a new method getFunction that fetches and writes the data to the client response. The final file looks like this.

Database Migrations

We are going to add a final addition to the project. When a database is tightly coupled with an application or service you can save yourself unfathomable levels of headache by properly handling migrations of said database. We are going to use migrate for that, and expand our db package.

I'll go through the following, and admittedly long, gist below the embedded code.

Right after the opening of the database, we add another function call to migrateDatabase that will in turn; start the migration process.

We will also add a MigrationLogger struct to handle the logging during the process, the code can be seen here and it's usage on ln 45.

The migrations are executed from normal sql-queries. The migration files are read from the folder seen at ln 37.

Each time the database is opened, all the unapplied database migrations will be applied. Thereby keeping the database up to date without any manuainterventionon.

This coupled with a docker-compose file - containing the database - makes development on multiple machines dead-simple.

Wrapping it up

So you've made it all the way down here 👏 👏

An undeployable microservice is of no use, therefore we will add a Dockerfile to package the application for easy distribution - then I promise to let you go.

The built image is a mere 10MB! 😱

The code is available below.

johan-lejdung / go-microservice-api-guide

REST API

Before cloning

Read the accompanying article here: https://medium.com/@johanlejdung/a-mini-guide-build-a-rest-api-as-a-go-microservice-together-with-mysql-fc203a6411c0

Change the module name if you are cloning it to a new location.

go mod init <your_module_name>

If you are using a private repository make sure to set GOPRIVATE with:

go env -w GOPRIVATE=github.com/repoURL/private-repo

Setup

Included here are a docker-compose file that can be used to spin up a MySQL database in docker for the porpose of running this code.

Run the code:

go run main.go

Try inserting a value with

curl -XPOST localhost:8080/endpoint -v

Then fetching a value with

curl localhost:8080/endpoint/1 -v

DEV Community: Johan Lejdung

A mini-guide - Go Modules and Private Repositories

Brief Background

Go Modules

Private Repositories

Pitching your idea - Things I've learned

Presenting in person 🎤

With slides in person 🌟

More than one? 😄😅

Online pitch 🧑‍💻

Time it! ⏱

Know your content 📚

Build story 💭

Keep it simple 🏕

A mini-guide — Middleware, and how it works in Go

johan-lejdung / go-microservice-middleware-guide

Middleware example

A mini-guide — Build a REST API as a Go microservice together with MySQL

Johan Lejdung ・ Aug 11 '19

A mini-guide — Build a REST API as a Go microservice together with MySQL

Setup the API

Database connection

Structs & Dependencies

Database Migrations

Wrapping it up

johan-lejdung / go-microservice-api-guide

REST API

Before cloning

Setup

Further reading