Dependency Management With Go Modules

1. Introduction
2. Project Setup
3. Dependencies
4. Project Links
5. Resources

Introduction

Go Modules is a dependency management system that makes dependency version information explicit and easier to manage. It is supported in Go 1.11 and above.
A module is a collection of Go packages stored in a file tree with a go.mod file at its root. The go.mod file defines the module’s module path, which is also the import path used for the root directory, and its dependency requirements, which are the other modules needed for a successful build.
In this article, I'll walk you through a simple CRUD REST API using Go Modules

Project Setup

For this guided project I would assume you already have Go installed on your local machine, if not please go through the golang documentation on how to get Go installed in your specific operating system. I would also assume you know the basics of Go, if not check out the Resources section of this article for links to some awesome resources.

To get started, create a new directory anywhere you like to build out the project. (You can do this through the terminal by typing the following):

mkdir bookstore

Now we're going to create our project structure.

cd into the directory you just created and type the following on your terminal

mkdir config controllers models drivers

This creates five (4) new directories in your current working directory. These directories will house the various modules and logic that will run our application.
Whilst still in our working directory let's create our application entry file main.go and our .env file

touch main.go .env

Dependencies

Go is battery loaded with most of the packages we will need for this project but we will be making use of three third-party packages

• pq: A pure Go Postgres driver for Go's database/sql package

• gorilla mux: A request router and dispatcher for matching incoming requests to their respective handler.

• gotenv: A package that loads environment variables from .env or io.Reader in Go.

To install our dependencies, we need to initialize Go Modules in the root directory where we created our entry file main.go.
To do this type the following on the terminal

go mod init main

This creates a go.mod file in your root directory that looks like this

module main

go 1.13

Now we can install our third-party packages like so:

go get github.com/lib/pq
go get github.com/gorilla/mux
go get github.com/subosito/gotenv

Once done downloading the packages you should see the installed third-party packages listed in the go.mod file and also creates a go.sum in your root directory

module main

go 1.13

require (
    github.com/gorilla/mux v1.7.4 // indirect
    github.com/lib/pq v1.4.0 // indirect
    github.com/subosito/gotenv v1.2.0 // indirect
)

If you are from a NodeJS background you can think of the root go.mod file as your package.json file just that holds a list of all dependencies used in your Go application while the go.sum file can be likened to the package.lock.json file which contains the expected cryptographic hashes of the content of specific module versions.

The major inbuilt Go packages we will be using in this project include the following

• database/sql This is a package that provides a generic interface around SQL (or SQL-like) databases and must be used in conjunction with a database driver (which is pq in our app)

• net/http This is a package that provides HTTP client and server implementations.

Models

We will set up our database models in the models directory we created earlier. The models will be handled as a module in our application using Go Modules. To do this we cd into our models directory and run the following on the terminal

touch book.go

This creates the file in which we will set up our book model like so:

package models

type Book struct {
    ID     int    `json:id`
    Title  string `json:title`
    Author string `json:author`
    Year   string `json:year`
}

In the above snippet, we have just created a package called models that holds our database model. With this package set, we can create a Go Module from your models package which can be used in any other part of our application. To do this type the following on the terminal

go mod init models

This creates a go.mod file in your models directory with the following information:

module models

go 1.13

This shows that the Go Module is for the models package and the version of Go I'm running on is go 1.13.
Making our models package a Go Module gives us the flexibility of importing it as a package in any other package we need to use it.

Drivers

This is where we will set up our database connections. We start by creating a drivers.go file in the drivers directory we created earlier

touch drivers.go

In the file, we flesh out our database connection like so

package drivers

import (
    "database/sql"
    "fmt"
    "log"
    "os"

    // third-party package
    _ "github.com/lib/pq"
)

var db *sql.DB

func logFatal(err error) {
    if err != nil {
        log.Fatal(err)
    }
}

// ConnectDB ...
func ConnectDB() *sql.DB {
    var (
        host     = os.Getenv("PG_HOST")
        port     = os.Getenv("PG_PORT")
        user     = os.Getenv("PG_USER")
        password = os.Getenv("PG_PASSWORD")
        dbname   = os.Getenv("PG_DB")
    )
    psqlInfo := fmt.Sprintf("host=%s port=%s user=%s "+
        "password=%s dbname=%s sslmode=disable",
        host, port, user, password, dbname)


    db, err := sql.Open("postgres", psqlInfo)
    logFatal(err)

    _, err = db.Exec("CREATE TABLE IF NOT EXISTS books (id serial, title varchar(32), author varchar(32), year varchar(32))")
    if err != nil {
        logFatal(err)
    }

    err = db.Ping()
    logFatal(err)

    log.Println(psqlInfo)

    return db
}

The underscore _ before the third-party package we imported implies that we are importing the package solely for its side-effects (initialization).
What this script does it initialize our database using a set of config variables from our .env which we will set up soon. At this point, we can set up our environment variables in our .env file and create our database.

PG_HOST='localhost'                   
PG_USER=postgres
PG_PASSWORD=<your password>
PG_DB=<your database name>
PG_PORT=5432


PORT="8080"

Now we need to create a Go Module out of our drivers package to make it available via import in other packages. Run

go mod init drivers

Config

The config directory will hold the database queries for our CRUD functionality. Let's start by creating a subdirectory book in our config directory that will hold our book queries. In the subdirectory create a bookConfig.go file. In the bookConfig.go file we'll write the logic for our database queries.

package bookdbconfig

import (
    "database/sql"
    "log"

    // models module
    "example.com/me/models"
)

// BookDbConfig database variable
type BookDbConfig struct{}

func logFatal(err error) {
    if err != nil {
        log.Fatal(err)
    }
}

// GetBooks ... Get all books
func (c BookDbConfig) GetBooks(db *sql.DB, book models.Book, books []models.Book) []models.Book {
    rows, err := db.Query("SELECT * FROM books")
    logFatal(err)

    defer rows.Close()

    for rows.Next() {
        err = rows.Scan(&book.ID, &book.Title, &book.Author, &book.Year)
        logFatal(err)

        books = append(books, book)
    }
    err = rows.Err()
    logFatal(err)

    return books
}

// GetBook ... Get a book
func (c BookDbConfig) GetBook(db *sql.DB, book models.Book, id int) models.Book {
    err := db.QueryRow("select * from books where id = $1", id).Scan(&book.ID, &book.Title, &book.Author, &book.Year)
    logFatal(err)

    return book
}

// AddBook ... Add a book
func (c BookDbConfig) AddBook(db *sql.DB, book models.Book) int {
    err := db.QueryRow("insert into books (title, author, year) values($1, $2, $3) returning id;", book.Title, book.Author, book.Year).Scan(&book.ID)
    logFatal(err)
    return book.ID
}

// UpdateBook ... Edit a book record
func (c BookDbConfig) UpdateBook(db *sql.DB, book models.Book) int64 {
    result, err := db.Exec("update books set title=$1, author=$2, year=$3 where id=$4 returning id;", &book.Title, &book.Author, &book.Year, &book.ID)

    rowsUpdated, err := result.RowsAffected()
    logFatal(err)
    return rowsUpdated
}

// RemoveBook ... remove a book
func (c BookDbConfig) RemoveBook(db *sql.DB, id int) int64 {
    result, err := db.Exec("delete from books where id=$1", id)
    logFatal(err)

    rowsDeleted, err := result.RowsAffected()
    logFatal(err)

    return rowsDeleted
}

Notice that we have imported the models module which we created earlier by mocking it as a dependency example.com/me/models. This is because there is no relative import in Go. Later on, we will wire up this mock imports in our root entry file and then this will all make sense.

As before we will also create a Go Module from our bookdbconfig

go mod init bookdbconfig

Controllers

Now we need to set up our controllers. This holds the logic and controls for the API endpoints we will need for our application. Let's start by creating a book.go file in the controllers directory we created earlier.
In the book.go file you just created in your controllers directory paste the following snippet:

package controllers

import (
    "database/sql"
    "encoding/json"
    "log"
    "net/http"
    "strconv"

    "example.com/me/bookdbconfig"

    "example.com/me/models"

    "github.com/gorilla/mux"
)

// Controller app controller
type Controller struct{}

var books []models.Book

func logFatal(err error) {
    if err != nil {
        log.Fatal(err)
    }
}

// GetBooks ... Get all books
func (c Controller) GetBooks(db *sql.DB) http.HandlerFunc {
    return func(w http.ResponseWriter, r *http.Request) {
        var book models.Book
        books = []models.Book{}

        bookStore := bookdbconfig.BookDbConfig{}

        books = bookStore.GetBooks(db, book, books)

        json.NewEncoder(w).Encode(books)
    }
}

// GetBook ... Get a single book
func (c Controller) GetBook(db *sql.DB) http.HandlerFunc {
    return func(w http.ResponseWriter, r *http.Request) {
        var book models.Book
        params := mux.Vars(r)

        id, err := strconv.Atoi(params["id"])

        logFatal(err)

        bookStore := bookdbconfig.BookDbConfig{}

        book = bookStore.GetBook(db, book, id)

        json.NewEncoder(w).Encode(book)
    }
}

// AddBook ... Add a single book
func (c Controller) AddBook(db *sql.DB) http.HandlerFunc {
    return func(w http.ResponseWriter, r *http.Request) {
        var book models.Book
        var bookID int
        json.NewDecoder(r.Body).Decode(&book)

        bookStore := bookdbconfig.BookDbConfig{}

        bookID = bookStore.AddBook(db, book)

        json.NewEncoder(w).Encode(bookID)
    }
}

// UpdateBook ... Update a book
func (c Controller) UpdateBook(db *sql.DB) http.HandlerFunc {
    return func(w http.ResponseWriter, r *http.Request) {
        var book models.Book
        json.NewDecoder(r.Body).Decode(&book)

        bookStore := bookdbconfig.BookDbConfig{}

        rowsUpdated := bookStore.AddBook(db, book)

        json.NewEncoder(w).Encode(rowsUpdated)

    }
}

// RemoveBook ... Remove/Delete a book
func (c Controller) RemoveBook(db *sql.DB) http.HandlerFunc {
    return func(w http.ResponseWriter, r *http.Request) {
        params := mux.Vars(r)

        id, err := strconv.Atoi(params["id"])

        logFatal(err)

        bookStore := bookdbconfig.BookDbConfig{}

        rowsDeleted := bookStore.RemoveBook(db, id)

        json.NewEncoder(w).Encode(rowsDeleted)
    }

}

Notice again that we have imported our models and bookdbconfig packages as dependencies again because we are making use of the packages in our controllers.
As before we will also initialize a Go Module in our controllers

go mod init controllers

Routing and Finishings

We are done with the basic functionalities of our API, now we need to set up our routes and wire-up all our modules.
In our entry file main.go let's paste the following snippet and create our endpoints:

package main

import (
    "database/sql"
    "log"
    "net/http"
    "os"

    "example.com/me/controllers"
    "example.com/me/drivers"

    "example.com/me/models"



    "github.com/gorilla/mux"

    "github.com/subosito/gotenv"
)

var books []models.Book

var db *sql.DB

func init() {
    gotenv.Load()
}

func logFatal(err error) {
    if err != nil {
        log.Fatal(err)
    }
}
func main() {
    db = drivers.ConnectDB()
    controller := controllers.Controller{}
    // port := os.Getenv("PORT")
    port := os.Getenv("PORT")

    if port == "" {
        log.Fatal("$PORT must be set")
    }
    router := mux.NewRouter()
    router.HandleFunc("/books", controller.GetBooks(db)).Methods("GET")
    router.HandleFunc("/books/{id}", controller.GetBook(db)).Methods("GET")
    router.HandleFunc("/books", controller.AddBook(db)).Methods("POST")
    router.HandleFunc("/books", controller.UpdateBook(db)).Methods("PUT")
    router.HandleFunc("/books/{id}", controller.RemoveBook(db)).Methods("DELETE")

    done := make(chan bool)
    go http.ListenAndServe(":" + port, router)
    log.Printf("Server started at port %v", port)
    <-done
    log.Fatal(http.ListenAndServe(":" + port, router))
}

Once again in our imports, we have imported the packages we need in this file as modules. At this point, if you have the vscode extensions for golang installed you should be getting linting errors in your code and if you try running the application like so it would fail. Let's try it out. To run your app type the following on the terminal

go run main.go

This is because we have not wired up the Go Modules we created in our root go.mod file. Let's do that now by making the following edit in our root go.mod file.

module main

go 1.13

require (
    example.com/me/bookdbconfig v0.0.0
    example.com/me/controllers v0.0.0
    example.com/me/drivers v0.0.0
    example.com/me/models v0.0.0

    github.com/gorilla/mux v1.7.4
    github.com/lib/pq v1.4.0 // indirect
    github.com/stretchr/testify v1.5.1 // indirect
    github.com/subosito/gotenv v1.2.0
)

replace example.com/me/bookdbconfig => ./config/book

replace example.com/me/models => ./models

replace example.com/me/controllers => ./controllers

replace example.com/me/drivers => ./drivers

Here we have added all the modules we created as dependencies using the replace keyword to reference the actual file path to the directory for each module which was initially being mocked (replace example.com/me/models => ./models).
Now if we run the app again we should get a response from the server.

2020/04/26 02:59:58 host=localhost port=5432 user=postgres password=<your password> dbname=bookstore sslmode=disable
2020/04/26 02:59:58 Server started at port 8080

And that's all there is to it, you can try out the endpoints on PostMan and tweak the API as you wish.

• Base URL: http://localhost:8080
• [GET]/books
• [GET]/books/{id}
• [POST]/books
• [PUT]/books
• [DELETE]/books/{id}

Final Project Structure

Project Link

Here is a link to a deployed and dockerized version of this project
Go Book API

Resources

Go basic knowledge
Go Modules