Mofi Rahman

Posted on Sep 4, 2019 • Edited on Mar 24, 2020

Golang Rest Api Build Your First Rest API with GO

#go #tutorial #api #restapi

Build Your First Rest API with GO

There is three part to this workshop.

API
Rest API
Rest API with GO

API

If you have been around a computer for long enough you probably heard of this thing. What is this API?

API stands for Application Program Interface. Like most thing in computer science the abbreviation doesn't help much.

What it actually means is it exposes functionality without exposing internals. If you program in a language that supports writing functions or methods (pretty much all programming languages) you would totally understand what I am talking about.



func addNumber(a, b int) int {
    // DO AMAZING MATH HERE
    // and return the result
}

Even if you are super new to go, you can tell this function is about adding two numbers and returning the result.

For the user of the function you just call the function and never worry about how the function is doing what it does (don't trust every function).

Thats all an API is. An API could be a function you wrote, or a function from a library or method from a framework, or a http endpoint.

Rest API

Most APIs written this days are web apis. Don't quote me on that one because I didn't do any research to get a proper number. 😁 But given the number of web services and web application I don't think I am too far off.

What is REST

REST is acronym for REpresentational State Transfer. It is architectural style for distributed hypermedia systems and was first presented by Roy Fielding in 2000 in his famous dissertation.

Like any other architectural style, REST also does have it’s own 6 guiding constraints which must be satisfied if an interface needs to be referred as RESTful. These principles are listed below.

Guiding Principles of REST

Client–server – By separating the user interface concerns from the data storage concerns, we improve the portability of the user interface across multiple platforms and improve scalability by simplifying the server components.
Stateless – Each request from client to server must contain all of the information necessary to understand the request, and cannot take advantage of any stored context on the server. Session state is therefore kept entirely on the client.
Cacheable – Cache constraints require that the data within a response to a request be implicitly or explicitly labeled as cacheable or non-cacheable. If a response is cacheable, then a client cache is given the right to reuse that response data for later, equivalent requests.
Uniform interface – By applying the software engineering principle of generality to the component interface, the overall system architecture is simplified and the visibility of interactions is improved. In order to obtain a uniform interface, multiple architectural constraints are needed to guide the behavior of components. REST is defined by four interface constraints: identification of resources; manipulation of resources through representations; self-descriptive messages; and, hypermedia as the engine of application state.
Layered system – The layered system style allows an architecture to be composed of hierarchical layers by constraining component behavior such that each component cannot “see” beyond the immediate layer with which they are interacting.
Code on demand (optional) – REST allows client functionality to be extended by downloading and executing code in the form of applets or scripts. This simplifies clients by reducing the number of features required to be pre-implemented.

To see an example of a REST API we can use

HTTP Verbs

These are some conventions HTTP apis follow. These are actually not part of Rest specification. But we need to understand these to fully utilize Rest API.

HTTP defines a set of request methods to indicate the desired action to be performed for a given resource. Although they can also be nouns, these request methods are sometimes referred as HTTP verbs. Each of them implements a different semantic, but some common features are shared by a group of them: e.g. a request method can be safe, idempotent, or cacheable.

GETThe GET method requests a representation of the specified resource. Requests using GETshould only retrieve data.

HEADThe HEAD method asks for a response identical to that of a GET request, but without the response body.

POSTThe POST method is used to submit an entity to the specified resource, often causing a change in state or side effects on the server.

PUTThe PUT method replaces all current representations of the target resource with the request payload.

DELETEThe DELETE method deletes the specified resource.

CONNECTThe CONNECT method establishes a tunnel to the server identified by the target resource.

OPTIONSThe OPTIONS method is used to describe the communication options for the target resource.

TRACEThe TRACE method performs a message loop-back test along the path to the target resource.

PATCHThe PATCH method is used to apply partial modifications to a resource.

THESE ARE ALL LIES.

Status Codes

1xx Information

2xx Success

3xx Redirects

4xx Client Error

5xx Server Error

This also has no actual meaning.

Terminologies

The following are the most important terms related to REST APIs

Resource is an object or representation of something, which has some associated data with it and there can be set of methods to operate on it. E.g. Animals, schools and employees are resources and delete, add, update are the operations to be performed on these resources.
Collections are set of resources, e.g Companies is the collection of Company resource.
URL (Uniform Resource Locator) is a path through which a resource can be located and some actions can be performed on it.

API Endpoint

This is what a API endpoint looks like.



https://www.github.com/golang/go/search?q=http&type=Commits

This URL can be broken into these parts

protocol	subdomain	domain	path	Port	query
http/https	subdomain	base-url	resource/some-other-resource	some-port	key value pair
https	www	github.com	golang/go/search	80	?q=http&type=Commits

Protocol

How the browser or client should communicate with the server.

Subdomain

Sub Division of the main domain

Domain

Unique reference to identify web site on the internet

Port

Port on the server the application is running on. By default its 80. So most cases we don't see it

Path

Path parameters in a Rest API represents resources.



https://jsonplaceholder.typicode.com/posts/1/comments

posts/1/comments

This path is representing the 1st posts resource'c comments

Basic structure is



top-level-resource/<some-identifier>/secondary-resource/<some-identifier>/...

Query

Queries are key value pairs of information, used mostly for filtering purposes.



https://jsonplaceholder.typicode.com/posts?userId=1

Parts after the ? is the query parameters. We have only one query here. userId=1

Headers

This was not part of the URL itself but header is a part of network component sent by the client or the server. Based on who sends it. There are two kinds of header

Request Header (client -> server)
Response Header (server -> client)

Body

You can add extra information to both the request to the server and to the response from the server.

Response Type

Usually JSON or XML.

Now a days it's mostly JSON.

Rest API with GO

This is why you are here. Or I hope this is why you are here.

Language Design in the Service of Software Engineering

This above article came out in 2012. But still pretty relevant to learn the ideology behind go.

If you are writing Rest API why should you choose go?

It's compiled. So you get small binaries.
It's fast. (slower than c/c++ or rust) but faster than most other web programming languages.
It's simple to understand.
It works really well in the microservices world for reason no 1.

net/http

The standard library in go comes with the net/http package, which is an excellent starting point for building RestAPIs. And most other libraries the adds some additional feature are also interoperable with the net/http package so understanding the net/http package is crucial to using golang for RestAPIs.

net/http

We probably don't need to know everything in the net/http package. But there are a few things we should know to get started.

The Handler Interface

I am never a proposer of memorizing something but as Todd Mcleod in his course mentions over and over. We need to memorize the Handler interface.



type Handler interface {
        ServeHTTP(ResponseWriter, *Request)
}

And here it is.

It has one method and one method only.

A struct or object will be Handler if it has one method ServeHTTP which takes ResponseWriter and pointer to Request.

With all our knowledge now we are ready to do some damage.

Let's Begin

I think now we are ready to get started.

That was a lot of theory. I promised you will build you first RestAPI.

Simple Rest API

So let's jump right into it.

In a folder where you want to write your go code



go mod init api-test

Create a new file, you can name it whatever you want.

I am calling mine main.go



package main

import (
    "log"
    "net/http"
)

type server struct{}

func (s *server) ServeHTTP(w http.ResponseWriter, r *http.Request) {
    w.Header().Set("Content-Type", "application/json")
        w.WriteHeader(http.StatusOK)
    w.Write([]byte(`{"message": "hello world"}`))
}

func main() {
    s := &server{}
    http.Handle("/", s)
    log.Fatal(http.ListenAndServe(":8080", nil))
}

Lets break down this code.

At the top we have our package main all go executable need a main package.

We have our imports. log for logging some error if it happens. net/http because we are writing a rest api.

Then we have a struct called server. It has no fields. We will add a method to this server ServeHTTP and that will satisfy the Handler interface. One thing you will notice in go we don't have to explicitly say the interface we are implementing. The compiler is smart enough to figure that out. In the ServeHTTP method we set httpStatus 200 to denote its the request was a success. We se the content type to application/json so the client understands when we send back json as payload. Finally we write



{"message": "hello world"}

To the response.

Lets run our server



go run main.go

If you had installed postman before, let's test our app with postman real quick.

Get returns us our message.

Great work!

But Wait.

Lets see what other HTTP verbs our application serves.

In postman we can change the Type of request we make. Click on the dropdown and select something else. Lets say we do post.

Now if we run the request, we get back the same result.

Well its not really a bug per se. But in most cases we probably want to do different things based on the request types.

Lets see how we can do that.

We will modify our ServeHTTP method with the following.



func (s *server) ServeHTTP(w http.ResponseWriter, r *http.Request) {
    w.Header().Set("Content-Type", "application/json")
    switch r.Method {
    case "GET":
        w.WriteHeader(http.StatusOK)
        w.Write([]byte(`{"message": "get called"}`))
    case "POST":
        w.WriteHeader(http.StatusCreated)
        w.Write([]byte(`{"message": "post called"}`))
    case "PUT":
        w.WriteHeader(http.StatusAccepted)
        w.Write([]byte(`{"message": "put called"}`))
    case "DELETE":
        w.WriteHeader(http.StatusOK)
        w.Write([]byte(`{"message": "delete called"}`))
    default:
        w.WriteHeader(http.StatusNotFound)
        w.Write([]byte(`{"message": "not found"}`))
    }
}

If our server is already running lets stop it with ctrl-c

Run it again.



go run main.go

Test it with postman or curl again.

One thing you may have noticed is that we are using our server struct literally for attaching a method to.

The go team knew this was an inconvenience and gave us HandleFunc Its a method on the http package that allows us to pass a function that has the same signature as the ServeHTTP and can serve a route.

We can clean up our code a little bit with this



package main

import (
    "log"
    "net/http"
)

func home(w http.ResponseWriter, r *http.Request) {
    w.Header().Set("Content-Type", "application/json")
    switch r.Method {
    case "GET":
        w.WriteHeader(http.StatusOK)
        w.Write([]byte(`{"message": "get called"}`))
    case "POST":
        w.WriteHeader(http.StatusCreated)
        w.Write([]byte(`{"message": "post called"}`))
    case "PUT":
        w.WriteHeader(http.StatusAccepted)
        w.Write([]byte(`{"message": "put called"}`))
    case "DELETE":
        w.WriteHeader(http.StatusOK)
        w.Write([]byte(`{"message": "delete called"}`))
    default:
        w.WriteHeader(http.StatusNotFound)
        w.Write([]byte(`{"message": "not found"}`))
    }
}

func main() {
    http.HandleFunc("/", home)
    log.Fatal(http.ListenAndServe(":8080", nil))
}

Functionality should be exactly the same.

Gorilla Mux

net/http built in methods are great. We can write a server with no external libraries. But net/http has its limitations. There is no direct way to handle path parameters. Just like request methods we have to handle path and query parameters manually.

Gorilla Mux is a very popular library that works really well to net/http package and helps us do a few things that makes api building a breeze.

Using Gorilla Mux

To install a module we can use go get

Go get uses git under the hood.

In the same folder you have your go.mod and main.go file run



go get github.com/gorilla/mux

We change our code to this



package main

import (
    "log"
    "net/http"

    "github.com/gorilla/mux"
)

func home(w http.ResponseWriter, r *http.Request) {
    w.Header().Set("Content-Type", "application/json")
    switch r.Method {
    case "GET":
        w.WriteHeader(http.StatusOK)
        w.Write([]byte(`{"message": "get called"}`))
    case "POST":
        w.WriteHeader(http.StatusCreated)
        w.Write([]byte(`{"message": "post called"}`))
    case "PUT":
        w.WriteHeader(http.StatusAccepted)
        w.Write([]byte(`{"message": "put called"}`))
    case "DELETE":
        w.WriteHeader(http.StatusOK)
        w.Write([]byte(`{"message": "delete called"}`))
    default:
        w.WriteHeader(http.StatusNotFound)
        w.Write([]byte(`{"message": "not found"}`))
    }
}

func main() {
    r := mux.NewRouter()
    r.HandleFunc("/", home)
    log.Fatal(http.ListenAndServe(":8080", r))
}

Looks like nothing really changed except for a new import and line 32.

HandleFunc HTTP Methods

But now we can do a little bit more with our HandleFunc Like making each function handle a specific HTTP Method.

It looks something like this



package main

import (
    "log"
    "net/http"

    "github.com/gorilla/mux"
)

func get(w http.ResponseWriter, r *http.Request) {
    w.Header().Set("Content-Type", "application/json")
    w.WriteHeader(http.StatusOK)
    w.Write([]byte(`{"message": "get called"}`))
}

func post(w http.ResponseWriter, r *http.Request) {
    w.Header().Set("Content-Type", "application/json")
    w.WriteHeader(http.StatusCreated)
    w.Write([]byte(`{"message": "post called"}`))
}

func put(w http.ResponseWriter, r *http.Request) {
    w.Header().Set("Content-Type", "application/json")
    w.WriteHeader(http.StatusAccepted)
    w.Write([]byte(`{"message": "put called"}`))
}

func delete(w http.ResponseWriter, r *http.Request) {
    w.Header().Set("Content-Type", "application/json")
    w.WriteHeader(http.StatusOK)
    w.Write([]byte(`{"message": "delete called"}`))
}

func notFound(w http.ResponseWriter, r *http.Request) {
    w.Header().Set("Content-Type", "application/json")
    w.WriteHeader(http.StatusNotFound)
    w.Write([]byte(`{"message": "not found"}`))
}

func main() {
    r := mux.NewRouter()
    r.HandleFunc("/", get).Methods(http.MethodGet)
    r.HandleFunc("/", post).Methods(http.MethodPost)
    r.HandleFunc("/", put).Methods(http.MethodPut)
    r.HandleFunc("/", delete).Methods(http.MethodDelete)
    r.HandleFunc("/", notFound)
    log.Fatal(http.ListenAndServe(":8080", r))
}

If you run this it should still do the exact same thing.

At this point you might be wondering how is doing the same thing with more lines of code a good thing?

But think of it this way. Our code became much cleaner and much more readable.

Clear is Better than Clever

Rob Pike

Subrouter



func main() {
    r := mux.NewRouter()
    api := r.PathPrefix("/api/v1").Subrouter()
    api.HandleFunc("", get).Methods(http.MethodGet)
    api.HandleFunc("", post).Methods(http.MethodPost)
    api.HandleFunc("", put).Methods(http.MethodPut)
    api.HandleFunc("", delete).Methods(http.MethodDelete)
    api.HandleFunc("", notFound)
    log.Fatal(http.ListenAndServe(":8080", r))
}

Everything else stays the same except we are creating something called a sub-router. Sub-router are really useful when we want to support multiple resources. Helps us group the content as well as save us from retyping the same path prefix.

We move our api to api/v1 . This way we can create v2 of our api if need be..

Path and Query Parameter



package main

import (
    "fmt"
    "log"
    "net/http"
    "strconv"

    "github.com/gorilla/mux"
)

func get(w http.ResponseWriter, r *http.Request) {
    w.Header().Set("Content-Type", "application/json")
    w.WriteHeader(http.StatusOK)
    w.Write([]byte(`{"message": "get called"}`))
}

func post(w http.ResponseWriter, r *http.Request) {
    w.Header().Set("Content-Type", "application/json")
    w.WriteHeader(http.StatusCreated)
    w.Write([]byte(`{"message": "post called"}`))
}

func put(w http.ResponseWriter, r *http.Request) {
    w.Header().Set("Content-Type", "application/json")
    w.WriteHeader(http.StatusAccepted)
    w.Write([]byte(`{"message": "put called"}`))
}

func delete(w http.ResponseWriter, r *http.Request) {
    w.Header().Set("Content-Type", "application/json")
    w.WriteHeader(http.StatusOK)
    w.Write([]byte(`{"message": "delete called"}`))
}

func params(w http.ResponseWriter, r *http.Request) {
    pathParams := mux.Vars(r)
    w.Header().Set("Content-Type", "application/json")

    userID := -1
    var err error
    if val, ok := pathParams["userID"]; ok {
        userID, err = strconv.Atoi(val)
        if err != nil {
            w.WriteHeader(http.StatusInternalServerError)
            w.Write([]byte(`{"message": "need a number"}`))
            return
        }
    }

    commentID := -1
    if val, ok := pathParams["commentID"]; ok {
        commentID, err = strconv.Atoi(val)
        if err != nil {
            w.WriteHeader(http.StatusInternalServerError)
            w.Write([]byte(`{"message": "need a number"}`))
            return
        }
    }

    query := r.URL.Query()
    location := query.Get("location")

    w.Write([]byte(fmt.Sprintf(`{"userID": %d, "commentID": %d, "location": "%s" }`, userID, commentID, location)))
}

func main() {
    r := mux.NewRouter()

    api := r.PathPrefix("/api/v1").Subrouter()
    api.HandleFunc("", get).Methods(http.MethodGet)
    api.HandleFunc("", post).Methods(http.MethodPost)
    api.HandleFunc("", put).Methods(http.MethodPut)
    api.HandleFunc("", delete).Methods(http.MethodDelete)

    api.HandleFunc("/user/{userID}/comment/{commentID}", params).Methods(http.MethodGet)

    log.Fatal(http.ListenAndServe(":8080", r))
}

Lets look at the params functions on line 36. We handle both path param and query params.

With this you now know enough to be dangerous.

Bookdata API

In Kaggle there is a dataset for bookdata. Its a csv file with about 13000 books. We will use it to make our own bookdata api.

books.csv

You can look at the file ☝🏼 there.

Clone Repo

Let's get started.

In a separate folder



git clone https://github.com/moficodes/bookdata-api.git

Tour of the Code

There are two packages inside the code. One is called datastore, one is called loader.

Loader deals with converting the csv data into an array of bookdata objects.

Datastore deals with how we access the data. It's mainly an interface that has some methods.

Run app

From the root of the repo

run



go run .

EndPoints

The App has a few Endpoints

All api endpoints are prefixed with /api/v1

To reach any endpoint use baseurl:8080/api/v1/{endpoint}



Get Books by Author
"/books/authors/{author}" 
Optional query parameter for ratingAbove ratingBelow limit and skip

Get Books by BookName
"/books/book-name/{bookName}"
Optional query parameter for ratingAbove ratingBelow limit and skip


Get Book by ISBN
"/book/isbn/{isbn}"

Delete Book by ISBN
"/book/isbn/{isbn}"

Create New Book
"/book"

Deploy app to cloud

This step is completely optional. But if you want to run you go app in the cloud somewhere IBM Cloud has an excellent paas solution using Cloud Foundry.

If you want to follow along

Click Here to Get Started
Install IBM Cloud CLI

Once you have an IBM Cloud account and the IBM Cloud CLI installed,

From terminal



ibmcloud login



ibmcloud target --cf

Open the manifest.yaml in the root of the cloned repository.

Change the app name to something you like. After the change the file should look something like this



---
applications:
- name: <your-app-name>
  random-route: true
  memory: 256M
  env:
    GOVERSION: go1.12
    GOPACKAGENAME: bookdata-api
  buildpack: https://github.com/cloudfoundry/go-buildpack.git

Then from the root of the repo run



ibmcloud cf push

Wait a few minutes and voila! It should be running.

To find your app url



ibmcloud cf apps

You should see your app as running and also have the URL there.

From that url you can test out all the endpoints still work.

Test

You can test my running app mofi-golang-api-demo-appreciative-antelope.mybluemix.net (There is no front-end here, try a endpoint)

If you want to see all the books written by JK Rowling



https://mofi-golang-api-demo-appreciative-antelope.mybluemix.net/api/v1/books/authors/rowling

If you have any question, feel free to ping me @moficodes.

Top comments (16)

Jer • Oct 27 '19

Hey @moficodes ! I wanted to thank you for the article. It works as a great bridge between some of the more terse tutorials and examples out there and winding up with a working SPA+API running on Go.

So, thanks for the work you put in ! I greatly appreciate it.

Mofi Rahman • Oct 28 '19

I am glad you liked it. 🥰

Ryan McCormick • Mar 3 '20

Great primer for writing REST APIs in golang. I am a go newbie and stumbled on to this article. Quick note about some of the example code for typo fix or for anyone else working through the example:

The beginning of the exercise has the w.WriteHeader and w.Header().Set lines flip-flopped.
while running the example at the beginning I kept getting "text/plain" for my response content type instead of "application/json". It looks to be correct elsewhere but w.Header().Set should be called before w.WriteHeader. According to docs: "Changing the header map after a call to WriteHeader (or Write) has no effect unless the modified headers are trailers".

Mofi Rahman • Mar 24 '20

Thanks for catching that.

Fixed it.

Steve Klassen (They/Them) • Jan 3 '20

There's a typo on this line:

log.Fatal(http.ListenAndServe(":8080", r))

Should be:

log.Fatal(http.ListenAndServe(":8080", api))

Mofi Rahman • Jan 3 '20

r := mux.NewRouter()

is my mux router. api is a subrouter on r.

In this example using api instead of r would work.

But if i have other routes on r this would not work.

Steve Klassen (They/Them) • Jan 3 '20

Oops, good point. I think I hadn't used api yet and it was complaining about that.

Mofi Rahman • Jan 3 '20

one of the "best worst" feature of go. :D

chinglin • Jan 13 '20

I'd more concern about the following area

Api-doc generation ( go-swagger? )
The limit or rate control relate stuff.
How things integrate with auth token ( jwt stuff ).

dirtyfishtank • Jan 28 '20

Excellent article. Much appreciated!

Zenifer Cheruveettil • Sep 8 '20

good one, it helped!

thanks

zen

PVillela • Dec 27 '20

Thanks for the excellent tutorial.

Neelesh • Jul 24 '20

really awesome dude

ajeyprasand • Jan 14 '20

bro thanks for the article however why did u did not add any code in the stuct called server initially ??

Mofi Rahman • Mar 24 '20

type server struct{}

func (s *server) ServeHTTP(w http.ResponseWriter, r *http.Request) {
    w.Header().Set("Content-Type", "application/json")
        w.WriteHeader(http.StatusOK)
    w.Write([]byte(`{"message": "hello world"}`))
}

You are talking about this part I believe.

For http.Handle it just needs some type that implements the handler interface.

For example, this below is valid code.

type hotdog int

func (s *hotdog) ServeHTTP(w http.ResponseWriter, r *http.Request) {
    w.Header().Set("Content-Type", "application/json")
        w.WriteHeader(http.StatusOK)
    w.Write([]byte(`{"message": "hello world"}`))
}

func main() {
    var s hotdog
    http.Handle("/", &s)
    log.Fatal(http.ListenAndServe(":8080", nil))
}

Jiten Patel • Oct 17 '21

New to go lang. What an article hats off 🙇

View full discussion (16 comments)

Build Your First Rest API with GO

API

Rest API

What is REST

Guiding Principles of REST

HTTP Verbs

Status Codes

Terminologies

API Endpoint

Protocol

Subdomain

Domain

Port

Path

Query

Headers

Body

Response Type

Rest API with GO

net/http

The Handler Interface

Let's Begin

Simple Rest API

Gorilla Mux

Using Gorilla Mux

HandleFunc HTTP Methods

Clear is Better than Clever

Subrouter

Path and Query Parameter

Bookdata API

Clone Repo

Tour of the Code

Run app

EndPoints

Deploy app to cloud

Test

Read next

Building WordPress APIs: Connecting IoT Devices to Your Site

From LocalHost to Public Endpoint - Quickly Share Your Work With zrok

Day 17: Github Actions

While Loops