How to build a CRUD REST API with Go, Gin and Fauna

Faruq — Mon, 07 Dec 2020 13:50:26 +0000

APIs act as software intermediaries allowing different applications to communicate with one another. They are popularly used for facilitating communication between the client side of an application and its server side. In this article, you will learn how to build a REST API for a reading list. With this API, a client will be able to add reading items to a reading list, update items, delete items and of course get items from the list and it will be built using the Go programming language, Gin and Fauna. Before we start looking into some code, let's take a look at the components of our tech stack.

Go is an open source programming language built by Google for handling some of their internal work and over the years, it has grown to become a popular server-side language.

Gin is a Go web framework that make it easier to write web apps in Go and greatly reduces the verbosity of your code. Though a large percentage of the Go community would prefer to write Go web apps without a framework, I have found Gin to be very helpful.

Fauna, which is the database that our little project will be using, is a powerful relational NoSQL database that can also serve as a data API when building serverless applications. Fauna supports the use of GraphQL for manipulating and querying the data in your database and its own native query language, FQL(Fauna Query Language).

FQL is a powerful querying language which allows you to perform conditional querying using if-like functions, run transactions, perform a union(returning a combination) of data returned from different indexes and so on. You can learn more about FQL here.

We will make use of FQL to interact with our Fauna database throughout our project.

Prerequisites

A basic knowledge of building backend applications with Go is required to follow along with this article.
A basic knowledge of Fauna and FQL. You can learn the basics here.

Installing necessary dependencies

First of all, we need to create a new project. You can create one right away in your GOPATH or use Go Modules to create the project outside your GOPATH. Once that is done, you need to install the following dependencies:

Gin
Fauna Go driver
godotenv (for storing secrets in a .env file)
Go playground's validator package for validating the fields of our models before storing them in the database

go get -u github.com/gin-gonic/gin
go get github.com/fauna/faundb-go/v3/faunadb 
go get github.com/joho/godotenv
go get github.com/go-playground/validator/v102

Now that you have your dependencies installed, we can take a look at the project folder structure and architecture

Project structure and architecture

After installing your dependencies, you should create the following folders and files:

As you can see from the image above, we have 4 packages:

Controllers - this is where we write request handlers to handle the HTTP requests to our API
Customerrors - this is where we write our custom error for representing app errors
Database - here, we create a FaunaDB interface to wrap the basic queries we will be creating in simple methods
Models - this is where we define the structs for the entities(for example, a reading item) in our app

As I said earlier, we will create a Fauna interface whose methods are a wrapper around basic queries that I believe are necessary for any CRUD app interacting with Fauna. We will also create a faunaDB struct that will implement this interface. This faunaDB struct will depend on a FaunaClient(which is available in FaunaDB Go driver) and a database secret(which will be discussed soon).

For the controllers package, we will have a controller struct whose methods will act as request handlers. The controller struct will depend on a Fauna interface and as such, will have that as a struct field. We will then pass the controller struct's methods to Gin for handling the requests.

All of this will be clearer and make more sense as you progress in this article. A Github repository link will also be provided at the end of this article to make the code easier to understand.

Setting up Fauna

Before we actually begin writing code, we need to set up our Fauna database and get a key secret which will allow us to access our database and perform operations on it from our code.

Create an account on Fauna if you do not have one and create a database(you can learn how to do that here). Once you have created a database, you should be redirected to your database’s dashboard which should look like so:

Now, we want to create an access key and obtain the corresponding secret which we can use the access our database from our code. To do that, open the Security tab on the left and click the "New Key" button. Select the database you just created, change the role to Server (because we will be using this access key from the server) and type in a name for the key if you prefer.

Once you have done that, click Save and copy the key secret displayed. You can only view this secret once so you cannot copy it later. Also, because our access key has the server role, this means anybody with the key has unrestricted access to our database and can cause serious damage if they happen to be a bad actor so we need to make sure we store our secret in a safe and secure place(environment variables for example).

For the sake of this tutorial, we will use a .env file (which you should have created by now) to store our secret. You should store it like so:

FAUNA_DB_SERVER_SECRET=your_secret

Writing application code

With that done, we can now start writing actual code.

Open the database/database.go file and create two constant variables called ReadingItemCollection and ReadingItemByTypeIndex.

const (
   ReadingItemCollection = "reading_items"
   ReadingItemByTypeIndex = "reading_items_by_type"
)

The ReadingItemCollection variable refers to the name of the collection where all the reading item documents will be stored and the ReadingItemByTypeIndex variable refers to the name of the index that we will use to filter the reading items by their type.

Now, as I said earlier, we are going to create a Fauna interface which will later be used by our controller and a faunaDB struct to implement the interface:

package database

import (
   "fmt"
   f "github.com/fauna/faunadb-go/v3/faunadb"
   "github.com/gozaddy/crud-api-fauna/models"
   "strconv"
)

...

type faunaDB struct {
   client *f.FaunaClient
   secret string
}

type FaunaDB interface {
   Init() error
   NewID() (string, error)
   FaunaClient() *f.FaunaClient
   GetDocument(collection string, documentID string) (f.Value, error)
   AddDocument(collection string, object models.Model) (f.Value, error)
   DeleteDocument(collection string, documentID string) error
   UpdateDocument(collection string, documentID string, update interface{}) error
}

The Init method initializes our Fauna interface by creating a new Fauna client using the secret provided and assigning that client to the client field of the faunaDB struct. In the init method, we also check if the reading_items collection exists and then create the collection and the reading_items_by_type index if it does not. To write a query, we use the Query method of the FaunaDB client.

Here's the code implementation for the Init method:

 func (fdb *faunaDB) Init() error{
   fdb.client = f.NewFaunaClient(fdb.secret)
   res, err := fdb.client.Query(
      f.If(
         f.Exists(f.Collection(ReadingItemCollection)),
         "Exists!",
         f.Do(
            f.CreateCollection(
               f.Obj{
                  "name": "reading_items",
               },
            ),
            f.CreateIndex(
               f.Obj{
                  "name": ReadingItemByTypeIndex,
                  "source": f.Collection(ReadingItemCollection),
                  "terms": f.Arr{f.Obj{"field": f.Arr{"data", "type"}}},
               },
            ),
         ),
      ),
   )
   if err != nil{
      return err
   }
   fmt.Println(res)
   return nil
}

As you can see, in the first line, we created a new FaunaDB client and assigned it to the client field of faunaDB. Then, in the second line we ran an FQL query that uses the If FQL function to check if the reading_items collection exists and returns "Exists!" if it does or run a transaction (with the Do function) consisting of two functions(one to create the collection and another to create the filter by type index) if it does not.

To create the index, we used the CreateIndex function. As you may already know, the terms property of an index allows us to filter results from a collection based on the value of a particular field of its documents(the type property of the data property of the document in this case).

By the way, the Obj type is simply a wrapper around map[string]interface{} and the Arr type is simply a wrapper around []interface{}.

After running the query, we then check for errors and handle them appropriately.

Generating new document IDs

The next method is the NewID method which we will be using to generate new IDs for every new document that we create. This method will make use of the NewId FQL function.

 func (fdb *faunaDB) NewID() (string, error) {
   res, err := fdb.client.Query(f.NewId())
   if err != nil{
      return "", err
   }
   var id string
   if err = res.Get(&id); err != nil{
      return "", err
   }
   return id, nil

After running the query and getting a response, we used the Get method of the response to decode the response into a native Go string which we returned from the function.

The next method is the FaunaClient method which is a very simple method that returns the FaunaDB client so that we can perform other queries that are not among the FaunaDB interface's methods.

func (fdb *faunaDB) FaunaClient() *f.FaunaClient {
   return fdb.client
}

Getting a Document

In FQL, we make use of the Get function to retrieve a particular document from a FaunaDB collection. The ref of a document contains both the name of the collection the document belongs to and the document’s unique ID so to get a document, we need its collection name and document ID. Our GetDocument method will therefore take those two values as arguments.

func (fdb *faunaDB) GetDocument(collection string, documentID string) (f.Value, error) {
   return fdb.client.Query(f.Get(f.RefCollection(f.Collection(collection), documentID)))
}

Note: The RefCollection method returns a ref based on a collection name and document ID.

Adding a document to a collection

if you notice, from the Fauna interface, the AddDocument method takes a collection name and a type of Model. But what is Model?
In our app’s code, Model is an interface which all models (types that will be stored in the database) must implement.

Head over to models/models.go and add this

package models

type Model interface{
   UniqueID() string
   Validate() error
}

The Model interface ensures that each entity being saved in the database has an ID by implementing the UniqueID method which should return the ID of the entity, and has a Validate method which can be used by methods and functions in other packages to validate the entity’s fields before adding it to the database.

To add a document to a collection, we make use of the Create FQL function.

func (fdb *faunaDB) AddDocument(collection string, object models.Model) (f.Value, error) {
   if err := object.Validate(); err != nil{
      return nil, err
   }

   return fdb.client.Query(f.Create(f.RefCollection(f.Collection(collection), object.UniqueID()), f.Obj{"data": object}))
}

Deleting a document

To delete a document, we use the Delete function and pass the ref of the document as an argument.

func (fdb *faunaDB) DeleteDocument(collection string, documentID string) error {
   _, err := fdb.client.Query(f.Delete(f.RefCollection(f.Collection(collection), documentID)))
   return err
}

Updating a document

Now we have the UpdateDocument method. This is used to update a document given its collection name, document ID and an empty interface type which contains the new data we are updating the document with. This method will make use of the Update FQL function.

func (fdb *faunaDB) UpdateDocument(collection string, documentID string, update interface{}) error {
   _, err := fdb.client.Query(
      f.Update(
         f.RefCollection(f.Collection(collection), documentID),
         f.Obj{"data": update},
      ),
   )
   return err
}

Now that we have implemented the FaunaDB interface, let's write the model for a reading item. Models represent entities that will be used throughout our application. In this case, ReadingItem which represents a single reading item is our only model.

In models/models.go, import the validator package and append the following:

type ReadingItem struct{
   ID string `json:"id" fauna:"id" validate:"required"`
   Title string `json:"title" fauna:"title" validate:"required"`
   Link string `json:"link" fauna:"link" validate:"omitempty,url"`
   Type string `json:"type" fauna:"type" validate:"required"`
   Author string `json:"author" fauna:"author" validate:"required"`
}

func NewReadingItem(id, title, link, itemType, author string) ReadingItem {
   return ReadingItem{
      id,
      title,
      link,
      itemType,
      author,
   }
}

func (r ReadingItem) Validate() error {
   return validator.New().Struct(r)
}

func (r ReadingItem) UniqueID() string {
   return r.ID
}

The fauna field tag is used to encode native Go structs into FQL expressions and to decode values from FQL queries into native Go structs. They are pretty similar to how the json field tag work. The validate field tag, on the other hand, is used to add validation rules to struct fields which can later be validated like we did in the Validate method of ReadingItem. You can learn more about the validator package here.

Error Handling

Now that we have our models ready, we can move to the API layer which is where we handle requests by responding with the appropriate responses. But before we start writing the code for the API layer, we need to write that of the customerrors package. Each method of the controller struct returns an error so we need to write our custom errors to store other error information like status code and so on. Here's the code for the custom errors package:

package customerrors

import "strconv"

type AppError struct{
   StatusCode int
   ErrorText string
}

func (ae AppError) Error() string{
   return "status code: "+strconv.Itoa(ae.StatusCode)+", error message: "+ae.ErrorText
}

func NewAppError(statusCode int, errorText string) AppError{
   return AppError{statusCode, errorText}
}

API Layer

With that taken care of, we can finally move to the API layer. Here’s what our controller should look like:

package controllers

import (
   "fmt"
   f "github.com/fauna/faunadb-go/v3/faunadb"
   "github.com/gin-gonic/gin"
   "github.com/gozaddy/crud-api-fauna/customerrors"
   "github.com/gozaddy/crud-api-fauna/database"
   "github.com/gozaddy/crud-api-fauna/models"
   "github.com/mitchellh/mapstructure"
   "net/http"
)

type controller struct{
   DB database.FaunaDB
}


func NewController(db database.FaunaDB) *controller{
   return &controller{db}
}

Let's begin by writing the endpoint to add a new reading item to the reading list.
We will call the method for this, AddReadingItem.

func (ctrl *controller) AddReadingItem(c *gin.Context) error {
   var req struct{
      Title string `json:"title" form:"title" binding:"required"`
      Link string `json:"link" form:"link" binding:"omitempty,url"`
      Type string `json:"type" form:"type" binding:"required"`
      Author string `json:"author" form:"author" binding:"required"`
   }

   err := c.ShouldBind(&req)
   if err != nil{
      return customerrors.NewAppError(http.StatusBadRequest, err.Error())
   }

   //generate item id
   id, err := ctrl.DB.NewID()
   if err != nil{
      return err
   }

   //create new reading item
   item := models.NewReadingItem(id, req.Title, req.Link, req.Type, req.Author)

   //save new reading item to FaunaDB
  _, err := ctrl.DB.AddDocument(database.ReadingItemCollection, item)
   if err != nil{
      return err
   }
   c.JSON(200, gin.H{
      "message": "New reading item successfully created!",
   })

   return nil
}

The json and form field tags are used to bind the request body to our req struct depending on the content-type of the request body(JSON or x-www-form-urlencoded). You can learn more about this here. The binding field tag is used to validate the fields of the request's body. Gin uses the validator package to achieve this so it works exactly the same way we saw earlier in this article but with a different field tag.

The shouldBind method does the actual binding of the request body to our struct and returns an error when one of the request body's fields does not pass validation. In this case, we can now return an AppError with the Bad request status code and the error message. In a real app, you would have to parse the error from shouldBind returned to get a more understandable error message.

We then create a new document ID using the NewID method and a new reading item based off the request body which we add to our database with the AddDocument method.

As you can see, writing our controller method is very straightforward because of the background code we already wrote in the other packages. Here's the endpoint for getting an individual reading item.

func (ctrl *controller) GetOneReadingItem(c *gin.Context) error {
   var item models.ReadingItem
   val, err := ctrl.DB.GetDocument(database.ReadingItemCollection, c.Param("id"))
   if err != nil{
      if ferr, ok := err.(f.FaunaError); !ok{
         return err
      } else{
         if ferr.Status() == 404 {
            return customerrors.NewAppError(ferr.Status(),"The reading item with the provided ID does not exist")
         } else if ferr.Status() == 400 {
            return customerrors.NewAppError(ferr.Status(), err.Error())
         }
         return err
      }
   }

   err = val.At(f.ObjKey("data")).Get(&item)
   if err != nil{
      return err
   }

   c.JSON(200, gin.H{
      "message": "Reading item retrieved!",
      "data": item,
   })
   return nil
}

Here, we simply got the document with the ID specified in the endpoint path, checked the error returned and handled each error case by returning the appropriate error from the method.
The data returned from FaunaDB is in the form of:

We need to find a way to get the value of the data property and decode it into a Go type. To accomplish this, we made use of the At and Get methods. The At method allows to select a particular property from the document while Get does the decoding. We then return the item and a response message with the JSON which is used for replying clients with JSON-encoded data(content-type: application/json).

For the update endpoint:

func (ctrl *controller) UpdateOneReadingItem(c *gin.Context) error {
   var req struct{
      Title string `json:"title" form:"title" binding:"omitempty" fauna:"title" mapstructure:"title,omitempty"`
      Link string `json:"link" form:"link" binding:"omitempty,url" fauna:"link" mapstructure:"link,omitempty"`
      Type string `json:"type" form:"type" binding:"omitempty" fauna:"type" mapstructure:"type,omitempty"`
      Author string `json:"author" form:"author" binding:"omitempty" fauna:"author" mapstructure:"author,omitempty"`
   }

   err := c.ShouldBind(&req)
   if err != nil{
      return customerrors.NewAppError(http.StatusBadRequest, err.Error())
   }

   var update map[string]interface{}

   err = mapstructure.Decode(req, &update)
   if err != nil{
      return err
   }

   fmt.Println(update)

   err = ctrl.DB.UpdateDocument(database.ReadingItemCollection, c.Param("id"), update)
   if err != nil{
      if ferr, ok := err.(f.FaunaError); !ok{
         return err
      } else{
         if ferr.Status() == 404 {
            return customerrors.NewAppError(ferr.Status(),"The reading item with the provided ID does not exist")
         } else if ferr.Status() == 400 {
            return customerrors.NewAppError(ferr.Status(), err.Error())
         }
         return err
      }
   }

   c.JSON(200, gin.H{
      "message": "Reading item updated successfully!",
   })

   return nil
}

The only thing new in this method is the use of the mapstructure library. Mapstructure allows us to decode maps into native Go structs and vice versa. A PATCH request will be used to access this endpoint and as is common with PATCH requests, the client only needs to include the new values of the resource they are attempting to update in the request body unlike in a PUT request where the entire new resource data is included in the request body. So req contains all the fields of the resource(in this case, a reading item) and uses omitempty to skip validation for fields that are empty. The only problem is that, if we are only updating some of the fields of the resource, some fields of req will be empty and this would lead to a bug as we would be changing some of the fields of our resource to empty strings.

Mapstructure solves this by using omitempty as one of the values of the mapstructure field tag. This ignores any empty string and converts the valid fields of our struct to a map which we can now pass to our UpdateDocument method.

Let's now move on to the DeleteOneReadingItem method which handles requests for deleting reading items.

func (ctrl *controller) DeleteOneReadingItem(c *gin.Context) error {
   err := ctrl.DB.DeleteDocument(database.ReadingItemCollection, c.Param("id"))
   if err != nil{
      if ferr, ok := err.(f.FaunaError); !ok{
         return err
      } else{
         if ferr.Status() == 404 {
            return customerrors.NewAppError(ferr.Status(),"The reading item with the provided ID does not exist")
         } else if ferr.Status() == 400 {
            return customerrors.NewAppError(ferr.Status(), err.Error())
         }
         return err
      }
   }

   c.JSON(200, gin.H{
      "message": "Reading item deleted successfully",
   })

   return nil
}

Now, let us move to the final endpoint - the endpoint for getting all the reading items. With this endpoint, we can get all the reading items stored in our database or all the reading items of a particular type so this is where we will make use of the index we created.

func (ctrl *controller) GetAllReadingItems(c *gin.Context) error {
   itemType := c.Query("type")
   var match f.Expr

   if itemType == "" {
      match = f.Documents(f.Collection(database.ReadingItemCollection))
   } else{
      match = f.MatchTerm(f.Index(database.ReadingItemByTypeIndex), itemType)
   }
   var items []models.ReadingItem
   val, err := ctrl.DB.FaunaClient().Query(
      f.Map(
         f.Paginate(
            match,
         ),
         f.Lambda("docRef", f.Select(f.Arr{"data"},f.Get(f.Var("docRef")))),
      ),
   )
   if err != nil{
      return err
   }

   err = val.At(f.ObjKey("data")).Get(&items)
   if err != nil{
      return err
   }

   c.JSON(200, gin.H{
      "message": "Reading items retrieved successfully",
      "data": items,
   })

   return nil
}

In the first line, we used the Query method to get the query parameter called type. If the user did not specify a query parameter, we use the Documents function to get all the documents in the collection otherwise, we use the ReadingItemByTypeIndex to get the reading items that are of the specified type.

We later use the Lambda function to get the individual documents instead of only returning the references.

With that done, we just need to set up our server in server.go.

package main

import (
   "github.com/gin-gonic/gin"
   "github.com/gozaddy/crud-api-fauna/controllers"
   "github.com/gozaddy/crud-api-fauna/customerrors"
   "github.com/gozaddy/crud-api-fauna/database"
   "github.com/joho/godotenv"
   "log"
   "os"
)


func handle(f func(c *gin.Context) error) gin.HandlerFunc {
   return func(context *gin.Context){
      if err := f(context); err != nil{
         if ae, ok := err.(customerrors.AppError); ok{
            context.JSON(ae.StatusCode, gin.H{
               "message": ae.ErrorText,
            })
         } else {
            log.Println(err.Error())
            context.JSON(500, gin.H{
               "message": "Internal server error",
            })
         }
      }

   }
}

func init(){
   //load .env file with godotenv so we can access our FaunaDB secret
   err := godotenv.Load()
   if err != nil{
      log.Fatalln("Error loading .env file:"+ err.Error())
   }
}

func main(){
   fdb := database.NewFaunaDB(os.Getenv("FAUNA_DB_SERVER_SECRET"))
   err := fdb.Init()
   if err != nil{
      log.Fatalln(err)
   }


   controller := controllers.NewController(fdb)
   router := gin.Default()
   router.GET("/api/", func(c *gin.Context) {
      c.JSON(200, gin.H{
         "message": "Welcome!",
      })
   })
   router.GET("/api/items", handle(controller.GetAllReadingItems))
   router.POST("/api/items", handle(controller.AddReadingItem))
   router.GET("/api/items/:id", handle(controller.GetOneReadingItem))
   router.PATCH("/api/items/:id", handle(controller.UpdateOneReadingItem))
   router.DELETE("/api/items/:id", handle(controller.DeleteOneReadingItem))

   //run our server on port 4000
   _ = router.Run(":4000")
}

In this file, we just set up our server with Gin, load our .env file and write the handle function which responds to the client based on the errors received from controller's methods.

Now, we can start our server with go run . and test the different endpoints with Postman or a similar tool.

Conclusion

You should now know how to build a simple CRUD API using Go, Gin and Fauna. You can find the code for this API here. You can also learn more about FaunaDB using its amazing documentation.

DEV Community: Faruq