DEV Community πŸ‘©β€πŸ’»πŸ‘¨β€πŸ’»

Cover image for Caching in Golang using Redis
Francisco Mendes
Francisco Mendes

Posted on

Caching in Golang using Redis

I believe that caching data in our application is not the ideal solution for everyone. In this article I taught how we could do in-memory caching in our application, in case you're interested.

However sometimes we like to decouple the responsibilities in our application and cache is one of them. So I decided to create a simple example of how to use Redis to cache our application data.

The idea of today's application is to make a request to the JSONPlaceholder API and we will get a user according to the id that is provided in the parameters.

The framework I'm going to use is Fiber, if you're familiar with Express.js you'll feel right at home, but the reason for using it is that it's very minimalistic and extremely intuitive.

The library I will be using in this article as a Redis client is go-redis/redis as it is quite simple to set up and use.

Let's code

First let's install the following packages:

go get github.com/gofiber/fiber/v2
go get github.com/go-redis/redis/v8
Enter fullscreen mode Exit fullscreen mode

Then let's create a simple API:

package main

import "github.com/gofiber/fiber/v2"

func main() {
    app := fiber.New()

    app.Get("/", func(c *fiber.Ctx) error {
        return c.SendString("It is working πŸ‘Š")
    })

    app.Listen(":3000")
}
Enter fullscreen mode Exit fullscreen mode

To run the API use the following command:

go run .
Enter fullscreen mode Exit fullscreen mode

If you are testing our Api, you will receive the It is working πŸ‘Š message in the body of the response.

Now if we are going to get a single user, we will have the following json in the body of the response:

{
  "id": 1,
  "name": "Leanne Graham",
  "username": "Bret",
  "email": "Sincere@april.biz",
  "address": {
    "street": "Kulas Light",
    "suite": "Apt. 556",
    "city": "Gwenborough",
    "zipcode": "92998-3874",
    "geo": {
      "lat": "-37.3159",
      "lng": "81.1496"
    }
  },
  "phone": "1-770-736-8031 x56442",
  "website": "hildegard.org",
  "company": {
    "name": "Romaguera-Crona",
    "catchPhrase": "Multi-layered client-server neural-net",
    "bs": "harness real-time e-markets"
  }
}
Enter fullscreen mode Exit fullscreen mode

Now we have to create our struct which we'll call User, but first let's create a file called utils.go. This is because as you may have noticed, we are going to have a lot of data. The struct will contain the following fields:

// @utils.go
package main

// This is the user structure according to the JSONPlaceholder API response body.
type User struct {
    ID       int    `json:"id"`
    Name     string `json:"name"`
    Username string `json:"username"`
    Email    string `json:"email"`
    Address  `json:"address"`
    Phone    string `json:"phone"`
    Website  string `json:"website"`
    Company  `json:"company"`
}

type Address struct {
    Street  string `json:"street"`
    Suite   string `json:"suite"`
    City    string `json:"city"`
    Zipcode string `json:"zipcode"`
    Geo     `json:"geo"`
}

type Geo struct {
    Lat string `json:"lat"`
    Lng string `json:"lng"`
}

type Company struct {
    Name        string `json:"name"`
    CatchPhrase string `json:"catchPhrase"`
    Bs          string `json:"bs"`
}
Enter fullscreen mode Exit fullscreen mode

This way we can make some changes to our endpoint, first we will add the id parameter. Then we will get the value of it using the c.Params() function.

app.Get("/:id", func(c *fiber.Ctx) error {
    id := c.Params("id")
    // ...
})
Enter fullscreen mode Exit fullscreen mode

Now we will make the http request to fetch only a user according to the id that is passed in the parameters. And we will return that same user.

app.Get("/:id", func(c *fiber.Ctx) error {
    id := c.Params("id")
    res, err := http.Get("https://jsonplaceholder.typicode.com/users/" + id)
    if err != nil {
        return err
    }

    defer res.Body.Close()
    body, err := ioutil.ReadAll(res.Body)
    if err != nil {
        return err
    }

    user := User{}
    parseErr := json.Unmarshal(body, &user)
    if parseErr != nil {
        return parseErr
    }

    return c.JSON(fiber.Map{"Data": user})
})
Enter fullscreen mode Exit fullscreen mode

Now if you test our Api, not forgetting to indicate the id parameter, we will get the data from the respective user.

As you may have noticed, we are constantly receiving data directly from the JSONPlaceholder API.

But as soon as we receive data from a certain user, we want to persist that same data in memory for a certain period of time. In order to consume the data from the cache.

But first of all, let's go back to the utils.go file and create a function called toJson(), which will take a buffer as an argument and will return the user's json.

// @utils.go
package main

import "encoding/json"

//...

// Converts from []byte to a json object according to the User struct.
func toJson(val []byte) User {
    user := User{}
    err := json.Unmarshal(val, &user)
    if err != nil {
        panic(err)
    }
    return user
}
Enter fullscreen mode Exit fullscreen mode

Now we can import the go-redis/redis library and let's create our Redis client. In this article I will only pass the address in the client's Options.

Just like we will import the package context that will be needed to work with our client.

package main

import (
    "context"
    "encoding/json"
    "io/ioutil"
    "net/http"

    "github.com/go-redis/redis/v8"
    "github.com/gofiber/fiber/v2"
)

var cache = redis.NewClient(&redis.Options{
    Addr: "localhost:6379",
})

var ctx = context.Background()

// ...
Enter fullscreen mode Exit fullscreen mode

Now on our endpoint we will have to cache the buffer we got from the response body. For that we will use the cache.Set() function which needs four arguments, the first is the context.

The second argument is the key, which in this case will be the id, while the third element will be the body. Finally we have to specify that the data will persist for ten seconds in the cache.

app.Get("/:id", func(c *fiber.Ctx) error {
    // ...

    cacheErr := cache.Set(ctx, id, body, 10*time.Second).Err()
    if cacheErr != nil {
        return cacheErr
    }

    return c.JSON(fiber.Map{"Data": user})
})
Enter fullscreen mode Exit fullscreen mode

Still on the endpoint, we still need to remove the logic we have with json.Unmarshal() and replace it with the toJson() function we created.

For that we will pass the body of the answer as the only argument of the function and we will return the function data in the answer of our Api.

app.Get("/:id", func(c *fiber.Ctx) error {
    // ...

    cacheErr := cache.Set(ctx, id, body, 10*time.Second).Err()
    if cacheErr != nil {
        return cacheErr
    }

    data := toJson(body)
    return c.JSON(fiber.Map{"Data": data})
})
Enter fullscreen mode Exit fullscreen mode

Now we will have all cached, but we haven't finished this yet because we still need to create a middleware.

What this middleware will do is check if the key already exists in the cache, if it does we will return the data we have stored in the cache.

But if the key doesn't exist in the cache, we will execute the next method of our route, which in this case will be the execution of the http request.

func verifyCache(c *fiber.Ctx) error {
    // ...
}
Enter fullscreen mode Exit fullscreen mode

First we have to get the value of the id parameter. Then we will use the cache.Get() function which takes two arguments.

The first argument is the context and the second argument is the key which in this case is the id. And we're still going to want the return value to be a buffer.

func verifyCache(c *fiber.Ctx) error {
    id := c.Params("id")
    val, err := cache.Get(ctx, id).Bytes()
    // ...
}
Enter fullscreen mode Exit fullscreen mode

If the key does not exist we will proceed to the next method to perform the http request, using the c.Next() function.

func verifyCache(c *fiber.Ctx) error {
    id := c.Params("id")
    val, err := cache.Get(ctx, id).Bytes()
    if err != nil {
        return c.Next()
    }
    // ...
}
Enter fullscreen mode Exit fullscreen mode

Otherwise we will use the toJson() function again to convert the buffer to json and we will return it.

func verifyCache(c *fiber.Ctx) error {
    id := c.Params("id")
    val, err := cache.Get(ctx, id).Bytes()
    if err != nil {
        return c.Next()
    }

    data := toJson(val)
    return c.JSON(fiber.Map{"Cached": data})
}
Enter fullscreen mode Exit fullscreen mode

Last but not least we need to add middleware to our route, like this:

app.Get("/:id", verifyCache, func(c *fiber.Ctx) error {
    // ...
})
Enter fullscreen mode Exit fullscreen mode

The final code should look like the following:

package main

import (
    "context"
    "io/ioutil"
    "net/http"
    "time"

    "github.com/go-redis/redis/v8"
    "github.com/gofiber/fiber/v2"
)

var cache = redis.NewClient(&redis.Options{
    Addr: "localhost:6379",
})

var ctx = context.Background()

func verifyCache(c *fiber.Ctx) error {
    id := c.Params("id")
    val, err := cache.Get(ctx, id).Bytes()
    if err != nil {
        return c.Next()
    }

    data := toJson(val)
    return c.JSON(fiber.Map{"Cached": data})
}

func main() {
    app := fiber.New()

    app.Get("/:id", verifyCache, func(c *fiber.Ctx) error {
        id := c.Params("id")
        res, err := http.Get("https://jsonplaceholder.typicode.com/users/" + id)
        if err != nil {
            return err
        }

        defer res.Body.Close()
        body, err := ioutil.ReadAll(res.Body)
        if err != nil {
            return err
        }

        cacheErr := cache.Set(ctx, id, body, 10*time.Second).Err()
        if cacheErr != nil {
            return cacheErr
        }

        data := toJson(body)
        return c.JSON(fiber.Map{"Data": data})
    })

    app.Listen(":3000")
}
Enter fullscreen mode Exit fullscreen mode

Here is the code for the utils.go file:

package main

import "encoding/json"

// This is the user structure according to the JSONPlaceholder API response body.
type User struct {
    ID       int    `json:"id"`
    Name     string `json:"name"`
    Username string `json:"username"`
    Email    string `json:"email"`
    Address  `json:"address"`
    Phone    string `json:"phone"`
    Website  string `json:"website"`
    Company  `json:"company"`
}

type Address struct {
    Street  string `json:"street"`
    Suite   string `json:"suite"`
    City    string `json:"city"`
    Zipcode string `json:"zipcode"`
    Geo     `json:"geo"`
}

type Geo struct {
    Lat string `json:"lat"`
    Lng string `json:"lng"`
}

type Company struct {
    Name        string `json:"name"`
    CatchPhrase string `json:"catchPhrase"`
    Bs          string `json:"bs"`
}

// Converts from []byte to a json object according to the User struct.
func toJson(val []byte) User {
    user := User{}
    err := json.Unmarshal(val, &user)
    if err != nil {
        panic(err)
    }
    return user
}
Enter fullscreen mode Exit fullscreen mode

Now, if we are going to test the performance of our app, we might notice a big difference in its performance. Without the use of the cache each request took an average of 456ms, after the implementation of the cache each request took an average of 7ms.

Conclusion

As always, I hope you found it interesting. If you noticed any errors in this article, please mention them in the comments. πŸ™Œ

Hope you have a great day! πŸ‘‹

Top comments (1)

Collapse
 
dreamfly2012 profile image
menghuiguli

thanks very much, it help me a lot~

🌚 Life is too short to browse without dark mode