Cache the data using Redis in Elixir/Phoenix

#elixir #redis #github

Recently I stumble upon a problem where I had to cache the data which I was getting from the database. But for this, you need to have a little bit of understanding of genservers and supervisors.

Let's start with Redis and why we need it?

The distributed cache is a widely-adopted design pattern for building robust and scalable applications. By serving data from the cache, and instead of fetching it from slower devices or re-computing it, the application’s overall performance is improved. The cache also relieves
some of the load on external data sources as well as improves the availability of the application during outages.

Redis is the most popular distributed caching engine today. It is a production-proven and provides a host of capabilities that make it the ideal distributed caching layer for applications. Redis Cloud and RLEC provide additional capabilities that extend the abilities of open source
Redis to make it even more suitable for caching purposes.

Redix

Redix provides the main API to interface with Redis. We will use this library to interact with redis server. But we'll get to this later.

def start_link(url) do
   GenServer.start_link(__MODULE__, {url})
end

start_link here starts a process that connects to Redis. Each Elixir process started with this function maps to a client TCP connection to the specified Redis server.

def init({url}) do
  Logger.info("connect to url #{url}");
  case Redix.start_link(url) do
  {:ok, conn} -> {:ok, conn}
  {:error, err} -> {:error, err}
  end
end

So here start_link function will invoke the init function. Now just run this

iex -S mix phx.server
Database.start_link("redis://127.0.0.1:6379")

iex(2)> Database.start_link("redis://127.0.0.1:6379")
[info] connect to url redis://127.0.0.1:6379
{:ok, #PID<0.448.0>}
iex(3)> {:ok, pid} =  Database.start_link("redis://127.0.0.1:6379") 
[info] connect to url redis://127.0.0.1:6379
{:ok, #PID<0.452.0>}

Now here you're doing a pattern matching. On the left side, it is a tuple that has the first argument as atom :ok and second has a variable. If you check the variable it would return a process id. Now you're connected to Redis server and you have a process running with that pid.

Now to set a key-value command in Redix. You have to use some binaries if you're not using one of those binaries it will suggest to you use those binaries. Now you're confused here about the binaries let me tell you the Elixir Compiler expects certain parameters to pass in case if you miss something it would tell you to use those binaries.

Okay let's get back to code so it would suggest you to something like this

 Redix.command!(conn, ["SET", "mykey", "foo"])
 "OK"

Now I wanted to set a key and value. But I wanted the value to be the data that I was fetching from the database. But here the problem is the data which I was getting database was a map. So I figured if I could somehow set a normal key and value then I will resolve the other problem.

So here what I did

 def set(conn, key, value) do
   GenServer.call(conn, {:set, key, value})
 end

 def handle_call({:set, key, value}, _from, state ) do
   reply = Redix.command(state, ["SET", key, value])
   {:reply, {:ok, reply}, state}
 end

What I did here is I have a set function that is expecting a conn, key, and value which you want to set. The set function will invoke a handle_call function which is where the main logic happens and you will able to set a key and value. That's the beauty of genserver because it let you maintain the state. If you run this it would give you something like this

iex(6)> Database.set(pid, "onetwo", "three")
# {:ok, {:ok, "OK"}}

But now what I want is to fetch a database from the database for which I want to create a function which will take a pid as first argument and id which I wanted to fetch.

def userone(pid, id) do
  GenServer.call(pid, {:userone, id})
end

So I created a function that expects that argument and will invoke the handle_call function.

def handle_call({:userone, id}, _from, state) do
  user = Accounts.get_user!(id)
  {:reply, user, state}
end

Now this will get the single user from the database. Just call this function and it would return a map. But our set function expects an only string, not a map. So we need to figure out the way to convert a map to a string.

There are many libraries that can help you to convert the map to string but I used the Poison module which you have to add under your dependencies.

def handle_call({:set, key, value}, _from, state) when is_map(value) do
 value = Poison.encode!(value)
 reply = Redix.command(state, ["SET", key, value])
 {:reply, {:ok, reply}, state}
end

So I did exactly that so I used an Elixir guard function and if the value is map go-ahead and converts that map to a string. Now if you see it will convert that map to string and will save it to Redis.

Now if you want to see how can you get the data which you've just set. Don't worry there is a function for that. But first, let's see how it works in redix.

 Redix.command(conn, ["GET", "mykey"])

Using this command I created another function that will just get the key which you've just set.

 # get the key function
 def get(pid, key) do
   GenServer.call(pid, {:get, key})
 end

 # handle call for get function
 def handle_call({:get, key}, _from, state) do
   reply = Redix.command(state, ["GET", key])
   {:reply, {:ok, reply}, state}
 end

This is how it will look like in the end

 # iex(6)> Database.set(pid, "onetwo", "three")
 # {:ok, {:ok, "OK"}}
 # iex(7)> Database.get(pid, "onetwo")
 # {:ok, {:ok, "three"}}
 # iex(8)>

Some useful information about redix connection:-

Redix tries to be as resilient as possible: it tries to recover automatically from most network errors.

If there's a network error when sending data to Redis or if the connection to Redis drops, Redix tries to reconnect. The first reconnection attempt will happen after a fixed time interval; if this attempt fails, reconnections are attempted until successful, and the time interval between reconnections is increased exponentially.

Here how it will look like in the end.

defmodule Gorm.Database do
  use GenServer
  require Logger
  alias Gorm.Accounts

  # "redis://localhost:6379/3"

  #connect the redis server
  def start_link(url) do
    GenServer.start_link(__MODULE__, {url})
  end

 def init({url}) do
    Logger.info("connect to url #{url}");
    case Redix.start_link(url) do
    {:ok, conn} -> {:ok, conn}
    {:error, err} -> {:error, err}
    end
 end

 # checking the connection if it's connected to redis or not
def check(pid) do
   GenServer.call(pid, :check)
end

 # handle call for check function
def handle_call(:check, _from, state) do
  checking = Redix.command!(state, ["PING"])
  {:reply, checking, state}
end

# get the key function
def get(pid, key) do
  GenServer.call(pid, {:get, key})
end

# handle call for get function
def handle_call({:get, key}, _from, state) do
  reply = Redix.command(state, ["GET", key])
  {:reply, {:ok, reply}, state}
end

# set the key function
def set(conn, key, value) do
  GenServer.call(conn, {:set, key, value})
end

#handle call for set function
def handle_call({:set, key, value}, _from, state ) do
  state = Exq.enqueue(Exq, "q1", SetWorker, [key, value])
  {:reply, state, state}
end

def handle_call({:set, key, value}, _from, state) when is_map(value) do
  value = Poison.encode!(value)
  reply = Redix.command(state, ["SET", key, value])
  {:reply, {:ok, reply}, state}
end

 # Get the single user from the database
 def userone(pid, id) do
   GenServer.call(pid, {:userone, id})
 end

 # Get the list of user from the database
 def list(pid) do
   GenServer.call(pid, :list)
 end

 # handle call for the list function
 def handle_call(:list, _from, state) do
   my_models = Accounts.list_users()

  {:reply, my_models, state}
 end

 # handle call for the single user function
  def handle_call({:userone, id}, _from, state) do
    user = Accounts.get_user!(id)
    {:reply, user, state}
  end
end

If you want to see a full code and how I've implemented it please go to my github.

github.com/siddhant3030/Micro/gorm