Luís Costa

Posted on Jun 1, 2018

Redis - An Introduction

#redis #caching #database #ruby

What is Redis?

Redis is an in-memory key-value data store that you can integrate with your application to store volatile data. It’s volatile because once you quit Redis, all the data you’ve stored is lost.

It works by storing some value in a key. You can later on request “Give me the value for key ‘some_key’”.

It supports a wide range of data types such as strings, hashes, lists, sets, bitmaps, etc…

Do I need it?

Redis is typically used as a caching mechanism. For instance, let’s say your application has an endpoint that performs a computational-heavy task and returns a large json to the user. If you repeat this work over and over for every request, you are most likely introducing a bottleneck in your client-server communication. A popular solution for this is to perform the work once and then store the result in Redis. The next time the user makes the same request, your application simply checks if the result is cached and returns it.

Of course this pattern requires that you follow a set of rules:

Each result stored should be uniquely namespaced. In the example below, the result should cached per user id (for example) and any other additional parameters that can identify two requests as being the same.

You should enforce a TTL (time to live) in each result that you store in Redis.
A TTL of 15 minutes means that after 15 minutes, the result is wiped from Redis. In the example below, this means that 15 minutes after the initial request, a new user request would trigger the heavy computation on the server again. This depends on your application and user needs, so take some time to think about this.

Don’t overuse Redis. This is important. Redis is useful for some tasks, not all tasks. Use a proper database when appropriate. For instance, you should not store user credentials and preferences in Redis.

If you’re working with huge datasets, be very careful when doing queries on Redis. If your application performs several reads and/or writes to Redis per minute, one wrong query to Redis (for instance, listing all the keys currently existent) may introduce downtime, possibly breaking your entire application. This, of course, depends on the overall infrastructure of your application.

For the remainder of this blog post, I’m going to talk about the major data types that Redis supports and the basic commands for each. I will be using the Ruby implementation of Redis (which you can find here) and show you, as an example, how to set up Redis in your Rails application. There are ports of Redis for almost any programming language, and the public API should be pretty similar among them, so setting it up for your preferred development environment should be a simple matter of reading the README file in the official repository.

Setting up Redis in your Rails app

Simply append the redis to your local Gemfile

gem 'redis'

And then create a new initialiser class in your config/initializer folder called redis.rb with:

Redis.current = Redis.new(host: 'localhost', port: 6379)

You can also pass additional options depending on how you have configured redis such as a password, a path to a specific unix socket, a driver and a specific database name. Check the README file over at the official ruby gem for more information on how to configure redis.

General commands

These are commands that are not specific to a data type, rather to any redis key itself

Deleting a key
Redis.current.del('some key')
Checking if a key exists
Redis.current.exists('some key')
Get all keys in the database
Redis.current.keys

!! WARNING !! : Please avoid at all costs running the command above in your production redis server, because this operation has a O(N) complexity, where N is number of keys you currently have in Redis.

Setting a TTL in a key Redis.current.expire('some key', 5000)

The second argument is the expire time, in milliseconds. This means that the key 'some key+ will expire in 5 seconds.

Renaming a key Redis.current.rename('some key', 'new key')

If you don’t know or don’t remember which data structure is currently held by a key, you can ask redis to inform you:

Redis.current.type('key') => 'String', 'Set', 'Hash', ...

If you really want to perform a scan over all your keys in Redis (for instance, to check if a key with a particular pattern in its name exist), the correct way to do it is with the scan command.

For instance, let’s say our current Redis database holds 1 billion keys and you want to know if there’s any key with name pattern some:cache:key. You do not, I repeat, you do NOT execute the command Redis.current.keys and iterate over the returned collection to find it. Not only will you probably kill your application memory, but you will also probably kill redis.

You’ll use scan for this. This command allows you to iteratively look for keys in the database using a cursor. You provide it with an initial cursor, redis will scan the next N keys (configurable), and then return you the next cursor (and the keys scanned) that you should use in the next call.

For a correct use of this command, you should stop scanning whenever redis returns 0 as the next cursor. This means that there’s no more keys to scan. In our example above, to find a key that contains the pattern some:cache:key we would use scan in the following way:

Redis.current.scan(0, match: '*some:cache:key*')

The asterics before and after the pattern means ‘I don’t care whatever is before or after this particular pattern, so the key random:some:cache:key:random would match.

The first call would return, for example, the iterator 1000 and whataver keys match the pattern. Given that the iterator is not 0, we can continue:

Redis.current.scan(1000, match: '*some:cache:key*')

This call would return the iterator 10, so we continue:

Redis.current.scan(10, match: '*some:cache:key*')

This call would return the iterator 0, so we know that there are no more keys that match this pattern. It goes without saying that you should keep a list of the keys that have matched, from the result of each scan command.

Strings / Integers

In this section we’l learn the basic operations around storing and retrieving string and integer values from redis.

Storing a new string value in a key
Redis.current.set('mykey', 'myvalue')
Retrieving the value of the key mykey
Redis.current.get('mykey') => 'myvalue'
Setting a new string value in a key, unless it already exists
Redis.current.setnx('mykey', 10) => false (because the key 'mykey' already exists)

Redis.current.setnx('another key', 10) => 'OK'

Hashes

A hash in Redis is a data structure that can hold several key — value pairs (just like a hash in Ruby)

Creating a new key/value pair in a hash:
Redis.current.hset('myhash', 'my hash field', 10)
Retrieving the value of the field my hash field in the hash myhash :
Redis.current.hget('myhash', 'my hash field')

There’s also a way to create a hash with multiple fields in a single command:

Redis.current.hmset('myhash', 'field 1', 'value field 1', 'field 2', 'value field 2', 'field 3', 'value field 3')

Here I’m showing examples with strings, but of course you can also store integers, and even json dumps, for instance:

Redis.current.hmset('myhash', 'field 3', '{\" key \": \" value \"}')

Likewise, you can also retrieve the values from multiple fields in a single command

Redis.current.hmget('myhash', 'field 2', 'field 3') => {'field 2' => 'value field 2', 'field 3' => 'value field 3'}

Deleting one or more fields from a hash Redis.current.hdel('myhash', 'field 2', 'field 3')

Retrieving the full hash stored in a key
Redis.current.hgetall('myhash') => {'field 1' => 'value field 1', 'field 2' => 'value field 2', 'field 3' => 'value field 3'}

Lists

Prepend a new element to a list (add to the beginning of the list):
Redis.current.lpush("mylist", 10)
Append a new element to a list (add to the end of the list):
Redis.current.rpush("mylist", JSON.dump({ some_hash_key: 2}))
Removing elements from a list
Redis.current.lrem("mylist", 1, 10)

The method lrem accepts three arguments. The first argument is the name of the key holding the list, the third argument is an element that we want to delete from the list, and the second argument is an integer that:

When 0, it will remove all elements in the list equal to the element we want to remove. For example, if we had several elements in the list with the value 10 calling Redis.current.lrem("mylist", 0, 10) would remove all elements in the list with the value 10
When positive, will remove the first element matching the element that we want to remove, starting from the head of the list to the tail
When negative, will remove the first element matching the element that we want to remove, starting from the tail of the list to the head
These last two options are useful if, for example, you’d like to remove the first or the last occurrences of a given element in a list
Getting the elements of a list
Redis.current.lrange("mylist", 0, -1)

The second and third argument of this method specifies the range of elements that we want. When using 0 and -1, it means that we want all elements starting from the beginning until the end. If we’d like only the first two elements we’d have to write Redis.current.lrange("mylist", 0, 1) , and so on.

If you don’t know the current size of your list:
Redis.current.llen("mylist")

Sets

Sets are useful if want to keep a data structure with unique elements.

Adding new elements to a set Redis.current.sadd("myset", "myelement")

Note that if the element “myelement” already exists in the set “myset”, it is ignored and thus not added

Removing elements from a set
Redis.current.srem("myset", "myelement")
Getting all current members of a set
Redis.current.smembers("myset")
Checking if an element is already in a set
Redis.current.sismember("myset", "myelement")

Final thoughts

The list of commands presented here is just a quick “cheasheet” that you can use to quickly look up the most common use cases of Redis. For a list of complete commands, check the official redis documentation.

Finally, please be aware of how you’re using redis. If redis is on a different machine than the one running your application, consider that each redis call is a network request. Also, be very careful with commands that return everything from a list, like smembers , lrange , hgetall , etc. If your application is very big, you may not only flood your application memory, but also possibly block redis for a while. Always, always, use iteration based search, such as scan .

Thanks for reading!

Top comments (5)

Rafal Pienkowski • Jun 3 '18

Great blog post. An easy and gentle introduction to the Redis for everyone who would like to give it a try. I like that you've mentioned that Redis storage isn't a silver bullet for increasing performance of our applications and has some drawbacks.

Thanks.