Getting Started Upstash Redis with Rails

Introduction

Have you ever wondered if there is an open source database that has a fast response time and key-value storage base?

The answer is yes. Introducing Redis, an open source database, in-memory, key-value data store and it is most-loved database by developers for five years running. Redis is commonly used as a primary database, cache, queue, and message broker. Redis is an open source (BSD licensed) that is used primarily as an application cache and quick response database and because the data is stored in memory rather than on a disk, Redis delivered unparalleled speed, reliability, and performance.

Redis provides data structures such as strings, hashes, lists, sets, sorted sets, bitmaps, streams, and much more. More about Redis data types. To achieve top performance, Redis can persist the data either by periodically dumping the dataset to disk or by appending each command to a disk-based log. Redis is written in ANSI C and works on most POSIX systems like Linux, and Mac OS X without external dependencies. Redis can be used in almost every programming language. You can find the Redis Clients page.

Upstash

Upstash is a cloud database provider with over 35 thousand users (per date) and with total over 48 thousand databases active. Upstash products including Redis, Kafka, and QStash and they offer a free plan which includes Max 10,000 commands daily, TLS Encryption, and Persistence storage which is perfect for the developer who wants to try their products without being charged. The best thing in my opinion is the pricing. Pay as you go, meaning that we will pay only for what we use with per-request pricing. Even if you upgrade to a paid plan, you will only pay $0.2 (regional) per 100K commands (More about Upstash Redis Pricing).

In this post, I’m going to give you a starter guide on how to use Upstash Redis, and later we will integrate it with Ruby on Rails application.

Upstash Console

Before starting to use Upstash Redis, the first thing you have to do is create an account in Upstash. Go over to https://console.upstash.com/login and create an account. Then you can log in to the Upstash Console. You will see something like this page

Start creating a database by clicking the Create database button, you will see a modal and fill up the form. For the Type, select Regional and select the Region that is closest to your application so you can access the database with low latency.

You will be redirected to the database details

There are some features you can use on this page:

• Details: In this section, you can see the details of your database like the Region, Endpoint, Password, Port, and TLS/SSL status.
• Usage: You can see the total number of requests, today’s bandwidth, daily requests, and the total cost.
• CLI: This is just exactly the Redis CLI but in the browser, try to type ping. If successful, you will get the response PONG.
• Data Browser: You can view the data you stored in the database on this page.
• Backup: You can configure the backup configuration on this page. Upstash will only keep the latest backup and overwrites the previous one. This feature can be enabled only for paid users.
• Quickstarts: If you don’t know how to start using Upstash Redis, this is one of the good places to start. You can see how to get started in some frameworks or edge functions.

Redis API Compatibility

Before start install the Redis client in the application, first we need to understand the Redis API compatibility in Upstash. Currently, Upstash supports Redis client protocol up to version 6.2. The following table shows the list of supported Redis commands:

Feature	Supported?	Supported Commands
String	✅	APPEND - DECR - DECRBY - GET - GETDEL - GETEX - GETRANGE - GETSET - INCR - INCRBY - INCRBYFLOAT - MGET - MSET - MSETNX - PSETEX - SET - SETEX - SETNX - SETRANGE - STRLEN
Bitmap	✅	BITCOUNT - BITFIELD - BITFIELD_RO - BITOP - BITPOS - GETBIT - SETBIT
Hash	✅	HDEL - HEXISTS - HGET - HGETALL - HINCRBY - HINCRBYFLOAT - HKEYS - HLEN - HMGET - HMSET - HSCAN - HSET - HSETNX - HSTRLEN - HRANDFIELD - HVALS
List	✅	BLMOVE - BLPOP - BRPOP - BRPOPLPUSH - LINDEX - LINSERT - LLEN - LMOVE - LPOP - LPOS - LPUSH - LPUSHX - LRANGE - LREM - LSET - LTRIM - RPOP - RPOPLPUSH - RPUSH - RPUSHX
Set	✅	SADD - SCARD - SDIFF - SDIFFSTORE - SINTER - SINTERSTORE - SISMEMBER - SMEMBERS - SMISMEMBER - SMOVE - SPOP - SRANDMEMBER - SREM - SSCAN - SUNION - SUNIONSTORE
SortedSet	✅	BZPOPMAX - BZPOPMIN - ZADD - ZCARD - ZCOUNT - ZDIFF - ZDIFFSTORE - ZINCRBY - ZINTER - ZINTERSTORE - ZLEXCOUNT - ZMSCORE - ZPOPMAX - ZPOPMIN - ZRANDMEMBER - ZRANGE - ZRANGESTORE - ZRANGEBYLEX - ZRANGEBYSCORE - ZRANK - ZREM - ZREMRANGEBYLEX - ZREMRANGEBYRANK - ZREMRANGEBYSCORE - ZREVRANGE - ZREVRANGEBYLEX - ZREVRANGEBYSCORE - ZREVRANK - ZSCAN - ZSCORE - ZUNION - ZUNIONSTORE
Transactions	✅	DISCARD - EXEC - MULTI - UNWATCH - WATCH
Generic	✅	COPY - DEL - EXISTS - EXPIRE - EXPIREAT - KEYS - PERSIST - PEXPIRE - PEXPIREAT - PTTL - RANDOMKEY - RENAME - RENAMENX - SCAN - TOUCH - TTL - TYPE - UNLINK
Connection	✅	AUTH - HELLO - ECHO - PING - QUIT - RESET - SELECT
Server	✅	ACL - DBSIZE - FLUSHALL - FLUSHDB - TIME
Scripting	✅	EVAL - EVALSHA - SCRIPT EXISTS - SCRIPT LOAD - SCRIPT FLUSH
Pub/Sub	✅	SUBSCRIBE - PSUBSCRIBE - UNSUBSCRIBE - PUNSUBSCRIBE - PUBLISH - PUBSUB
Cluster	❌
Geo	❌
HyperLogLog	❌
Stream	❌

Go to Upstash Redis® API Compatibility to check the latest version.

Time to Play

Creating Application

Let’s start creating a Rails application and integrating Upstash Redis. Because we are going to create a Rails application, we need to make sure Ruby is installed in your machine.

$ ruby -v
ruby 3.1.2p20 (2022-04-12 revision 4491bb740a) [arm64-darwin21]

$ rails -v
Rails 7.0.4

I’m using ruby 3.1.2 which is the latest stable version and Rails 7.0 version. Let’s start by creating the application

$ rails new try-upstash

This command will create a rails application with the default setting.

Now install the Redis Ruby in the application, open Gemfile, and add the redis gem

gem "redis", "~> 5.0"

Open your terminal and run bundle install to install the gems.

Now it’s time to create some models and creating some data.

# create product model
$ rails g model Product name description:text price:float
      invoke  active_record
      create    db/migrate/20221229082154_create_products.rb
      create    app/models/product.rb
      invoke    test_unit
      create      test/models/product_test.rb
      create      test/fixtures/products.yml

# create user model
$ rails g model User email:string:uniq name
      invoke  active_record
      create    db/migrate/20221229084021_create_users.rb
      create    app/models/user.rb
      invoke    test_unit
      create      test/models/user_test.rb
      create      test/fixtures/users.yml

Open db/seeds.rb and create some data for product and user model

# Create 10 products
(1..10).each do |i|
  Product.create(name: "Product #{i}", price: 100, description: "Description")
end

# Create 3 users
(1..3).each do |i|
  User.create(email: "email#{i}@example.com", name: "User #{i}")
end

Then run the seed by running rails db:seed in your terminal. To make sure the data is created we can use GUI Database or using rails console. For now, I will use the rails console, open your terminal, and type rails c, you will see the rails console and you can run ruby script or rails specific command in this console. Let’s try to get the total number of Product and User.

The database seed is success, now we have 10 products and 3 users.

#1 Use Case - Saving model attribute in Upstash Redis

Imagine that your application needs to store a boolean value to indicate whether the user has been completing the onboarding process or not. You could add a new column in your table named onboarded with the type boolean, and every time you want to verify the onboarding, you will access the table very often. This is fine if you don’t have a lot of data and the traffic is not high. But, your application is growing and the application starting slow. One of the solutions here is to save the onboarded value in the Redis database and because Redis stores the data in memory, rather than on a disk, you will get top speed and performance. You may have a question like this

“If the server crashes, will the Redis data be lost?”

The answer depends on the Redis cloud you are using, but in this case, if you are using Upstash Redis the data will be still there in the Durable Storage such as EBS in AWS. In Upstash, the data is reloaded to memory from block storage in case of a server crash. Upstash also offers Multi Zone Replication to provide you with high availability and better scalability and this can be enabled for all paid users.

Let’s back to the application, open the Gemfile and add kredis and dotenv

gem "kredis", "~> 1.3"
gem "dotenv-rails"

Run bundle install and then initiate the kredis with ./bin/rails kredis:install. This will create a file in config/redis/shared.yml . You can configure the redis connection in this file. By default the configuration will read the env variable REDIS_URL in the development and production environment and fallback to default connection redis://127.0.0.1:6379/0 if the env variable doesn’t exist.

development: &development
  url: <%= ENV.fetch("REDIS_URL", "redis://127.0.0.1:6379/0") %>
  timeout: 1

Now for the env file, create .env in your root project and add REDIS_URL variable. You need to grab the value from your database details.

and then change the redis:// to rediss:// in order to use TLS for the connection

REDIS_URL=<valuehere>

Now the connection is complete, back to our use case where we want to save the onboarded status for each user. We will use kredis in this case to make our life easier by connecting redis database to the application model. Open the user model app/models/user.rb and then add the kredis_boolean attribute because the type we need is a boolean.

class User < ApplicationRecord
  kredis_boolean :onboarded
end

If you need other than boolean type, in kredis we can use list, unique_list, enum, counter, etc.

Now you have onboarded method in your instance model. Let’s try it out in the rails console.

• Open your terminal and run rails c.
• Grab the first user with u = User.first

• Now you have the instance of User model for the first user with id 1. You can get the onboarded value from the redis database with u.onboarded.value

As you can see, kredis will automatically map the key for the user with the format model_name_plural:id:attribute_name. You can find it in the kredis repo.

The default value is nil and we want to change the value to true or false based on our needs.

• Set the onboarded value to true because the user has been completed the onboarding

• To make sure the value has been changed, you can use the previous command to get the value of onboarded

• Now the onboarded is true which means the user has been completed the onboarding.

In Upstash, you can also see all the data you have inserted into redis database with Data Browser feature. Go back to Upstash Console and then click your redis database. You will see the Data Browser. You can view the value based on the key, delete and set the expiration of your data key.

Or if you want to try to get the value with Redis CLI, switch to CLI page and you can type the command there. For example, I want to get the value of the first user and then use GET command.

So in your application if you want to check whether the user is onboarded, simply check the value of onboarded attribute and it’s available for the instance of User model.

u = User.first
if u.onboarded.value
  # run some code if the user is onboarded
else
  # run some code if the user has not been onboarded
end

#2 Use Case - Caching with Upstash Redis

In this second use case, we will use Upstash Redis for caching. Let’s consider that we need to cache a particular value or some query result from an external application (API). It is common that the response of the API is JSON object and we want it to cache the result for some time and save it to the redis database. In this case, we can use Low-Level Caching in Rails by using the Rails.cache.fetch method. This method does both writing and reading to the cache. As long as the information is serializable like JSON then the Rails cache works just fine.

Here is an example, consider that we have product and each product instance has method to looks up the competitor price from other website or API and then we want to save it for only 8 hours after that the value should no longer exist in the database.

First, configure the cache stores to use redis as a database. Open the development environment file config/environments/development.rb and change the store to redis

- config.cache_store = :memory_store
+ config.cache_store = :redis_cache_store, { url: ENV["REDIS_URL"] }

and because caching is disabled by default in development environment, for testing purpose we need to enabled it with command rails dev:cache

Let’s add competitor_price in the Product model

class Product < ApplicationRecord
  def competitor_price
    Rails.cache.fetch("#{cache_key_with_version}/competitor_price", expires_in: 8.hours) do
      # call the API here
    end
  end
end

That is the basic usage for our use case. So here is how it works, when you call competitor_price, the block will be executed in the event that the cache key is not available in the database or the cache key has expired and the return value of the block will be written to the cache under the given cache key.

So in this case when first time calls competitor_price it will call the API and the result of the API will be written to the cache, the second time you call competitor_price it will get the value from the cache and after 8 hours the cache key will be deleted automatically.

cache_key_with_version is a Rails method that generates cache key based on model’s class name, id, and updated_at attributes. So the resulting cache key will be something like products/1-20221229085409206814/competitor_price

For the testing, let’s update the competitor_price method and change the expires to 30 seconds and write the static value from the block

class Product < ApplicationRecord
  def competitor_price
    Rails.cache.fetch("#{cache_key_with_version}/competitor_price", expires_in: 30.seconds) do
      sleep 3
      {
        "prices": [
          { "name": "Amazon", "price": 12.00 },
          { "name": "Ebay", "price": 11.00 },
        ]
      }
    end
  end
end

You may notice that I added sleep 3 to illustrate the process when calling the external API.

Let try out in the rails console, open your terminal and run rails c

• First grab the first product and assign to variable product

• Now call the competitor_price with product.competitor_price, you will notice that it will take some time around 3 seconds to complete the process because this is the first time which means the block will be executed. Remember we added sleep 3 before.

• Now let’s call the method again and the block will not be executed, instead, the resulting value will be from the cache. You don’t need to wait for 3 seconds to get the value.
• And after 30 seconds, the cache key will be deleted automatically. You can check in the Data Browser in Upstash Console

This method is useful when you need to cache a particular query or value and you need to get the value quite often in your application without sacrificing the response time.

Summary

Upstash Redis is compatible with almost all Redis API and can be used for popular use cases such as:

• Query Caching
• Session Caching
• Message Broker
• Queues, etc

Upstash Redis also can be used as a database without worrying the data will be lost if the server crashes, thanks to Durable Storage. Upstash Global Database replicates the database and is distributed across multiple regions and the user is routed to the nearest region to minimize the latency and optimize the performance.

References

• https://docs.upstash.com/redis
• https://guides.rubyonrails.org/caching_with_rails.html
• https://developer.redis.com/explore/what-is-redis/