Redis Server: Deploying on Kubernetes for Kids

Hi there! BotKids here.
Today, I'll explain how to deploy a Redis Stack Server on Kubernetes while keeping it kids-friendly 😊.

Before starting, please make sure you have:

• A working Kubernetes Cluster
• kubectl installed

A Redis Stack Server is like a Redis with superpowers! These superpowers come from extra features called "modules." Our Redis Stack Server has five amazing modules:

1. RedisTimeSeries: A cool way to remember things that happened in the past, like a time machine!

2. RedisBloom: A super-fast way to find and count things without having to remember everything.

3. RediSearch: A search engine for Redis, like a detective that helps you find stuff quickly.

4. RedisJSON: A way to store and play with JSON data, like having a toy box for JSON.

5. RedisGraph: A way to explore connections between things, like a map of your friendships.

Before we start deploying our Redis Stack Server, let's first prepare our Kubernetes playground.

Step 1: Create the Redis namespace
A namespace is like a big room where we keep our Redis Stack Server and its friends. We want to create a room called "redis". To create the Redis namespace, run this command:

kubectl create namespace redis

Now we have a big room called "Redis" for our Redis Stack Server!

Step 2: Check and create the "standard" storage class
A storage class is like a recipe that tells Kubernetes how to make a special kind of storage for our data. We need a recipe called "standard". First, let's check if we already have this recipe.

kubectl get storageclass standard

If you see the "standard" storage class in the results, great! We already have the recipe. If not, don't worry. We can create it ourselves. Save the following YAML code as a file named "standard-storageclass.yaml":

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: standard
provisioner: kubernetes.io/gce-pd
parameters:
  type: pd-standard
reclaimPolicy: Retain

Now, run this command to create the "standard" storage class:

kubectl apply -f standard-storageclass.yaml

That's it! We have our "standard" recipe ready for use.

Now that we have our Redis namespace and the "standard" storage class, let's deploy our Redis Stack Server with superpowers:

Step 3: Create the ConfigMap
A ConfigMap is like a little box where we store some settings. In this case, we're storing settings for the Redis master and slave nodes. To create the ConfigMap, save the YAML code provided as a file named "configmap.yaml":

apiVersion: v1
kind: ConfigMap
metadata:
  name: redis-stack-configuration
  namespace: redis
  labels:
    app: redis
data:
  master.conf: |
    maxmemory 1000mb
    maxmemory-policy allkeys-lru
    maxclients 20000
    timeout 300
    dbfilename dump.rdb
    dir /data
    appendonly yes
    save ""
  slave.conf: |
    slaveof redis-stack-0.redis-service.redis 6379
    maxmemory 1000mb
    maxmemory-policy allkeys-lru
    maxclients 20000
    timeout 300
    dir /data
    appendonly yes
    save ""

Then, run this command:

kubectl apply -f configmap.yaml

So what did we "apply" ?

Let's imagine Redis is a magical toy box. We use the ConfigMap to set some rules about how our toy box works. Here are the rules we set:

maxmemory:Our toy box can only hold a certain number of toys. We set a limit on how many toys it can hold (1000 toy-sized spaces in our case).

maxmemory-policy: If our toy box gets too full, we need to remove some toys to make room for new ones. We choose the "allkeys-lru" policy, which means we'll remove the toys that we haven't played with in a while first.

maxclients: Our toy box can have many friends playing with it at the same time. We set a limit on how many friends can play together (20,000 friends in our case).

timeout: If a friend stops playing with our toy box and goes to do something else, we want to give their spot to another friend who wants to play. We set a rule that says if a friend hasn't played with the toy box for 300 seconds, they'll have to wait their turn again.

dbfilename: We give a special name to the list of toys inside our toy box, so we know what's inside.

dir: We pick a special room in our house where we'll keep our toy box and the list of toys.

appendonly: We decide to write down every time we add a new toy to our toy box in a special notebook. This way, if anything happens to our toy box, we can look at the notebook and know exactly which toys were inside.

save: We also have a camera that takes pictures of our toy box from time to time. But since we have our special notebook, we decide not to use the camera for now. So, we tell the camera not to take any pictures by providing an empty string.

These rules in the ConfigMap help us manage our magical Redis toy box and make sure everything runs smoothly and efficiently.

About "Master-Slave" concept
In Redis, we have a concept called "master-slave." Let's think of the master as a wise teacher and the slaves as students who follow the master. The master holds all the important information, and the students want to learn from the master.
When we use a master-slave setup in Redis, we have one master Redis server and one or more slave Redis servers. The master Redis server is the main place where all the data is stored, and the slave Redis servers are like backup copies that hold the same data as the master.
Whenever the master Redis server gets new information, it tells the slave Redis servers about it. The slaves then update their copies of the information to match the master. This way, if something happens to the master, we still have the slave Redis servers that know all the same information, so we don't lose any data.
Now, in our ConfigMap, we have two different configurations - one for the master and one for the slave. The master configuration is for the wise teacher (the main Redis server), and the slave configuration is for the students (the backup Redis servers). These configurations help set up the master-slave relationship and make sure the master and slave Redis servers know their roles.
So, in our Redis clubhouse, the master is the superhero who takes care of all the important stuff, and the slaves are like sidekicks who follow the superhero and help when needed. This way, if our superhero is busy or needs help, the sidekicks are always there to save the day!

Step 4: Create the Service
A Service is like a phonebook for our application. It helps other parts of our app find and talk to Redis (our toy box). Save the YAML code provided as a file named "service.yaml".

apiVersion: v1
kind: Service
metadata:
  name: redis-service
  namespace: redis
  labels:
    app: redis
spec:
  ports:
    - port: 6379
  clusterIP: None
  selector:
    app: redis

Then, run this command:

kubectl apply -f service.yaml

So, what did we "apply" this time ?
Remember, a Redis service is like a magical phonebook for our Redis toy box. This phonebook helps our friends (other parts of our application) find and connect to Redis. Here's how our phonebook works:

ports: Our phonebook has a special number that our friends can call to talk to our toy box. We set the number to 6379, which is the favorite number of Redis.

selector: Our phonebook also needs to know which toy box it's helping. So, we give it a magical sticker that says "app: redis." Our phonebook looks for toy boxes with the same magical sticker and helps connect our friends to the right one.

clusterIP: None: Remember how our phonebook usually gives our friends a single phone number to call, which connects them to the toy box? When we set clusterIP: None, we're telling our magical phonebook not to use a single phone number for our toy box. Instead, it creates a special kind of phonebook called a "headless service."

A headless service is like a magical phonebook that doesn't have one phone number for all our friends to call. Instead, it gives each friend a direct connection to Redis, so they can talk to it without going through a shared phone number. This can be helpful when Redis needs to work closely with our friends, and they need a faster and more direct way to connect. By using a headless service, we make sure that our friends can quickly and easily talk to our Redis whenever they want.

Step 5: Create the StatefulSet
A StatefulSet is like a blueprint for building a house. It defines the structure of our Redis deployment. Save the YAML code provided as a file named "statefulset.yaml".

apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: redis-stack
  namespace: redis
spec:
  serviceName: "redis-service"
  replicas: 2
  selector:
    matchLabels:
      app: redis
  template:
    metadata:
      labels:
        app: redis
    spec:
      initContainers:
      - name: init-redis
        image: redis/redis-stack-server:latest
        command:
        - /bin/bash
        - -c
        - |
          set -ex
          # Extract pod ordinal index from the hostname to generate the redis server-id.
          hostname_regex='-([0-9]+)$'
          if [[ `hostname` =~ ${hostname_regex} ]]; then
            ordinal=${BASH_REMATCH[1]}
          else
            exit 1
          fi

          # Copy the appropriate redis config files from the config map to their respective directories.
          config_map_path="/mnt"
          config_destination="/etc/redis-config.conf"

          if [[ ${ordinal} -eq 0 ]]; then
            cp "${config_map_path}/master.conf" "${config_destination}"
          else
            cp "${config_map_path}/slave.conf" "${config_destination}"
          fi
        volumeMounts:
        - name: redis-claim
          mountPath: /etc
        - name: config-map
          mountPath: /mnt/
      containers:
      - name: redis
        image: redis/redis-stack-server:latest
        ports:
        - containerPort: 6379
          name: redis-stack
        command:
          - redis-stack-server
          - "/etc/redis-config.conf"
        volumeMounts:
        - name: redis-data
          mountPath: /var/lib/redis-stack
        - name: redis-claim
          mountPath: /etc
      volumes:
      - name: config-map
        configMap:
          name: redis-stack-configuration
  volumeClaimTemplates:
  - metadata:
      name: redis-claim
    spec:
      accessModes: [ "ReadWriteOnce" ]
      resources:
        requests:
          storage: 1Gi
      storageClassName: standard
  - metadata:
      name: redis-data
    spec:
      accessModes: [ "ReadWriteOnce" ]
      resources:
        requests:
          storage: 1Gi
      storageClassName: standard

Then, run this command:

kubectl apply -f statefulset.yaml

We're building a house for our Redis, and we're using a special blueprint called a "statefulset." This blueprint has a lot of important parts that make sure our house is built perfectly for Redis:

serviceName: This connects our house blueprint to the magical phonebook we made before. This way, our friends can find our house easily when they want to play with Redis.

replicas: This tells us how many houses we want to build for our Redis. Right now, we want to build 2 houses.

initContainers: These are like little helpers that get our houses ready before Redis moves in. We have one helper here who sets up the rules for Redis, depending on if it's the leader (master) or a helper (slave).

containers: This is where we put Redis in houses. We make sure Redis knows the rules set up by our little helper.

volumes: These are like storage rooms in our houses where we keep important things. In this case, we have a storage room for the list of rules we made for Redis earlier (the ConfigMap).

volumeClaimTemplates: These are like asking our house builder to add more storage rooms. Here, we ask for two more storage rooms: one for our list of rules and another for the actual Redis data.

By understanding these important parts of our house blueprint (the statefulset), we can make sure our house is built perfectly for our Redis toy box. This way, Redis can work together with our friends and make our playtime a lot more fun!

Step 6: Check if everything works !
Check our Configmap

$ kubectl get configmap -n redis
NAME                        DATA   AGE
redis-stack-configuration   2      50s

Check our Service

$ kubectl get service -n redis
NAME          TYPE       CLUSTER-IP EXTERNAL-IP PORT(S)  AGE
redis-service ClusterIP  None       <none>      6379/TCP 55s

Check our StatefulSet

$ kubectl get statefulset -n redis
NAME          READY   AGE
redis-stack   2/2     2m19s

Check pods

kubectl get pods -n redis   
NAME            READY   STATUS    RESTARTS   AGE
redis-stack-0   1/1     Running   0          3m12s
redis-stack-1   1/1     Running   0          2m52s

How these components work together:

1. The ConfigMap stores the Redis settings, acting as a central place for both master and slave configurations.

2. The Service acts as a phonebook, helping other parts of the app find and talk to the Redis nodes.

3. The StatefulSet is the blueprint that describes the structure of our Redis deployment:

• It connects to the Service using the serviceName attribute.

• It specifies how many Redis instances we want with the replicas attribute.

• It uses an initContainer to prepare the Redis configuration files for master and slave nodes based on the pod's hostname.
  It defines the main container running the Redis server, telling it to use the configuration file prepared by the init container.

• It sets up volumes to store data, including the ConfigMap and storage for the Redis data.
  When you apply these files with kubectl apply -f, Kubernetes builds the Redis house using the blueprint (StatefulSet), the phonebook (Service), and the little box for settings (ConfigMap).

I hope this simpler explanation helps you understand the key components and their interactions in deploying a Redis StatefulSet on Kubernetes. If you have any questions, feel free to ask! Happy coding! 😊

Comments

[Hilman Maulana]

how do connect to the master?
When try to connect and try to write some data it is always like this

redis-service:6379> SET mykey "Hello"
(error) READONLY You can't write against a read only replica.

[Bot-kids]

Hello !
Sorry for the late reply, hope you found out since then.

And to answer you, this is happening because you are connected to the "slave" Redis replica which is a read only instance.
To resolve this issue, you need to ensure that your write operations are directed to the Redis master instance.

If you connect to Master using following command, you should be able to perform write operations :

kubectl exec -n redis -it redis-stack-0 -- redis-cli

Here is the command to connect to slave instance :

kubectl exec -n redis -it redis-stack-1 -- redis-cli

To check the role, please run in Redis CLI session :

ROLE

• Master Instance (redis-stack-0):

1) "master"
2) (integer) 0
3) 1) "slave0:ip=10.3.50.119,port=6379,state=online,offset=144165,lag=1"

• Replica Instance "slave" (redis-stack-1):

1) "slave"
2) "redis-stack-0.redis-service.redis"
3) (integer) 144165

To perform write operations, ensure you're connecting to the master Redis instance. Here's how you can do it :

redis-cli -h redis-stack-0.redis-service.redis -p 6379

SET mykey "Hello"

Please let me know if it helps or not :)

[Yuneui Jeong]

This post helped me a lot! I've written a Helm chart based on your work so that I can configure further.

GitHub Link: github.com/uniglot/redis-stack-rep...

[Bot-kids]

Glad it helped you ! What do you think about kids oriented explanation ?

[Yourfriendlyfriend]

Good post !