DEV Community

loading...

Redis: replication, part 3 - redis-py and work with Redis Sentinel from Python

Arseny Zinchenko
DevOps, cloud and infrastructure engineer. Love Linux, OpenSource, and AWS.
Originally published at rtfm.co.ua on ・7 min read

Still on the subject about Redis replication and Redis Sentinel – a couple of examples using the redis-py library for Python.

Previous series posts:

  1. Redis: replication, part 1 – an overview. Replication vs Sharding. Sentinel vs Cluster. Redis topology
  2. Redis: replication, part 2 – Master-Slave replication, and Redis Sentinel

All Redis clients for Python can be found here – redis.io/clients.

At first, will use redis-py without Sentinel, just to check how it’s working.

Then – will spin up Sentinel and will check masters and slaves discovery.

A working environment aka Redis replication and Redis Sentinel are already configured in the previous post – here is just Python examples.

redis-py and Redis

Install redis-py on Debian:

root@redis-0:/home/admin# apt install python-redis

Also, pip can be used but I’d prefer to use normal repositories.

Check it:

root@redis-0:/home/admin# python
Python 2.7.13 (default, Sep 26 2018, 18:42:22)
[GCC 6.3.0 20170516] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import redis
>>> r = redis.Redis(host='localhost', port=6379, db=0)
>>> r.get('test')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib/python2.7/dist-packages/redis/client.py", line 880, in get
return self.execute_command('GET', name)
File "/usr/lib/python2.7/dist-packages/redis/client.py", line 573, in execute_command
return self.parse_response(connection, command_name, **options)
File "/usr/lib/python2.7/dist-packages/redis/client.py", line 585, in parse_response
response = connection.read_response()
File "/usr/lib/python2.7/dist-packages/redis/connection.py", line 582, in read_response
raise response
redis.exceptions.ResponseError: NOAUTH Authentication required.

Okay, works.

Find authorization in documentation:

>>> help(redis.Redis)

Add the password:

>>> r = redis.Redis(host='localhost', password='foobared', port=6379, db=0)
>>> r.get('test')
'test'

Okay.

redis-py and Redis Sentinel

Now run Sentinel instances if they are not started yet and let’s try Redis via Redis Sentinel cluster client.

Sentinel’s configs /etc/redis/sentinel.conf on hosts now are:

sentinel monitor redis-test 52.58.69.184 6379 2
sentinel down-after-milliseconds redis-test 6001
sentinel failover-timeout redis-test 60000
bind 0.0.0.0
sentinel auth-pass redis-test foobared

After Sentinel instance start – it will update config, so now on the redis-1 (which is the first slave here) config looks like:

admin@redis-1:~$ cat /etc/redis/sentinel.conf
sentinel myid e3e62e6577aa975f93346dad3d4f8e25833fd8f1
sentinel monitor redis-test 52.29.101.118 6379 2
sentinel down-after-milliseconds redis-test 6001
bind 0.0.0.0
sentinel failover-timeout redis-test 60000
# Generated by CONFIG REWRITE
port 26379
dir "/home/admin"
sentinel auth-pass redis-test foobared
sentinel config-epoch redis-test 1
sentinel leader-epoch redis-test 1
sentinel known-slave redis-test 52.58.69.184 6379
sentinel known-slave redis-test 35.159.18.26 6379
sentinel known-sentinel redis-test 172.31.46.202 26379 e0fe655c59aa3cc32eab1c0858c52418700abe79
sentinel known-sentinel redis-test 172.31.41.39 26379 07a450af0d2f178410b78ee0f5ae99ce1cd0ac62
sentinel current-epoch 1

Check Sentinel cluster status:

root@redis-0:/home/admin# redis-cli -p 26379 info sentinel
Sentinel
sentinel_masters:1
sentinel_tilt:0
sentinel_running_scripts:0
sentinel_scripts_queue_length:0
sentinel_simulate_failure_flags:0
master0:name=redis-test,status=ok,address=52.58.69.184:6379,slaves=2,sentinels=3

And use redis-py:

>>> from redis.sentinel import Sentinel
>>> sentinel = Sentinel([('localhost', 26379)], socket_timeout=0.1)

You can get a master’s IP using the discover_master() method:

>>> sentinel.discover_master('redis-test')
('52.58.69.184', 6379)

And slaves:

>>> sentinel.discover_slaves('redis-test')
[('52.29.101.118', 6379), ('35.159.18.26', 6379)]

To work with a master use the master_for() method:

| master_for(self, service_name, redis_class=, connection_pool_class=, **kwargs)

| Returns a redis client instance for the “service_name“ master.

>>> master = sentinel.master_for('redis-test', socket_timeout=0.1)

But if call master now – it will tell us we aren’t authorized:

>>> master.set('test-key', 'test-value')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib/python2.7/dist-packages/redis/client.py", line 1072, in set
return self.execute_command('SET', *pieces)
File "/usr/lib/python2.7/dist-packages/redis/client.py", line 573, in execute_command
return self.parse_response(connection, command_name, **options)
File "/usr/lib/python2.7/dist-packages/redis/client.py", line 585, in parse_response
response = connection.read_response()
File "/usr/lib/python2.7/dist-packages/redis/sentinel.py", line 55, in read_response
return super(SentinelManagedConnection, self).read_response()
File "/usr/lib/python2.7/dist-packages/redis/connection.py", line 582, in read_response
raise response
redis.exceptions.ResponseError: NOAUTH Authentication required.

So during creating the master object – add password:

>>> master = sentinel.master_for('redis-test', socket_timeout=0.1, password='foobared')

Check now:

>>> master.set('test-key', 'test-value')
True

Also, you can get a whole master’s configuration:

>>> master.config_get()
{'appendonly': 'no', ... 'slave-announce-port': '0'}

Or in a more readable view:

>>> for i in master.config_get():
...   print(i)
...
appendonly
requirepass
daemonize
protected-mode
zset-max-ziplist-entries
zset-max-ziplist-value
dir
slave-serve-stale-data
cluster-require-full-coverage
slowlog-log-slower-than
masterauth
rdbchecksum
...

On a salve – check the key we added previously:

admin@redis-1:~$ redis-cli -a foobared get test-key
"test-value"

Work with slaves is similar to master:

>>> slave = sentinel.slave_for('redis-test', socket_timeout=0.1, password='foobared')
>>> slave.get('test-key')
'test-value'

Master change and Sentinel failover

Let’s check out the master at this moment:

>>> sentinel.discover_master('redis-test')
('52.58.69.184', 6379)

And eventually let’s try to stop Redis master on the redis-0 host:

root@redis-0:/home/admin# systemctl stop redis-server

Or by:

root@redis-0:/home/admin# redis-cli -a foobared DEBUG sleep 600

(if you’ll just kill the redis-server process with the kill – Sentinel will restart it again).

Check Sentinel’s logs:

10633:X 01 Apr 13:46:10.430 # +sdown master redis-test 52.58.69.184 6379
10633:X 01 Apr 13:46:10.486 # +odown master redis-test 52.58.69.184 6379 #quorum 2/2
10633:X 01 Apr 13:46:10.486 # +new-epoch 1
10633:X 01 Apr 13:46:10.486 # +try-failover master redis-test 52.58.69.184 6379
10633:X 01 Apr 13:46:10.488 # +vote-for-leader e3e62e6577aa975f93346dad3d4f8e25833fd8f1 1
10633:X 01 Apr 13:46:10.492 # 07a450af0d2f178410b78ee0f5ae99ce1cd0ac62 voted for e3e62e6577aa975f93346dad3d4f8e25833fd8f1 1
10633:X 01 Apr 13:46:10.492 # e0fe655c59aa3cc32eab1c0858c52418700abe79 voted for e3e62e6577aa975f93346dad3d4f8e25833fd8f1 1
10633:X 01 Apr 13:46:10.559 # +elected-leader master redis-test 52.58.69.184 6379
10633:X 01 Apr 13:46:10.559 # +failover-state-select-slave master redis-test 52.58.69.184 6379
10633:X 01 Apr 13:46:10.611 # +selected-slave slave 52.29.101.118:6379 52.29.101.118 6379 @ redis-test 52.58.69.184 6379
10633:X 01 Apr 13:46:10.612 * +failover-state-send-slaveof-noone slave 52.29.101.118:6379 52.29.101.118 6379 @ redis-test 52.58.69.184 6379
10633:X 01 Apr 13:46:10.664 * +failover-state-wait-promotion slave 52.29.101.118:6379 52.29.101.118 6379 @ redis-test 52.58.69.184 6379
10633:X 01 Apr 13:46:11.498 # +promoted-slave slave 52.29.101.118:6379 52.29.101.118 6379 @ redis-test 52.58.69.184 6379
10633:X 01 Apr 13:46:11.498 # +failover-state-reconf-slaves master redis-test 52.58.69.184 6379
10633:X 01 Apr 13:46:11.557 * +slave-reconf-sent slave 35.159.18.26:6379 35.159.18.26 6379 @ redis-test 52.58.69.184 6379
10633:X 01 Apr 13:46:12.550 * +slave-reconf-inprog slave 35.159.18.26:6379 35.159.18.26 6379 @ redis-test 52.58.69.184 6379
10633:X 01 Apr 13:46:12.550 * +slave-reconf-done slave 35.159.18.26:6379 35.159.18.26 6379 @ redis-test 52.58.69.184 6379
10633:X 01 Apr 13:46:12.632 # -odown master redis-test 52.58.69.184 6379
10633:X 01 Apr 13:46:12.632 # +failover-end master redis-test 52.58.69.184 6379
10633:X 01 Apr 13:46:12.632 # +switch-master redis-test 52.58.69.184 6379 52.29.101.118 6379
10633:X 01 Apr 13:46:12.632 * +slave slave 35.159.18.26:6379 35.159.18.26 6379 @ redis-test 52.29.101.118 6379
10633:X 01 Apr 13:46:12.632 * +slave slave 52.58.69.184:6379 52.58.69.184 6379 @ redis-test 52.29.101.118 6379
10633:X 01 Apr 13:46:18.647 # +sdown slave 52.58.69.184:6379 52.58.69.184 6379 @ redis-test 52.29.101.118 6379

Check from Python:

>>> sentinel.discover_master('redis-test')
('52.29.101.118', 6379)

Master was changed – 52.29.101.118 – it’s the redis-1 host.

Check status from the redis-1:

admin@redis-1:~$ redis-cli -a foobared info replication
Replication
role:master
connected_slaves:1
slave0:ip=35.159.18.26,port=6379,state=online,offset=76847,lag=0
...

Check the Redis node status on the redis-0 host – it must become a slave now:

root@redis-0:/home/admin# redis-cli -a foobared info replication
Replication
role:slave
...

And try to add a new key from Python on the redis-0 host using the same master object we created before:

>>> master.set('test-key2', 'test-value2')
True

Check:

admin@redis-2:~$ redis-cli -a foobared get test-key2
"test-value2"

Done.

Similar posts

Discussion (0)