ETCD Multinode cluster using Docker

#etcd #docker

In this post, I will be showing how to create a 3-node cluster with each node represented by a docker container. First, lets create our nodes IP.

sudo ifconfig en0 alias 192.168.10.10/24 up
sudo ifconfig en0 alias 192.168.11.10/24 up
sudo ifconfig en0 alias 192.168.12.10/24 up

Once they are created, you can use script below which is a modified version of this

REGISTRY=quay.io/coreos/etcd
# available from v3.2.5
# REGISTRY=gcr.io/etcd-development/etcd

# For each machine
ETCD_VERSION=latest
TOKEN=my-etcd-token
CLUSTER_STATE=new
NAME_1=etcd-node-0
NAME_2=etcd-node-1
NAME_3=etcd-node-2
HOST_1=192.168.10.10
HOST_2=192.168.11.10
HOST_3=192.168.12.10
CLUSTER=${NAME_1}=http://${HOST_1}:2380,${NAME_2}=http://${HOST_2}:2380,${NAME_3}=http://${HOST_3}:2380
DATA_DIR=/mydir/etcd #make sure to change this

# For node 1
THIS_NAME=${NAME_1}
THIS_IP=${HOST_1}
docker run \
  -d \
  -p ${THIS_IP}:2379:2379 \
  -p ${THIS_IP}:2380:2380 \
  --volume=${DATA_DIR}/${THIS_NAME}:/etcd-data \
  --name ${THIS_NAME} ${REGISTRY}:${ETCD_VERSION} \
  /usr/local/bin/etcd \
  --data-dir=/etcd-data --name ${THIS_NAME} \
  --initial-advertise-peer-urls http://${THIS_IP}:2380 --listen-peer-urls http://0.0.0.0:2380 \
  --advertise-client-urls http://${THIS_IP}:2379 --listen-client-urls http://0.0.0.0:2379 \
  --initial-cluster ${CLUSTER} \
  --initial-cluster-state ${CLUSTER_STATE} --initial-cluster-token ${TOKEN}

# For node 2
THIS_NAME=${NAME_2}
THIS_IP=${HOST_2}
docker run \
  -d \
  -p ${THIS_IP}:2379:2379 \
  -p ${THIS_IP}:2380:2380 \
  --volume=${DATA_DIR}/${THIS_NAME}:/etcd-data \
  --name ${THIS_NAME} ${REGISTRY}:${ETCD_VERSION} \
  /usr/local/bin/etcd \
  --data-dir=/etcd-data --name ${THIS_NAME} \
  --initial-advertise-peer-urls http://${THIS_IP}:2380 --listen-peer-urls http://0.0.0.0:2380 \
  --advertise-client-urls http://${THIS_IP}:2379 --listen-client-urls http://0.0.0.0:2379 \
  --initial-cluster ${CLUSTER} \
  --initial-cluster-state ${CLUSTER_STATE} --initial-cluster-token ${TOKEN}

# # For node 3
THIS_NAME=${NAME_3}
THIS_IP=${HOST_3}
docker run \
  -d \
  -p ${THIS_IP}:2379:2379 \
  -p ${THIS_IP}:2380:2380 \
  --volume=${DATA_DIR}/${THIS_NAME}:/etcd-data \
  --name ${THIS_NAME} ${REGISTRY}:${ETCD_VERSION} \
  /usr/local/bin/etcd \
  --data-dir=/etcd-data --name ${THIS_NAME} \
  --initial-advertise-peer-urls http://${THIS_IP}:2380 --listen-peer-urls http://0.0.0.0:2380 \
  --advertise-client-urls http://${THIS_IP}:2379 --listen-client-urls http://0.0.0.0:2379 \
  --initial-cluster ${CLUSTER} \
  --initial-cluster-state ${CLUSTER_STATE} --initial-cluster-token ${TOKEN}

Once the cluster is created, you can use the command below to check this cluster status.

$ docker exec -it etcd-node-0 /bin/sh
/ # etcdctl -w table --endpoints=192.168.11.10:2379,192.168.10.10:2379,192.168.12.10:2379 endpoint status
+--------------------+------------------+---------+---------+-----------+-----------+------------+
|      ENDPOINT      |        ID        | VERSION | DB SIZE | IS LEADER | RAFT TERM | RAFT INDEX |
+--------------------+------------------+---------+---------+-----------+-----------+------------+
| 192.168.11.10:2379 | 4c3c38d6652b5d75 |  3.2.28 |   25 kB |     false |         5 |         17 |
| 192.168.10.10:2379 | a00eaaf5194c573d |  3.2.28 |   25 kB |      true |         5 |         17 |
| 192.168.12.10:2379 | fdd13ca43538c5a2 |  3.2.28 |   25 kB |     false |         5 |         17 |
+--------------------+------------------+---------+---------+-----------+-----------+------------+