Etcd Backup and Restore (3)

Last post I have show steps to restore etcd, but when we edit the /etc/kubernetes/manifest/etcd.yaml file, there are some confuse items among the /var/lib/etcd on hostPath and mountVolumne.

For example, the --data-dir in the command:

etcdctl snapshot restore --data-dir

this is the dir on the host machine.

And the parameter also appear inside the /etc/kubernetes/manifest/etcd.yaml file. This is the command running inside the container, so they are totally diffferent. So here let's give a more detailed explanation.

1. Saving the snapshot

etcdctl snapshot save ./stored.db ...

• This command connects to the running etcd
• Takes a snapshot of its current state
• Saves it as stored.db in current directory on host machine

1. Restoring the snapshot

etcdctl snapshot restore ./stored.db --data-dir /var/lib/etcd-restore

• Takes the stored.db snapshot file
• Unpacks/expands it into a complete etcd data directory structure
• Places this expanded data directory at /var/lib/etcd-restore on host machine
• This directory now contains all the database files etcd needs to run

1. Container mounting and startup

# In etcd.yaml
volumes:
- hostPath:
    path: /var/lib/etcd-restore  # make a new dir for Host machine to use it for store etcd data
    type: DirectoryOrCreate
  name: etcd-data
...
volumeMounts:
- mountPath: /var/lib/etcd      # Container directory
  name: etcd-data

• When etcd pod starts, Kubernetes:
  1. Sees the hostPath volume configuration
  2. Mounts host's /var/lib/etcd-restore directory
  3. Makes it appear at /var/lib/etcd inside container

1. etcd process startup

command:
- etcd
- --data-dir=/var/lib/etcd

• etcd process starts inside container
• Looks for its data at /var/lib/etcd (which is actually host's /var/lib/etcd-restore mounted)
• Finds and uses the restored data

So the complete data path is:

1. stored.db (snapshot file)
   ↓
2. /var/lib/etcd-restore/* (expanded data on host)
   ↓ (mount)
3. /var/lib/etcd/* (same data, visible in container)
   ↓ (etcd process)
4. Running etcd using restored data