AWS EBS Multi-attach Clustered Storage System with GlusterFS

Clustered Storage System with GlusterFS on AWS EC2 Instances

This guide describes how to set up a clustered storage system using GlusterFS on two AWS EC2 instances, utilizing EBS Multi-Attach for shared storage.

Prerequisites

• Minimum 2 EC2 (minimum c5.large) instances
• Both instances in the same AWS region and availability zone
• Tested on AWS Provided Ubuntu 24.04 LTS
• EBS Volume Type: Provisioned IOPS SSD (io2) to support Multi-Attach
• SSH access to EC2 instances (ensure both instances have public IPs)
• External EBS Multi-Attach enabled for volumes

Solution Overview

Steps

1. Create EC2 Instances and EBS Volumes
2. Attach EBS Volumes to Both Instances
3. Format and Mount EBS Volumes
4. Install GlusterFS and Set Up Cluster
5. Create GlusterFS Volume and Mount Shared Storage
6. Auto-mount EBS Volume on Instance Reboot

Respective Diagram

Step 1: Create EC2 Instances and EBS Volumes

1.1 Create EC2 Instances

• Create 2 EC2 instances in the same region and availability zone on aws console.
• Assign public IPs to the instances so that they can be accessed.

1.2 Create EBS Volumes (io2)

• Create EBS volumes with the type Provisioned IOPS SSD (io2).
• Ensure that these volumes are in the same availability zone as your EC2 instances.
• Enable Multi-Attach for both volumes so they can be attached to multiple instances.

1.3 Attach EBS Volumes to EC2 Instances

• Attach the created EBS volumes to both EC2 instances.
• Goto EBS Volume then Select the EBS Volume (io2) then on the upper right corner Select Action then Attach Volume.

---

Step 2: Format and Mount EBS Volumes

2.1 Verify EBS Volume Connection

• SSH into both EC2 instances and verify the connection of the EBS volume by running:
  

  lsblk
  
  

• Ensure that the volume is attached to the instance.

2.2 Format the EBS Volume

• Format the EBS volume once on either of the instances (only format it once for the shared file system):
  

  sudo mkfs.xfs /dev/nvme1n1  # Replace 'nvme1n1' with your disk name
  
  

2.3 Mount the EBS Volume

• Create a directory to mount the EBS volume:
  

  sudo mkdir /home/ubuntu/data
  
  

• Mount the EBS volume to the newly created directory:
  

  sudo mount /dev/nvme1n1 /home/ubuntu/data
  
  

• Verify the mount:
  

  df -h /home/ubuntu/data
  
  

---

Step 3: Install GlusterFS

3.1 Install GlusterFS on Both Instances

• Add the GlusterFS repository and install the GlusterFS server package:
  

  sudo add-apt-repository ppa:gluster/glusterfs-10
  sudo apt-get update
  sudo apt-get install -y glusterfs-server
  
  

3.2 Start GlusterFS Service

• Start the GlusterFS service on both instances:
  

  sudo systemctl start glusterd
  sudo systemctl enable glusterd
  
  

---

Step 4: Set Up GlusterFS Cluster

4.1 Peer Probe

• Choose one instance as the primary and the other as the secondary.

• On the primary instance, run the following command to add the secondary instance to the cluster:
  

  sudo gluster peer probe <secondary_instance_privateIP>
  
  

• If successful, the output will display: Success.

---

Step 5: Create GlusterFS Volume

5.1 Create Shared Volume

• On the primary instance, create a GlusterFS volume called shared-volume using the mounted EBS volumes. This volume will be replicated across both instances:
  

  sudo gluster volume create shared-volume replica 2 transport tcp <primary_instance_privateIP>:/home/ubuntu/data <secondary_instance_privateIP>:/home/ubuntu/data force
  
  

5.2 Start the GlusterFS Volume

• Start the shared-volume on from any instances once:
  

  sudo gluster volume start shared-volume
  
  

---

Step 6: Mount GlusterFS Volume

6.1 Mount the GlusterFS Volume on Both Instances

• Create a mount point directory (e.g., /mnt/shared):
  

  sudo mkdir /mnt/shared
  
  

• Mount the shared-volume GlusterFS volume to this directory:
  

  sudo mount -t glusterfs <primary_instance_privateIP>:/shared-volume /mnt/shared
  
  

• Verify that the GlusterFS volume is mounted correctly by creating or updating a file in the /mnt/shared directory on either instance.

---

Step 7: Auto-Mount EBS Volume on Reboot

7.1 Add to /etc/fstab

• Edit /etc/fstab to automatically mount the EBS volume on reboot:
  

  echo '/dev/nvme1n1 /home/ubuntu/data xfs defaults 0 0' | sudo tee -a /etc/fstab
  
  

---

Conclusion

You have successfully set up a clustered storage system using GlusterFS with shared EBS volumes in AWS. The shared storage is now accessible from both EC2 instances, and the GlusterFS volume ensures that data is synchronized between the two instances.