DEV Community: Oluwole Suliat Adefowoke

Elastic File System (EFS) Multi-Instance Deployment: Streamlining File Sharing Across AWS Instances

Oluwole Suliat Adefowoke — Fri, 19 Jan 2024 19:00:11 +0000

In this article, we'll dive into AWS Elastic File System (EFS), a robust storage service that sets itself apart by enabling simultaneous mounting on multiple instances, offering a heightened level of flexibility. If you missed our exploration of Elastic Block Store (EBS), you can find it here.

Now, let's proceed with a comprehensive overview of the steps involved:

1. Create EC2 Instances

Similar to the process for Elastic Block Store (EBS), I provisioned servers with no special configurations, running on eu-north-1b.

2. Create File System

In the AWS console, search for Elastic File System service and click on "Create file system".

Choose between quick creation with default settings or customize for specific configurations

For customization, follow the detailed steps below:

Step 1: File System Settings

File Name: efs-demo
File System Type: Regional (redundant across multiple Availability Zones)
Automatic Backups: Disable (for now)
Lifecycle Management: Default
Performance Settings: Default Click on Next.

Step 2: Network Access

Specify Mount Targets for each Availability Zone where instances are running.
Select the VPC and subnet ID of your instances. (EFS is a VPC specific service)
Specify a security group which will act as a firewall for Mount Targets. Click on Next

Step 3: File System Policy

This is very optional setting but you can configure different file system policies if you want but for this demo, we leave as default and proceed to review and create the file system

3. Mount File System

Access the terminal with connections to both Instances A & B.
Starting with Instance-A, when working with EFS, Amazon has made it really easy if you're using an Amazon Linux Image, you can install a package they created called Amazon-EFS-utils
To install this package, Run sudo yum install -y amazon-efs-utils
Repeat this command on Instance-B.

Now that we installed the EFS package, we want to mount that file system we created in the console.
To see a list of installed efs packages; Run sudo mount. + tab

We can see a .efs after running that command because we installed the amazon efs utils package.

To mount a file system, you need 2 things

efs instance id
a folder/directory to mount on To get the efs instance id, go in to the console, click on the efs instance and copy the id as indicated in the image below

Now that we have all we need, to mount file system, Run sudo mount.efs efs-instance-id /efsdemo/

SIDE NOTE: I went through this so you don't have to

If encountering mounting errors, ensure that the mount target security group has an inbound rule allowing NFS access from the EC2 security group. Edit the inbound rule accordingly.

Once the error is resolved, running df -h should confirm the successful mounting of the file system at /efsdemo.

At this point, we technically can use our EFS file systems but the changes made would not persist through a reboot

4. Persist Data

To ensure data persistence through reboots, open the fstab file using the command sudo nano etc/fstab.
Provide essential details such as the file system ID, the mount point, the file system type, and additional configurations sourced from the AWS EFS Mount Configuration

After completing these configurations, a system reboot should automatically mount the file system, ensuring seamless data persistence.

To verify the persistence, create a file within the mounted directory using the command:
sudo nano efsfile.txt and input the desired content.

5.Simultaneous Mount on Instance-B

Unlike EBS, EFS supports simultaneous mounting on multiple instances. Proceed to the second instance and replicate the process performed on Instance-A. Execute the following commands:

sudo mkdir /efsdemo
sudo mount.efs fs-0b058364663762fc9 /efsdemo
df -h
cd /efsdemo/
ls 
cat efsfile.txt

The successful execution of these commands on Instance-B demonstrates that the file created on Instance-A is accessible on Instance-B.

Additionally, creating a new file efsfile2.txt on Instance-B is reflected when inspecting the contents on Instance-A.

This capability underscores the flexibility and convenience offered by AWS Elastic File System for simultaneous data access across multiple instances.

Exploring AWS's Elastic Block Store: A Step-by-Step Beginner's Guide

Oluwole Suliat Adefowoke — Mon, 15 Jan 2024 17:10:38 +0000

In this article, we will provide a step-by-step beginner's guide on how to effectively utilize AWS's Elastic Block Store (EBS).

Things to Note

Before we delve into the detailed explanation, it's crucial to keep the following points in mind:

EC2 Instance and EBS Availability Zone:

Ensure that your EC2 instance is in the same availability zone as your EBS. This alignment is vital for seamless interaction between the two.

EBS Volume Independence:

Unlike EC2 instances, EBS volumes are not tied to a specific instance. This characteristic allows for flexibility in managing storage resources.

Now, let's proceed with a comprehensive overview of the steps involved:

Steps

1. Create EC2 Instances

For this demonstration, I have provisioned a couple of servers with no special configurations. However, it's essential to highlight the importance of the Availability Zone (AZ). Both instances in the image below are deployed in eu-north-1b. As EBS is specific to an Availability Zone, any volume attached must be created in the same zone as the instances.

2. Create EBS Volume

Navigate to the EBS page within the EC2 window. Scroll down to "Elastic Block Store" and click on "Volume." Even if you have existing volumes, click "Create volume" for a new one. Configure the volume settings based on your specifications. For this demo, we'll use the minimum requirements, as shown in the image below:

Remember to create the volume in the same availability zone as your instances. Once created, wait for the volume state to change to "attached" before proceeding.

3. Attach Volume to EC2 Instance

In the volume details, click on the "Actions" button and select "Attach Volume". Specify the instance by clicking the dropdown.
As in the image below, we're attaching it to Instance-A.

In the image above, we can see the device name. When we attach the EBS Volume, that's the name it's going to show up as. We can also see a message saying on newer Linux kernels, it may rename it to /dev/xvdf through /dev/xvdp. So it's going to be either /dev/sdf or /dev/xvdf depending on how our operating system handles it.

However, it is also important to note that with the advent of NVMe storage, the naming convention has changed to better reflect the underlying technology. This naming scheme is more specific and aligned with the characteristics of NVMe storage, providing a clearer representation of the device's identity in the system. The shift to the "nvmeXnY" naming convention is part of the Linux kernel's adaptation to better support and differentiate NVMe storage devices. But that's what to look out for once we log into our instances.

We go into the terminal, and we have a connection with both instances. Since we attached the EBS Volume to Instance-A, that's what we'll be working with for now.

4. Create File System

On Instance-A, to take a look at all the block devices connected to it, Run: lsblk

We can see from the image above that our Operating system named the block device as "nvme1n1". In this case, it suggests it's an NVMe (Non-Volatile Memory Express) device. The number "1" indicates the index of the NVMe controller, and "n1" suggests it is the first NVMe namespace under that controller. The one that is 8G is our root device where the OS was installed, and the one that's 20G is our EBS Volume.

At this point, we can't do anything with it. We have to create a file system, then mount it. First, let's verify if there's a file system on there. Run: sudo file -s /dev/nvme1n1
Don't forget to specify the name of YOUR block device; i.e., either /dev/xvdf, /dev/nvme1n1, or /dev/sdf

If the command returns "data" like in the image above, that means there's no file system available.
To create a file system, type in the command sudo mkfs. + tab to show all the different file systems you can create. For this demo, we'll be using .ext4.
Run: sudo mkfs.ext4 /dev/nvme1n1

Once the file system is created and we run the command again, we can see that it doesn't return "data" like earlier as indicated in the image above.

5. Mount File System

After the file system has been created, the next step is to mount it. First, we need a directory that we're going to mount this file system to. Run: sudo mkdir /ebs-demo
And if you run ls /, you should see our ebs-demo folder.

To mount the file system in the directory,
Run: sudo mount /dev/nvme1n1 /ebs-demo/

Now if we run lsblk again, we can see it has been mounted on the ebs-demo folder. And if we run df -h, we can see our file system /dev/nvme1n1 has been mounted on /ebs-demo.

At this point, our configurations are set. We have our EBS Volume, created file system, and mounted it to a folder. However, our changes in data wouldn't persist through a reboot.

6. To Persist Data

Cd into the folder /ebs-demo and run sudo blkid to get the UUID for our file system.

Copy the UUID as indicated in the image above and use it to modify the FS tab file so that it can read the configurations upon a reboot. Run: sudo nano /etc/fstab.

All we have to do is edit the file by adding the UUID number, the specific directory where we mounted the file system, type of file system, and specify to use default configuration settings as indicated in the image above.

At this point, if we reboot our machine, it should automatically mount to our file system.

Now that everything is up and running, we can create a file inside the directory that we mount our file system.
Run: sudo nano ebsfile.txt and put in message you want in the text file.

7. Detach from Instance A and Attach to Instance B

Returning to the AWS console, our goal is to demonstrate that an EBS Volume is not tied exclusively to a specific instance. Although the option to detach the volume may initially appear greyed out, it is indeed possible to detach and reattach it to another instance.

To execute this, follow these steps:

Navigate to the AWS EC2 Instances dashboard.
Locate and stop Instance-A.
Once stopped, proceed to detach the EBS volume. Allow a few moments for the operation to complete.
After detaching from Instance-A, reattach the volume to a different instance, in this case, Instance-B.

Once the volume is successfully attached to Instance-B, return to the terminal connected to Instance-B.

Run the following command to check the available block devices, similar to what was done on Instance-A: lsblk.

You should observe the 20GB block device named "nvme1n1", as seen previously. To ensure the presence of a file system,
execute: sudo file -s /dev/nvme1n1

Now, you have the flexibility to mount this file system wherever needed. For the purposes of this demo, create a directory and mount the file system using:

sudo mkdir /ebs2-demo
sudo mount /dev/nvme1n1 /ebs2-demo/

Navigate into /ebs2-demo/, list the files, and confirm the existence of the previously created ebsfile.txt. If you view the content of this file, you will find the data persisted from Instance-A.

This demonstration illustrates the dynamic nature of EBS Volumes, allowing detachment, reattachment to different instances, and seamless continuity of data operations.
As a concluding step, you can proceed to delete both instances and the associated EBS volume.