Top 7 AWS EFS Performance Tuning Tips

AWS Elastic File System (EFS) is a scalable, cloud-based file system that you can use in AWS. It is designed for Linux-based applications. You can use it as a stand-alone file system, in combination with on-premises resources, or with other AWS services, such as EC2. In this article, you will learn seven techniques for optimizing EFS performance.

How EFS Works

EFS is based on the NFSv4 file system structure protocol. This mirrors most on-premises structures and enables you to smoothly and simply migrate files. When using EFS, you can choose between Standard Access and Infrequent Access.

Standard access is designed for frequently accessed items and provides low-latency access. Infrequent Access is designed for long-term storage and data that isn’t often needed. It provides lower-cost storage in exchange for higher-latency.

EFS does not require storage provisioning, allowing you to scale as needed. It is a pay-for-use service.

Key features of EFS include:

• Shared storage—you can simultaneously access files from up to 1000 EC2 instances across multiple regions and availability zones (AZs). You can also access files from offsite via AWS Direct Connect or virtual private network (VPN).
• Scalable performance—IOPS and throughput scale with usage and number of attached instances, up to 500k IOPS and 10 GB/s throughput. Scaling is automatic, ensuring that performance matches size and that you aren’t paying for unnecessary resources.
• Security and compliance—you can protect EFS with existing security infrastructures, including Identity and Access Management (IAM) and virtual private cloud (VPC) security groups. You can also define individual file permissions using POSIX. The service includes built-in compliance for common regulatory standards, including SOC, PCI DSS, and HIPAA.

Top 7 AWS EFS Performance Tuning Tips

EFS is designed to optimize performance capabilities for you. However, there are some additional steps you can take to ensure that you are really getting the best possible performance. Below are a few tips that can help.

Choose Your Performance Mode

EFS does not use instances. Instead, I/O limits are determined by the performance mode you choose to use. Your options are General Performance and Max I/O. General Performance provides high-scalability and low-latency while Max I/O can scale to higher throughput in exchange for higher-latency.

Regardless of which option you choose, storage volumes start at .5MB/s baseline throughput and include 7.2 minutes of 100MB/s burst credits. The only way to permanently increase your throughput is to increase your file system size.

While you can wait until the size increases naturally, you can also force this growth by writing dummy data to your system. This data can be used to force your limit to whatever you need and can be overwritten as you add live data. If you use this method, be sure not to include dummy data in backups to avoid paying for wasted resources.

Enable Asynchronous Write

Unless you have a pressing need for synchronously writing to your file system, you should enable the asynchronous write feature. This feature enables you to buffer pending write operations onto EC2 instances. This buffering helps you eliminate the need for a round trip between your client and EFS for each write operation. Shorter trips equal lower latency and faster operations.

Monitor With Metrics

You can and should monitor your EFS resources using AWS CloudWatch metrics. CloudWatch is a service that automatically collects metrics data for you. You can access this data from the AWS console, CLI, or via API.

Use metrics data to stay up-to-date on the performance of your system and to alert yourself to any drops in performance or bottlenecks. If you do see performance issues, you can use the metrics data that CloudWatch provides to identify the problem and correct it.

In particular, be sure to watch your burst credits metric. Burst credits are used to temporarily boost performance during times of high traffic. If you run out you will see a sudden, steep drop in performance.

Separate Operations by Latency

Depending on the workloads you are running, you may benefit from separating latency-sensitive operations from those that are not. Doing so provides you with separate throughput caps and burst credits for each volume created.

Separating your operations also enables you to set different performance modes if needed. While this often cannot provide the same performance as you would see from locally mounted files, it can provide more consistent performance.

Avoid App Code

EFS not designed to be used for managing codebases or deploying applications. It cannot provide the read volume or speed needed for these operations. Rather, it is designed for massively shared storage, containing exported data files, asynchronous logs, and media assets. If you need to run applications, particularly in production, you are better off sticking to containers or local file systems.

Select Proper Mount Options

Most users do not need to adjust the default mount options set by AWS. However, if you have benchmarks and tests showing that you can get better performance from changes, you can adjust as needed. If you do take this option, make sure that you mount your file system using the DNS name. This ensures that your data is mounted in the same AZ as your EC2 instances.

With EFS, you can use NFS version 4.0 or 4.1. NFSv4.1 typically provides higher performance so use this version whenever possible. Additionally, you can increase the size of your NFS client’s read/write buffer to further increase speeds.

Use Lifecycle Management Policies

Lifecycle management policies are designed to help you tier your storage to reduce costs. These policies enable you to automatically move files between Standard and Infrequent Access tiers based on file access.

To create lifecycle policies, you can use the EFS console. During policy creation, you can specify parameters for movement, including the number of access requests and time since last access. When policies are triggered, files are moved transparently to users and remain easily accessible from any location.

Conclusion

EFS is a useful file system, but like all AWS services, it needs configuration and fine-tuning for optimal results and cost-savings. The most obvious, but often missed, optimization opportunity is the performance mode configuration. You can actually choose your own mode, and ensure that you’re using the mode that serves your project best.

Hopefully, the tips in this article can help you refine EFS until you get the results you're interested in. Be sure to continue experimenting with optimization techniques. Maintenance is a long-lasting marathon that requires continuous monitoring and dynamic optimization.