Cache Layers vs Storage Classes for Performance

Improving S3 Performance Through Caching vs. Storage Classes

Amazon S3 Express One Zone launched for the general public in November 2023, targeting AWS customers with extremely high-performance storage needs. It’s built to handle workloads that exceed the performance limits of S3 Standard, also offering persistent storage. However, its use cases are seemingly similar to traditional S3 caching scenarios. So does this mean Amazon S3 Express One Zone renders S3 caching as no longer necessary?

S3 Storage Class Performance

• Speed: Performance speeds 10x faster than S3 Standard
• Scale: Up to 2 million reads and 200,000 writes per second
• Transaction Costs: Read and write operations cost just 25% compared to S3 Standard
• S3 API Interface: Same interface as any typical S3 bucket

S3 Express One Zone is clearly made for high TPS workloads. Its improved performance advantage, however, are not free—the trade offs in both cost and durability must be weighed before using it in high-performance systems.

• Storage Costs: Storage cost 3-4x more than S3 Standard
• Redundancy: Data is stored in only one AZ (as the name "One Zone" suggests), resulting in lower durability than S3 Standard
• Network Dependency: Performance gains are greatest when your compute resources and storage are in the same AZ; these benefits drop substantially if they are separated

The performance boost has some compromises and is most effective when in the same AZ. For globally distributed compute resources, the performance advantages of Express One Zone become less noticeable as performance declines. If these trade-offs are acceptable, you can start using Express One Zone with the official AWS documentation.

S3 Cache Layers

Choosing the right cache layer depends on the specific needs of your use case. Whether you require a high-performance cache for specific tasks or a global cache to lower latency from different regions, there are plenty of options to explore.

CloudFront CDN

CloudFront CDN is Amazon's content delivery network that caches S3 objects at edge locations worldwide.

• Works like a standard CDN to speed up content delivery: CloudFront caches content at edge locations near end users, reducing the distance data needs to travel.
• Cuts latency to single-digit milliseconds: By storing content at optimal edge locations, CloudFront sends data significantly quicker than direct S3 access.
• Especially valuable for high-traffic, globally distributed users: CloudFront's distributed architecture shines with global audiences, managing high request volumes with solid performance. But these performance advantages drop when users are located close to your servers.
• Limited benefits when compute resources and users are co-located: When your servers and users are in the same region or availability zone, CloudFront's benefits are not as significant because the network distance is already so short.

Although both Amazon S3 Express One Zone and CloudFront CDN can serve as an S3 cache, their overlap in effective use cases is rather limited. A CDN is more about scaling your infrastructure globally while enhancing security, whereas Amazon S3 Express One Zone is tailored for high-performance local workloads.

Archil

Archil is a third-party service built specifically for high-performance S3 caching, delivering performance on par with Amazon S3 Express One Zone. It serves as a POSIX-compliant S3 cache for applications that demand both performance and flexibility. Mounted as a file storage system on your EC2 instance, Archil caches data from S3 and functions as a write-back cache by combining multiple writes into a single operation to help lower expenses.

Both Amazon S3 Express One Zone and Archil support demanding, high TPS workloads that need ultra-low latency. But Archil comes with a number of unique advantages:

• POSIX Compliance: Developers can interact with S3 as a POSIX-compliant file system, making the cache easier to integrate without major code changes
• Pay-As-You-Use Model: S3 Express One Zone is persistent storage, so you’re charged the storage premium continuously. Archil is as a true cache, reducing costs on infrequently accessed data by removing data after the TTL expires

As the more dynamic performance option, Archil enables you to transfer S3 objects in and out of your cache faster. Unlike S3 Express One Zone, which requires data to be pre-stored in that tier for any performance gains, Archil caches only accessed data. If you’re unsure which S3 files will need fast access or find it too costly to migrate your entire bucket to S3 Express One Zone, Archil is an ideal choice.

Other S3 Caching Solutions

Apart from Amazon S3 Express One Zone, there are not many other managed high-performance solutions that can effectively speed up S3 for high-demand workloads. While other AWS caching options are available, they have considerable drawbacks:

• ElastiCache/Redis: Works well as a cache, but its performance declines and cost increases as the size of objects grow, generally making it unsuitable for objects over 128 MB
• MinIO: Functions as an S3-compatible object storage system that can be self-hosted, providing performance similar to Amazon S3 Express One Zone. However, it needs manual infrastructure management and doesn’t integrate smoothly with AWS services. To fully leverage its performance benefits, your compute must be on-prem with MinIO as network latency will degrade performance.
• Compute-adjacent EBS volumes: Though not necessarily a cache by definition, high-performance instances with attached EBS volumes can act as a makeshift cache for frequently accessed data, though this requires custom setup.

S3 Cache or Storage Class, Who Wins?

The choice between S3 Express One Zone and a dedicated cache layer comes down to your particular use case. S3 Express One Zone is best suited for workloads that require constant data access, whereas Archil excels with unpredictable data access patterns by offering more cost-effective eviction and storage of infrequently accessed data. Although there are a few managed solutions for boosting S3 performance, S3 Express One Zone and Archil are both dependable and resilient services capable of the most intensive workloads.