S3 as a File Storage

#aws #s3 #solutionsarchitect #cloudcomputing

Well, Well, Well, AWS just launched S3 Files.

What it is? It is a S3 Bucket as a File System. The First and Only in Cloud. It makes your buckets accessible as file systems. Why is it a huge deal? Why this has excited a lot of DevOps and Architects? How does it solves your cost issues. lets see.

Before we move for those who doesn't understand or know the diff in Object storage vs File System let me brief you

File Storage - Just like your Laptop Drive, File system, Folder Editable files. Extremally low latency
Use - work on it, process it, collaborate. e.g. Render farm → 50 nodes writing frames to shared output directory, IoT device writes readings continuously

Object Storage - no folders, no hierarchy just keys. What looks like a folder path (finance/2026/april/) is just a prefix in the key name. cheap storage, infinetly scalable. But you cannot Change a file (You will have to reupload)
Use - Store it, Share it at scale

How they changed S3 into file server.

You can create S3 bucket as a File system
You can access the S3 as file system from your EC2 or EKS or Lambda
Changes to data on the file system are automatically reflected in the S3 bucket
Files can be attached to multiple compute resources enabling data sharing across cluster

What are your Use Cases:

Production Applications - A document processing service that reads uploaded PDFs, annotates them, and saves results — all via file operations.
The data lives in S3. The app mounts it via S3 Files. No download-process-upload loop needed.
Agentic AI Agents using Python libraries and CLI tools - An AI agent running Python cannot call S3 SDK for every operation. It uses libraries like pandas, numpy, PIL, ffmpeg — all of which expect file paths, not S3 URIs.
Multiple agents can share the same mounted bucket simultaneously. One agent writes. Another reads
ML Training Pipelines
Without S3 Files:
Step 1: Copy 500 GB from S3 to EFS/EBS → takes 30 minutes
Step 2: Train on local copy
Step 3: Upload results back to S3

With S3 Files:
Mount the S3 bucket. Start training immediately.

When not to use S3 File and stay with FSX..

When something new is introduced people often take it as a magical solution for everything. Here we also need to understand the boundary of our use cases and when it is wise to keep using FSX.

Workloads migrating from on-premises NAS environments- on-premises apps mount it over the network via NFS or SMB. S3 Files Only supports NFS not the SMB They may need Active Directory authentication while S3 files uses POSIX. FSX was originally the best choice and nothing changes.
HPC and GPU Clusters - For high-performance computing (HPC) and GPU cluster storage, Amazon FSx for Lustre is must. S3 files doesn't have enough throughput and IOPS

Use the Basic map below to decide.

Do you need file system access to data in S3?
├── Yes → Is data already in / belongs in S3?
│ ├── Yes → Use S3 Files
│ └── No → Consider EFS or FSx
│
└── No → Regular S3 API is fine

Are you migrating from on-premises NAS?
├── Windows/SMB based → FSx for Windows File Server
├── NetApp based → FSx for NetApp ONTAP
├── ZFS based → FSx for OpenZFS
└── Generic NFS based → EFS or S3 Files

Is it extreme HPC / GPU cluster?
└── Yes → FSx for Lustre

Does your app need specific file system features?
├── Multi-protocol + enterprise features → FSx for NetApp ONTAP
├── ZFS snapshots + clones → FSx for OpenZFS
├── Windows ACLs + Active Directory → FSx for Windows File Server
└── None of the above, data is in S3 → S3 Files

Cost Comparison Side - Side

Now coming to Pricing between S3 Files and the EFS

Assumptions
Total Data: 1 TB
Active Working data: 200 GB (20% hot)
Monthly Reads: 500 GB

With EFS the Cost Comes to - 200-250$ per Month
With S3 the Cost is - 30-50$ per month

In Conclusion S3 Files is revolutionizing for Large Data Sets and eliminating the need for the EFS for your applications. However be wise to use where it is required and not pasted everywhere.

Additional Reads: