Using Amazon EFS with AWS Lambda: Persistent File Storage

Unlock persistent, low-latency storage in AWS Lambda using Amazon EFS. This 2025-ready guide covers real-world use cases, step-by-step Terraform and CloudFormation examples, performance tuning, cost comparisons (vs S3, /tmp, Elasticache), and DevOps best practices for scalable serverless architecture.

AWS Lambda + EFS: Scalable File Storage for Serverless Workloads

When we think of AWS Lambda, we often imagine stateless, short-lived functions with tight constraints on storage and memory. But what if your function needs to read or write persistent data across multiple invocations? Enter Amazon EFS (Elastic File System) — AWS’s fully managed NFS solution that can be mounted directly to your Lambda functions within a VPC.

What This Means in Simple Terms

By mounting EFS to your Lambda function, you unlock:

• Persistent storage — survives between invocations
• Shared access — multiple Lambdas, containers, and EC2s can access the same filesystem
• Large file support — process GB-level datasets, ML models, PDFs, images, and more
• Faster ML inference or image manipulation — with mounted models or binaries

Why /tmp Isn’t Enough in Lambda

Lambda’s /tmp directory:

• Is ephemeral — data is wiped once the execution environment is reclaimed
• Has a hard 512MB size limit
• Cannot be shared across Lambda instances or invocations

If your application needs persistent storage, multi-function collaboration, or handling large files, /tmp becomes a serious bottleneck.

What Is Amazon EFS?
Amazon EFS is a fully managed, elastic, network file system accessible from:

• EC2
• ECS/Fargate
• Lambda (via VPC & access point)

With EFS:

• You can store unlimited files with standard POSIX permissions
• Mount the same filesystem across services
• Pay only for what you use (GB/month + I/O if using provisioned mode)

Why Use Amazon EFS with Lambda?

EFS makes serverless Lambda functions stateful and collaborative. Key advantages:

• Persistent Storage – Data stays even after Lambda shuts down
• Large File Support – No 512MB cap like /tmp
• Shared Access – Share data across multiple Lambda functions and invocations
• Zero Manual Scaling – Automatically grows with usage
• POSIX File Permissions – Secure multi-tenant file access

Cost Comparison: EFS vs S3 vs /tmp vs ElastiCache

Storage Type	Pricing	Pros	Cons
EFS (Standard)	~\$0.30/GB/month + I/O (provisioned if enabled)	Scalable, shared, persistent, POSIX	Latency > S3, costlier
S3	~\$0.023/GB/month	Durable, cheap, static hosting	Not writable by Lambda directly without SDK
Lambda /tmp	Free up to 512MB	Fastest, local	Ephemeral, size-limited
ElastiCache	~\$0.02/GB/hour	Low-latency, real-time caching	In-memory only, not persistent

How to Use EFS with Lambda – Step-by-Step

Prerequisites:

• Existing Lambda function
• VPC with private subnets
• EFS in the same region and VPC

Setup Steps:

• Create EFS File System
• Create EFS Access Point
• Configure Mount Targets (across AZs)
• Update Security Groups (allow NFS from Lambda to EFS)
• Attach Lambda to VPC & Mount via Access Point

Python Sample (Writing to EFS):

with open("/mnt/efs/mylog.txt", "a") as f:
    f.write("Function invoked at: {}\n".format(datetime.now()))

Real-World Use Case

Let’s say you have a startup running ML inference via Lambda. The ML model file is 400MB. Storing it in /tmp (limited to 512MB) or S3 (slower, not POSIX-compliant) doesn’t scale. With EFS mounted, your Lambda loads the model instantly from the shared file system — improving cold start times and inference speed drastically.

When Should You Use Lambda with EFS?

Use Case	Should Use EFS?
ML model loading	Yes
Image/Video rendering (FFmpeg, PIL)	Yes
Data sharing across Lambdas	Yes
Large file access during function	Yes
Small, stateless functions	No
Low-latency DB-style caching	Prefer ElastiCache or DynamoDB

Cost Comparison: EFS vs S3 vs /tmp vs ElastiCache

Feature	EFS	S3	/tmp	ElastiCache
Persistent	Yes	Yes	No
Shared	Yes	Yes	No	Yes
POSIX-Compliant	Yes	No	Yes	No
Max File Size	∞	∞	512MB	N/A
Speed	Fast	Slower	Fast	Fast
Pricing Model	Per GB/month + I/O	Per request + storage	Free	Per node/hour

Reference: EFS Pricing, S3 Pricing, ElastiCache Pricing

Terraform Script to Mount EFS with Lambda

Here’s a production-ready snippet:

# Create a VPC (or use an existing one)
resource "aws_vpc" "main" {
  cidr_block = "10.0.0.0/16"
}

# EFS File System
resource "aws_efs_file_system" "lambda_efs" {
  performance_mode = "generalPurpose"
  lifecycle_policy {
    transition_to_ia = "AFTER_7_DAYS"
  }
  throughput_mode = "bursting"
}

# Create Mount Targets in each AZ subnet
resource "aws_efs_mount_target" "efs_mt" {
  for_each        = toset(["subnet-az1", "subnet-az2"])  # Replace with real subnet IDs
  file_system_id  = aws_efs_file_system.lambda_efs.id
  subnet_id       = each.key
  security_groups = [aws_security_group.efs_sg.id]
}

# Security Group for EFS (Allow NFS)
resource "aws_security_group" "efs_sg" {
  vpc_id = aws_vpc.main.id
  ingress {
    from_port   = 2049
    to_port     = 2049
    protocol    = "tcp"
    cidr_blocks = ["10.0.0.0/16"]
  }
}

# Lambda Function Role
resource "aws_iam_role" "lambda_exec" {
  name = "lambda-exec-role"
  assume_role_policy = jsonencode({
    Version = "2012-10-17",
    Statement = [{
      Action = "sts:AssumeRole",
      Effect = "Allow",
      Principal = {
        Service = "lambda.amazonaws.com"
      }
    }]
  })
}

# Lambda Function
resource "aws_lambda_function" "efs_lambda" {
  function_name = "efs_lambda_demo"
  runtime       = "python3.9"
  handler       = "index.handler"
  role          = aws_iam_role.lambda_exec.arn
  filename      = "lambda-deploy.zip"

  vpc_config {
    subnet_ids         = ["subnet-az1", "subnet-az2"]
    security_group_ids = [aws_security_group.efs_sg.id]
  }

  file_system_config {
    arn              = aws_efs_access_point.ap.arn
    local_mount_path = "/mnt/efs"
  }
}

# EFS Access Point
resource "aws_efs_access_point" "ap" {
  file_system_id = aws_efs_file_system.lambda_efs.id
  posix_user {
    uid = 1000
    gid = 1000
  }
  root_directory {
    path = "/lambda"
    creation_info {
      owner_gid   = 1000
      owner_uid   = 1000
      permissions = "755"
    }
  }
}

Explanation of Key Sections

• aws_efs_file_system: Creates the shared storage.
• aws_efs_mount_target: Mounts the EFS to subnets in your VPC (required).
• aws_efs_access_point: Simplifies access, ensures POSIX permissions.
• aws_lambda_function: The Lambda is now “wired” to use /mnt/efs as a real folder.
• vpc_config: Required, since Lambda with EFS must run inside a VPC.

Output & Testing

If done correctly, your Lambda will:

• Write/read files to /mnt/efs
• Persist across invocations
• Share data with other compute instances

Test tip: Add a print(os.listdir("/mnt/efs")) to verify it works.

Pre-Checks Before Using Lambda + EFS

Checklist Item	Reason
Must deploy Lambda inside a VPC	Required for EFS access
Ensure proper security groups	NFS (2049) must be open between Lambda and EFS
Avoid cold start bloat	Mounting adds latency; prefer provisioned concurrency for speed
Set correct POSIX permissions	Access Point must match your Lambda UID/GID

When Not to Use This Setup

• If your function runs fine within /tmp
• If you're doing low-latency key-value access → use DynamoDB or ElastiCache
• If you don’t want VPC complexity (adding NAT Gateway, etc.)

CloudFormation Snippet: Mounting EFS to Lambda

AWSTemplateFormatVersion: '2010-09-09'
Description: Attach Amazon EFS to AWS Lambda for shared persistent storage

Resources:

  MyVPC:
    Type: AWS::EC2::VPC
    Properties:
      CidrBlock: 10.0.0.0/16
      EnableDnsSupport: true
      EnableDnsHostnames: true
      Tags:
        - Key: Name
          Value: LambdaVPC

  MySubnet1:
    Type: AWS::EC2::Subnet
    Properties:
      VpcId: !Ref MyVPC
      CidrBlock: 10.0.1.0/24
      AvailabilityZone: !Select [ 0, !GetAZs "" ]
      MapPublicIpOnLaunch: true

  MySecurityGroup:
    Type: AWS::EC2::SecurityGroup
    Properties:
      GroupDescription: Allow Lambda access to EFS
      VpcId: !Ref MyVPC
      SecurityGroupIngress:
        - IpProtocol: tcp
          FromPort: 2049
          ToPort: 2049
          CidrIp: 0.0.0.0/0 # for public test only — restrict in prod

  MyEFS:
    Type: AWS::EFS::FileSystem
    Properties:
      Encrypted: true
      PerformanceMode: generalPurpose

  MyMountTarget:
    Type: AWS::EFS::MountTarget
    Properties:
      FileSystemId: !Ref MyEFS
      SubnetId: !Ref MySubnet1
      SecurityGroups:
        - !Ref MySecurityGroup

  MyAccessPoint:
    Type: AWS::EFS::AccessPoint
    Properties:
      FileSystemId: !Ref MyEFS
      PosixUser:
        Uid: "1000"
        Gid: "1000"
      RootDirectory:
        CreationInfo:
          OwnerUid: "1000"
          OwnerGid: "1000"
          Permissions: "750"
        Path: "/lambda"

  LambdaExecutionRole:
    Type: AWS::IAM::Role
    Properties:
      AssumeRolePolicyDocument:
        Version: '2012-10-17'
        Statement:
          - Effect: Allow
            Principal:
              Service: lambda.amazonaws.com
            Action: sts:AssumeRole
      ManagedPolicyArns:
        - arn:aws:iam::aws:policy/service-role/AWSLambdaVPCAccessExecutionRole
        - arn:aws:iam::aws:policy/AmazonElasticFileSystemClientReadWriteAccess

  MyLambdaFunction:
    Type: AWS::Lambda::Function
    Properties:
      FunctionName: MyEFSLambda
      Runtime: python3.12
      Handler: index.lambda_handler
      Code:
        ZipFile: |
          import os
          def lambda_handler(event, context):
              with open("/mnt/efs/test.txt", "w") as f:
                  f.write("Hello from Lambda using EFS!")
              return "File written to EFS"
      Role: !GetAtt LambdaExecutionRole.Arn
      VpcConfig:
        SubnetIds:
          - !Ref MySubnet1
        SecurityGroupIds:
          - !Ref MySecurityGroup
      FileSystemConfigs:
        - Arn: !GetAtt MyAccessPoint.Arn
          LocalMountPath: /mnt/efs

Explaining the Lambda + EFS CloudFormation Stack (Simplified Professional View)

This CloudFormation setup provisions a serverless Lambda function connected to Amazon EFS, enabling persistent, shared storage across invocations. Here's what each component does and why it matters:

1. Amazon EFS (AWS::EFS::FileSystem)

• Provides a durable, shared NFS file system.
• Ideal for ML models, media processing, or large binaries.

2. Mount Target

• Enables EFS access from a Lambda in a VPC.
• One target per AZ is needed.
• Ensures private, secure VPC routing to EFS.

3. Access Point

• Defines a safe mount path (e.g., /lambda).
• Sets POSIX permissions to prevent access issues.
• AWS-recommended for multi-function access and permission control.

4. Security Groups

• Allows port 2049 (NFS) from Lambda to EFS.
• Lambda’s SG is whitelisted by the EFS SG.
• Ensures secure, scoped connectivity inside the VPC.

5. IAM Role

• Grants VPC, EFS, and CloudWatch Logs access.
• Follows least-privilege policy: only elasticfilesystem:ClientMount, logs:*, etc.

6. Lambda Function

• Connects to private subnets + security group.
• Mounts EFS at /mnt/efs using the Access Point.
• Can read/write files across invocations — perfect for ML, media, or temp file sharing.

When to Use:

• ML inference with large model files.
• Persistent shared data (e.g., thumbnails, binaries).
• Temporary storage exceeding /tmp’s 512MB limit.

Pre-checks:

• Lambda must run in private subnets.
• Ensure port 2049 is open between SGs.
• Use Access Points to avoid permission issues.
• Role must have correct EFS + VPC permissions.

Recommendations & Best Practices

Scenario	Use Lambda + EFS?	Why?
ML inference with >100MB models	Yes	Faster load time
Sharing temp files between functions	Yes	Persistent access
Simple file logging	Use CloudWatch	Cheaper
Real-time caching	Use ElastiCache	Lower latency

Key Concepts and Terminology

Term	Description
Lambda	Serverless compute function that runs code in response to events
EFS (Elastic File System)	Scalable, network-based file storage for AWS compute services
VPC (Virtual Private Cloud)	Isolated network environment for deploying AWS resources
Access Point	Named mount path with identity and permissions
Throughput Mode	Controls how EFS scales I/O (Bursting or Provisioned)
/tmp	Lambda's 512MB ephemeral local storage, cleared after container reset
Elasticache	In-memory key-value store (e.g., Redis) used for caching/shared data
Cold Start	Latency on first invocation due to provisioning resources

References

• AWS Lambda EFS Official Guide
• Amazon EFS Pricing
• Amazon S3 Pricing
• ElastiCache Pricing
• AWS Terraform EFS + Lambda Example
• EFS Performance Tips
• Lambda + EFS Access Point Best Practices