Secure Data Sharing: AWS Lambda Writing to S3 Across Accounts

Introduction

In the multi account AWS environment. It is common to have services hosted in one account and the data ingested to the different account. For instance, there is a lambda function in one AWS account and read or write data to the S3 bucket in a different AWS account.

This blog explains the detailed steps to configure S3 bucket policy and IAM role for lambda function to achieve cross account S3 access with AWS Lambda.

The Scenario

• Account A (1111111111): Hosts the Lambda function.
• Account B (2222222222): Owns the target S3 bucket (account_b_bucket).

The goal is to allow AWS lambda function in Account A to put an object in the S3 bucket of Account B.

Architecture Overview

The setup involves three main components

1) IAM role (account_a_lambda_role) in account A and the custom policy attached to the role to access the S3 bucket in account B.

Attach this policy to the Lambda execution role (replace the bucket name):

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "CrossAccountS3",
      "Action": ["s3:GetObject", "s3:PutObject"],
      "Effect": "Allow",
      "Resource": ["arn:aws:s3:::account_b_bucket/*"]
    }
  ]
}

This will grant permissions on the lambda to request access to the S3 bucket in account B.

2) Trust policy of Lambda Execute Role allows lambda function to assume the role.

Attach this policy to the Lambda execution role:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": { "Service": "lambda.amazonaws.com" },
      "Action": "sts:AssumeRole"
    }
  ]
}

3) Bucket Policy in Account B that allows IAM role attached to the lambda function in account A to put the object in the bucket.

Configure the bucket policy in account B to trust account A's Lambda role. Replace the account number, lambda role and bucket name.

{
  "Id": "ExamplePolicy",
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "CrossAccountBucket",
      "Action": ["s3:GetObject", "s3:PutObject"],
      "Effect": "Allow",
      "Resource": ["arn:aws:s3:::account_b_bucket/*"],
      "Principal": {
        "AWS": ["arn:aws:iam::1111111111:role/account_a_lambda_role"]
      }
    }
  ]
}

Execution Flow

The Lambda in Account A is invoked.

It assumes the IAM role account_a_lambda_role.

The role policy allows S3 actions on the target bucket.

The bucket policy in Account B authorizes the role ARN.

The Lambda can now securely access the cross-account bucket.

Sample Lambda code

Replace the "TARGET_BUCKET" in the below code with your target bucket name.

import boto3
import os
from datetime import datetime

# Initialize S3 client
s3_client = boto3.client("s3")

def lambda_handler(event, context):
    # Get bucket name from environment variable
    bucket_name = os.environ["TARGET_BUCKET"]

    # Generate file content
    timestamp = datetime.utcnow().strftime("%Y-%m-%d_%H-%M-%S")
    file_content = f"Hello from Lambda!\nTimestamp: {timestamp}\n"

    # File name to upload
    file_name = f"lambda_output_{timestamp}.txt"

    try:
        # Upload file content to S3
        s3_client.put_object(
            Bucket=bucket_name,
            Key=file_name,
            Body=file_content.encode("utf-8")
        )

        return {
            "statusCode": 200,
            "body": f"File {file_name} successfully uploaded to {bucket_name}"
        }
    except Exception as e:
        return {
            "statusCode": 500,
            "body": f"Error uploading file: {str(e)}"
        }

Conclusion

By configuring the correct IAM role policies and S3 bucket policies, we can ensure the secure communication between the lambda in one AWS account and S3 bucket in a different AWS account.

This pattern is not limited to the Lambda function. It can be applied to other services like EC2, ECS tasks etc.