Unlock the Power of Enhanced Filtering with S3 Batch Operations

#aws #s3 #batch

What is Amazon S3 Batch Operations?

It allows you to perform batch operations on large sets of objects in Amazon S3.
You can select objects based on prefixes, tags, and metadata.
Actions that can be performed include copying, archiving, applying access controls, and adding object tags.
The goal is to simplify and automate repetitive tasks.

Any news on Batch Operations?

You can now manage objects within an entire bucket, based on prefix, suffix, creation date, or storage class.
You can quickly apply the operation to all the matching objects.

Benefit of Batch Operations

You can easily perform one-time or batch workloads:
Copying objects between staging and production buckets
Invoking an AWS Lambda function to convert file types
Restoring archived backups from S3 Glacier storage

Create a replication rule

Set the filtering: file name prefix

Create a CSV inventory configuration for batch operation

Manifest.csv contains all objects' names for the batch operation

generative-ai-cities,aws_team_1.png
generative-ai-cities,aws_team_2.png
generative-ai-cities,aws_team_3.png
generative-ai-cities,aws_team_1_large.png
generative-ai-cities,aws_team_2_large.png
generative-ai-cities,aws_team_3_large.png

Create an S3 batch operation using a CSV inventory configuration

Select "copy" to replicate objects to the destination bucket.

Create an S3 batch operation using an S3 replication configuration

Filter objects by creation date
You can specify filters to reduce the scope of replicated objects.
These filters work in conjunction with existing filters in your replication configuration.
If no filters are specified, all objects defined by the replication configuration will be replicated.