S3 cross region replication

The situation

• Data size: 1 PB (massive).
• Source: S3 bucket in us-west-1.
• Destination: S3 bucket in us-east-1.

• Constraints:

  • Already in AWS (data is in S3).
  • On-prem DC restrictions: can’t use Snowball.
  • It’s a one-time copy, not continuous replication.

---

1. aws s3 sync command

• Pros: Straightforward, flexible, and works across Regions.
• Cons: Requires significant time and compute (you need EC2 instances or similar to run it), and moving 1 PB over the public internet (or even across AWS backbone) will take a long time. Could be expensive in terms of data transfer costs.

1. Amazon S3 Batch Replication (S3 Replication with batch operations)

• Pros:

  • Runs inside AWS, no need to pull/push from external compute.
  • Leverages cross-Region replication features.
  • Scales automatically, optimized for huge datasets (PB scale).
  • You can configure one-time replication and then delete the rule after the copy finishes.

• Cons: Slightly more setup effort than s3 sync.

---

✅ Correct answer

• Best solution: S3 Batch Replication (one-time) — optimized for PB-scale cross-Region copies within AWS.
• Why not s3 sync: It’s slower, less scalable, and requires external compute resources.
• Why not S3TA: It only accelerates internet uploads/downloads, not bucket-to-bucket copies.