Managing S3 Bucket using Lifecycle Configuration

Do you remember that I migrated my workload to AWS? I migrated the workload to AWS Lambda and stored the data into Amazon S3. However, the size of unused objects increases. I want to reduce the size to decrease the cost. As a developer who loves to code, I will think about writing the code for the cleaning-up process. But, it will give me more overhead to manage the code and ensure no bugs. Well, it is sure a lot of work! Let's find another solution! We can try Lifecycle Configuration. What is it? You can change the object class or delete the object based on your defined rules. Imagine you want to move the object class from standard into infrequent access for data whose last access was 30 days ago. You can use Lifecycle Configuration. For more information about Lifecycle Configuration, please visit this site.

What should we add?

I want to manage the Lifecycle Configuration and S3 in my previous Terraform codes. If you want to know more about my repository, please visit here. Please also read my post when creating the repository.

My First Experience Learning Terraform and Amazon ECS + AWS Fargate

Bervianto Leo Pratama for AWS Community Builders ・ Apr 10 '22

#aws #terraform #beginners #devjournal

I will use Expiring objects to remove the files. Please see here to explore more.

I only need to add the aws_s3_bucket and aws_s3_bucket_lifecycle_configuration resources. You may look at the code here.

variable "bucket_name" {
  type     = string
  nullable = false
}


resource "aws_s3_bucket" "cert_bucket" {
  provider = aws.ap-southeast-3

  bucket = var.bucket_name

}

resource "aws_s3_bucket_lifecycle_configuration" "cert_bucket_lifecycle" {
  provider = aws.ap-southeast-3

  bucket = aws_s3_bucket.cert_bucket.id

  rule {
    id = "deletion-rule-1"

    filter {
      prefix = "merged/"
    }

    expiration {
      days = 30
    }

    # ... other transition/expiration actions ...

    status = "Enabled"
  }
}

Please check the PR too.

feat: add deletion policy #95

berviantoleo posted on Nov 19, 2023

View on GitHub

Notes

• Please take note that those expired objects won't be deleted immediately. It might delay the deletion. If you check the expiration date property, you will see when the object will be deleted.

• I have an existing bucket. So, I'll need to import the state. You may use this command: terraform import aws_s3_bucket.cert_bucket tutorial1-berv. Don't forget to rename the resource id. :)

• Please take note that my bucket is non-versioned. You need to add noncurrent_version_expiration if you use a versioning-enabled bucket.

---

It's a cool feature! Anyway, Which features of S3 do you like? Feel free to share here.

Update (21 November 2023):

• The objects were deleted.

Thank you

Thank you for reading. Please give any feedback. I will be happy to have any valuable comments from you.