DEV Community

Cover image for S3 Lifecycle Rules, Step by Step: Move Old Data to Cheaper Storage Without Surprise Fees
CloudSkill Consulting LLC
CloudSkill Consulting LLC

Posted on

S3 Lifecycle Rules, Step by Step: Move Old Data to Cheaper Storage Without Surprise Fees

#aws #s3 #finops #cloud

Most S3 buckets start the same way. Someone creates a bucket, an app writes files into it, and nobody looks at it again. Months later the bucket holds years of logs, old exports and copies of files nobody remembers. Every byte sits in the default storage class, S3 Standard, and every byte is billed at that rate.

S3 Lifecycle rules fix this. A lifecycle rule tells Amazon S3 to move objects to a cheaper storage class after a number of days, or to delete them when they are no longer needed. You write the rule once, and S3 applies it every day.

The catch is that cheaper classes come with conditions. Some charge for a minimum number of days. Some charge to read data back. Some charge per object moved. A careless rule can cost more than it saves. This guide from CloudSkill Consulting LLC walks through S3 lifecycle rules one step at a time, so you can lower storage cost without surprise fees.

Step 1: Learn the storage classes in plain words

A storage class is a price plan for an object. Each object in S3 has one. The data is the same; what changes is the price to store it, the price to read it, and how fast you can get it back.

Here are the classes this guide uses, in order from "used often" to "almost never used":

  • S3 Standard (STANDARD): the default. No minimum storage time and no retrieval fee. Best for data you read often.
  • S3 Standard-IA (STANDARD_IA): "IA" means infrequent access. Cheaper to store, but there is a fee each time you read data, a 30-day minimum storage charge, and a 128 KB minimum billable size (see the S3 pricing and storage-class pages linked in Sources).
  • S3 Glacier Instant Retrieval (GLACIER_IR): for data read about once a quarter, still with millisecond access. It has a 90-day minimum.
  • S3 Glacier Flexible Retrieval (GLACIER): archive storage. You must restore an object before you can read it, and that takes minutes to hours. It has a 90-day minimum.
  • S3 Glacier Deep Archive (DEEP_ARCHIVE): the lowest-cost archive, with restores measured in hours. It has a 180-day minimum.

There is also S3 Intelligent-Tiering (INTELLIGENT_TIERING). It watches how each object is used and moves it between tiers for you, for a small monitoring fee per object. It has no retrieval fees. It is a good choice when you honestly don't know how often data is read.

Diagram of S3 storage classes as a staircase from Standard down to Deep Archive, each step labelled with its minimum storage duration
Figure 1: Each step down is cheaper to store, but has a longer minimum stay and slower or paid reads.

Step 2: Find out what is in the bucket before you write a rule

Never write a lifecycle rule from memory. Look at the data first. Two questions matter:

  1. How big are the objects? Small objects behave differently. By default, S3 Lifecycle does not transition objects smaller than 128 KB to any storage class. AWS explains why: you pay a transition request for each object, so for small objects the request cost can outweigh the savings.
  2. How often is each prefix read? A prefix is the start of an object key, like logs/ or exports/2025/. Logs written once and almost never read are good candidates. Product images that a website loads all day are not.

AWS gives you tools for this. S3 Storage Class Analysis watches access patterns on a bucket or prefix and helps you decide when to move data from Standard to Standard-IA. S3 Storage Lens shows storage use across buckets. Turn one of them on and give it time before you choose your day counts.

A good habit: write down each prefix, who owns it, how often it is read, and how long it must be kept. "How long it must be kept" is a business question, not a technical one. Ask the data owner.

Step 3: Understand the three rules that cause surprise fees

These three rules are where most lifecycle mistakes come from.

Rule A: minimum storage duration. Standard-IA has a 30-day minimum, Glacier Instant Retrieval and Glacier Flexible Retrieval have 90 days, and Deep Archive has 180 days (minimum storage charges, per the S3 pricing and storage-class pages). If you delete, overwrite or move an object out before that time, you are charged for the rest of the minimum. A rule that moves logs to Glacier Flexible Retrieval and then deletes them 30 days later still pays for 90 days.

Rule B: small objects. Standard-IA and Glacier Instant Retrieval bill each object as at least 128 KB. Objects archived to Glacier Flexible Retrieval or Deep Archive also carry about 40 KB of extra metadata each: 8 KB billed at the Standard rate and 32 KB at the archive rate. Millions of tiny files can cost more after archiving than before.

Rule C: reads cost money and time. IA and Glacier classes charge per GB retrieved. Glacier Flexible Retrieval and Deep Archive objects are not readable until you restore them, and while restored you pay for both the archive copy and a temporary Standard copy.

Diagram showing three warning cards: minimum storage duration, small-object overhead, and retrieval fees, each with a short example
Figure 2: Check every lifecycle rule against these three conditions before you turn it on.

Step 4: Write a first rule that is hard to get wrong

Start with one prefix that is clearly "write once, read rarely", such as application logs. Here is a full lifecycle configuration in the JSON format the AWS CLI expects. It has three rules.

{
  "Rules": [
    {
      "ID": "logs-tier-then-expire",
      "Filter": { "Prefix": "logs/" },
      "Status": "Enabled",
      "Transitions": [
        { "Days": 30, "StorageClass": "STANDARD_IA" },
        { "Days": 120, "StorageClass": "GLACIER" }
      ],
      "Expiration": { "Days": 400 }
    },
    {
      "ID": "old-versions-cleanup",
      "Filter": {},
      "Status": "Enabled",
      "NoncurrentVersionExpiration": {
        "NoncurrentDays": 30,
        "NewerNoncurrentVersions": 3
      }
    },
    {
      "ID": "abort-stuck-uploads",
      "Filter": {},
      "Status": "Enabled",
      "AbortIncompleteMultipartUpload": { "DaysAfterInitiation": 7 }
    }
  ]
}
Enter fullscreen mode Exit fullscreen mode

Here is what each part does, in plain words:

  • logs-tier-then-expire applies only to keys starting with logs/. After 30 days the objects move to Standard-IA. After 120 days they move to Glacier Flexible Retrieval. After 400 days they expire. The gaps are chosen on purpose: 90 days in Standard-IA is more than its 30-day minimum, and 280 days in Glacier Flexible Retrieval is more than its 90-day minimum. No early-deletion charge applies. Since July 2026 you can move objects to Standard-IA from day 0. We wait 30 days because new logs are read most often in their first weeks.
  • old-versions-cleanup matters only if versioning is on. When you overwrite or delete an object in a versioned bucket, the old copy stays as a noncurrent version, and you keep paying for it. This rule permanently deletes noncurrent versions 30 days after they become noncurrent, but always keeps the 3 newest ones. Be sure that matches your recovery needs, because deleted versions can't be brought back.
  • abort-stuck-uploads cleans up multipart uploads that started but never finished. Their parts are stored and billed, yet they don't appear as normal objects. AWS notes that expiration rules don't remove incomplete multipart uploads; you need this separate action.

An empty Filter ({}) means "every object in the bucket".

S3 rounds lifecycle timing to the next midnight UTC. An object created at 10:30 AM UTC with a 3-day rule becomes eligible at midnight UTC four days later. Transitions to Glacier also run in the background, so the physical move can lag behind the date. You are still billed at the destination rate from the date the rule is met.

Diagram of one log object's timeline: day 0 in Standard, day 30 to Standard-IA, day 120 to Glacier Flexible Retrieval, day 400 expired
Figure 3 (example rule): One object's life under the logs/ rule, with each stay longer than that class's minimum.

Step 5: Apply it safely with the AWS CLI

First, save any rules that already exist. This step matters. The AWS CLI documentation says put-bucket-lifecycle-configuration replaces the whole existing configuration. Any rule you leave out of your file is gone.

# 1. Back up the current configuration (an error here usually means none exists yet)
aws s3api get-bucket-lifecycle-configuration \
  --bucket example-app-data > lifecycle-backup.json

# 2. Check your new file is valid JSON
jq . lifecycle.json

# 3. Apply the new configuration
aws s3api put-bucket-lifecycle-configuration \
  --bucket example-app-data \
  --lifecycle-configuration file://lifecycle.json

# 4. Read it back and compare
aws s3api get-bucket-lifecycle-configuration --bucket example-app-data
Enter fullscreen mode Exit fullscreen mode

If an older rule existed, merge it into lifecycle.json before step 3. A bucket has one lifecycle configuration with up to 1,000 rules.

To see stuck uploads before and after, list them:

aws s3api list-multipart-uploads --bucket example-app-data
Enter fullscreen mode Exit fullscreen mode

Step 6: Check the result, then widen slowly

Lifecycle rules don't act the moment you save them. Give it a few days, then check:

  1. Billing: in AWS Cost Explorer, group S3 cost by usage type. Storage cost for the moved classes should rise while Standard storage falls.
  2. Request and retrieval costs: watch for a jump in transition requests (many small objects) or in retrieval fees (data that is read more than you thought).
  3. Complaints: if someone needs an archived object, they will find it isn't instantly readable. That tells you the day count was too short for that prefix.

When the first prefix looks right, add the next one as a new rule. One prefix at a time keeps every change easy to undo: set its Status to Disabled and apply again.

Diagram of a four-step loop: measure the bucket, write one rule, apply with backup, check the bill, then repeat for the next prefix
Figure 4: Grow your lifecycle configuration one prefix at a time.

A short checklist

  • Know each prefix's size, read pattern and required retention.
  • Leave small objects in Standard or Intelligent-Tiering.
  • Make every stay longer than the class's minimum duration.
  • Add AbortIncompleteMultipartUpload to every bucket.
  • On versioned buckets, decide how many old versions to keep, then expire the rest.
  • Back up the current configuration before every put.

At CloudSkill Consulting LLC, we test lifecycle rules like this on a small prefix before applying them to a whole bucket. Storage tiers are often the first place a cloud bill can shrink. We do this kind of FinOps check for teams on AWS, Azure and Google Cloud. If a rule here behaves differently in your account, write to info@cloudskillconsulting.com.

Sources (official documentation, checked Oct 17, 2026)

Top comments (0)