How to Recover an Entire Folder in S3 After Accidental Deletion

Accidentally deleting an entire folder (prefix) in Amazon S3 can be a heart-stopping moment. Fortunately, if versioning is enabled on your S3 bucket, there's a straightforward way to recover your data. This blog post will guide you through restoring your deleted folder using a simple shell script.

Prerequisite

To successfully recover your deleted folder, you must enable versioning on the target bucket. This is a critical prerequisite. If versioning was not enabled at the time of deletion, recovery isn't possible this time. However, consider enabling versioning for future protection.

For more information on enabling versioning, refer to the AWS documentation on S3 versioning.

Recovery Method

Here is a step-by-step guide on how to recover your deleted folder:

1. Ensure AWS CLI is installed: You'll need the AWS Command Line Interface (CLI) to execute the recovery script. You can install it here.

2. Create the recovery script: Save the following script as delete_deletemarker_of_s3.sh.
   

   #!/bin/bash
   
   # Get the time one hour ago in ISO 8601 format
   one_hour_ago=$(date -u -d '1 hour ago' +'%Y-%m-%dT%H:%M:%SZ')
   
   # List all the latest delete markers
   delete_markers=$(aws s3api list-object-versions --bucket my-bucket --prefix my-folder/ --query 'DeleteMarkers[?IsLatest==`true`].[Key, VersionId, LastModified]' --output text)
   
   # Delete only the delete markers set within the last hour
   while IFS=$'\t' read -r key version_id last_modified; do
     if [[ "$last_modified" > "$one_hour_ago" ]]; then
       echo "Deleting delete marker for $key with version ID $version_id, set at $last_modified"
       aws s3api delete-object --bucket my-bucket --key "$key" --version-id "$version_id"
     fi
   done <<< "$delete_markers"
   

3. Execute the recovery script: Run the following command to execute the script.
   

   bash delete_deletemarker_of_s3.sh
   

Explanation of the Script

• Get the time one hour ago: The script calculates the time one hour prior to the current time in ISO 8601 format.
  

  one_hour_ago=$(date -u -d '1 hour ago' +'%Y-%m-%dT%H:%M:%SZ')
  

• List latest delete markers: It lists all the latest delete markers for the specified bucket and prefix.
  

  delete_markers=$(aws s3api list-object-versions --bucket my-bucket --prefix my-folder/ --query 'DeleteMarkers[?IsLatest==`true`].[Key, VersionId, LastModified]' --output text)
  

• Delete recent delete markers: The script iterates through the delete markers and deletes those that were set within the last hour.
  

  while IFS=$'\t' read -r key version_id last_modified; do
    if [[ "$last_modified" > "$one_hour_ago" ]]; then
      echo "Deleting delete marker for $key with version ID $version_id, set at $last_modified"
      aws s3api delete-object --bucket my-bucket --key "$key" --version-id "$version_id"
    fi
  done <<< "$delete_markers"
  

By deleting the delete markers, the script cancels the deletion, effectively restoring the deleted objects. The condition to check if the delete marker was set within the last hour ensures that only recent accidental deletions are recovered, preventing the restoration of objects that were intentionally deleted in the past.

Running the Script in Amazon CloudShell

For ease of use, you can execute this script in Amazon CloudShell:

1. Start Amazon CloudShell from the AWS console.
2. Upload the delete_deletemarker_of_s3.sh file to CloudShell.
3. Run the script using the command provided above.

Final Notes

Make sure to replace my-bucket with your actual bucket name and my-folder/ with your actual folder name (prefix) in the script.

By following this guide, you can quickly recover from accidental deletions in S3, provided you have versioning enabled. If not, take this opportunity to enable versioning on your important buckets to safeguard against future accidental deletions.