Delete Multiple Versioning-Enabled S3 Buckets by boto3

Have you ever tried to delete multiple non-empty S3 buckets with versioning-enabled on AWS Console? How irritating it is to go through each of every S3 bucket? To delete the bucket on AWS console, you have to:

1. delete every objects in the bucket
2. empty the bucket by typing the bucket name
3. deleting the bucket by typing 'delete'

You have to go through 2 confirmations. Let's say you have up to 20 buckets to delete, you have to go through total up to 40 confirmations to make by going through each of every bucket, which is totally a hell. And hence, I choose to do it programmatically. I will walk you through this post to write a simple script to delete multiple buckets.

Prerequisite

1. AWS CLI installed. Download from here.
2. Configure AWS credentialds in AWS CLI. Can refer this document to learn how to configure your AWS CLI. Make sure you have the access to list buckets, delete versioned objects in the buckets and buckets.
3. Have your Git installed and enabled on your local machine.
4. pip installed. pip is a package manager for Python. Just like npm for NodeJS. Refer to this document to install pip.

Note that in this tutorial, I am using my administration account. It is always recommended to provide the least privilege permissions to the user.

Walkthrough

In this project, we will be using:

• boto3: AWS SDK in Python
• inquirer: To ease the process of asking end user questions, parsing, validating answers, managing hierarchical prompts and providing error feedback

1. Clone the repository and go to s3 directory in terminal.

git clone https://github.com/andylim0221/aws_scripts.git

cd aws_scripts/s3

1. Install the necessary packages such as boto3 and inquirer.

pip install -r requirements.txt

1. Run the script with your profile or default profile.

python delete_bucket.py # default profile

# or with profile any
python delete_bucket.py --profile any 

And you will see the similar lists as below :

You can choose the buckets you want to delete by pressing space bar and navigating by up arrow and down arrow button. The selected options will be shown in yellow font color. By clicking space bar again on the selected buckets will remove it from the options.

Once you have finished selecting, press Enter button and go to next step.

You will see the question to confirm if you want to delete the buckets you chose earlier. By typing delete, the action is confirmed and it will proceed to delete the buckets.

And now you just have to wait for the deletion to happen and in the meanwhile, have a good cup of coffee.

When the script is done, check your buckets by listing your buckets to see if the selected buckets are deleted.

aws s3 ls #list all buckets

Note that, if you want to cancel the action or reselect the buckets, you have to kill the script by Ctrl-C.

Troubleshooting

When you list the buckets, sometimes the deleted buckets are still there, the reason why it is still in the list because the list is not updated immediately. You can check in your AWS Console and you will see the deleted buckets are shown as below :

Give it up to 24 hours, the bucket should just disappear from the list.

Conclusion

In this one simple script, I can :

• remove objects
• remove versioned objects
• empty objects
• delete buckets

Of course this can be improved in security or efficiency, but it should be doing well for the job. Let me know in the comments if you found bugs or issues from this solution.

Happy coding! 💻

Comments

[IanB]

Very nice, super easy, Thanks!