DEV Community

Rohit Shrivastava for AWS Community Builders

Posted on • Originally published at Medium

#2 Shorticle: How to read data from S3 bucket using python

To read data from an Amazon S3 bucket using Python, you can utilize the boto3 library, which is the official AWS SDK for Python. Follow the steps below to get started:
Install the boto3 library by running the following command in your Python environment:

pip install boto3
Enter fullscreen mode Exit fullscreen mode

Import the required modules in your Python script:

import boto3
Enter fullscreen mode Exit fullscreen mode
  1. Create a boto3 S3 client or resource object by providing your AWS access key ID and secret access key (or by using an IAM role with appropriate permissions). For example:
s3_client = boto3.client('s3', aws_access_key_id='YOUR_ACCESS_KEY', aws_secret_access_key='YOUR_SECRET_KEY')
Enter fullscreen mode Exit fullscreen mode
  1. Use the client or resource object to interact with your S3 bucket. To read an object (file) from the bucket, you can use the get_object() method:
response = s3_client.get_object(Bucket='your-bucket-name', Key='your-object-key')
Enter fullscreen mode Exit fullscreen mode
  1. In the above code snippet, replace 'your-bucket-name' with the name of your S3 bucket, and 'your-object-key' with the key (path) of the object you want to read.

  2. Access the content of the object through the response object. For example, to read the object's content as text:

object_content = response['Body'].read().decode('utf-8')
Enter fullscreen mode Exit fullscreen mode
  1. You can also read the content as bytes by omitting the decode('utf-8') part. Process or use the data as needed. Here’s a complete example that demonstrates reading a text file from an S3 bucket:
import boto3
# Create an S3 client
s3_client = boto3.client(‘s3’, aws_access_key_id=’YOUR_ACCESS_KEY’, aws_secret_access_key=’YOUR_SECRET_KEY’)
# Read an object from the bucket
response = s3_client.get_object(Bucket=’your-bucket-name’, Key=’your-object-key’)
# Read the object’s content as text
object_content = response[‘Body’].read().decode(‘utf-8’)
# Process or use the content as needed
print(object_content)
Enter fullscreen mode Exit fullscreen mode

Remember to replace 'YOUR_ACCESS_KEY', 'YOUR_SECRET_KEY', 'your-bucket-name', and 'your-object-key' with your actual credentials and bucket/object information.

Note: It’s recommended to follow AWS security best practices and avoid hardcoding your access keys in your code. Instead, consider using AWS credentials stored in a secure manner, such as environment variables, AWS profile configuration, or EC2 instance roles, depending on your execution environment.

Top comments (0)