Uploading Files Using Pre-Signed URLs to a Specific Storage Class

Here’s a step-by-step guide on how to implement file uploads using pre-signed URLs to a specific storage class, specifically with AWS S3. I’ll cover how to generate a pre-signed URL in Python and how to use it in Postman.

Architecture:

Create an IAM User:

• Sign in to the AWS Management Console.
• Navigate to IAM (Identity and Access Management):
• Open the IAM Console.
• Create a New User:
• Click on Users in the sidebar.
• Click the Add user button.
• Enter a user name (e.g., s3-uploader).
• Select Programmatic access for the access type to generate an access key ID and secret access key.
• Click Next: Permissions.

Create an S3 Bucket:

• Navigate to S3:
• Open the S3 Console.
• Create a New Bucket:
• Click on Create bucket.
• Enter a unique bucket name (e.g., data-from-resign).
• Choose a region.
• Configure options as needed (default settings are usually sufficient for this example).
• Click Create bucket.

[ Good Read: Comparison between Mydumper, mysqldump, xtrabackup]

You can check more info about: Pre-Signed URLs to a Specific Storage Class
.

• DevOps Services.

• Containerization Tools.

• Cloud Platform Engineering Services.

• DevOps Consultant.

Edit Cross-origin resource sharing (CORS):

[

{

    "AllowedHeaders": [

        "*"

    ],

    "AllowedMethods": [

        "PUT"

    ],

    "AllowedOrigins": [

        "https://example.com"

    ],

    "ExposeHeaders": []

}

]

Attach a Custom Policy to the User:

• Create a Custom Policy:
• In the IAM Console, go to Policies.
• Click Create policy.
• Select the JSON tab and enter a policy that grants permission to upload files to your specific bucket. For example:

{

    "Version": "2012-10-17",

    "Statement": [

        {

            "Sid": "VisualEditor0",

            "Effect": "Allow",

            "Action": "s3:PutObject",

            "Resource": "arn:aws:s3:::data-from-presign/*"

        }

    ]

}

- Click Next: Tags (optional) and then Next: Review.
- Provide a name (e.g., S3UploadPolicy) and description.
- Click Create policy.

## Attach the Policy to the User:

- Go to Users and select the user you created (s3-uploader).
- Click the Permissions tab.
- Click Add Permissions.
- Select Attach policies directly.
- Search for and select the policy you created (S3UploadPolicy).
- Click Next: Review and then Add permissions.

## Generate Programmatic Access Credentials:

- Get Access Keys
- Go to Users and select the user (s3-uploader).
- Click the Security credentials tab.
- Under Access keys, click Create access key.
- Download the CSV file containing the Access key ID and Secret access key or copy them. These are needed for programmatic access.

## Generate a Pre-Signed URL:
Using the AWS SDK (Boto3 for Python), generate a pre-signed URL. Here’s a Python script to do this:

import boto3
import botocore

# Assuming your S3 bucket name and image file name
ACCESS_KEY = 'access_key'
SECRET_ACCESS_KEY = 'secret_key'
BUCKET_NAME = 'data-from-presign'
OBJECT_KEY = 'image.png'
STORAGE_CLASS =  'ONEZONE_IA'

# Initialize a session using the AWS SDK for Python (Boto3)
session = boto3.Session(
    aws_access_key_id=ACCESS_KEY,
    aws_secret_access_key=SECRET_ACCESS_KEY,
    region_name='ap-south-1'  # Specify the region where your bucket is located
)