How to Mount an S3 Bucket into an ECS Container

Bira — Sat, 26 Aug 2023 02:57:05 +0000

Introduction:

In this blog post, we will show you how to mount an S3 bucket into an ECS container. This allows you to access the files in the S3 bucket from within the container.

Steps:

Create an S3 bucket.
Create an IAM role that allows the ECS container to access the S3 bucket.
Create an ECS task definition that mounts the S3 bucket into the container.
Deploy the ECS task definition.

Detailed Steps:

To create an S3 bucket, you can use the AWS Console, the AWS CLI, or the AWS SDKs.
To create an IAM role, you can use the AWS Console, the AWS CLI, or the AWS SDKs. The IAM role must have the following permissions:

    - s3:ListBucket
    - s3:GetObject
    - s3:PutObject

To create an ECS task definition, you can use the AWS Console, the AWS CLI, or the AWS SDKs. The task definition must specify the following:

    - The container image that you want to use.
    - The mount point for the S3 bucket.
    - The IAM role that you created in step 2.

To deploy the ECS task definition, you can use the AWS Console, the AWS CLI, or the AWS SDKs.

Example:

Here is an example of an ECS task definition that mounts an S3 bucket:

{
  "taskDefinition": {
    "family": "my-task-family",
    "containerDefinitions": [
      {
        "name": "my-container",
        "image": "my-container-image",
        "mountPoints": [
          {
            "containerPath": "/data",
            "sourceVolume": "my-s3-volume",
            "readOnly": true
          }
        ]
      }
    ],
    "volumes": [
      {
        "name": "my-s3-volume",
        "host": {
          "sourcePath": "/mnt/my-s3-bucket"
        },
        "dockerVolumeConfiguration": {
          "driver": "s3fs",
          "options": {
            "s3Url": "https://s3.amazonaws.com/my-bucket",
            "accessKeyId": "my-access-key-id",
            "secretAccessKey": "my-secret-access-key"
          }
        }
      }
    ]
  }
}

In this example, the container is named my-container and it is using the image my-container-image. The S3 bucket is mounted at the /data path in the container. The S3 bucket is also configured with the IAM role that you created in step 2.

Conclusion:

In this blog post, we showed you how to mount an S3 bucket into an ECS container. This allows you to access the files in the S3 bucket from within the container.

I hope this helps!

Inserting test data into DynamoDB local

Bira — Fri, 18 Aug 2023 03:18:49 +0000

DynamoDB is a fully managed NoSQL database service that offers high performance, scalability, and durability. It is a popular choice for storing large amounts of data that need to be accessed quickly.

One way to insert data into DynamoDB is to use a CSV file. This can be a convenient way to load data into DynamoDB, especially if you have a large amount of data to load.

In this blog post, we will show you how to insert test data into DynamoDB local using Python and CSV.

Prerequisites

Before you begin, you will need the following:

A Python development environment
The boto3 library
A CSV file with your test data

Step 1: Create a DynamoDB Local instance

To start, you need to create a DynamoDB Local instance. This can be done by following the instructions in the DynamoDB Local documentation: https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/DynamoDBLocal.html.

Step 2: Create a DynamoDB table

Once you have created a DynamoDB Local instance, you need to create a DynamoDB table. This can be done by using the DynamoDb Admin (https://www.npmjs.com/package/dynamodb-admin) or NoSQL Workbench (https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/workbench.html)

Step 3: Generate Data using Excel.

You can easily regenerate requisite data from excel or Google Sheets and download as CSV files and then import into your DynamoDB

Step 4: Load the CSV file into DynamoDB

Now that you have created a DynamoDB table, you can load the CSV file into DynamoDB. The following code loads the CSV file test_data.csv into the test_table table:

import boto3
import csv

# Create a DynamoDB client
dynamodb = boto3.resource('dynamodb', endpoint_url='http://localhost:8000')

# Open the CSV file
with open('./csv/user.csv', 'r') as csvfile:

    # Create a reader object
    reader = csv.reader(csvfile, delimiter=',')

    # Get the header
    header = next(reader, None)

    # Iterate over the rows in the CSV file
    for row in reader:

        # Create a dictionary to represent the item
        item = {}
        for i, column in enumerate(header):
            item[column] = row[i]
            print(column, row[i])

        # Put the item in the DynamoDB table
        dynamodb.Table('test_table').put_item(Item=item)

Step 5: Verify the data in DynamoDB

Once you have loaded the CSV file into DynamoDB, you can verify the data by querying the test_table table using the tools mentioned in the step 2.

Conclusion

In this blog post, we showed you how to insert test data into DynamoDB local using Python and CSV. We hope this helps you get started with DynamoDB.

For more information on DynamoDB, please visit the DynamoDB documentation: https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/.

DEV Community: Bira