How to test AWS services locally with Moto library?

Currently, I'm working on coding Python services that work with AWS S3 buckets. However, for unit testing parts of these services, I don't want to create a separate AWS S3 bucket just for my unit tests.

It would require managing AWS credentials and it would take a bunch of time to set up for multiple team members. What if there is a way to Mock AWS S3 service on local that does not require any steps from the above?

That's how I found Moto library for Python.

• Run the example below on Replit.

import os
from moto import mock_s3
import boto3
import pprint

BUCKET_NAME = "test_bucket"

os.environ['AWS_ACCESS_KEY_ID'] = 'testing'
os.environ['AWS_SECRET_ACCESS_KEY'] = 'testing'
os.environ['AWS_SECURITY_TOKEN'] = 'testing'
os.environ['AWS_SESSION_TOKEN'] = 'testing'


with mock_s3():
    client = boto3.client('s3', region_name='us-east-1')
    client.create_bucket(Bucket=BUCKET_NAME)

    def write_value_to_s3(key, value):
        s3 = boto3.client('s3')
        key = key.split("/")
        bucket_name = key[0]
        key_in_the_bucket = "/".join(key[1:])
        s3.put_object(Bucket=bucket_name, Key=key_in_the_bucket, Body=value)


    path_1 = f"{BUCKET_NAME}/a/d"
    path_2 = f"{BUCKET_NAME}/b/c"

    write_value_to_s3(path_1, "")
    write_value_to_s3(path_2, "")

    client = boto3.client("s3")

    object_page = client.list_objects_v2(
        Bucket=BUCKET_NAME,
        Prefix="a",
        Delimiter="/"
    )

    pprint.pprint(object_page)

    object_page = client.list_objects_v2(
        Bucket=BUCKET_NAME,
        Prefix="",
        Delimiter="/"
    )

    pprint.pprint(object_page)

    path_3 = f"{BUCKET_NAME}/a"
    write_value_to_s3(path_3, "")

    object_page = client.list_objects_v2(
        Bucket=BUCKET_NAME,
        Prefix="",
        Delimiter="/"
    )

    pprint.pprint(object_page)

    path_4 = f"{BUCKET_NAME}/a/b"
    write_value_to_s3(path_4, "")

    object_page = client.list_objects_v2(
        Bucket=BUCKET_NAME,
        Prefix="a/",
        Delimiter="/"
    )

    pprint.pprint(object_page)


    path_5 = f"{BUCKET_NAME}/a/b/c"
    write_value_to_s3(path_5, "")

    object_page = client.list_objects_v2(
        Bucket=BUCKET_NAME,
        Prefix="a/b",
        Delimiter="/"
    )

    pprint.pprint(object_page)

And here you go 💪

Comments

[Shwetabh Shekhar]

I have been using moto for mocking a lot of AWS services and it works pretty well. Thanks for sharing this!