Boost Your Development Efficiency! Simulate S3 with a Custom Amazon S3 Mock Application

💡Original japanese post is here.
https://zenn.dev/tttol/articles/13032ef69d8333

Introduction

I've developed a mock application for Amazon S3 called "MOS3" (pronounced "mɒsˈθri"). Similar to LocalStack, MOS3 is an S3-like application that runs in a local environment.

MOS3 is a GUI application that allows you to manage files directly from your browser. You can upload files without having to use CLI commands like aws s3 cp.

Moreover, MOS3 can handle S3 requests from the AWS SDK, making it possible to simulate S3 operations locally.

You can find the source code here:
https://github.com/tttol/mos3

Background

I am an application engineer, and I frequently develop web applications using Java and other technologies.

Many of the applications I create use S3 as external storage, and it's often necessary to connect to S3 even for local development. While I sometimes use a development AWS account with actual S3 buckets, setting up an account can be cumbersome for small-scale applications. In such cases, I've used LocalStack as a mock for S3.

LocalStack is an excellent and convenient open-source software, but I have noticed some challenges:

• It's primarily operated via CLI, making GUI-based file management difficult.
  • There is a GUI feature, but it didn't meet my requirements.
• LocalStack provides resources beyond S3, but I rarely need anything other than S3, making it overly complex for my needs.

Given these challenges, I thought, "Why not create something that perfectly fits my needs?" And thus, MOS3 was born.

Features

Use as a Local Replacement for S3

MOS3 runs at http://localhost:33333/s3.
※ For detailed installation instructions, please refer to the README.

MOS3 can be used as a standalone S3-like local file server, but it also accepts requests from the AWS SDK. For example, if you send a request using the getObjects method from the AWS SDK for Java, MOS3 will respond with the specified key's object.

To call MOS3 from the AWS SDK for Java, you need to set the endpoint of the S3Client to http://localhost:3333. Here’s a code sample:

S3Client s3 = S3Client.builder()
        .region(region)
        .endpointOverride(new URI("http://localhost:3333"))
        .build();

Caution

When specifying localhost as the endpoint, you may need to enable path-style access for it to work correctly.

S3Configuration s3Configuration = S3Configuration.builder()
        .pathStyleAccessEnabled(true) // enable path style access
        .build();

S3Client s3 = S3Client.builder()
        .region(region)
        .endpointOverride(new URI("http://localhost:3333"))
        .serviceConfiguration(s3Configuration)
        .build();

File Management via Browser

MOS3 allows you to manage files and directories through your browser.

You can upload files by clicking the "New File" button.

You can create new directories by clicking the "New Dir" button.

You can download files by clicking on them.

You can delete files or directories by clicking the trash can icon on the right.

Linking with Your Local PC Directory

docker run -p 3333:3333 -v ./upload:/app/upload -it --rm tttol/mos3:latest

The above is the command to run MOS3. (Quoted from the README)
MOS3 runs on a Docker container.

The key point here is the use of the -v option to mount the volume. Files uploaded to MOS3 are internally stored in the /app/upload directory. By mounting it with -v ./upload:/app/upload during container startup, files on MOS3 are synchronized with the ./upload directory. Therefore, you can manage MOS3 files directly from Mac Finder or Windows Explorer.

Future Features

MOS3 is still a work in progress. Below are some of the features I plan to implement in the future.

Handling Requests from AWS SDKs Other Than Java

I often use Java, so I have mainly implemented support for requests from the AWS SDK for Java. However, the behavior with other SDKs is not guaranteed, as some work and others do not. I plan to extend support to other SDKs in the future, but this remains a future task.

Handling AWS CLI Requests

Handling AWS CLI requests for commands like ls, mb, cp, and rm is still a work in progress. While ls and cp are minimally supported, handling other commands is a future task.

Conclusion

MOS3 is open-source software (MIT License). Issues and pull requests are welcome.
I will continue to improve MOS3 to make it an even more user-friendly application.