Upload files in your web app to an AWS S3 bucket

Hi Sparta!

In this article I will share with you a file storage module that can help you store your user's files (profil's pictures, PDF documents...) in your app with a scalable architecture.

We could have stored all those data to a custom server, but AWS S3 buckets are giving us the scalability and I'll show you here how to use them.

On my side, I used this module on top of the React/Node/MySQL starter. This starter has already been presented to you in this article.

What does it bring?

• Open source code
• Documentation and "Quick Start"
• Complete integration of the process in your React / NestJS (it can be easily adapted to a node backend)
• AWS S3 configurations
• 5 hours of work saved :D

Prerequisite

By getting the web starter I was mentioning above, you'll be able to plug this file upload module directly on top of it and have a functional project with an authentication and a file storage in less than 20 minutes.

Note that using the starter is not mandatory, you can also use the module as standalone. The integration won't be as easy as with the starter, but it should still be simple to integrate it in your already created project :)

The starter is available here.
The module is available here.

How does it work?

The S3 buckets is a public cloud storage resource available in Amazon Web Services' (AWS). You can manually upload files into the bucket and you can, of course, do it programmatically. Here is the high picture workflow of the process:

1. Frontend asks the backend for an URL where it can saves a given file.
2. Backend ask with it's credentials AWS to get signed URL that can be used to upload a file, and returns its to the frontend.
3. Frontend uses the URL to save the file to the bucket.
4. (optional) Your frontend can ask your backend to save in database the URL where your image/file has been uploaded in order to display it in your app :)

Setup the file storage process

• Create your S3 bucket here.
• Get your credentials. Go to IAM console, from the navigation menu click on Users, select your user or create a new user, create an access key, get the Key ID and the Key secret and keep them in save place, we'll see them soon ;)
• Grant the permissions for your users to modify your bucket (grant right AmazonS3FullAccess). IAM > Create autorisation > S3
• Add the CORS policy in S3 > autorisations

[
    {
        "AllowedHeaders": [
            "*"
        ],
        "AllowedMethods": [
            "PUT"
        ],
        "AllowedOrigins": [
            "*"
        ],
        "ExposeHeaders": [],
        "MaxAgeSeconds": 3000
    }
]

• Authorize public access to your S3 bucket
• Add your environment variables in the backend using the credentials your get during the second step:

s3: {
        url: "https://[bucket].s3.eu-west-3.amazonaws.com/userUploads",
        bucket: "[bucket]",
        region: "[region]",
        host: "s3.eu-west-3.amazonaws.com",
        accessKeyId: "XXXXXXXXXXXXXXXXXX",
        secretAccessKey: "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX"
    }

• Your backend can now ask AWS for a signed URL (and give it back to the frontend):

npm install aws-sdk@2.799.0

    async getS3Url(fileExtension: string): Promise<string> {
        let signedUrl = await this.s3.getSignedUrlPromise('putObject', {
            Bucket: this.configService.environment.s3.bucket,
            Key: `pictures/${uuid.v4()}.${fileExtension}`,
            ContentType: 'application/octet-stream',
            ACL: 'public-read',
            Expires: 604800
        });
        return signedUrl;
    }

• Finally, the frontend can use the signedUrl to store a file:

    ...

    const headersContent = {
        "Content-Type": file.type,
        Accept: "application/json"
    };
    const headers = new Headers(headersContent);
    const response = await fetch(
        new Request(signedUrl, {
            method: "PUT",
            body: file,
            headers
        })
    );

The entire tutorial and a step by step integration on the starter with an example screen is available here.

Conclusion

I hope this module will help you saving your some time while trying save user's files in your project.
If you have any question, I'll be present as usual in the comment section !

Links:

• The platform sharing the starter and it's modules : Fast Modular Project
• Module "File uploads to AWS S3" repository here.

Do not hesitate to pin and like if you appreciated the article ❤️