ReactNative Large file upload using Amplify S3

Nikhil Walvekar — Sun, 10 Jan 2021 12:30:27 +0000

Currently while using Amplify library to upload a to AWS S3 we have to read that file into memory and pass it to Storage.put call. If the file which we are uploading is large for example a video, it causes memory issues and the ReactNative app crashes.
The Amplify Storage plugin supports Managed upload. It will divide the file into small chunks and upload them in batches.
But there is no support to read the file in small chunks.

The Amplify library supports custom plugins, using which we can connect with services other than AWS or add a wrapper to AWS services.

To support the uploading of large files using Amplify, I have used the same mechanism as a custom plugin.

NPM Library amplify-s3-chunk-upload

Usage

Install

npm i -s amplify-s3-chunk-upload

Configure Storage plugin in App.js (ReactNative)

import { StorageChunkUpload } from 'amplify-s3-chunk-upload';
import { Credentials } from '@aws-amplify/core';



// put following code after Amplify.configure

// Load StorageChunkUpload Plugin
const storagePlugin = new StorageChunkUpload({}, Credentials);
Storage.addPluggable(storagePlugin);
storagePlugin.configure(config);

File upload call


// get File stats
const { size } = await RNFS.stat(fileURI);

// here we are simulating an array of bytes
const fileObject = {
  // set the size
  size: size,

  // here we will read file as per bodyStart & bodyEnd, this will avoid reading complete file in the memory.
  slice: (bodyStart, bodyEnd) => {
    // Here in this sample code, we are using react-native-fs to read files.
    return RNFS.read(fileURI, bodyEnd - bodyStart, bodyStart, 'base64')
      .then((data) => Buffer.from(data, 'base64'))
      .catch((error) => {
        // Log error if required.
      });
    },
  };

  // Upload call, for parameters, refer to Amplify docs.
  const result = await Storage.put(`Your-file-name.mp4`, fileObject, {
    contentType: 'video/mp4',
    level: 'protected',
    provider: 'StorageChunkUpload',
  });

Since we are making standard Storage.put call and the underlying code also uses the same Amplify Library code, you can pass all other parameters such as progressCallback etc.

We have this in action for the last couple of days.
Please feel free to share any feedback.
NPM
GitHub

DEV Community: Nikhil Walvekar

ReactNative Large file upload using Amplify S3

Usage

Install

Configure Storage plugin in App.js (ReactNative)

File upload call