DEV Community

loading...

ReactNative Large file upload using Amplify S3

walvekarnikhil profile image Nikhil Walvekar ・2 min read

Currently while using Amplify library to upload a to AWS S3 we have to read that file into memory and pass it to Storage.put call. If the file which we are uploading is large for example a video, it causes memory issues and the ReactNative app crashes.
The Amplify Storage plugin supports Managed upload. It will divide the file into small chunks and upload them in batches.
But there is no support to read the file in small chunks.

The Amplify library supports custom plugins, using which we can connect with services other than AWS or add a wrapper to AWS services.

To support the uploading of large files using Amplify, I have used the same mechanism as a custom plugin.

NPM Library amplify-s3-chunk-upload


Usage

Install

npm i -s amplify-s3-chunk-upload
Enter fullscreen mode Exit fullscreen mode

Configure Storage plugin in App.js (ReactNative)

import { StorageChunkUpload } from 'amplify-s3-chunk-upload';
import { Credentials } from '@aws-amplify/core';



// put following code after Amplify.configure

// Load StorageChunkUpload Plugin
const storagePlugin = new StorageChunkUpload({}, Credentials);
Storage.addPluggable(storagePlugin);
storagePlugin.configure(config);

Enter fullscreen mode Exit fullscreen mode

File upload call


// get File stats
const { size } = await RNFS.stat(fileURI);

// here we are simulating an array of bytes
const fileObject = {
  // set the size
  size: size,

  // here we will read file as per bodyStart & bodyEnd, this will avoid reading complete file in the memory.
  slice: (bodyStart, bodyEnd) => {
    // Here in this sample code, we are using react-native-fs to read files.
    return RNFS.read(fileURI, bodyEnd - bodyStart, bodyStart, 'base64')
      .then((data) => Buffer.from(data, 'base64'))
      .catch((error) => {
        // Log error if required.
      });
    },
  };

  // Upload call, for parameters, refer to Amplify docs.
  const result = await Storage.put(`Your-file-name.mp4`, fileObject, {
    contentType: 'video/mp4',
    level: 'protected',
    provider: 'StorageChunkUpload',
  });


Enter fullscreen mode Exit fullscreen mode

Since we are making standard Storage.put call and the underlying code also uses the same Amplify Library code, you can pass all other parameters such as progressCallback etc.


We have this in action for the last couple of days.
Please feel free to share any feedback.
NPM
GitHub

Discussion (5)

Collapse
viprocket1 profile image
loljohndoe • Edited

hey,
Im getting this error.
ERROR [ERROR] 44:48.50 axios-http-handler [Error: Request failed with status code 403] ERROR [ERROR] 44:48.91 AWSS3ProviderManagedUpload - error happened while uploading a part. Cancelling the multipart upload [Error: Request failed with status code 403] ERROR [ERROR] 44:48.135 axios-http-handler [Error: Request failed with status code 403] ERROR [ERROR] 44:48.162 axios-http-handler [Error: Request failed with status code 403] ERROR [ERROR] 44:48.188 axios-http-handler [Error: Request failed with status code 403] ERROR [ERROR] 44:48.288 axios-http-handler [Error: Request failed with status code 404] WARN [WARN] 44:48.304 StorageChunkUpload - error uploading [Error: Upload was cancelled.undefined] ERROR [Error: Upload was cancelled.undefined]

Collapse
walvekarnikhil profile image
Nikhil Walvekar Author

Let me know if this fix works for you
github.com/walvekarnikhil/amplify-...

Collapse
viprocket1 profile image
loljohndoe

Im using the patched version 2.0.1 but im still getting the error. please tell me if i miss anything in the thread.

Collapse
orsinidennis profile image
Dennis Orsini

I get a 403 error when passing credentials from amplify (v. 3.3.24) and trying to upload with the plugin, using storage without the plugin works with no such error.

Collapse
walvekarnikhil profile image
Nikhil Walvekar Author

Sorry for the late reply, I somehow missed the comment.
There was a similar issue reported and a new version was published.
github.com/walvekarnikhil/amplify-...

Let me know if this fix works for you.

Forem Open with the Forem app