DEV Community: Umakant Vashishtha

Buffered vs Streaming Data Transfer

Umakant Vashishtha — Sun, 24 Dec 2023 17:47:58 +0000

Introduction

In this article, we will see different methods of data transfer and how to efficiently transfer data from/to a server using HTTP protocol.

With HTTP protocol, the client can either transfer data to the server (upload) or request certain data from the server (download).

Client/Server can choose to transfer the data in different chunks, especially if the size of the data is more than a few kilobytes to ensure that the packet loss is minimal.

For download operations, server can choose to send the data in chunks while for upload operations, client can choose to send the data in chunks.

When request data is accumulated in memory and then processed, we call it buffered processing.

In contrast, when the data is processed at the time each chunk is received, we call it streaming processing.

By processing, it could be anything depending on the request, from storing in the file system to transmitting to some other server. For simplicity, we will only consider storing the data in the file system.

We will compare the two approaches and see which one is better in terms of performance and memory usage.
We will also see how to implement both approaches in Node.js.

Buffered Data Transfer

As soon as the client starts sending the data, server stores all byte chunks in the memory, the chunks are concatenated into a buffer (the chunks array in the code below) and when the request is finished sending all chunks, a close event is emitted and then saved to the file system. Eventually, garbage collector will free up the memory in the next run.

Let’s take a look at the following code snippet that implements the buffered upload.

import fs from "fs";
import crypto from "crypto";

async function bufferedUploadHandler(req, res) {
  let uuid = crypto.randomUUID();
  let bytesRead = 0;
  let chunks = [];

  req.on("data", (chunk) => {
    bytesRead += chunk.length;
    // add chunks into an array, file content is gauranteed to be in order
    chunks.push(chunk);
  });

  req.on("close", () => {
    // Put all the chunks into a buffer together
    let data = Buffer.concat(chunks).toString();
    // Save the content to the file on disk
    fs.writeFileSync(`./data/file-buffered-${uuid}.txt`, data, {
      encoding: "utf-8",
    });
    res.end(`Upload Complete, ${bytesRead} bytes uploaded.`);
  });
}

Streaming Data Transfer

As can be seen from the flow diagram, the chunks from the request body are written directly to the file system with the help of fs.createWriteStream and pipe and not stored in the memory for the complete timeline of the request.

This is possible because request implements Readable Stream interface and the content can be directly written to the file system using Writable Stream interface from fs.

The code snippet below that implements the file streaming upload.

import fs from "fs";
import crypto from "crypto";

async function streamedUploadHandler(req, res) {
  let uuid = crypto.randomUUID();

  let fsStream = fs.createWriteStream(`./data/file-streamed-${uuid}.txt`);
  req.pipe(fsStream); // req.read() -> fsStream.write()

  req.on("close", () => {
    fsStream.close();
    res.end(`Upload Complete, ${bytesRead} bytes uploaded.`);
  });

  let bytesRead = 0;
  req.on("data", (chunk) => {
    bytesRead += chunk.length;
  });
}

Let’s use the above in a basic http server.

import { createServer } from "http";

const server = createServer(async (req, res) => {
  const { method, url } = req;
  console.log(req.method, req.url);

  if (method === "POST" && url == "/upload-buffer") {
    await bufferedUploadHandler(req, res);
    return;
  } else if (method === "POST" && url == "/upload-stream") {
    await streamedUploadHandler(req, res);
    return;
  }
});

server.listen(3000);

console.log("Server listening on http://localhost:3000");

Benchmark

We will now compare the two approaches in terms of performance (response time) and memory usage, by firing a number of concurrent requests in parallel.

For the comparison, to also analyze the memory usage of server in each case, we can change the response to return peak memory usage of the server as below:

let peakMemory = 0;

// Update every two milliseconds
setInterval(() => {
  let memoryUsage = process.memoryUsage();
  if (memoryUsage.rss > peakMemory) {
    peakMemory = memoryUsage.rss; // Resident Set Size
  }
}, 2);

async function streamedUploadHandler(req, res) {
  let uuid = crypto.randomUUID();

  let fsStream = fs.createWriteStream(`./data/file-streamed-${uuid}.txt`);
  req.pipe(fsStream); // req.read() -> fsStream.write()
  req.on("close", () => {
    fsStream.close();
    res.end(peakMemory.toString());
  });
}

We are interested in the peak memory usage and not just the current memory usage.

Client Side Code

For firing parallel requests to the server, I used the following code which fires 200 requests in parallel, each containing a file of roughly 15 MB.

Once all requests are complete, we calculate the average, minimum and maximum time taken for the requests to complete, and also the maximum memory usage of the server.

The script is run multiple times, with different number of parallel requests each for buffered upload and streaming upload to compare the results.

import http from "http";
import fs from "fs";

async function fileUploadRequest(fileName, requestPath = "/upload-stream") {
  return new Promise((resolve, reject) => {
    let start = Date.now();
    const req = http.request(
      {
        method: "POST",
        hostname: "localhost",
        port: "3000",
        path: requestPath,
        headers: {
          Accept: "*/*",
        },
      },
      function (res) {
        const chunks = [];

        res.on("data", function (chunk) {
          chunks.push(chunk);
        });

        res.on("end", function () {
          const body = Buffer.concat(chunks).toString();
          let end = Date.now();
          resolve({
            time: end - start,
            memoryUsage: Number(body),
          });
        });
      }
    );

    const a = fs.createReadStream(fileName);

    a.pipe(req, { end: false });

    a.on("end", () => {
      req.end();
    });

    req.on("error", function (error) {
      reject(error);
    });
  });
}

async function fireParallelRequests(count, path = "/upload-stream") {
  const promises = [];
  for (let i = 0; i < count; i++) {
    promises.push(fileUploadRequest("./data.csv", path));
  }

  let metrics = await Promise.all(promises);

  let latencies = metrics.map((m) => m.time);
  let min = Math.min(...latencies);
  let max = Math.max(...latencies);
  let avg = latencies.reduce((a, b) => a + b, 0) / latencies.length;
  let maxMemoryUsage = Math.max(metrics.map((m) => m.memoryUsage));

  console.log("Total Requests:", count);
  console.log("URL:", path);
  console.log("Min Time:", min);
  console.log("Max Time:", max);
  console.log("Avg Time:", avg);
  console.log(
    "Max Memory Usage:",
    `${Math.round((maxMemoryUsage / 1024 / 1024) * 100) / 100} MB`
  );
}

async function main() {
  await fireParallelRequests(200, "/upload-stream");
  // await fireParallelRequests(200, "/upload-buffer");
}

main();

Here are the results from 200 concurrent requests for both buffered and streaming upload:

Parameter	Buffered Data Transfer	Streaming Data Transfer
Total Requests	200	200
Min Time (ms)	1975	1297
Max Time (ms)	34609	31680
Avg Time (ms)	13061	3995
Max Memory Usage (MB)	2889.66 MB	276.39 MB

As can be seen, the difference in memory usage is quite apparent, as the buffered upload uses almost 10 times more memory than streaming upload. In fact, the memory usage for buffered upload is so high that it can cause the server to crash if the number of concurrent requests is high enough.

Also, the difference in average latency is noticeable where the streaming upload is as much as 3 times faster than buffered upload.

The following charts depict the time and memory usage for both buffered and streaming upload for different number of concurrent requests.

The memory usage for buffered upload increases linearly with the number of requests, while for streaming upload, it remains almost constant.

Conclusion

In this article, we saw how to implement buffered and streaming data transfer in Node.js and compared the two approaches in terms of performance and memory usage.

We saw that streaming upload is much faster than buffered upload and also uses much less memory.

We usually have limited memory available on the server and we want to process as many requests as we can, so streaming upload is the way to go.
But sometimes, we may want to implement buffered upload, for example, if we want to process the data in some way before saving it to the file system.

References

S3 File Upload in Node.js and React - Build React App

Umakant Vashishtha — Sun, 15 Oct 2023 17:07:43 +0000

In this three parts series, we are learning how to add secure file upload feature in your application using AWS S3, Node.js and React with Signed URLs.

Check out the previous parts if you haven't

S3 File Upload in Node.js and React - Setting Up S3 Bucket

Umakant Vashishtha ・ Oct 8

#node #react #aws #javascript

S3 File Upload in Node.js and React - Setting Up Node.js App

Umakant Vashishtha ・ Oct 8

#node #javascript #react #aws

Understanding the flow
Setting up React App
- Setting up App Config
- Setting Up API Client
Testing the Signed URL in a Playground Component
Reusable Components
Keep in Mind
- Limiting File Uploads
- Maintaining Storage

In this part, we will build the react application to upload files directly to AWS S3 using Signed URLs generated from our node.js application.

If you prefer video tutorials, here is the series on YouTube.

Understanding the flow

Fig: File Upload Complete Flow

The above diagram shows the complete flow of the file upload process.

User selects the file to upload
React application sends a request to the node.js server to generate a signed URL
Node.js server generates a signed URL by calling AWS S3 APIs with AWS Credentials and sends it back to the react application.
React application uses the signed URL to upload the file directly to AWS S3.

Once the file is uploaded, we can save the URL in the database and use it to display the file in the application.

Setting up React App

Let's start by creating a new react application using create-react-app command.

  npx create-react-app client

Once the application is created, we need to install the following dependencies.

  npm install axios @emotion/react @emotion/styled @mui/icons-material @mui/material

We will use axios to make API calls to our node.js server, @emotion and @mui for creating the styled components.

Setting up App Config

We will create a new file config.js in the src folder to store the configuration for our application.

// src/config/index.js

const config = {
  API_BASE_URL: process.env.REACT_APP_API_BASE_URL,
};

export default config;

We will use REACT_APP_API_BASE_URL environment variable to store the base URL of our node.js server.

Create a new file .env.development.local in the root of the project and add the following content.

REACT_APP_API_BASE_URL=http://localhost:3010/api

Setting Up API Client

We will create a new file api/index.js in the src folder to create an API client using axios to make API calls to our node.js server and to AWS S3.

// src/api/index.js
import axios from "axios";
import config from "../config";

const apiClient = axios.create({
  baseURL: config.API_BASE_URL,
});

export async function getSignedUrl({ key, content_type }) {
  const response = await apiClient.post("/s3/signed_url", {
    key,
    content_type,
  });

  return response.data;
}

export async function uploadFileToSignedUrl(
  signedUrl,
  file,
  contentType,
  onProgress,
  onComplete
) {
  axios
    .put(signedUrl, file, {
      onUploadProgress: onProgress,
      headers: {
        "Content-Type": contentType,
      },
    })
    .then((response) => {
      onComplete(response);
    })
    .catch((err) => {
      console.error(err.response);
    });
}

Testing the Signed URL in a Playground Component

We will create a new component Playground in the src/components folder to test the signed URL flow.

// src/components/Playground.js

import React, { useState } from "react";
import { getSignedUrl, uploadFileToSignedUrl } from "../api";

const Playground = () => {
  const [fileLink, setFileLink] = useState("");
  const onFileSelect = (e) => {
    const file = e.target.files[0];

    const content_type = file.type;

    const key = `test/image/${file.name}`;

    getSignedUrl({ key, content_type }).then((response) => {
      console.log(response);

      uploadFileToSignedUrl(
        response.data.signedUrl,
        file,
        content_type,
        null,
        () => {
          setFileLink(response.data.fileLink);
        }
      );
    });
  };
  return (
    <div>
      <h1>Playground</h1>

      <img src={fileLink} />

      <input type="file" accept="*" onChange={onFileSelect} />
    </div>
  );
};

export default Playground;

We will add the Playground component to the App.js file to test the flow.

// src/App.js

import React from "react";
import Container from "@mui/material/Container";
import Playground from "./components/Playground";

function App() {
  return (
    <div className="App">
      <Container style={{ display: "flex", justifyContent: "center" }}>
        <Playground />
      </Container>
    </div>
  );
}

Reusable Components

We can wrap the above functionality into a hook component that can be reused in multiple parts of your application where file upload is required, as below:

// src/hooks/useFileUpload.js

import { useCallback, useState } from "react";
import { getSignedUrl, uploadFileToSignedUrl } from "../api";

function getKeyAndContentType(file, prefix = "documents") {
  const [fileName, extension] = file.name.split(".");

  // to generate unique key everytime
  let key = prefix + `/${fileName}-${new Date().valueOf()}.${extension}`;

  let content_type = file.type;

  return { key, content_type };
}

export default function useFileUpload(onSuccess, prefix) {
  const [uploading, setUploading] = useState(false);
  const [uploadProgress, setUploadProgress] = useState(null);

  const uploadFile = useCallback((file) => {
    if (file) {
      const { key, content_type } = getKeyAndContentType(file, prefix);

      getSignedUrl({ key, content_type }).then((response) => {
        const signedUrl = response.data?.signedUrl;
        const fileLink = response.data?.fileLink;

        if (signedUrl) {
          setUploading(true);
          uploadFileToSignedUrl(
            signedUrl,
            file,
            content_type,
            (progress) => {
              setUploadProgress((progress.loaded / progress.total) * 100);
            },
            () => {
              onSuccess(fileLink);
              setUploading(false);
            }
          ).finally(() => {
            setUploadProgress(0);
          });
        }
      });
    }
    // eslint-disable-next-line
  }, []);

  return {
    uploading,
    uploadProgress,
    uploadFile,
  };
}

The usage for this hook would look something like this:

// src/components/EditAvatar.js
import React, { useEffect, useState } from "react";

import { MenuItem, Menu } from "@mui/material";
import useFileUpload from "../hooks/useFileUplaod";

function EditAvatar({ inputId, image, name, onChange, prefix = "avatars" }) {
  const { uploadFile } = useFileUpload(onChange, prefix);

  const [file, setFile] = useState(null);

  useEffect(() => {
    uploadFile(file);
    // Do NOT put uploadFile function as dependency here
    // eslint-disable-next-line
  }, [file]);

  return (
    <div>
      <img src={image} alt={name} className="edit-avatar" />
      <input
        type="file"
        accept="image/jpeg, image/png"
        onChange={(e) => {
          setFile(e.target.files[0]);
        }}
        id={inputId}
        className="edit-file-input"
      />
      <div className="edit-menu-button">
        <Menu>
          <label htmlFor={inputId}>
            <MenuItem>Upload New</MenuItem>
          </label>
          {image && (
            <a href={image} target="_blank" rel="noreferrer">
              <MenuItem>Preview</MenuItem>
            </a>
          )}
          <MenuItem onClick={() => onChange(null)}>Remove</MenuItem>
        </Menu>
      </div>
    </div>
  );
}

You can find the rest of the code here:

https://github.com/umakantv/yt-channel-content/tree/main/file-upload-using-s3

Thank you for reading, please follow/subscribe if you liked the content, I will share more such in-depth content related to full-stack development.

Keep in Mind

Limiting File Uploads

File upload should be limited so that adversaries can't abuse the feature and bloat your s3 bucket with unnecessary files. Some things you can do to be careful are:

Creating Signed URL behind auth
Rate limiting on creating Signed URLs with IP Address
Specific key prefix format for each user/organization, E.g. - * public/assets//,
- public/assets//avatar/

Maintaining Storage

Using S3 Lifecycle Rules
Using S3 Object Expiry
Deleting unused files in S3 bucket when replaced with new files

Happy learning. :)

S3 File Upload in Node.js and React - Setting Up Node.js App

Umakant Vashishtha — Sun, 08 Oct 2023 18:23:57 +0000

Creating AWS Access Key for Programmatic Access
Setting up Node.js application
Setting up Config
Signed URL Route

In this three parts series, we are learning how to add secure file upload feature in your application using AWS S3, Node.js and React with Signed URLs.

Check the first part if you haven't

S3 File Upload in Node.js and React - Setting Up S3 Bucket

Umakant Vashishtha ・ Oct 8

#node #react #aws #javascript

In this part, we will setup node.js application and use AWS SDK to generate S3 Signed URL using AWS access credentials
that will be used in our react application to upload files directly.

If you prefer video tutorials, here is the series on YouTube.

Creating AWS Access Key for Programmatic Access

We will need AWS access keys to send programmatic calls to AWS from AWS SDKs
To create AWS access keys, head over to IAM dashboard in AWS.

First go to User Groups page and create a user group
- Select S3 Full Access Permission from the list of permissions in Attach Permission Policies tab and assign to this group
Then go to the Users page and create a user and add the user to the User group created just now.
Then select the created just and go to the Security Credentials tab and under Access keys section for the user, generate new access keys.
Make sure to copy/download the secret key, it will not be shown again, we will need it in the next step

Setting up Node.js application

Start a new npm project
Install the following libraries with the given command:

  npm i express dotenv cors @aws-sdk/client-s3@3.400.0 @aws-sdk/s3-request-presigner@3.400.0

We are using specific versions for aws-sdk so that the code works as expected.

Setting up Config

Create a .env.local file in the root of your project with the following variables:


PORT = 3010

# AWS IAM Config
AWS_ACCESS_KEY_ID = <YOUR_AWS_ACCESS_KEY_ID>
AWS_SECRET_KEY = <YOUR_AWS_SECRET_KEY>

# AWS S3 Config
AWS_S3_BUCKET_NAME = <yt-file-upload-tutorial>

Create a new file config/index.js to load configurations for the node.js app.

import { config as loadConfig } from "dotenv";

loadConfig({
  path: ".env.local",
});

const config = {
  PORT: parseInt(process.env.PORT, 10) || 3010,
  AWS: {
    AccessKeyId: process.env.AWS_ACCESS_KEY_ID,
    AWSSecretKey: process.env.AWS_SECRET_KEY,
    BucketName: process.env.AWS_S3_BUCKET_NAME,
    Region: "ap-south-1",
  },
};

export default config;

Generating Signed URL

Setup express application, with a GET route handler for /api/s3/signed_url route.
Next add a controller in a new file utils/s3.js as shown below:

// utils/s3.js
import { PutObjectCommand, S3Client } from "@aws-sdk/client-s3";
import { getSignedUrl } from "@aws-sdk/s3-request-presigner";
import config from "../config/index.js";

const s3 = new S3Client({
  region: config.AWS.Region,
  credentials: {
    accessKeyId: config.AWS.AccessKeyId,
    secretAccessKey: config.AWS.AWSSecretKey,
  },
});

const BUCKET_NAME = config.AWS.BucketName;

export async function createPresignedPost({ key, contentType }) {
  const command = new PutObjectCommand({
    Bucket: BUCKET_NAME,
    Key: key,
    ContentType: contentType,
  });

  const fileLink = `https://${BUCKET_NAME}.s3.${config.AWS.Region}.amazonaws.com/${key}`;

  const signedUrl = await getSignedUrl(s3, command, {
    expiresIn: 5 * 60, // 5 minutes - default is 15 mins
  });

  return { fileLink, signedUrl };
}

Note how the above code uses the config from the previous step.

Use the function in the route handler as shown below

import express from "express";
import { createPresignedPost } from "../utils/s3.js";

const s3Router = express.Router();

s3Router.post("/signed_url", async (req, res) => {
  try {
    let { key, content_type } = req.body;
    key = "public/" + key;
    const data = await createPresignedPost({ key, contentType: content_type });

    return res.send({
      status: "success",
      data,
    });
  } catch (err) {
    console.error(err);
    return res.status(500).send({
      status: "error",
      message: err.message,
    });
  }
});

export default s3Router;

I am using the s3Router in my express app as app.use('/api/s3', s3Router).

We can test this by curl with the following command:

curl  -X POST \
  'http://localhost:3010/api/s3/signed_url' \
  --header 'Accept: */*' \
  --header 'Content-Type: application/json' \
  --data-raw '{
  "key": "images/a.png",
  "content_type": "image/png"
}'

This should return a response as below:

{
  "data": {
    "signedUrl": "https://yt-file-upload-tutorial.s3.ap-south-1.amazonaws.com/public/images/a.png?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Content-Sha256=UNSIGNED-PAYLOAD&X-Amz-Credential=AKIA6DIIPAXK4E2THWRC%2F20230830%2Fap-south-1%2Fs3%2Faws4_request&X-Amz-Date=20230830T205324Z&X-Amz-Expires=300&X-Amz-Signature=ef0c4faa9fb48f25ba52e821a0453b2c25ee5e75f27eb60484cf23b1434c46d4&X-Amz-SignedHeaders=host&x-id=PutObject",
    "fileLink": "https://yt-file-upload-tutorial.s3.ap-south-1.amazonaws.com/public/images/a.png"
  }
}

In the next part, I will show how to create secure Signed URLs from the node.js back-end application using the security credentials to allow file uploads from our font-end application.

Thank you for reading, please subscribe if you liked the video, I will share more such in-depth content related to full-stack development.

Happy learning. :)

S3 File Upload in Node.js and React - Setting Up S3 Bucket

Umakant Vashishtha — Sun, 08 Oct 2023 11:01:17 +0000

In this three part series, I will show you how to add secure file upload feature in your application using AWS S3, Node.js and React.

If you prefer video tutorials, here is the series on YouTube.

What is Static Content?

Images, PDFs, Audios, Videos, etc are all static content that don’t change often. They are stored on a file server and served through a unique URL.

Example: https://yt-file-upload-tutorial.s3.ap-south-1.amazonaws.com/public/test/image/profile.png

Uploading files is very important feature in most of the applications and we need disk space to store these files.
Our users upload files on our application like profile picture, documents, etc., we have to store these somewhere and they should be accessible by a URL.

Let's setup S3 and understand what S3 is?

S3 (Simple Storage Service) is a static file storage and server provided by AWS cloud provider, and we will use it to store and serve our application's static content.
S3 has buckets in while files are arranged in different directories.

Why should we use S3?

S3 is performant and fast
It is reliable
It provides secure way to upload files and we can control which files are public and where the files can be uploaded from
It is cheaper than building and maintaining our own file server

AWS gives you 12 months of free-tier when you sign-up and S3 provides 5GB storage free under Free-Tier.

Creating S3 Bucket

Let's create an S3 bucket and upload some files to it and then we will try to access those files by a URL.

Go to S3 Dashboard and open Buckets page.
Click Create Bucket and add a name.
Under Block Public Access settings for this bucket settings, select allow public access.
Submit the form to create the bucket.

Upload a sample file

Create a folder and name it public.
Inside the public folder, upload any file, preferably an image file.
Click on the file name to see the properties.

Sample File: https://yt-file-upload-tutorial.s3.ap-south-1.amazonaws.com/public/profile.png

Some of the properties for this file are:

Base Path: https://yt-file-upload-tutorial.s3.ap-south-1.amazonaws.com
key: public/images/profile/34534534.png
Metadata:
Content-Type: image/png

S3 buckets can have both private and public files, so we can control which files are publicly accessible.

If you try to open this image in a browser using the complete path, it won't be accessible because it's not allowed for public read. So let's change that setting.

Updating Policy for Public Read Access on a prefix URL

Go to bucket home page and open the Permissions tab. Inside that, under the Edit Bucket Policy section, edit the setting to allow public read for files in public folder using the below JSON setting. MAKE SURE to use your bucket name.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "AddPerm",
            "Effect": "Allow",
            "Principal": "*",
            "Action": "s3:GetObject",
            "Resource": "arn:aws:s3:::YOUR_BUCKET_NAME/public/*"
        }
    ]
}

The above can be found in the documentation here: https://repost.aws/knowledge-center/read-access-objects-s3-bucket

Updating CORS policy

We will need to do a bit more setup to update CORS policy to allow uploads from certain domains.

To do that, again head over to the permissions tab and then under Cross-Origin Resource Sharing section, edit the setting using the following json properties.
You can add any number of domains to allow GET and PUT requests.

[
    {
        "AllowedHeaders": [
            "*"
        ],
        "AllowedMethods": [
            "PUT"
        ],
        "AllowedOrigins": [
            "http://localhost:3000"
        ],
        "ExposeHeaders": []
    },
    {
        "AllowedHeaders": [
            "*"
        ],
        "AllowedMethods": [
            "GET"
        ],
        "AllowedOrigins": [
            "*"
        ],
        "ExposeHeaders": []
    }
]

In the next part, I will show how to create secure Signed URLs from the node.js back-end application using the security credentials to allow file uploads from our font-end application.

Thank you for reading, please subscribe if you liked the video, I will share more such in-depth content related to full-stack development.

Happy learning. :)

DEV Community: Umakant Vashishtha

Buffered vs Streaming Data Transfer

Introduction

Buffered Data Transfer

Streaming Data Transfer

Benchmark

Client Side Code

Conclusion

References

S3 File Upload in Node.js and React - Build React App

S3 File Upload in Node.js and React - Setting Up S3 Bucket

Umakant Vashishtha ・ Oct 8

S3 File Upload in Node.js and React - Setting Up Node.js App

Umakant Vashishtha ・ Oct 8

Table of Contents

Understanding the flow

Setting up React App

Setting up App Config

Setting Up API Client

Testing the Signed URL in a Playground Component

Reusable Components

Keep in Mind

Limiting File Uploads

Maintaining Storage

S3 File Upload in Node.js and React - Setting Up Node.js App

Table of Contents

S3 File Upload in Node.js and React - Setting Up S3 Bucket

Umakant Vashishtha ・ Oct 8

Creating AWS Access Key for Programmatic Access

Setting up Node.js application

Setting up Config

Generating Signed URL

S3 File Upload in Node.js and React - Setting Up S3 Bucket

What is Static Content?

Let's setup S3 and understand what S3 is?

Why should we use S3?

Creating S3 Bucket

Upload a sample file

Updating Policy for Public Read Access on a prefix URL

Updating CORS policy