Route your S3-served static HTMLs beautifully

Marton Veto — Mon, 18 Feb 2019 10:00:38 +0000

If you are that developer how doesn't play with the big boys and you don't have a fancy React, Angular or insert any fancy frontend framework here which is popular when you read this article site, which supports client-side routing, but you have some static HTML files (for whatever reason) and you'd like to host it in AWS S3, this article is for you!

What I'll show you is how to do some basic routing through S3. Your users don't have to add the ugly .html postfix to the end of their request if they want to reach the /products.html page, but they will be able to type /products.

🙀🙀🙀

How cool is that? Less typing, more revenue! Let's implement it!

But first, the boring task:

create a bucket in S3, with the default settings. We'll adjust everything later. Upload your static files into the bucket (or mine).

Configuring the permissions

In order for your customers to access content at the website endpoint, you must adjust the public access settings of the bucket.
- Go to Permissions > Public access settings
- Click Edit and make sure that the following 2 options are unselected:
  - Block new public bucket policies: this allows you to add a new public policy into your bucket. We will do this in the next step.
  - Block public and cross-account access if bucket has public policies: this allows users to access the content of a public bucket. If you have this option set only the creator of the bucket and other AWS services have access to this bucket. This is a pretty new feature of S3 (late 2018).
Now we are able to add a bucket policy to our bucket.
- Go to Permissions > Bucket policy
- Add the following policy document, which allows you to request every object from the bucket:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": "*",
            "Action": "s3:GetObject",
            "Resource": "arn:aws:s3:::<YOUR_S3_BUCKET_NAME>/*"
        }
    ]
}

Configuring the static website hosting

Now that we have every permissions setting, let's do the interesting part! Assume that I have the following HTML pages:

index.html, which is my landing page
about.html, products.html
and error.html, which is my error handling page

I want to have the following routing set:

/ -> index.html
/about -> about.html
/products -> products.html
everything else -> error.html

The first and last rule is easy to set up because S3 supports it from out-of-the-box with the Static website hosting feature. On your bucket page go to Properties > Static website hosting. Set the Index document field to index.html and the Error document to error.html (or howsoever you named your these files). And now the trickier part: you have to set up some Redirection rules.

It's configured via XML... 😬

But it's not that bad. Bear with me.

You have to set up RoutingRules, which consists of 2 things:

a Condition where we provide that what kind of request do we want to catch, eg. /products
and where this request will be Redirected, eg. to /products.html.

You will have a RoutingRules parent and as many RoutingRule as the number of HTML sites you want to redirect. So this is how it will look like in the end if I add routing to my two static files:

<RoutingRules>
  <RoutingRule>
    <Condition>
      <KeyPrefixEquals>about</KeyPrefixEquals>
      <HttpErrorCodeReturnedEquals>404</HttpErrorCodeReturnedEquals>
    </Condition>
    <Redirect>
      <ReplaceKeyWith>about.html</ReplaceKeyWith>
    </Redirect>
  </RoutingRule>

  <RoutingRule>
    <Condition>
      <KeyPrefixEquals>products</KeyPrefixEquals>
      <HttpErrorCodeReturnedEquals>404</HttpErrorCodeReturnedEquals>
    </Condition>
    <Redirect>
      <ReplaceKeyWith>products.html</ReplaceKeyWith>
    </Redirect>
  </RoutingRule>
</RoutingRules>

The most important part in this is the HttpErrorCodeReturnedEquals element. When you require a resource from an S3 bucket with a key like "products", S3 won't find it because it doesn't exist (obviously we only have HTML files in our buckets) and it will return a 404 (Not Found) error, which will be propagated to the client, and we won't even access the Redirect rule.

That's all. But do not forget, at last in your URL, your users will still see that they're browsing the products.html and not /products. But it will ease their user experience with eliminating the need of typing .html after every request.

Routing to external sites

With S3 routing rules, you have the ability to route your request to external websites. If your users request the /facebook resource from your page, you can redirect them to your Facebook page, https://facebook.com/YOUR_FACEBOOK_ACCOUNT_POSTFIX. This is how you can set up rules like this:

<RoutingRule>
  <Condition>
    <KeyPrefixEquals>facebook</KeyPrefixEquals>
    <HttpErrorCodeReturnedEquals>404</HttpErrorCodeReturnedEquals>
  </Condition>
  <Redirect>
    <HostName>facebook.com</HostName>
    <ReplaceKeyPrefixWith>/YOUR_FACEBOOK_ACCOUNT_POSTFIX</ReplaceKeyPrefixWith>
  </Redirect>
</RoutingRule>

Postscript

I hope this article helped you to do basic routing with S3. When I had to do the same thing, I spent a couple of hours while I found the best solution. I wish you won't have to spend that much time on it. You can find every code I pasted above in a Git repo.

If you find any error in the above-written stuff, please blame me publicly in the comment section. Or if it helped you, just drop a "thanks". I'll appreciate it.

See ya. 👋

File upload with AWS Lambda and S3 in Node

Marton Veto — Mon, 11 Feb 2019 21:56:06 +0000

If you have a Lambda function in Node and want to upload files into S3 bucket you have countless options to choose from. In this article, I'll present a solution which uses no web application frameworks (like Express) and uploads a file into S3 through a Lambda function. The HTTP body is sent as a multipart/form-data.

The code

For parsing the multipart/form-data request I use the lambda-multipart package. This package can parse both text and file content and this is how I use it:

const parseMultipartFormData = async event => {
  return new Promise((resolve, reject) => {
    const parser = new Multipart(event);

    parser.on("finish", result => {
      resolve({ fields: result.fields, files: result.files });
    });

    parser.on("error", error => {
      return reject(error);
    });
  });
};

In the files list I will have a list of Buffer objects.

This is how I call it and loop through all the files to upload them:

  const { fields, files } = await parseMultipartFormData(event);

  await Promise.all(
    files.map(async file => {
      await uploadFileIntoS3(file);
    })
  );

And finally, uploading a file into S3:

const uploadFileIntoS3 = async file => {
  const ext = getFileExtension(file);
  const options = {
    Bucket: process.env.file_s3_bucket_name,
    Key: `${uuidv4()}.${ext}`,
    Body: file
  };

  try {
    await s3.upload(options).promise();
  } catch (err) {
    console.error(err);
    throw err;
  }
};

I use the uuid library to get a unique identifier what I'll use as the name of the file. Be aware that if your files are Buffer objects you can pass them to the upload method of the S3 SDK, but you cannot pass Buffer objects to the putObject method! In the catch block you should add some meaningful error handling. I just logged the error and re-threw it to be able to see it on the caller side.

You can add some file verifications to check the MIME type of the files and the size. ⚠️ But watch out, currently, Lambda has multiple limitations. One of them is that it only supports HTTP requests which are less than 6MB in size, so if you want to upload files which are bigger than this limit, you cannot use this solution.

Don't forget to create some IAM role (and associate it to the Lambda function) to be able to put an object into the S3 bucket.

This is how I'm getting the extension of the file:

const getFileExtension = file => {
  const headers = file["headers"];
  if (headers == null) {
    throw new Error(`Missing "headers" from request`);
  }

  const contentType = headers["content-type"];
  if (contentType == "image/jpeg") {
    return "jpg";
  }

  throw new Error(`Unsupported content type "${contentType}".`);
};

And basically, that's all. You can find the full source code here. I am using the Serverless Framework for deploying my Lambda functions and for creating an S3 bucket.