DEV Community

Cover image for Hosting static sites with Cloudflare R2 and MinIO Client
Brett Weir for BrettOps

Posted on • Originally published at brettops.io on

Hosting static sites with Cloudflare R2 and MinIO Client

There are countless services nowadays for hosting static sites: GitHub, GitLab, Netlify, Surge, Porkbun, DigitalOcean, even Cloudflare.

If there are so many ways to get a static site online, why would anyone bother with setting up a plain ol' S3 bucket?

Well, there are lots of reasons:

  • Hosting multiple versions of a site. If you want v1, v2, and v3 at the same time, but don't want to commit your built sites to Git, then you need a writable location.

  • Hosting many sites from one domain. Maybe you're a hosting service! Or maybe you just provide hosting for multiple users in your company. You can build out a consistent workflow on top of S3 and host all the content in a single location.

  • Total control of how the sites are published. Maybe you want to vary available content by region, add authentication, use server-side analytics, or just configure how content is cached by the CDN. Building your own custom workflow will give you access to all the levers you need.

In this article, we'll develop a recipe for using Cloudflare R2 as a static site hosting service. You will:

  • Create a simple static site,

  • Publish the site to Cloudflare R2 with MinIO Client, and

  • Use Cloudflare Transform Rules to make your bucket behave more like a web server.

By the end of it, you will be the proud owner of a many-headed static site hydra that you'd never know was a simple S3 bucket underneath.

Prerequisites

The easiest way to meet all the prerequisites for this tutorial is to complete the previous tutorial in this series, Serve static assets with Cloudflare R2.

Here's a summary of the things you'll need:

  • A domain name proxied by Cloudflare.

  • A Cloudflare account.

  • An R2 bucket configured for public access.

For this tutorial, I created an R2 bucket called sites and made it accessible at sites.brettops.io. The domains and paths you will use for this article will be different, but the steps should otherwise be the same for you.

Step 0: Build a static site (optional)

If you came here because you already have a static site that you're ready to publish, use that and skip this section. For everyone else, you can set up an example site with me.

I'll be using MkDocs, because it's fast and simple and generates some nice boilerplate so that the site isn't completely empty. MkDocs is written in Python, so you'll need Python installed (which probably isn't an issue if you're on Linux).

Install the mkdocs package:

pip install mkdocs
Enter fullscreen mode Exit fullscreen mode

This installs the mkdocs command. You can test that mkdocs is available by doing the following:

mkdocs --version
Enter fullscreen mode Exit fullscreen mode
$ mkdocs --version
mkdocs, version 1.4.2 from /home/brett/.pyenv/versions/3.10.7/lib/python3.10/site-packages/mkdocs (Python 3.10)
Enter fullscreen mode Exit fullscreen mode

Create a new mkdocs.yml project in the current directory:

mkdocs new .
Enter fullscreen mode Exit fullscreen mode
$ mkdocs new .
INFO - Writing config file: ./mkdocs.yml
INFO - Writing initial docs: ./docs/index.md
Enter fullscreen mode Exit fullscreen mode

One more thing: let's add a subpage for this site. You'll see why this is important later in the article:

cat > docs/about.md <<EOF
# About

Some more info about that.
EOF
Enter fullscreen mode Exit fullscreen mode

You can run a local dev server to see your changes in action:

mkdocs serve
Enter fullscreen mode Exit fullscreen mode
$ mkdocs serve
INFO - Building documentation...
INFO - Cleaning site directory
INFO - Documentation built in 0.05 seconds
INFO - [23:22:45] Watching paths for changes: 'docs', 'mkdocs.yml'
INFO - [23:22:45] Serving on http://127.0.0.1:8000/
Enter fullscreen mode Exit fullscreen mode

Visit your local dev server at http://127.0.0.1:8000/. Our fancy new docs site isn't going to be anything to write home about, but it'll do the job.

New MkDocs static site served locally.

When you're satisfied with your site, you can build a finished site for hosting like so:

mkdocs build
Enter fullscreen mode Exit fullscreen mode
$ mkdocs build
INFO - Cleaning site directory
INFO - Building documentation to directory: /home/brett/Projects/examples/mkdocs-site/site
INFO - Documentation built in 0.05 seconds
Enter fullscreen mode Exit fullscreen mode

This will create a site/ directory that contains our finished site, which is what we'll publish to Cloudflare R2.

Step 1: Deploy the site with MinIO Client

MinIO Client is far and away the best S3 command line tool I've found.

It's written in Go, so getting it onto your system is easy, and it supports a ton of commands that alternatives such as aws s3 or s3cmd simply don't have.

First, download and install the mc tool:

curl -O https://dl.min.io/client/mc/release/linux-amd64/mc
sudo install mc /usr/local/bin/
Enter fullscreen mode Exit fullscreen mode
$ curl -O https://dl.min.io/client/mc/release/linux-amd64/mc
  % Total % Received % Xferd Average Speed Time Time Time Current
                                 Dload Upload Total Spent Left Speed
100 24.9M 100 24.9M 0 0 10.2M 0 0:00:02 0:00:02 --:--:-- 10.2M
$ sudo install mc /usr/local/bin/
Enter fullscreen mode Exit fullscreen mode

For more installation options, see the official quickstart.

The mc command should be usable at this point:

mc --version
Enter fullscreen mode Exit fullscreen mode
$ mc --version
mc version RELEASE.2023-06-15T15-08-26Z (commit-id=bf3924b58341eb7a71785653a29bf26ca9fac95e)
Runtime: go1.19.10 linux/amd64
Copyright (c) 2015-2023 MinIO, Inc.
License GNU AGPLv3 <https://www.gnu.org/licenses/agpl-3.0.html>
Enter fullscreen mode Exit fullscreen mode

mc allows you to configure a connection by creating an alias. You can have as many aliases configured as desired.

Use the mc alias set command to configure your R2 connection:

mc alias set NAME https://XXXXXX.r2.cloudflarestorage.com/ YYYYYY ZZZZZZ
Enter fullscreen mode Exit fullscreen mode

Where:

  • NAME is the desired name for your alias. You'll have to type this often, so it's better to keep it short. I'll call mine r2.

  • XXXXXX is your Cloudflare account ID.

  • YYYYYY is your Cloudflare R2 access key ID.

  • ZZZZZZ is your Cloudflare R2 secret access key.

Once you've configured an alias, you can test it out or access it by prefixing the desired path with the alias:

mc ls r2/
Enter fullscreen mode Exit fullscreen mode
$ mc ls r2/
[2023-06-17 18:38:11 UTC] 0B sites/
Enter fullscreen mode Exit fullscreen mode

Hey, that's the sites bucket! Let's try accessing it!

$ mc ls r2/sites/
$
Enter fullscreen mode Exit fullscreen mode

The above prints nothing. That's good! There's nothing in the bucket!

At this point, we've verified that the bucket works. Now we can put some stuff in it.

Note: The MinIO docs say to test the connection like this:

mc admin info r2

This does not work with Cloudflare. I don't know why.

It probably has to do with the fact that AWS S3 buckets have the bucket name in the domain, whereas Cloudflare R2 buckets have the bucket name in the path. πŸ€”

For hosting a static site, far and away the best tool for the job is the mc mirror command, which synchronizes files between two locations:

mc mirror SOURCE TARGET
Enter fullscreen mode Exit fullscreen mode

In our case, we'll set it up to synchronize the local MkDocs site/ directory to the R2 bucket. We'll add the --overwrite flag so that it overwrites existing files if there are any differences, and we'll add the --remove flag so that it deletes files from the target that no longer exist in the source.

This will be great for when we create a pipeline to continuously publish changes to a site.

Double-check your command before running. You can very easily delete any existing data in the bucket if you're not careful with these commands.

mc mirror site r2/sites/latest/ --overwrite --remove
Enter fullscreen mode Exit fullscreen mode
$ mc ls r2/sites/
$ mc mirror site r2/sites/latest/ --overwrite --remove
...site/sitemap.xml.gz: 1.38 MiB / 1.38 MiB ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 556.07 KiB/s 2s
Enter fullscreen mode Exit fullscreen mode

If we browse to the published location, we'll be able to access the individual files we just uploaded:

Root  raw `index.html` endraw  of the published MkDocs site.

We're not done yet though.

Step 2: Rewrite trailing slashes

You may have noticed that if you try to click on the About link, you get an error:

Clicking on the **About** link leads to an error.

Web servers these days will almost always rewrite a URL that ends in a trailing slash (/) to an index.html file at the same path. In other words:

https://brettops.io/

Is the same page as:

https://brettops.io/index.html

This allows you to visit sites.brettops.io/latest/ and the contents of sites.brettops.io/latest/index.html, which is how it worked when we tested our site locally. Cloudflare doesn't do this by default, which is why our About link leads to nowhere.

We can tell Cloudflare to behave like this when serving our R2 site, using Rewrite URL Rules. That way, our links will work, and we'll be able to access our site at sites.brettops.io/latest/.

Go to the Cloudflare dashboard for your domain, and click Rules, then Transform Rules in the sidebar:

Navigate to the **Transform Rules** page.

Click the Create rule button under the Rewrite URL tab:

Click the **Create rule** button.

Add an actually good name for your rule. It's the only way you'll be able to remember what it does without reading through your rule expressions:

Add a good name for your rule.

Under If..., select Custom filter expression, and add the following expressions to the Expression Builder with an And between them:

Field Operator Value
Hostname equals sites.brettops.io
URI Path ends with /

Configure the expression to match requests for the Rewrite URL rule.

Alternatively, you can edit the expression manually by clicking Edit expression and add the following:

(http.host eq "sites.brettops.io" and ends_with(http.request.uri.path, "/"))
Enter fullscreen mode Exit fullscreen mode

Under Then..., then under Path, select the Rewrite to... option, select Dynamic, and add the following expression (see screenshot image below for reference):

concat(http.request.uri.path, "index.html")
Enter fullscreen mode Exit fullscreen mode

This uses the concat function to append index.html to the URLs of matched request.

And under Query, select Preserve.

Rewrite the path, but preserve the query string.

When you're ready, click Deploy. Then your new rule will be live:

The new Rewrite URL rule is live.

At this point, you should be able to navigate to your site's URLs and see that they're accessible without adding index.html to the path:

[ raw `sites.brettops.io/latest` endraw ](https://sites.brettops.io/latest) is accessible.

Following the About link takes us to the About page, no problem.

Congratulations, we've built a fully functional static site hosting service!

Conclusion

Cloudflare R2 is deeply integrated with Cloudflare and easy to get started with. MinIO Client makes working with S3 clean and obvious. Together, they provide a slim, bare-bones hosting solution that is highly adaptable to different needs and use cases.

What's better, all of the steps taken in this article are easy to automate in a CI pipeline, allowing you to build a general solution for your team or company that scales with your users.

For simple use cases, I'd still rather use an off-the-shelf solution, but this is one of those tools that you keep in your toolbox, because you never know when you're going to need it. Sometimes, all you really need is something you can hack on and make your own.

Top comments (0)