Hetzner finally released Object Storage 🎉

#devops #webdev #cloud #tutorial

Hetzner finally launched their own S3-compatible object storage, adding to their existing block storage and file storage solutions. And because I am a huge fan of Hetzner, and even bigger customer through my PaaS company Sliplane, I want to show you what S3 is, if its the right tool for you, and how to get started with it!

At the end of the blog post, you will have learned:

What S3 is
Who exactly Hetzner is
When you should use S3
How to get started with S3
How to use the Hetzner S3 with the Minio CLI

Leeeet's go!

What is S3 and why do I need it?

S3 stands for Simple Storage Service and was initially a product by AWS. AWS S3 was a huge success, which lead to many companies offering competing products with the same interfaces, leading to so called "S3-Compatible" services.

Generally, S3 belongs to the category of "Object storage" (the middle column). Object storage is highly scalable, decoupled from your compute, and offers a flat hierarchy of objects (that's just your data + some metadata) in something called a bucket.

S3 is awesome for write once, read many (WORM) use cases. Imagine a social media platform, where every user has a profile picture. Most users change their picture maybe once per year, but it gets viewed everytime someone looks at a profile. Thats write once, read many!

There are many such use-cases. Backups, images, analytical data, etc. It's usually also cheaper than using something like block storage, but for that you are trading some performance.

Who exactly is Hetzner?!

Ok, so S3 is originally from AWS, but there are S3-compatible products. And one of those products just got released as a beta version by my favorite cloud hosting provider, Hetzner.

Hetzner is probably the biggest german cloud provider that has experienced a crazy boom in the last year or so. They have super good prices, awesome support (some might disagree :D), and are very actively listening to their customers.

I liked them so much that I decided to build an entire PaaS platform on mostly Hetzner infrastructure 👀 💕

Get started with Hetzner S3

Enough blabla, lets get cracking and actually take a look at how it it works.

I assume you already have a Hetzner account at this point (If not, what are you waiting for?! (Feel free to take my 20 Euro referral link)

Depending on the time you are reading this blog post you might have to apply for the Beta. I got accepted pretty quickly, I hope this is the same for you. After getting accepted you will see an empty S3 dashboard:

You can then create a bucket by simply giving a unique name:

After creating the bucket it will be empty, you can then create some credentials to use with the CLI or some programming SDK. Make sure to copy both the access key and the secret key, you will need it later!

Using the CLI

Minio has a very nice CLI tool called mc that can be used to interact with any S3-compatible storage. To get started, you can install it on macOS using Homebrew:

brew install minio/stable/mc

On Linux, you can download the binary from the Minio Client Releases page and make it executable:

wget https://dl.min.io/client/mc/release/linux-amd64/mc
chmod +x mc
./mc --help

On Windows, you can download the ZIP file from the Minio Client Releases page and extract it to a folder of your choice. Then add the binary to your PATH:

mc.exe --help

If that works, we can finally get started. First, we need to configure an alias with the access key and secret key you previously copied. After setting the alias, you can run mc alias list to see if it worked.

mc alias set hetzner https://fsn1.your-objectstorage.com <access_key> <secret_key> --api "s3v4" --path "off"

mc alias list

Now go into any folder with a file you want to upload. In my case, I wanted to upload a file called README.md, to the alias hetzner, into the bucket sliplane, with the object name README.md. You could also add more / into the path, for example: /hetzner/sliplane/README.md

mc cp README.md hetzner/sliplane/README.md
mc ls hetzner/sliplane
mc cat hetzner/sliplane/README.md

You can then run mc ls hetzner/sliplane (dont forget to change your bucket name) to see if the file uploaded successfully. You can also use the cat command to print the file, of course only if its readable :)

Conclusion

I truly hope you learned something new and got excited to build some awesome stuff with S3. Is there anything that you would like me to cover? What are you going to build with S3? I'd love to hear from you in the comments :)

Cheers,

Jonas, Co-Founder of Sliplane