From "The Cloud" to my cloud: I'm back to self-hosting

In the past 1-2 years, it feels "the cloud" has become a huge mess. So, I took a moment to really think about how much it's taking over our digital tools and services. In my opinion, it's creating a lot of value, but the really big players often don't have the right intentions.

I decided to make a big change. Instead of letting those huge cloud companies handle all my data, I've switched to self-hosted, open-source solutions. In this blog, I'm sharing my journey from relying on the big players to creating my own cloud.

Things I'll cover in this blog post:

• RSS feeds are still relevant
• Nextcloud, Rclone, Storj
• Tailscale & Wireguard

Personal journey

I used to be all about building my own stuff and reinventing the wheel whenever possible. But like a lot of people, I got sucked into the convenience of using Dropbox and Google Drive. It was easy, didn't take a lot of time, and for a while, it pulled me away from that whole DIY mentality I had going in my early tech days. But it didn't stick.

After seeing too many horror stories about cyber attacks, weird privacy policies, and just being fed up with corporate services, I decided to go back to doing things my way, by self-hosting what I use most. This time around, unlike several years ago, I know what I'm doing.

I want to be in control of my own data again be able to make my own choices.

Why open-source

Open source isn't just about access to the source code; it's a philosophy. It's freedom, transparency, and control over the tech you use in your life.

By adopting open-source software, I choose to take responsibility for my digital privacy and make software work around my life, not the other way around.

No longer do I blindly trust algorithms or hand over my information to these large players without really thinking about it first. Instead, I choose to build a cloud that has everything I need, everything I want to do myself, and has the stuff I want control over.

Building my cloud

When it comes to building a cloud, there are a few things you need. In my case, the Raspberry Pi 4 for this post in 2020, a layer of security, and software to run to make my life a little easier. Bonus points for a backup strategy.

Security

I got started with security by setting up secure and super simple remote access with Tailscale. It's a very straightforward piece of software that lets you create a VPN mesh network using Wireguard. You can set up connections between several devices in literal minutes.

Tailscale is open-source, but has a hosted version of their software. As I'm not willing to compromise on security and I'm by no means a security expert, I'm trusting their hosted service to help set up my VPN mesh network. After the initial connection, all VPN traffic is peer-to-peer between the devices, so my phone is contacting with my Raspberry Pi directly.

With Tailscale I can access my server at home from anywhere in the world. No need to forward any ports in my router or expose the Raspberry Pi to the internet. Win-win!

RSS

Then I took back my RSS feed, which is pretty straightforward but super powerful. It gave me back control over what I consume online. Unlike those AI news feeds that throw random stuff at you, my RSS feed lets me subscribe to content I enjoy and value. I use Miniflux for this. It's very basic, fast, and straight to the point. That's all I'm asking for. No distraction, just interesting content.

Storage

One of the things I mentioned in the beginning and I've mentioned before on this website is that I'm a huge fan of Nextcloud. It's a self-hosted version of Dropbox, but with a lot of very useful plugins you can install.

Last time (in 2020) I set up an instance of it on my Raspberry Pi using Snap. This time around, I'm using Docker Compose instead. This has 2 reasons: It's my comfort zone and resource management.

With Docker Compose I can set very specific resource limitations for Nextcloud on my Raspberry Pi. This has the benefit that it simply doesn't freeze anymore. The mini-computer never runs out of resources, stays relatively cool, and is very stable.

In 2020, I used an 8TB external hard drive for this to work, but I've chosen to do this with a 500GB SSD this time. The Raspberry Pi boots off of this SSD as well, so I no longer need a Micro SD card, which is unreliable when it comes to systems that do a lot of writing (logging for example).

Backups

One thing that everyone does after it goes wrong for the first time is backing up your files. I've been bitten by this before, so I was prepared this time. Nextcloud files are stored on the local SSD, but every night, I upload everything to Storj using Rclone. Rclone is smart enough to only update files that have been added or updated but it also deletes files that have been deleted in Nextcloud.

The huge benefit of using Rclone is that it doesn't take a lot of resources and is finished very quickly if there haven't been any file changes in the past 24 hours. If, for some reason, the SSD on the Raspberry Pi breaks, I've got a backup of everything.

The Challenges

The transition from using cloud services to migrating to this new setup was much easier than I expected. I downloaded my whole Dropbox folder on my desktop and uploaded it to my Raspberry Pi using Rsync, a nice built-in file-copying tool for Linux. After that went well, I did the same for Google Drive.

After indexing all of those files in Nextcloud, I downloaded the Nextcloud app on my phone, turned on auto upload, and watched my photos appear. But this is where I encountered a problem: Nextcloud wants to resize all images on the fly to generate thumbnails. This was a performance killer, so I found Preview generator and these problems were solved.

This generates previews of these images and stores them on the SSD for later. It takes a bit more disk space, but it's worth it as this makes the Nextcloud instance more stable and much faster.

The result

I've now got a very stable and secure cloud that "just works". Granted, it "just works" because I've learned a lot about Docker and servers in the past few years. That doesn't take away that I have full control over my data and have the option to pick and choose between open-source software and proprietary software. I'm never locked in, and I think that's what Open Source is all about.

Open source software for everyone

Oh, before I forget! I've found an incredible website full of open-source software that I think you'll enjoy: selfh.st. You can find a lot of different types of software that could help you move away from "the cloud".