DEV Community: Roelof Jan Elsinga

From "The Cloud" to my cloud: I'm back to self-hosting

Roelof Jan Elsinga — Thu, 04 Apr 2024 10:00:00 +0000

From "The Cloud" to my cloud: I'm back to self-hosting

In the past 1-2 years, it feels "the cloud" has become a huge mess. So, I took a moment to really think about how much it's taking over our digital tools and services. In my opinion, it's creating a lot of value, but the really big players often don't have the right intentions.

I decided to make a big change. Instead of letting those huge cloud companies handle all my data, I've switched to self-hosted, open-source solutions. In this blog, I'm sharing my journey from relying on the big players to creating my own cloud.

Things I'll cover in this blog post:

RSS feeds are still relevant
Nextcloud, Rclone, Storj
Tailscale & Wireguard

Personal journey

I used to be all about building my own stuff and reinventing the wheel whenever possible. But like a lot of people, I got sucked into the convenience of using Dropbox and Google Drive. It was easy, didn't take a lot of time, and for a while, it pulled me away from that whole DIY mentality I had going in my early tech days. But it didn't stick.

After seeing too many horror stories about cyber attacks, weird privacy policies, and just being fed up with corporate services, I decided to go back to doing things my way, by self-hosting what I use most. This time around, unlike several years ago, I know what I'm doing.

I want to be in control of my own data again be able to make my own choices.

Why open-source

Open source isn't just about access to the source code; it's a philosophy. It's freedom, transparency, and control over the tech you use in your life.

By adopting open-source software, I choose to take responsibility for my digital privacy and make software work around my life, not the other way around.

No longer do I blindly trust algorithms or hand over my information to these large players without really thinking about it first. Instead, I choose to build a cloud that has everything I need, everything I want to do myself, and has the stuff I want control over.

Building my cloud

When it comes to building a cloud, there are a few things you need. In my case, the Raspberry Pi 4 for this post in 2020, a layer of security, and software to run to make my life a little easier. Bonus points for a backup strategy.

Security

I got started with security by setting up secure and super simple remote access with Tailscale. It's a very straightforward piece of software that lets you create a VPN mesh network using Wireguard. You can set up connections between several devices in literal minutes.

Tailscale is open-source, but has a hosted version of their software. As I'm not willing to compromise on security and I'm by no means a security expert, I'm trusting their hosted service to help set up my VPN mesh network. After the initial connection, all VPN traffic is peer-to-peer between the devices, so my phone is contacting with my Raspberry Pi directly.

With Tailscale I can access my server at home from anywhere in the world. No need to forward any ports in my router or expose the Raspberry Pi to the internet. Win-win!

RSS

Then I took back my RSS feed, which is pretty straightforward but super powerful. It gave me back control over what I consume online. Unlike those AI news feeds that throw random stuff at you, my RSS feed lets me subscribe to content I enjoy and value. I use Miniflux for this. It's very basic, fast, and straight to the point. That's all I'm asking for. No distraction, just interesting content.

Storage

One of the things I mentioned in the beginning and I've mentioned before on this website is that I'm a huge fan of Nextcloud. It's a self-hosted version of Dropbox, but with a lot of very useful plugins you can install.

Last time (in 2020) I set up an instance of it on my Raspberry Pi using Snap. This time around, I'm using Docker Compose instead. This has 2 reasons: It's my comfort zone and resource management.

With Docker Compose I can set very specific resource limitations for Nextcloud on my Raspberry Pi. This has the benefit that it simply doesn't freeze anymore. The mini-computer never runs out of resources, stays relatively cool, and is very stable.

In 2020, I used an 8TB external hard drive for this to work, but I've chosen to do this with a 500GB SSD this time. The Raspberry Pi boots off of this SSD as well, so I no longer need a Micro SD card, which is unreliable when it comes to systems that do a lot of writing (logging for example).

Backups

One thing that everyone does after it goes wrong for the first time is backing up your files. I've been bitten by this before, so I was prepared this time. Nextcloud files are stored on the local SSD, but every night, I upload everything to Storj using Rclone. Rclone is smart enough to only update files that have been added or updated but it also deletes files that have been deleted in Nextcloud.

The huge benefit of using Rclone is that it doesn't take a lot of resources and is finished very quickly if there haven't been any file changes in the past 24 hours. If, for some reason, the SSD on the Raspberry Pi breaks, I've got a backup of everything.

The Challenges

The transition from using cloud services to migrating to this new setup was much easier than I expected. I downloaded my whole Dropbox folder on my desktop and uploaded it to my Raspberry Pi using Rsync, a nice built-in file-copying tool for Linux. After that went well, I did the same for Google Drive.

After indexing all of those files in Nextcloud, I downloaded the Nextcloud app on my phone, turned on auto upload, and watched my photos appear. But this is where I encountered a problem: Nextcloud wants to resize all images on the fly to generate thumbnails. This was a performance killer, so I found Preview generator and these problems were solved.

This generates previews of these images and stores them on the SSD for later. It takes a bit more disk space, but it's worth it as this makes the Nextcloud instance more stable and much faster.

The result

I've now got a very stable and secure cloud that "just works". Granted, it "just works" because I've learned a lot about Docker and servers in the past few years. That doesn't take away that I have full control over my data and have the option to pick and choose between open-source software and proprietary software. I'm never locked in, and I think that's what Open Source is all about.

Open source software for everyone

Oh, before I forget! I've found an incredible website full of open-source software that I think you'll enjoy: selfh.st. You can find a lot of different types of software that could help you move away from "the cloud".

How do you choose a database type for a software project?

Roelof Jan Elsinga — Fri, 13 Oct 2023 10:00:00 +0000

How do you choose a database type for a software project?

In the world of software development, databases are the heart of most applications. These essential tools serve as the backbone of every web application, storing and retrieving data that powers the dynamic content we interact with. When starting or expanding a software project, one of the most important decisions you'll make is choosing the right database type.

In this blog post, we will explore the strengths and use cases of three popular database systems: MySQL, MongoDB, and Neo4j. By understanding the advantages and intricacies of each, you can make an informed decision tailored to your project's unique needs.

What is the role of a database in a software project?

Before we look at the different database types, let's take a moment to think about the role that databases play in software development. A database is the repository where data is stored, managed, and retrieved by your application. Whether it's a blog, an e-commerce platform, or a social media network, databases are at the heart of all software projects.

A well-chosen database system can benefit performance, scalability, and data integrity. It ensures that your application can efficiently store and retrieve the information it needs to operate seamlessly.

If you've always used a single database type, you might be wondering why you should even think about this. Every database stores data that you can use. Why does the database type matter? Well, let's look at 3 different databases a little more closely to find out.

MySQL: The Relational Database

MySQL, a popular open-source relational database management system, is a dependable choice for many software applications. It excels in structuring data into tables with predefined schemas, making it ideal for projects that require structured, predictable data.

Use Cases : MySQL shines when your application demands structured data with fixed relationships. It's a great fit for content management systems, e-commerce websites, and data-driven applications.
Tips for Effective Data Modeling in MySQL : Effective data modeling in MySQL involves designing a clear schema that anticipates your application's data needs. Normalization is key to avoid data redundancy, and well-indexed tables improve query performance.
Choose this database if : Your project relies on structured, predictable data with well-defined relationships, and you need a robust, ACID-compliant database for data integrity and consistency.

In conclusion, MySQL offers a reliable, structured way to store data and is particularly effective when dealing with predictable data and well-defined relationships. Its adherence to ACID principles guarantees data integrity and consistency. Now, let's delve into the realm of NoSQL databases, starting with MongoDB, and explore how it contrasts with relational databases.

MongoDB: The NoSQL Database

MongoDB, a leading NoSQL database, takes a different approach. It's a document-oriented database that stores data in flexible, schema-less documents, making it an excellent choice for projects that require dynamic data structures.

Use Cases : MongoDB is well-suited for applications with unstructured or semi-structured data, such as social networks, content management systems with variable data, and IoT platforms.
Tips for Effective Data Modeling in MongoDB : Effective data modeling in MongoDB involves understanding your data's structure and designing your documents to match the way your application accesses data. It offers the flexibility to quickly adapt to changing business requirements.
Choose this database if : Your project involves dynamic, semi-structured data, and you value flexibility, scalability, and quick development iteration. It's an excellent fit for applications where the data will change quite often.

In essence, MongoDB's flexibility and scalability make it a strong contender for projects with evolving data requirements. However, bear in mind that every project is unique, and the "best" database depends on your specific needs and the nature of your data. Now that we've covered relational and document-oriented databases, let's move on to exploring the world of graph databases, starting with Neo4j.

Neo4j: The Graph Database

Neo4j is a graph database designed to handle complex relationships and highly interconnected data. It's an excellent choice for projects that need to navigate intricate networks or analyze graph-like data.

Use Cases : Neo4j is an indispensable tool for applications dealing with social networks, recommendation engines, fraud detection, and any scenario where relationships between data, not the data itself are a core component.
Tips for Effective Data Modeling in Neo4j : In Neo4j, data is modeled as nodes and relationships. Effective modeling involves identifying the entities, relationships, and properties in your data and defining them as nodes and edges in a graph.
Choose this database if : Your project revolves around complex relationships and you need to perform graph-based queries or deep analysis. It's the ideal choice for applications that deal with unpredictable and even unknown depths or relationships between entities.

In conclusion, Neo4j is fantastic at handling complex relationships and interconnected data, making it a game-changer for projects that deal with extensive data interlinking and analysis. It's the go-to choice for uncovering insights from relationships and navigating through previously unknown data efficiently.

Conclusion

In conclusion, the choice of a database system is an important aspect of software development. MySQL, MongoDB, and Neo4j each offer specific advantages and use cases. By carefully evaluating your project's needs, you can make sure your data is stored and managed optimally, so you can quickly and easily develop your apps.

Using Caddy for automatic SSL certificates with Cloudflare

Roelof Jan Elsinga — Sat, 23 Sep 2023 10:00:00 +0000

Using Caddy for automatic SSL certificates with Cloudflare

Cloudflare has been a blessing and a curse when it comes to taking care of SSL certificates for your websites and web applications. Cloudflare does most of the heavy lifting and makes sure you're protected from attacks. However, it's also a curse. Let me explain!

Why Cloudflare and generating SSL certificates are a challenge

Cloudflare serves as a proxy between the internet and your webserver. This is great because they can filter suspicious behavior and protect your web server from attacks. The attacker will never be able to determine the IP address of your server, because this is hidden by Cloudflare.

However, this is also a problem. When generating SSL certificates, the provider needs to be able to verify that the requesting server is actually who they say they are. Certbot for example, will need an external server to reach your server to verify that the domain name points to the requesting server. Only then if it can verify this, will it generate an SSL certificate for that domain.

The problem is that you still have Cloudflare in the middle of this interaction. The request from the external server will never reach your web server, so it can't verify the domain name points to your server. You can circumvent this by temporarily turning off the Cloudflare proxy and performing this check, but now you're not protected. And what happens when it's time to renew your certificate? Will you temporarily disable the Cloudflare proxy every time? I don't think so.

How does Caddy generate SSL certificates behind Cloudflare?

So I hear you asking: "Even if you use Caddy to automatically generate these SSL certificates, you'll run into the same problems". And you would be right. However, Caddy has a very nice plugin you can install that interacts with the Cloudflare API to solve DNS challenges for LetsEncrypt.

The benefits of these are fantastic because you get:

HTTPS between your server and Cloudflare AND HTTPS between Cloudflare and your visitors.
Zero, yes truly Zero, maintenance when it comes to SSL certificates. Caddy will automatically generate new certificates and you won't have to mess with Cloudflare, ever.

If Caddy itself didn't make you excited about web servers, the thought of zero maintenance and no messing around with Cloudflare will. Let's look at how we can integrate this.

How to integrate the Cloudflare DNS module in Caddy with Docker

The docker part of this integration is optional, but I prefer to run all of my applications in containers, including my Caddy installation. So you can follow that part or skip over it, your choice.

First, we'll need to use caddyserver/xcaddy to build a custom binary for Caddy. Caddy is written in Go, so we'll need to compile our binary to include the Cloudflare plugin.

You can do so by using this command:

$ xcaddy build \
    --with github.com/caddy-dns/cloudflare

This command will create a binary that includes the base Caddy installation and add the Cloudflare DNS module. In a multi-stage docker image this will look like this:

FROM caddy:2.6.1-builder AS caddy-builder

RUN xcaddy build \
    --with github.com/caddy-dns/cloudflare

FROM caddy:2.6.1-alpine

COPY --from=caddy-builder /usr/bin/caddy /usr/bin/caddy

The reason I'm putting this in a multi-stage docker image, rather than exposing it in 1 stage is the final image size. The caddy builder image is quite large, as it includes all the development software. You won't need any of this in production. All you'll need is the resulting Go binary at /usr/bin/caddy.

One of the downsides is that building your custom Caddy binary isn't the quickest process. Luckily, you won't have to do this very often, so it's a great idea to publish your own Caddy base image (the Dockerfile above) and include that when you're building your final application.

As an example of this, I'll create a base image from the Dockerfile above and call this: roelofjanelsinga/caddy-cloudflare

I can now use this for hosting my web app like so:

FROM php:8.1-cli AS build-env
COPY --chown=www:www . /var/www/html
WORKDIR /var/www/html

# Perform some other build steps, like "npm run prod" and "composer install"

FROM roelofjanelsinga/caddy-cloudflare:latest

COPY --from=build-env /var/www/html /var/www/html
COPY ./docker/caddy/Caddyfile /etc/caddy/Caddyfile

Again, this is a multi-stage build to have the smallest final image as possible. In the second step, I'm using the custom Caddy base image, I'm copying my project files from the build step into the final image, and I'm including a custom Caddyfile with my domain configuration. This image can be served and will automatically take care of SSL.

Getting the Cloudflare API Token

In the custom Caddyfile, we'll need to add an entry to tell Caddy we want to use Cloudflare for our DNS challenges. Let me give you an example of my configuration first:

https://mydomain.com {

    root * /var/www/html/public
    encode zstd gzip
    file_server

    tls {
        dns cloudflare {env.CF_API_TOKEN}
        resolvers 1.1.1.1
    }

}

Keep in mind, that this is a barebones, non-production configuration. It's just here to illustrate how to make this process work.

In the TLS configuration, we've noted that Cloudflare should be used for DNS challenges and you're seeing an environment variable for a Cloudflare API token.

Let's see how to get that token:

Log in to Cloudflare and go to the domain you want to enable Caddy for.
On the right, you'll see a section with the "API" header. Click "Get your API token":
Under the API tokens block, click "Create Token":
On the "User API Tokens" page, scroll to the bottom and press "Get started" in the Create Custom Token section:
Give your token a descriptive name, and add 2 permissions:
1. Zone - Zone - Read
2. Zone - DNS - Edit
Click "Continue to summary" and you should now see your API token.

Using the Cloudflare API Token with Caddy

Now that you have the API token, the easiest way to use this is by including it in your docker-compose settings:

But you can supply this environment key in any way that works for you. Now, after you start your caddy container, you'll notice that it's instantly trying (and succeeding) to perform DNS challenges through Cloudflare and generate SSL certificates for your application.

You'll now have automatic SSL certificates as long as you use Caddy and you'll never have to mess with Cloudflare again. Pretty easy, right?

Quick tip: Reduce your Docker Image size when using the League Flysystem s3 adapter

Roelof Jan Elsinga — Sun, 22 Jan 2023 11:00:00 +0000

Quick tip: Reduce your Docker Image size when using the League Flysystem s3 adapter

If you're deploying your PHP applications as Docker images and you're interacting with S3 as your filesystem with Flysystem, you'll know that your Docker image won't be small. Flysystem is a fantastic library to interact with the filesystem and adding support for interacting with an S3 bucket is very easy, but it comes with the downside of having to include the massive AWS PHP SDK.

The current AWS PHP SDK includes all classes for every AWS service, even if you're never planning on using anything else besides S3. The author of Flysystem (Frank de Jonge) has already marked this as an issue, as the entire AWS PHP SDK is 29mb. You can find that discussion here: Reducing package size.

This massive SDK is difficult to explain when you only need a tiny sliver of the functionality, so let's fix that!

Remove any unused dependencies

As a response to this, the AWS team has proposed a possible solution, which requires very little from you as a developer, but saves you 28mb! The AWS team has created a callback that you can add to your composer.json and removes all the services you don't plan on using: Removing Unused Services.

These are all the changes you'll need to make to your composer.json and it'll instantly make your Docker image size smaller without breaking any of the great functionality or league/flysystem:

{
  "require": {
    "league/flysystem-aws-s3-v3": "^1.0"
  },
  "scripts": {
    "pre-autoload-dump": [
      "Aws\\Script\\Composer\\Composer::removeUnusedServices"
    ]
  },
  "extra": {
    "aws/aws-sdk-php": [
      "S3"
    ]
  }
}

The pre-autoload-dump hook allows you to add a hook to the composer process, which in this case reads the composer.json file and determines which services you'd like to preserve. It'll remove all other services from the SDK. There are a few services which will always be included, no matter if you use them or not:

The AWS team has marked these namespaces as "unsafe to delete", so they'll always be included in the dependencies.

The configuration in the extra key contains a list of the namespaces you'd like to preserve. As I'm only using the S3 namespace, this is the only service I'm listing. Technically, I don't need to do this, as this namespace will never be removed by this Composer hook, but I like to be explicit about my dependencies.

So if you're trying to make your Docker image smaller and you're using Flysystem with the S3 adapter, be sure to implement this in your project! It'll save you many precious megabytes!

How to enable step debugging in PHP with Xdebug 3 and PHPStorm

Roelof Jan Elsinga — Thu, 29 Dec 2022 11:00:00 +0000

How to enable step debugging in PHP with Xdebug 3 and PHPStorm

Are you looking to debug your PHP code more efficiently? You can do this very easily with Xdebug 3 and PHPStorm. With this setup, you will be able to step through your code line by line, allowing for better visibility into how your program is executing and helping you identify any issues quickly. In this guide, I will show you how to enable step debugging in PHP with Xdebug 3 and PHPStorm.

These are the steps we're going to do to enable step debugging in PHPStorm:

Configure Xdebug 3
Configure PHPStorm
Testing it out

It's surprisingly simple to enable step debugging for PHP, so let's get started!

Configure Xdebug 3 to enable step debugging

The first step to enabling step debugging for PHP is to configure Xdebug 3. In this guide, I assume you've already installed PHP and Xdebug 3, so if you haven't, do so before continuing this process.

First, we'll need to find out which configuration we need to change to enable step debugging for Xdebug 3. An easy way to do this is by running an example PHP application, in my example Laravel, and placing phpinfo(); in the main script (public/index.php for Laravel). This shows a list of all loaded configurations in the "Additional .ini files parsed" section. Here you'll find a listing for Xdebug as well.

Usually, this is loaded from /etc/php/8.1/cli/conf.d/20-xdebug.ini. Of course the path depends on your PHP version and operating system. In my case, I'm running PHP 8.1 on Ubuntu, so yours might differ.

Find the path to your Xdebug configuration and open this in a text editor (or gedit, nano, vim) and change the contents of this file from:

zend_extension=xdebug.so

zend_extension=xdebug.so
xdebug.start_with_request=yes
xdebug.client_port=9003
xdebug.client_host=127.0.0.1
xdebug.mode=debug
xdebug.idekey=PHPSTORM

This configuration tells Xdebug to send information to port 9003 on your local machine. Now we'll configure PHPStorm to start an Xdebug client on port 9003.

Configure PHPStorm to enable step debugging

Now that we've configured Xdebug 3, we'll need to configure PHPStorm to receive debugging information from Xdebug. We'll need to set up an Xdebug Client inside PHPStorm. We can configure this by going to the PHPStorm settings: Ctrl + Alt + S. In the search bar, search for debug. You'll see the following screen:

In the Xdebug section, be sure that the Debug port is set to port 9003, and the Can accept external connection option is checked. Press "Apply" and "OK". Now you're ready to test your step debugging!

Test the step debugging with Xdebug 3 and PHPStorm

You've configured Xdebug 3 and PHPStorm, so now it's time to test your new debugging abilities! Run your PHP development server (php artisan serve for Laravel), place a breakpoint in your code, and Start Listening for PHP Debug Connections:

Once you've started listening, you can go to your application in the browser and you should get a notification in PHPStorm. You will need to select the project to assign the notification to, but after that, you'll get nice breakpoints in PHPStorm:

You can now place breakpoints anywhere and start debugging your PHP applications like a programming ninja!

Conclusion

With the help of Xdebug 3 and PHPStorm, you can now debug your code like a pro. You'll be able to quickly identify any issues that may arise in your applications with ease. With a few simple steps, I've shown you how to configure step debugging for PHP so that you can save time and energy when developing complex projects. As long as you follow these instructions, it should only take minutes before start seeing great results from using this powerful toolset!

Why you should (still) add an RSS feed to your content website in 2023

Roelof Jan Elsinga — Fri, 23 Dec 2022 11:00:00 +0000

Why you should (still) add an RSS feed to your content website in 2023

Do you want a simple way to keep up with all the latest news and updates from your favorite content websites? Well, RSS feeds are here to help! An RSS feed (or Really Simple Syndication) is like an online newspaper that collects stories from multiple sources. It's a great way for readers to quickly find out what's new without having to visit each individual website. And even though it may seem outdated in 2022, adding an RSS feed to your content website can be hugely beneficial in 2023 and beyond. In this blog post, I'll explain why you should still add an RSS feed to your website in this day and age.

These are the topics we're going to look at:

What is an RSS feed?
How do I add an RSS feed to my website?
What are the benefits of adding an RSS feed to a website?
Is it worth adding an RSS feed in 2023?
Are there any downsides to using an RSS Feed for websites?

Let's get started!

What is an RSS feed?

Have you ever heard of an RSS feed? It stands for "Really Simple Syndication" and it's a great way to get the latest news and updates from multiple sources, without having to go to each website individually. An RSS feed is like creating an XML-based file (called a "feed") that contains all your website's recent content. People can subscribe to the feed, so their favorite news reader will automatically update whenever new content is published.

Back in the early 2000s, RSS was developed by Netscape Communications Corporation as an alternative to other web syndication formats. This version - which was then called RSS 0.91 - quickly became popular with content publishers.

A few years later, software developers further improved the format into what we now call RSS 2.0. This version had more control over how information was displayed to news readers, and it became even more popular because of its flexible nature and ease of use compared to other web syndication formats.

Nowadays, people use RSS feeds for blogs, news sites, podcasts, and more - helping us all stay up-to-date on the latest content from our favorite sources!

How do I add an RSS feed to my website?

Adding an RSS feed to your website is quite easy. If you're using a CMS to maintain your website, chances are you'll already have an RSS feed on your website. You can check this by going to yourwebsite.com/rss.xml. If you see an XML document, you've already got an RSS feed on your website and you can skip to the next section.

If you don't already have an RSS feed, creating one is quite simple. Let's look at a basic example:

<?xml version="1.0" encoding="utf-8"?>
<rss xmlns:dc="http://purl.org/dc/elements/1.1/" version="2.0">
    <channel>
        <description>My website name</description>
        <title>My website name</title>
        <link>https://mywebsite.com</link>
        <pubDate>Fri, 23 Dec 2022 12:00:00 GMT</pubDate>
        <item>
            <description>This is a description of my blog post.</description>
            <title>The title of my blog post</title>
            <link>https://mywebsite.com/blog/my-great-post/</link> <guid>https://mywebsite.com/blog/my-great-post/</guid>
            <pubDate>Fri, 23 Dec 2022 12:00:00 GMT</pubDate>
            <media:content xmlns:media="http://search.yahoo.com/mrss/" url="https://mywebsite.com/image.jpg" medium="image" type="image/jpeg" width="640" height="426"/>
        </item>
    </channel>
</rss>

Each RSS feed starts with an XML version declaration, has an RSS wrapper, and contains channel and item information.

The channel contains information about your website: title, description, URL (link), and the publish date of this feed (pubDate). The items contain similar items: title, description, URL (link), pubDate, and a media object.

There are many more options, but these are the basics of an RSS feed for a blog.

You can find the full RSS 2.0 spec at W3C.

What are the benefits of adding an RSS feed to a website?

Adding an RSS feed to your website can be hugely beneficial in 2023 and beyond. Here are some of the greatest benefits:

Increased visibility : By providing an RSS feed, you make it easier for users to find new content on your website without having to visit every page manually. This makes it easier for them to stay updated with your latest posts and updates.
Improved discoverability : RSS feeds enable content to be syndicated across different platforms, which can help increase your visibility in search results and boost organic traffic to your website.
Higher engagement rates : By providing an RSS feed, you make it easier for users to stay engaged with your content. As a result, you can increase the amount of time users spend on your website, which can lead to higher conversion rates and a better overall user experience.
Easier content promotion : RSS feeds make it easier for you to promote your content through social media sites like Twitter, Facebook, and LinkedIn. It also allows you to build automation on new items in your RSS feed. For example, this post is automatically published to LinkedIn and Twitter, because the RSS feed has been updated with this blog post.

In short, adding an RSS feed to your website in 2023 can help you increase your visibility, engagement, and overall user experience. So don't wait any longer - start setting up your RSS feed today!

Is it worth adding an RSS feed in 2023?

RSS used to be the best way to get your content out into the world. Social media took this advantage away because its content was much easier to digest than setting up an RSS feed. However, with more incredible (no-code) tools out there than ever, the role of the RSS feed is coming back!

RSS feeds can be used for more than just content promotion. They can be used to automate content publishing across different platforms, helping you get the most out of your content and reach a wider audience. You can automatically cross-post your blog posts to many platforms at once, you can synchronize your e-commerce offerings to Google shopping, your Facebook product catalog, and Instagram shopping. The best part is that you don't need to set up API access for this automation! All you need is your trusty RSS feed.

All in all, no doubt adding an RSS feed to your website in 2023 will bring many benefits and help you build a successful online presence.

Are there any downsides to using an RSS Feed for websites?

Like any technology, RSS feeds come with certain drawbacks. The biggest one is that most users nowadays don't use RSS readers. So, you'll have to rely on other methods (such as email newsletters) to notify your followers of new content and updates. Additionally, if you're running a blog or an e-commerce site, you need to make sure your RSS feed is up to date with the latest content. If it's outdated, it can be a turn-off for visitors and they may not engage with your website.

Overall, an RSS feed can be hugely beneficial if you use it carefully and keep it updated. It's best to use this technology in combination with other methods such as email newsletters, social media posts, and blogs to get the most out of your content.

Conclusion

Adding an RSS feed to your website in 2023 can provide numerous benefits such as increasing visibility, discoverability, and engagement. However, you should make sure that your RSS feed is always up to date to get the most out of it. Additionally, you should use other methods such as email newsletters and social media posts in combination with your RSS feed to achieve the best results.

Overall, if used correctly, an RSS feed can be a great tool to help grow your website's online presence and reach more people with your content. So don't wait any longer - start setting up your RSS feed today!

Why Caddy might be the best start for your next software project

Roelof Jan Elsinga — Sat, 22 Oct 2022 10:00:00 +0000

Why Caddy might be the best start for your next software project

Do you dread setting up a webserver for your project(s) and do you wish you could skip this altogether and it'll still magically work? Then Caddy might be the perfect webserver for your next project!

In this post, we're going to look at a few of the benefits of Caddy that instantly sold it as the replacement for Nginx and Apache. These are the topics we're going to look at:

What is Caddy?
How does Caddy compare to other webservers?
Why should you use Caddy for your next software project?
How do you get started with Caddy?

Let's dive right in and see what Caddy is all about!

What is Caddy?

Caddy is a webserver, like Nginx and Apache, that routes traffic from the internet to your application, static files, or acts as a reverse proxy.

But it's so much more than just a webserver, it also automatically generates SSL certificates for your website, takes care of caching, and its configuration is hysterically simple compared to Nginx & Apache.

Caddy takes care of the things that you dread doing in projects with Nginx and Apache. Sure, it's not difficult to set up SSL for those webservers, but it's still something you have to think about. Caddy does all this work for you, so all you have to do is point it to your application and it just works.

The Caddy team has put together an excellent illustration of what Caddy is in technical terms:

How does Caddy compare to other webservers?

So how does Caddy compare to other webservers like Nginx and Apache? I think the most important thing to look at for most developers is performance. How fast is it compared to the other webservers? There are a lot of benchmarks on the internet that compare Caddy with Nginx, for example 35 Million Hot Dogs: Benchmarking Caddy vs. Nginx. This benchmark shows something very interesting:

Nginx is marginally faster than Caddy at serving requests
Caddy is multi-threaded and will start to use latency when it's getting hit with massive numbers of HTTP requests...but it will never fail to fulfill the request.
Nginx is single-threaded (in this test) and will start to refuse connections once it's at its capacity
Nginx only outperforms Caddy with a special performance-optimized configuration, while Caddy stays at a "stock" configuration

Reading that benchmark is worth your time if you're interested to learn more about the performance comparison.

I've unfortunately not been able to find a good benchmark that includes Apache as well. If I do happen to find one, I will update this and post a link to the comparison.

So if you look at the performance of Caddy compared to other available webservers, we can see that it's not just comparable with Nginx, but sometimes even outperforms it. The biggest takeaway from reading that benchmark is the fact that Caddy is so fast with a "stock configuration". You don't have to spend time trying to optimize your configuration to get excellent speeds like you would have to with Nginx. This is what I want! I want the webserver to just work and get out of my way. Caddy accomplishes this for me!

Why should you use Caddy for your next software project?

I've already mentioned a few reasons why you should use Caddy for your next software project, but let's list them again:

It's comparable, if not faster, than Nginx
Its configuration is incredibly simple
You don't have to think about SSL certificates
It's multi-threaded, so it'll actually make use of the server you're running it on

Let's add a few excellent benefits to using Caddy for your next project!

First of all, Caddy doesn't have any dependencies. It's a compiled Go binary that can run anywhere. This makes it ideal for using it in a container in your Docker or Kubernetes environment.

Secondly, it offers things like load-balancing traffic to backend services, health checks, circuit breaking, and caching out-of-the-box. For me, this is a clear benefit, because this saves a lot of time...time you can spend on building your application.

Lastly, the biggest benefit of using Caddy is that it has built-in directives for using fastcgi. This helps you to route traffic to your PHP application with a single line in your configuration file.

But how do you get started with Caddy? Let's find out!

How do you get started with Caddy?

Getting started with Caddy is simple! You can use the CLI to serve your configuration and start a server right then and there, or you can use Docker. I prefer to use Docker because it fits perfectly in my projects like that.

Let's look at the configuration to serve traffic to our PHP/Laravel application:

(laravel) {
    root * /var/www/html/public
    encode zstd gzip
    file_server
}

(redirect_clean_url) {
    handle_path /index.php* {
        redir {uri} permanent
    }
}

:8000 {
    import laravel
    import redirect_clean_url

    handle {
        try_files {path} {path}/ /index.php?{query}
        php_fastcgi my_php_fpm_backend:9000
    }
}

The code block starting with (laravel) and (redirect_clean_url) are snippets, essentially reusable blocks of configuration that you can apply to any configuration by importing them: "import laravel" and "import redirect_clean_url". If you've hosted a Laravel application before, you'll know that you have to route all traffic to the index.php file in the public folder of your project. The "laravel" snippet sets the document route to that public folder, enables gzip and zstd encoding, and tells Caddy to serve static files from the public folder (images, css files, etc.).

The redirect_clean_url snippet is an important snippet to improve the SEO of your application. Routing all traffic to your index.php file is great, but it also causes a problem: both https://example.com and https://example.com/index.php are valid paths. This causes ugly URLs (https://example.com/index.php/blog for example) and Google could mark this as duplicate content. To prevent all of this, we want to redirect all traffic from /index.php to a clean URL without index.php in it. This snippet takes care of redirecting /index.php/blog to /blog. It does this with a 301: take note of the "permanent" keyword.

The ":8000" code block is our actual website, in this case, localhost:8000. You can also replace this with example.com and it'll automatically generate SSL certificates for your domain and serve it through HTTPS. In the handle code block, we're telling Caddy to look for static files first and if it can't find those, redirect the traffic to the index.php in the public folder. When the traffic is routed to PHP, you can specify the FPM backend using the "php_fastcgi" directive. In my case, this routes the traffic to a PHP-FPM container in the same docker environment, but this could also be http://127.0.0.1:9000 or any other place you're running your FPM server.

Now you've got a fully functional webserver for your PHP application including an SSL certificate. That's really it. You're done. It's that easy.

The simplicity of this configuration file, the excellent performance of the web server, and the fantastic documentation make Caddy my preferred webserver.

Conclusion

Caddy is an excellent web server for several reasons: it's comparable to Nginx in terms of speed, its configuration is much simpler, and it doesn't require any thought about SSL certificates. It takes all the tedious work away when setting up a webserver and helps you to get back to developing your application or website! If you're looking for a new web server for your next project, look no further than Caddy.

5x performance increase: A simple trick to speed up your PHP application

Roelof Jan Elsinga — Thu, 30 Jun 2022 10:00:00 +0000

5x performance increase: A simple trick to speed up your PHP application

Have you been optimizing your PHP application but hit a wall and can't seem to get any more performance?

I've got a little (known) trick you can try! This simple addition increased the performance of one of my PHP application from 16.5 req/sec to 83 req/sec and even works in a Docker container!

What is this trick? OPcache!

What is OPcache?

OPcache is a PHP extension that compiles and caches your PHP scripts, so it can use the cached version of your application when you run the program. This is a huge speed improvement, because PHP is not a compiled language. Normally, PHP compiles your application and then executes it every time you do a request on your application. With OPcache, we can skip the compilation part of the request cycle.

Using OPcache in Docker

I run all of my PHP applications in Docker, so I will focus on how to make this work for a Docker image. You can replicate most of the steps for your own system, so if you're not using Docker, you can still follow along.

The configuration

This is the basic configuration we're using for our OPcache extension. This config file should replace the default opcache.ini at: /usr/local/etc/php/conf.d/opcache.ini.

[opcache]
opcache.enable=1
opcache.revalidate_freq=0
opcache.validate_timestamps=0
opcache.max_accelerated_files=10000
opcache.memory_consumption=192
opcache.max_wasted_percentage=10
opcache.interned_strings_buffer=16
opcache.fast_shutdown=1

This configuration will cache the compiled PHP scripts, even if you've made changes to them. For these changes to show up, you'll have to restart your PHP-FPM daemon. However, if you're running your PHP application in Docker, you won't have to worry about this. You should create a new Docker image when you make changes anyway, so you always start in production fresh after your made your changes.

If you want to use OPcache on your development environment, you'll want to see any changes you make right away, so you can set opcache.enable=0 or let the cache invalidate itself by using: opcache.validate_timestamps=1.

Install OPcache in Docker

Let's install OPcache on a php-fpm:alpine image:

FROM php:8.1-fpm-alpine

# Install build dependencies and the OPcache extension
RUN apk add --no-cache $PHPIZE_DEPS \
    && docker-php-ext-install opcache \
    && apk del $PHPIZE_DEPS

# Copy the opcache.ini into your Docker image    
COPY docker/php/opcache.ini /usr/local/etc/php/conf.d/opcache.ini

# Run your application
CMD php-fpm

This is a very minimal Dockerfile and might not even compile, but it does show the steps you'll need to take to enable OPcache in your application.

With this configuration, I've sped up my PHP application by 5x, increasing the requests per second from 16.5 to 83. I'm very satisfied with the results, so I hope it works for you as well.

How to migrate from Mailchimp to Postmark + Temporal

Roelof Jan Elsinga — Fri, 04 Mar 2022 11:00:00 +0000

Photo by Markus Winkler

How to migrate from Mailchimp to Postmark + Temporal

MailChimp is a great SaaS platform for sending e-mails to your subscribers and work with automation to sell to your mailing list. However, once you get a sizable audience and you want to customize your e-mail interactions, you'll be hit with a hefty price tag. Mailchimp is very expensive! If you've got a mailing list of 2000 contacts and you're on the cheapest paid plan, you're already paying $34 per month. This is a great option for a marketing team that has no access to a software developer, but I am one of those software developers.

Before we get started with the migration, let's introduce the 2 heroes of the story: Postmark and Temporal. If you haven't heard of these two before, here's a quick introduction. If you have, you can skip to the sections that you want to read:

What is Postmark?
What is Temporal?
Create e-mail templates in Postmark
Create Temporal workflows
1. The newsletter sending workflow
2. Handling unsubscribes from the newsletter

What is Postmark?

You might have heard about Postmark before, it's quite a well-known e-mail platform that allows you to send transactional and marketing e-mails, and even deal with incoming e-mails. In my experience, its delivery rate is better than a few other options out there. One of the biggest benefits is the fact that you can create templates in code and synchronize those with Postmark. This is a very important aspect for this entire process, because this allows me to send e-mails using a REST API.

What is Temporal?

The second piece of software that makes all of this possible is Temporal. Temporal is software that keeps track of the state of your code. In simple terms, it executes each line of code only once and can resume code execution after system crashes. This makes your code incredibly stable. If that hasn't convinced you of its power, perhaps the following example will clarify it.

Example

If you have a banking application and your customer is sending a payment to another customer and your program crashes halfway through: what happened to the payment? What will happen when you restart the application? Will this automatic process charge your customer twice? Temporal knows where the application was in its code execution and resumes from that exact line. Temporal will help you to never charge your customer twice, not even after a crash.

Delay code by weeks

Another great benefit of this state machine is that you can let processes take weeks or months if you want to. In normal code, you can put some type of delay (sleep, time.Sleep, etc.) in your code, but you wouldn't think of doing this for 14 days. Usually, you'd only delay your code by about 10 seconds or so.

But why is this important? This is important, because it drastically simplifies your code. Charging a customer for a subscription is now just a for-loop with a delay of 30 days after the charge. 30 days later, check if the customer is still subscribed, charge them, and delay for another 30 days. It's really that simple. Keep this in the back of your head, because this will become very important in this the scheduling of e-mails to send to your contacts.

One last thing to highlight before we can get started! You can schedule code execution based on a cron schedule. This will become the heart of the newsletter application. Enough talk, let's get to some code!

Create e-mail templates in Postmark

If you've ever sent e-mails from your code, you might have worked with local templates and sent ready-made HTML e-mails to your contacts. This is always a pain and I can't even begin to count the number of times an e-mail just doesn't look good on a mobile device or looks a little off on Gmail in Safari. Creating emails in HTML just takes you back to the "Good ol' HTML 4 days" where everything is a table and nothing is responsive. Let's not even try to attempt this and let Postmark handle this for you.

If you use the mailmason starter I've also used as a base, you'll see minimal HTML tables and mostly just simple HTML content. You can style your emails using simple CSS files and Mailmason will even generate the plain text versions of your emails.

After you've synchronized your templates with your Postmark server, you'll have templates with easy-to-use template aliases. You can use these template aliases, alongside the variables you'll need for your templates (first name, etc.) and send them as JSON to Postmark to send your email. Sending emails with a simple REST API call is a fantastic feeling.

For the sake of this post, let's imagine we're using a template like this:

<h1>Hello!</h1>
<p>
    I hope you've had a great week! This is what I've written for your this week, I hope you enjoy!
</p>

<img src="{{image_url}}" >

<br />

<h1>{{title}}</h1>

<p>{{description}}</p>

<a href="{{url}}" class="cta-button">Read &quot;{{title}}&quot;</a>

<p>
    I'm looking forward to sending you the next email soon, take care!
</p>

As you can see, we've got a few variables we can fill in using the REST API:

image_url
title
description
url

Let's give this template the alias "weekly-newsletter".

This is a very basic email template with minimal styling that'll be stored on the Postmark servers, rather than in your own application.

If you're interested in learning more about mailmason and my automatic process for synchronizing templates with Postmark, please reach out to me.

Let's move onto the heart of automation: Temporal.

Create Temporal workflows

Temporal, the state machine, has a concept called workflows. A workflow is a collection of individual steps that perform a (more complicated) task. A workflow should be deterministic, which means it has the same result every time it executes with the same input data.

Any nondeterministic behavior should be moved to an activity, because the result of an activity is recorded in the workflow history. You can interpret an activity as a step in your workflow. Some examples of code that's great for an activity is: sending an e-mail or fetching something from an API.

When we break this down for sending a newsletter, you can see a workflow as these steps:

Who do I send an e-mail to?
Is there something for me to send? If no, check again next week.
Tell Postmark to send contact X Template Y with variables Z
Done, check again next week

During this workflow, I want to skip the code as quickly as possible and check again next week. "Checking again next week" highlights one of the features Temporal offers for workflows: Cronjob workflows. These Cronjob workflows can be executed whenever you want to, in my case every saturday at 15:00 (3pm). This cron schedule looks like: "0 15 * * 6".

Starting (scheduling) a workflow in my code, which is written in Go, looks like this:

// Create an easy-to-use workflow ID
const WorkflowID = "newsletter-%s"

// Register the workflow with a string name
w.RegisterWorkflowWithOptions(EmailNewsletterWorkflow, workflow.RegisterOptions{
    Name: "email.newsletter",
})

type SubscribeRequest struct {
    Email string `json:"email"`
}

// Execute the workflow with a cron schedule
if _, err := s.client.ExecuteWorkflow(ctx, client.StartWorkflowOptions{
    ID: fmt.Sprintf(WorkflowID, "hello@example.com"),
    TaskQueue: Queue,
    CronSchedule: "0 15 * * 6",
}, "email.newsletter", SubscribeRequest{
    Email: "hello@example.com",
}); err != nil {
    return err
}

There is a lot of code that I'm omitting in this example, like setting up a Temporal worker and configuring the Temporal client. I'm also hardcoding the e-mail address in this example for simplicity's sake. The input data for the workflow, "SubscribeRequest", has json tags, because temporal stores this input data in the database and by specifying the keys it should use as json, you avoid some rare encoding and decoding issues.

The workflow ID

I'm highlighting the easy-to-use Workflow ID, because this will make it easy for us to stop the workflow in case the contact is unsubscribing from the mailing list. By specifying this workflow ID, you also prevent the workflow from running multiple times, in case someone accidentally (or on purpose) signs up for your mailing list multiple times. If you don't specify the workflow ID, it'll be assigned a random UUID, which makes it very difficult to cancel the workflow without saving this random workflow ID in another database.

The newsletter sending workflow

We've seen that we can execute a workflow based on a cron schedule, but the workflow doesn't do anything yet. Let's change that! In the workflow, we'll need to do 2 things:

Fetch the post we want to send our subscriber
Send the email

This workflow could look something like this:

type Post struct {
    Title string `json:"title"`
    Image string `json:"image"`
    Description string `json:"description"`
    URL string `json:"url"`
    PostedDaysAgo int `json:"posted_days_ago"`
}

w.RegisterActivityWithOptions(FetchLatestPost, activity.RegisterOptions{
    Name: "post.fetch-latest",
})

w.RegisterActivityWithOptions(SendPostmarkTemplate, activity.RegisterOptions{
    Name: "email.send",
})

func EmailNewsletterWorkflow(ctx workflow.Context, config SubscribeRequest) error {

    // We want to retry both of these activities a maximum of 10 times.
    ctx = workflow.WithActivityOptions(ctx, workflow.ActivityOptions{
        TaskQueue: Queue,
        StartToCloseTimeout: time.Minute,
        RetryPolicy: &temporal.RetryPolicy{
            InitialInterval: time.Second,
            BackoffCoefficient: 2.0,
            MaximumInterval: time.Minute,
            MaximumAttempts: 10,
        },
    })

    var latestPost Post

    // Call an endpoint within an activity to fetch the latest post
    if err := workflow.
        ExecuteActivity(ctx, "post.fetch-latest").
        Get(ctx, &latestPost); err != nil {
        return err
    }

    // We want to avoid sending emails if there wasn't a new post within the last 7 days 
    if latestPost.PostedDaysAgo > 7 {
        workflow.GetLogger(ctx).Info("latest post is posted more than 7 days ago, skipping...")
        return nil
    }

    if err := workflow.
        ExecuteActivity(ctx, "email.send", Config{
            TemplateAlias: "weekly-newsletter",
            Email: config.Email,
            From: "info@roelofjanelsinga.com",
            MessageStream: "newsletter",
            TemplateModel: map[string]interface{}{
                "title": latestPost.Title,
                "image": latestPost.Image,
                "description": latestPost.Description,
                "url": latestPost.URL,
            },
        }).
        Get(ctx, nil); err != nil {
        return err
    }

    return nil
}

This workflow fetches the latest post and checks whether it was posted in the past 7 days. If there hasn't been a new post in the past 7 days, the workflow returns and schedules a new workflow for next week.

If there was a new post in the past 7 days, the workflow executes an activity called "email.send" with the data the activity needs (Config)

This activity calls the Postmark API and sends the template with alias "weekly-newsletter" to the contact we've given to the workflow. In this post, we hardcoded this email as "hello@example.com". The TemplateModel map in the configuration are the variables you defined in your Postmark template. You can also choose to use a struct rather than a map, but I'm reusing this "email.send" activity for every workflow that sends emails, so this makes it easier to use for my application.

The email sending activity is quite straightforward and just converts the Config into an API call to Postmark:

type PostmarkResponse struct {
    To string
    SubmittedAt time.Time
    MessageID string
    ErrorCode int
    Message string
}

type PostmarkFailure struct {
    ErrorCode int
    Message string
}

func SendPostmarkTemplate(_ context.Context, config Config) (*PostmarkResponse, error) {

    client := sling.New().Base("https://api.postmarkapp.com")

    var success PostmarkResponse
    var failure PostmarkFailure

    _, err := client.
        Post("/email/withTemplate").
        Add("Accept", "application/json").
        Add("X-Postmark-Server-Token", "your-token-here").
        BodyJSON(Body{
            From: config.From,
            To: config.Email,
            TemplateID: config.TemplateID,
            TemplateAlias: config.TemplateAlias,
            TemplateModel: config.TemplateModel,
            MessageStream: config.MessageStream,
        }).
        Receive(&success, &failure)

    if err != nil {
        return &PostmarkResponse{
            ErrorCode: failure.ErrorCode,
            Message: failure.Message,
        }, err
    }

    return &success, nil
}

We can now subscribe our contacts to our mailing list and send them a weekly email! Unfortunately, not every contact will stay subscribed indefinitely, so we'll need to handle unsubscribes. Let's see how we can do that!

Handling unsubscribes from the newsletter

One of your contacts wants to unsubscribe from your mailing list, that's too bad! However, it's not difficult to implement this in our application, because we've thought about this when we started our initial workflow!

Remember the workflow ID? That very nice and easy to use workflow ID? That's going to make this unsubscribe process much...much easier!

When we executed the workflow for this contact, we've customized the workflow ID to contain the contact's email: newsletter-hello@example.com. Now, all we need to unsubscribe a contact from our newsletter is their email.

Since we're using a workflow with a cron schedule, we can't just cancel the workflow, because this causes a new workflow to be scheduled. We want to stop the workflows from being scheduled after the contact unsubscribes, so we'll need to terminate the workflow.

This is what it looks like:

type UnsubscribeRequest struct {
    Email string `json:"email"`
}

func (s service) Unsubscribe(ctx context.Context, payload UnsubscribeRequest) error {
    err := s.client.TerminateWorkflow(ctx, fmt.Sprintf(WorkflowID, payload.Email), "", "unsubscribed")

    if err != nil {
        s.logger.Error("Error terminating workflow", err)
        return err
    }

    return nil
}

The empty string that we pass to the TerminateWorkflow method represents the RunID. By leaving this RunID empty, Temporal assumes you want to terminate the latest run for this workflow.

In the code, you can also see "unsubscribed", this is the reason of termination. It's an optional thing to add, but it's nice if you're working in a team and wondering why your workflow is terminated.

Conclusion

In this post, I've described how I've migrated my newsletters from Mailchimp to Postmark + Temporal. Now, with Postmark and Temporal, I've got complete control over who, how, and what to send to my mailing list. The upside is that I can still do all things I could with Mailchimp, for a fraction of the cost. The downside of building all of this yourself is that you'll need to have technical expertise and solve any issues that arise by yourself.

I can live with those downsides, because working with both Postmark and Temporal is a delight! If something is unclear, Temporal has a great community to help you out.

If you have any questions about this process, don't hesitate to reach out!

Neo4j for SEO & UX: Easily create related content

Roelof Jan Elsinga — Sun, 18 Apr 2021 10:00:00 +0000

Neo4j for SEO & UX: Easily create related content

Improving your UX and SEO for your website takes time and effort. You need to give your content some context to help your readers and the search engines understand what it's about. What if you could automate this and do more with the same amount of effort?

3 months ago, I've written about using Neo4j, a Graph Database, for SEO and UX purposes. Neo4j might be complete overkill for the relatively small amount of data I put in it, but it has boosted my own productivity and internal link building strategy. Here's why!

Generating internal links

With a Graph database, you can make...well graphs. By linking pieces of content together, I've created a large cob web of content with dependencies, as you can see in this screenshot:

Neo4j screenshot of Plant care for Beginners content

As you can see, there are many, many connections between pieces of content. Each of these connections means that a piece of content is related to another piece of content. You can see where I'm going with this right? These connections help me to related page, which creates a giant web of internal links. These internal links help search engines to make sense of your content and give it context.

In a way, you're helping these search engines provide some context to your content. You don't need a database like Neo4j for this at all, you can do this manually as well. But it makes it a whole lot easier. By having a Graph database in place, which focuses on relationships between nodes (Plant, Article, Page, etc), you can quickly generate relevant internal content.

Content ideas are easier

If you've written a lot of content for a longer period of time, you'll know this feeling: "What do I write about now?". With a graph database, this becomes much easier.

If you can't think of anything to write about, you can start connecting the circles in the screenshot from earlier. Say you have Page A and Page B, they're great pages, but they have nothing in common, nothing connecting to them. You can now draw a line between Page A and Page B, which is called Page C and it'll serve as the common ground.

Before, you couldn't link from Page A to Page B, because they we're so different, but now you can link to it indirectly. You can link from Page A to Page C and then to Page B and vice versa. You've created content context, which is great for your readers, but also for the search engines trying to make sense of your website.

Automate your work

Again, you can do all of these things by hand. But doing this takes a lot of time and you will miss some connections that might be obvious to an automated system, You want great UX, you want great SEO, but you'll need great content for that. If you spend all your time manually creating connections between pieces of content, you'll never have time to actually write that content.

If you use Neo4j for your content, it will probably be massive overkill, but it will improve your workflow. At least it has for me. My normal writing process hasn't changed at all, but the impact it has on UX and SEO has been far greater. And that with the same effort as before.

My 2020 in review

Roelof Jan Elsinga — Wed, 30 Dec 2020 11:00:00 +0000

My 2020 in review

2020 has been a strange year for us all, but I don't want to linger on the negatives this year has brought. Instead, I'd like to highlight everything that went well for me. I'd like to invite you to do the same to end this strange year on a high. In this post, I'll highlight everything I've learned this year and everything I'm proud of. Quarantine has been a strange time, where all of a sudden I had a lot of extra time to spend on myself and my projects.

Highlights of my 2020

Golang

At the beginning of the year, I needed to make a few business processes faster that were written in PHP. I ran out of ideas on how to do this in PHP, so I gave Go a try. What a major success that turned out to be. The process execution time was shorten significantly: from 1 week to "just" 3 hours. Learning Golang turned out to have a major impact on my career., but more on that later.

The power and simplicity of Go convinced me to migrate a few other parts of a larger monolith application, written in PHP, to Go microservices.

Aloia CMS

My trusty CMS, Aloia CMS, got its first stable version this year (february) and has become a powerhouse for my projects. Its simplicity and extendability made it a very reliable base for my portfolio and Plant care for Beginners, which I'll talk about next. The latest version (3.3.0) brings model events when content is saved or deleted. This change alone makes it possible to build all kinds of automation scripts for the consuming Laravel application. For example, this allows me to easily synchronize data between the flat files and Neo4j.

In 2021, I hope to continue to improve Aloia CMS by making the performance even better and allowing for easier extensions for the built-in commands. This change will make it even nicer for developers to use.

Plant care for Beginners

2020 was THE year for my side projects! The extra time quarantine allowed me to write many plant care guides for Plant care for Beginners. In May, I enabled Google Adsense and since that point my side project actually started to make some money. At the beginning of December, I switched my ads from Adsense to Mediavine and I hope to stay with them for a while.

In the short term, I will migrate parts of the website from a purely file-based CMS (Aloia CMS) to Neo4j. The data will still be stored on disk, but content suggestions and content aggregate pages will be generated using Neo4j. This makes it much easier for myself to help visitors find the information they might be looking for.

CRO-tool

Sander had a great idea for a new side-project: CRO-tool. This project aims to help UX and CRO (conversion rate optimization) professionals to use psychology for their A/B experiments. Together, we've worked hard on getting an MVP off the ground, with success! We've sold a few early-bird lifetime licenses for our tool and we're planning to release around 20 new psychological theories of the next few months to keep adding value to our subscribers.

CRO-tool has taught me a lot about running a SaaS project on which people depend for their day job. The tool has to be as stable as possible at all times and changes should be made with care.

Ansible

Throughout this year, I've been looking at automating everything that could be automated. This includes deploying all websites I manage. Up to this point, this has always been a manual process. Using Ansible, I've created playbooks to automate the deployments of my websites. This helps to keep my mind at ease and prevents human error while deploying. I'm no longer at risk of forgetting something during deployments, because this has been automated.

Neo4j

Neo4j came on my radar quite late in the year, around mid-November. However, in this short time it has changed my views of what a database should be. A database should shape to your needs, not the other way around. If you're working with complex data, you'll often struggle with many joins in a query to get the data you need. Neo4j makes this much easier and the performance, even while doing many "joins", is many times better than MySQL. I've built a few projects with Neo4j already. One of which is a Go + GraphQL server and a Neo4j database. The other is an extension for Plant care for Beginners, using PHP and Neo4j together. Both have been a pleasure to work with so far.

A new job

I've found a new job this year! I've been working at Tubber for the past 5 years and I was ready for the next step. I've found the right place for me at Afosto. At this new job, I'll get to work with Kubernetes, Microservices, and a lot of Golang. I enjoyed working with Go so much that it ultimately resulted in finding a new job where I can use it more often. I hope to learn a lot about the topics I just mentioned in 2021 and using them in production.

Mastermind group

Since I've had more time to spend on my side-projects, having a Mastermind group has been an amazing experience. This mastermind group has given me new insights and ideas to use to improve my side-projects and make them profitable. It has been fine working on my side-projects by myself, but having others there as a soundboard and your best critics is great. "Fail fast" is really one of the things that I experience during the masterminds, because you have to to explain your problems and progress. By thinking about it and getting questions, you'll quickly know if your ideas are going to work or if they need a little more polish.

Fun statistics

These are some fun statistics for me about 2020:

I've written 76 blog posts on 4 websites:
- 32 on roelofjanelsinga.com
- 31 on plantcareforbeginners.com
- 10 on aloiacms.com
- 3 on blog.cro-tool.com
Launched 2 websites:
- cro-tool.com
- blog.cro-tool.com
Reached a peak of 63k visitors per month on Plant care for Beginners

A strange but productive year

2020 was a strange year, but it I've been very productive throughout it. I'm very happy with what I've accomplished and I hope to carry on this productive streak to 2021. I don't think I'll start any new projects in 2021, but who knows what will happen. I'm planning on improving what I've got now, instead of adding more to my list of projects. Quality over quantity.

I hope you've had a good year, considering the current state of the world. Even if you haven't had the time or energy to be productive this year, you've made it. Productive or not, I hope you learned something new or found something that brings you energy in some way. This year has highlighted how important mental health is, so work on that first, before trying to be "productive".

How to create a simple MQTT switch in Home Assistant

Roelof Jan Elsinga — Wed, 18 Nov 2020 11:00:00 +0000

How to create a simple MQTT switch in Home Assistant

When you're using Home Assistant for your home automation and you've got a few MQTT devices you might want to create simple switches for your devices. However, if you're like me, this simple task turned out to be a very tough task. This post is as much for you as it is for me, because I forget how to do this every time and each time I go through this it takes me hours to get going. In this short post, we're going to do 3 things:

Define an MQTT device as a sensor in Home Assistant (optional)
Define an MQTT device as as switch in Home Assistant
Create a simple on/off switch to toggle a state in your MQTT device

There are a few prerequisites when you go through this process:

You need to be able to edit the configuration.yaml file
You need to have the "Mosquitto broker" add-on installed in your Home assistant instance

Let's get right into it, so you can get back to building amazing automations.

1. Define an MQTT device as a sensor in Home Assistant (optional)

Defining your devices as a sensor is optional and doesn't have anything to do with creating a simple switch in Home Assistant, but it can allow you to create triggers based on the state (on or off) of your MQTT device in the future. So if you want to do this, you can go through this step, otherwise you can go to step 2.

To register your MQTT device as a sensor in Home Assistant, you need to define it in the configuration.yml file. Let's look at a basic example:

sensor:
  - platform: mqtt # This is an MQTT device
    name: "LED Switch 1" # Choose an easy-to-remember name
    state_topic: "home/office/led/get" # The topic to read the current state

After adding this sensor information, you can access the state of your MQTT device as "sensor.led_switch_1", or whichever name your specified: "sensor.whichever_name_your_chose". You can use this entity as a trigger to automate other things in the future.

2. Define an MQTT device as as switch in Home Assistant

Now the most important step of this whole post, defining an MQTT device as as switch in Home Assistant. To do this, open the configuration.yaml file again and add the following configuration:

switch:
  - platform: mqtt # Again, it's an MQTT device
    name: "LED Switch 1" # Choose an easy-to-recognize name
    state_topic: "home/office/led/get" # Topic to read the current state
    command_topic: "home/office/led/set" # Topic to publish commands
    qos: 1
    payload_on: 0 # or "on", depending on your MQTT device
    payload_off: 1 # or "off", depending on your MQTT device
    retain: true # or false if you want to wait for changes

The qos depends on your situation, but in short it means this:

0: A lot of messages are sent to the device and the connection is very stable
1: A message can be sent multiple times to ensure the MQTT received the message
2: A message can only be sent a maximum of one time and there is a handshake that makes sure the message is received

If you're building a simple switch, you can choose 1 or 2. Not a lot of messages will be sent and you want to make sure that your MQTT device received the message. You should now restart Home Assistant to make sure the configuration is loaded. Do this by going to: Configuration -> Server Controls -> Restart.

Now wait until your instance comes back online and you can move to the last step.

3. Create a simple on/off switch to toggle a state in your MQTT device

Now that we have registered your MQTT device as a switch, we can create a visual element for it on your dashboard. You can modify your dashboard by clicking the three dots at the top right of your dashboard and click "Edit dashboard".

If you've never edited your dashboard you'll get a message asking if you're sure you want to edit your dashboard. Just say yes and you'll have a screen like the screenshot below.

Click the orange button on the right to add a new element. You'll get an overview and we're interested in either the "Entities" or "Button" card from the screenshot below.

If you want a button that lights up when your MQTT device has the "on" state and is off when the state is "off", than choose the "Button" card. If you just want an on/off toggle, choose the "Entities" card. By creating a switch in step 2, you should now be able to easily create a visual element for your MQTT device and toggle its state by pressing a simple button in your dashboard. Like in the screenshot below.

MQTT switch off state

And when you toggle the switch or press the big lamp in your dashboard, you'll trigger the "on" state of the MQTT device. This will automatically update the state in your dashboard like the screenshot below.

MQTT switch on state

If you registered your MQTT device as a sensor in step 1, you can now trigger other automations based on the state of your MQTT device when you toggle your switch or press the button. I hope this helped you, I know this cost me hours to figure out by myself, so I'm already saving myself hours next time.