Implementing A/B Testing with Dokku and save money for your Side project

A/B testing is an invaluable technique for improving web applications and testing new features. However, it can be costly to run at scale, especially for side projects with limited resources. In this article, I'll show how you can implement A/B testing essentially for free using opensource tools like Dokku.

The Value of A/B Testing

If you're not familiar, A/B testing involves sending a percentage of your traffic to a "variant" version of your app to test how a change impacts metrics like conversion rate, engagement, etc. By gathering data on the variant, you can make informed decisions about whether to fully launch new features.

Running effective A/B tests usually requires significant infrastructure - you need to spin up duplicate environments and route traffic between them. At large companies, this is handled by expensive proprietary tools. But for small teams and side projects, the costs can be prohibitive.

This is where open source comes in! With some clever configuration, we can replicate A/B testing with minimal cost.

Introducing Dokku

Dokku is an open source platform built on Docker and Heroku that makes it easy to run apps on your own servers. It has a powerful routing system that we can leverage for A/B testing.

Some key advantages of Dokku:

• Free and open source
• Simple setup - runs on a single server
• Git-based deployment similar to Heroku
• Supports custom domains
• Traffic routing system for running multiple app versions

For our A/B testing needs, that traffic routing system will be perfect.

Implementing A/B Testing with Dokku

With Dokku, we can deploy two versions of our app and control the percentage of traffic each one receives.

Here's an overview of the setup:

1. Deploy version A of the app (the original)
2. Deploy version B of the app (with changes to test)
3. Configure Dokku to split traffic between them
4. Adjust traffic percentages based on test results
5. Fully launch version B if successful

Try to deploy first project:
https://dev.to/jackynote/deploy-spring-boot-3-application-to-dokku-environment-with-dockerfile-4ai1

Let's go through a specific example.

Say we have a side project called "Foodium" that helps users find restaurants. We want to test a new recommendation algorithm.

Here are the steps:

1. Deploy the current "Foodium" app to Dokku, with the name foodium-v1
2. Copy the code to foodium-v2. Implement the new algorithm. Deploy this version.
3. Configure routing to send 80% of traffic to foodium-v1 and 20% to foodium-v2:

dokku proxy:ports-add foodium-v1 http:80:5000 80:20
dokku proxy:ports-add foodium-v2 http:80:5001

1. Watch metrics on foodium-v2 to see if the new algorithm improves engagement.
2. If successful, gradually shift more traffic to foodium-v2 by adjusting the port mapping.
3. Finally, routing 100% to foodium-v2 to fully launch.

And that's it! We've implemented A/B testing and iterated on new features with no additional infrastructure. Dokku enabled us to set up dual versions of the app and control traffic all through easy config.

Wrapping Up

A/B testing is a modern necessity for improving web products. But it can be expensive and cumbersome to implement at scale. For side projects and small teams, using Dokku and other open source tools is a budget-friendly way to get the benefits of A/B testing. Just make sure to monitor metrics carefully and iterate as you learn from your tests.

So next time you want to trial a new feature on your app, consider using Dokku's traffic splitting approach. Careful testing will help take your product to the next level, without breaking the bank!