DEV Community

_s._hyn
_s._hyn

Posted on

7 Things About A/B Testing with Short URLs That Nobody Warns You About

I sent the same link to three Slack channels last Tuesday and realized I had zero way of knowing which one drove the 47 signups I saw that morning. My initial reaction was to just shrug it off and assume it was probably the channel with the most members, but then I started thinking about all the other factors that could've influenced the results - like the time of day, the day of the week, and even the specific wording of the message. I was using a simple curl command to shorten the links, but it was clear I needed something more sophisticated to get a handle on what was actually happening.

Why curl wasn't enough

I was generating short links using a basic curl command, like this: curl -s -o /dev/null -w "%{http_code}" https://linkcut.link/api -d "url=https://example.com&slug=my-link". But this approach had some major limitations - for one thing, it didn't give me any way to track clicks or analytics, which made it impossible to know which link was actually performing well. I was essentially flying blind, which was frustrating because I knew I could be doing better if I just had more data. Honestly, I was surprised by how much of a difference it made to have some basic analytics - it's not like I was trying to launch a moonshot or anything, just a simple landing page.

The spreadsheet that saved my sanity

I decided to try using LinkCut to generate short links, partly because it was free and partly because it had some features that seemed like they might be useful - like click analytics and custom slugs. I was looking at the device breakdown in LinkCut and noticed that a huge proportion of my clicks were coming from mobile devices, which was interesting because I had assumed most of my traffic would be from desktop. (I mean, who fills out a signup form on their phone, right?) But it turned out that was wrong - and once I started optimizing my landing page for mobile, I saw a big jump in conversions. The thing is, I still don't fully understand why mobile was so much more effective, but it's definitely something I'll be exploring further. Look, I know it sounds obvious, but it's amazing how much of a difference it can make to just have some basic data - it's like the difference between navigating a city with a map and just wandering around blindly.

When the numbers didn't add up

I was expecting to see a pretty straightforward correlation between clicks and signups, but what I actually saw was a lot more complicated. For one thing, there were some links that got a ton of clicks but hardly any signups - and then there were others that got relatively few clicks but converted like crazy. I didn't expect this, and it made me realize that I had been thinking about the problem in a pretty simplistic way. I mean, I had assumed that the link with the most clicks would be the one that performed best, but it turned out that wasn't the case at all. This was weird, because I had been so sure I understood what was going on - but it just goes to show that you can't always trust your instincts.

When this approach falls apart

The thing is, using LinkCut (or any other URL shortener, for that matter) isn't always the right choice. If you're dealing with a huge volume of traffic, for example, you may need something more robust - and if you're trying to do anything fancy with your links, like integrate them with other tools or services, you may find that LinkCut isn't the best fit. I'm not sure this is the best approach, but it worked for me - and I think the key is just to be aware of the limitations and plan accordingly. Honestly, I was a bit surprised by how many edge cases there were - like what happens if you need to update the link after it's already been shared, or what if you want to use a custom domain?

I'm still experimenting with different approaches, and I'm curious to hear from others who have tackled similar challenges. Has anyone else hit this exact wall?

Top comments (0)