How to Measure Retention - Zero to Startup Ep 9

So, you get sign ups, but are they coming back? This question is not easy and straight forward to answer.

It can be quite challenging to wrap you head around how to measure retention. That's why in this episode, I want to show you, how it can be done and how we do it for our hosting startup.

From Zero to Startup - The series

We are two passionate developers from Germany on a journey to building our own profitable SaaS startup from scratch. In this series I will share everything we learn along the way, from coming up with the idea, coding a prototype, getting our first users and scaling to infinity. We are starting from absolutely nothing but our coding skills. We will not rely on any external funding. We are Lukas Mauser and Jonas Scholz and this is Zero to Startup.

Step 1: Define how users get value

Users get value from your service if they keep using it.

But what does it even mean to use your service? First step is to define some key actions, that describe how your product is used.

Let's look at our hosting service as an example. Usage can be measured through key actions like:

• sign ins
• services created
• deploy attempts
• successful deploys
• ...

But it doesn't have to be restricted to actions.

Value can also be represented through metrics like monthly revenue or average service uptime. While revenue is a very strong indicator for value, it does not work in all cases. For example if your service is free to use...

Every business works a bit different, so start with defining what metric or usage parameter represents best, how much value your users get.

Step 2: Identify your north star and supporting metrics

We can simply track everything. But it would be hard to get an overview over what's going on. There is a concept of defining one single north star metric first and working backwards from there.

Ideally we want to find a north star, that represent best, what value our users get from using our product over time. In a lot of cases this means recurring revenue. But what if you offer a freemium pricing model?

For our hosting service, we thought about deployments for example. But someone can deploy an application only once, but they can still get a lot of value if they keep this service running for a year.

So service uptime is probably a better representation for the value our users get from our product.

Airbnb for example used number nights booked in a given time as their north star.

Take some time to think about how your users get value from your product.

Step 3: Tracking this metric over time

Now comes the tricky part, that is not intuitive at first.

Let's take the number of deploys as an example.

To get started, let's just look at one single user and track this metric every single day. Did he deploy something when he signed up? What about the day after? After two days, three days, four, five six, ...

And now, let's do that for every single user. The result will look something like this:

Now we realign the numbers a bit, moving all datapoints to the front and we get the number of deploys depending on account age:

Next, we calculate an average value from all users. Sum up all datapoints, and divide by the number of data available. We have a lot of data for the current day, because we can look at every single user that we have, but 3 weeks ago this number was lower and so on...

The further back we go, the less data we have, but that's not a huge problem.

If you get only a few sign ups per day, you can also summarize sign ups per week in order to get more data points.

Let's plot what we just calculated: average number of deploys over account age. The result is a retention graph, that ideally flattens out at some point and ideally looks something like this:

You read the graph like this:

The spike on the first day, means that users who just signed up show more activity simply because they try out a lot. The older the account age the less activity. Ideally you never want to loose activity completely.

If you graph hits 0 at some point and does not recover, your retention is bad.

The higher the curve flattens out, the better, because it means your users come back often to work with your service.

You can create graphs like this for different metrics, like revenue, service uptime, sign ins, etc. And easily see, how usage of your product changes over time.

Summary

To measure retention, first define important actions or metrics. Then, plot them for each user, from the day they signed up. Rearrange the data points, so that you look at your metric over account age and calculate the average from there.

The result is a retention graph that ideally flattens out at some point in time. You can create that graph for several important metrics of your startup and get an idea, how usage of your product looks like over time.

But is this data really meaningful? How much samples do you need in order to make a decision? More in the next episode!

ℹ️ Additional notes

We would love to hear some feedback from you! ❤️

Check out the product on https://sliplane.io?utm_source=zero-to-startup.

And feel free to comment or get in touch with us to talk about any topics you are interested in or questions you have about the project in general.

✌🏻