KPIs: How much data do you need? - Zero to Startup Ep 10

In the last episode, I talked about how we measure retention for our hosting startup. But since we are still in an early stage we only get a very limited amount of data. So the question arises, if this data even means anything? And in general, how much data do we need, in order to make some confident decisions? Here are my thoughts about it...

From Zero to Startup - The series

We are two passionate developers from Germany on a journey to building our own profitable SaaS startup from scratch. In this series I will share everything we learn along the way, from coming up with the idea, coding a prototype, getting our first users and scaling to infinity. We are starting from absolutely nothing but our coding skills. We will not rely on any external funding. We are Lukas Mauser and Jonas Scholz and this is Zero to Startup.

You will never have enough data

Our first big learning: In the beginning you will never have enough data! You can only read so much from single digit numbers in your metrics. And of course you can not draw any statistical conclusions from that.

Tracking your metrics as a tiny early stage startup serves a different purpose. It gives you hints to identify problems and opportunities. Not more and not less.

You are following individual users and try to understand how they use your app and what problems they face. It's your job to decide whether these individuals represent your target audience and whether that aligns with your longterm vision.

That's why it's so important to have a clear vision and good understanding of the audience that you want to server. You get hints from your metrics but you judge what to do with it by evaluation them how big the impact will be on your vision and target audience.

Why a few data points can be enough

After we launched our prototype and got our first few signups, we quickly became nervous, when only a few users deployed something through our app. Why are not more people deploying? Are there any bugs? Are they even human? What's going on??

So in the following weeks we started tracking some metrics around deploys. And after taking a look at them, we noticed, that our signup-deploy ratio wasn't that bad at all. Our nervousness mainly came from not knowing our numbers. And even though we only had a little bit of data to work with it was enough to calm our minds and reassure us, that at least our platform works.

So the second learning was: sometimes it's already enough to measure just a few successful events. Not that there is any statistical significant insight, but rather qualitative if everything is working for example or how even a few users are using your app.

Summary

As an early stage startup you will never have enough data to draw statistically significant conclusions.

But sometimes a few datapoints is all you need to identify problems and opportunities of your application.

Even though the data might not be much it can give some good qualitative insights and ideas to work with.

ℹ️ Additional notes

We would love to hear some feedback from you! ❤️

Check out the product on https://sliplane.io.

And feel free to comment or get in touch with us to talk about any topics you are interested in or questions you have about the project in general.

✌🏻