When developing your product – it is important to focus on what’s important. Often when collecting metrics for a new product we focus on ‘more is always better’ rather than the more useful metrics which might not look quite so good on paper.
Vanity Metrics
Think about the metrics you see companies bragging about. They’ve had 40,000 signups to their closed beta, they’ve got 100,000 downloads in the first week after launch. They sound really impressive but… don’t really mean anything. How many of those 100,000 downloads stick around after 3 days? How many actively engage with the app? And how many actually spend money in-app?
These big impressive numbers are Vanity Metrics. They sound amazing and a marketing department can shout them from the rooftops to try and entice more investors and more users. But for the product team – they don’t tell you anything actionable.
Actionable Metrics
Say you run an e-commerce website. You’ve got a decent userbase and revenue is steady. Unfortunately, your cart abandonment rate is also higher than you would like. You can see in your analytics that most customers are abandoning their carts at the stage where you ask them to input their shipping and billing addresses. You ask your UX team to see if there’s anything they can do to improve this. They come up with a solution – Give the user the option to use their shipping address as their billing address.
This is a pretty clear cut A/B test in the waiting. Does giving the user the option to use their shipping address as their billing address reduce the number of users who abandon their cart at this stage.
The result of this A/B test is an actionable metric. If abandonment at this stage is reduced in the changed version, you can roll out (action) this change to 100% of your users.
Tips on gathering actionable metrics
A/B Tests
The results of an A/B test are the most actionable of all metrics. They explicitly prove a hypothesis (or disprove it) and give you a defined action to take at the end of them.
A/B test can be used on the smallest of changes right down to changing the text on a signup button to see if it increases conversions.
Metrics are people too
While vanity metrics tend to be far removed from your customers, a good actionable metric will focus on your customers. You should be looking at what is happening on a per-customer basis rather than on abstract groups. So instead of looking at a total number of pageviews, look at average pageviews per new user vs returning users. These should be relatively consistent and safe from external influences or massive fluctuations in new users.
Cohort analysis
Speaking of per-customer metrics, cohort metrics are the cream of the crop.
Think about a product which has a free trial period followed (ideally) by the customer converting to a paid account. Your product also has key lifecycle events you track: downloading the app, creating a free account, using the app and finally converting to a paid account. To compare the cohorts you could create a monthly report and track what percentage of users who downloaded the app that month has completed each lifecycle event. If the completion rates remain stable for each cohort, we know that nothing significant has changed. But if the completion rates start increasing, we can track what else has happened that month to see what is causing the increase (like a discount on the paid account or new marketing campaign).
It’s important to note that there’s nothing wrong with collecting vanity metrics. They’re extremely useful in presentations, marketing materials, and potentially to higher stakeholders. However, it is only from actionable metrics that your product can grow and improve.
Top comments (0)