It's no secret that building a quality product is hard. But keeping your customers engaged and happy as you make product improvements is equally hard.
At any given time, there will be more than a handful of improvements you can make within your product. You’ll have some ideas. Your teammates will pitch in with what they think should be improved. And of course, your customers will have a few opinions of their own.
Any product improvement you make will not satisfy everyone. You need to pick your battles. But taking an informed decision on which specific improvement you should work on isn’t straight forward either.
So, how do you identify which opportunities to go after? And an equally important question to ask, how do you unearth where the scope for _ real _ improvements lie within your product?
What is product improvement?
Product improvement is the process of making changes within your product in such a way that it enables you to either improve engagement or retention within your product, and move the needle for your business.
4-step process to identify product improvement opportunities
1. Map feature usage with your product journey
Not every feature you built carries the same impact. There will be features that will make your users go “Wow!”. And there’ll be features that rarely get noticed. This is a function of the feature’s goal.
Every user who signed up for your product can be bucketed into six categories:
- Signed up
- Activated
- Engaged
- Retained
- Paying
- Delighted
Nearly all the features you build will have its primary goal of moving users one step closer to the delighted bucket. This allows you to directly map all your features to specific buckets mentioned above.
Of course, this requires you to have at least a rough idea of how users flow within your product. What feature they should use first, followed by what should be used next, all the way till they accomplish what they set out to achieve with your product.
By creating a timeline of all the top paths customers take within your product, you can gain insight into how your users sequentially perform events in your product, and identify when drop-offs happen or where unsuccessful behaviour occurs. If you’re using a tool like Mixpanel, Amplitude, or Heap, you should be able to get this data.
Once you have an idea of the most common paths users take within your product, you should be able to map each feature to your user journey and map them to the buckets mentioned above.
There’s a good chance that you will have multiple paths within your product. After all, your user’s journey is rarely linear. Which is why it is best to pick one path — usually the most frequently used — and focus on improving it first.
At the end of this step, you should have:
- a clear representation of what features your users use and when,
- where your users drop off, and
- how users flow within your app.
Takeaway: Map your users' journey within your product based on the features they use.
2. Measure feature success throughout the journey
Any good process that helps you identify opportunities will require you to gather data. A great process, however, goes a step further and takes a holistic approach.
Most teams when trying to understand a feature’s success simply track a few key events and look at how the numbers change on a daily, weekly, or monthly basis. This leads to teams over-indexing on single data, leaving them vulnerable to gaps, blind-spots, and biases.
Half the battle with identifying if a feature is used for its original intent is understanding of why users used them in the first place.
To truly understand how people are using your product, it isn’t enough to identify how each feature is being used, but for what purpose it is being used, why it’s being used, and if the feature was successful in achieving its goal.
By measuring the success of each feature throughout the user’s journey you identified in the previous step, you’ll get a clear picture of what needs to be improved. We wrote an in-depth guide on how to measure the success of a feature. 👇🏼
How To Measure The Success Of Your Feature
At the end of this step, you should have data points on:
- how people are using each feature in your product,
- why they’re using the feature,
- what purpose they’re using the feature for, and
- if the feature was successful in achieving what it was originally built for.
Takeaway: Measure the success of each feature your customers use in their journey.
3. Find the correlation between feature usages and business goals
The success of a feature is irrelevant if it can’t have a meaningful impact on your larger business goals.
When looking at data, most teams say, “people who performed X converted” or “people who converted did X”. This is an effect of thinking in a single dimension. They make the mistake of not considering people who performed X but didn’t convert, nor do they take into consideration the people who converted without performing X.
By using the Phi coefficient, a measure of association for two binary variables in statistics, we can take a multi-dimensional approach to find if there is any relationship between a specific feature usage within your app and the number of occurrences of a feature usage that is likely to maximize the outcome.
To get started with the Phi coefficient you need two things:
- an ideal frequency of usage for the feature, say n, and
- to what goal you’re trying to find a correlation. For example, second-week retention.
By digging into your analytics tool or your database, you should now be able to fill in the cells a, b, c, and d into the 2x2 matrix table below:
where,
n = ideal frequency of usage for the feature.
a = number of people who used the feature >= n times AND achieved the goal.
b = number of people who used the feature >= n times AND did not achieve the goal.
c = number of people who used the feature < n times AND achieved the goal.
d = number of people who used the feature < n times AND did not achieve the goal.
The ideal frequency of usage for a feature, n, depends largely on the product, how the feature impacts the product as a whole, the market you’re after, and your business model. So, if you don’t have a clear picture of what n is, it’s okay to start with a hypothesis and work from there.
Using this data, you can then find Phi’s correlation coefficient by using the above data in the below formula:
This determines the correlation strength between the ideal frequency of feature usage and the goal. The value you get can vary between -1 and +1, where +1 indicates a positive relationship between your feature and the goal, -1 indicates a negative relationship, and 0 indicates no relationship.
A lot of the times, a single feature by itself can not help you improve your goal of increasing engagement or improve retention. To counter this, we recommend identifying the correlation strength between all the features in the user journey that we identified from the first step.
A good example of this is an analytics dashboard. Say you’re building a social media management tool and you’re trying to increase the usage of your report. You might receive a ton of requests from your customers wanting different types of charts to visualize. However, if they aren’t scheduling enough posts to make the reports meaningful, adding more types of charts will not help you achieve your goal.
At the end of this step, you should have:
- a relationship between the number of occurrences of feature usage and the outcome.
Takeaway: Identify the relationship between the number of times a feature is used and your goal.
4. Identifying what improvement to work on
Now that you know what number of occurrences of every feature usage is likely to improve the outcome, how people use those features, when, and also why, we can focus on specific feature improvements backed by data.
There are two types of improvements you can make for your existing features in your product:
1. Adoption improvements
Adoption improvements are meant to target your current users who aren’t using it at the moment but would benefit from it. Three possible reasons why users did not adopt the feature:
- The feature didn’t have enough capabilities. These are users who say, “I’ll use X if it can do A, B, and C”.
- Discovering the feature was hard.
- Understanding how to use the feature was hard.
You should focus on improving adoption when there’s a positive correlation between the frequency of usage of a feature and the goal, but only a small number of people are using it.
2. Frequency improvements
Frequency improvements are meant to target users who are already using the feature but aren’t using it enough number of times. A possible reason why people aren’t using the feature enough number of times could be that users aren’t sure why they should use a feature.
You should focus on making a frequency improvement when there’s a positive correlation between the frequency of usage of a feature and the goal, but a large number of your users aren’t using the feature enough number of times.
Takeaway: Make product improvements based on the relationship between a feature and the goal, how many people adopted the feature, and how frequently they use the feature.
Product improvement examples at Zepel
1. Git Integration
A few months ago, Git integration in Zepel only allowed you to complete a linked item when you merged pull request. This wasn’t enough.
- Not all users were using it since it was difficult to find.
- Users who used it were able to only complete an item, but not update its statuses.
With the introduction of Git Workflow automation, teams today update their team members automatically based on their activity in their version control system. This led to an increase in the number of people who adopted the feature, enabling them to be in sync with the reality of what’s happening effortlessly.
2. Item Details Pop-up
The item details pop-up in Zepel is where a user story’s description, comments, and all its properties can be viewed in one shot. Plenty of times, while writing acceptance criteria in the description area, teams realize the current user story requires an additional subtask.
Previously, users couldn't view or add subtasks from within the details pop-up. This meant, our users had to go back and then add new subtasks. To solve this, we made a few design tweaks and enabled teams to add new subtasks from within the details pop-up.
While it sounds trivial, by removing an additional step and unnecessary friction, we were able to increase the frequency of items created.
Final thoughts on product improvement
Gone are the days where you can spew a few features every fortnight and win over customers. The only strategy to keep customers happy is by continually delivering value. And that means, consistently improving your existing features.
Truth is, the product improvements most of us do today are based on gut feeling, or at best, a response to the number of customers who shout and leave. That’s the equivalent of throwing spaghetti at a wall to see what sticks. While it may make for a fun evening, it's not a sustainable strategy. At some point, you’ve got to know your numbers and live by them. But even numbers can't guarantee good decision-making.
Without context, data is plain numbers represented with fancy charts.
By following a process that doesn’t just consider numbers, but also takes the big picture into account, you can make a better judgement on which features need improvements and priorities the right features.
Top comments (0)