Two numbers are worth knowing before you plan your next sprint. When Pendo analyzed usage data from 615 software products, it found that 80% of features are rarely or never used. And teams that measure every change find that most ideas don't work: at Microsoft, only about one in three tested ideas improved the metric it was meant to improve. The rest did nothing or made things worse.
Most companies never see this. They track their top-line goals, like revenue, churn and total signups, plus one or two obvious funnels. Very few measure whether each individual change actually worked. Without that, the changes that didn't work stay in the product, and nobody knows.
The usual product loop looks like this. You collect feedback from client surveys, reviews and the support channel. You turn the loudest themes into the next sprint. You ship. Then you go back to the same surveys and support tickets to see whether the update landed.
It works, sort of. But it has a blind spot: you never really know whether the change made things better. Feedback tells you what people feel. It rarely tells you what they did. Survey responses come late, and they come mostly from the happiest and the angriest users. Support tickets tell you what broke, not what quietly started working. "Nobody complained" is not the same as "it worked".
There is a better way: develop through experiments you can measure.
The core rule
Every change goes through the same four gates:
- Reason. If we want to change something, we must know what we are trying to improve and why.
- Measure. We must know how to measure that improvement in numbers, before we build anything. Change. We ship the change in a way that lets us measure it.
- Decide. We look at the numbers and either roll back, or adopt and build on the change.
- Then the loop starts again with a new goal. That's the whole method.
Full article is here: https://deniskoronets.com/post/development-through-experiments-measure-every-change
Top comments (0)