Houston, do we have a problem? A Product Manager's Guide

The first step to solve any problem is to acknowledge that there is a problem. - Zig Ziglar

It’s a nice quote. Not completely true though.
Acknowledging that there’s a problem is hard. But before you can acknowledge a problem, you’ll need to know if there is one.

This requires finding, gathering, and analyzing data first.

Step #1: Check your assumptions against the data

This step relies on observability. If your problem is currently a feeling, you’ll need something to back that up. Some data, somewhere. Downloads, number of users, revenue, search results, page views, NPS scores, surveys - gather everything you can get your hands on. Once you have all the data you can find, start with a comprehensive, year over year review of any and all data you can cobble together about usage, sentiment, and popularity of your feature.

After gathering all the data, sit with it. Look at it.

• Downloads: You had fewer downloads month over month for the last three quarters. Some of these were CI tools for testing, so true downloads were even less than this number.
• Usage: How many users do you have?
• Sentiment: Do they love your product or hate it?
• Demographics: What kind of users are they? Hobbyists? Students? Experts?
• What kind of questions are there about your product?
• How many support tickets or github issues? Questions on stackoverflow?
• How many visits are there to your documentation? how does this compare to your number of users?
• Search your product on google. Read 3 pages of results. What's the general mood? What's good?
• Revenue: How much revenue can you attribute to the product, this year, last year, and for its entire lifetime?
• Baggage quantification: what’s your backlog look like? How quickly do you respond to users when they complain? How big is your codebase?

If the following things are true, you have might have a problem:

• you have very few users
• you can't find anyone who loves your product or talks about it
• you have no questions about your product on SO or in your support team
• release notes have few views and no comments
• github has no issues filed
• documentation views are very low, OR, documentation views are very high and you have few users

If what you have is silence - creepy, overwhelming silence - your product is the problem. Ditto for if you have loud complaining everywhere.

Step #2: Present the data without fanfare or excuses to the engineers who built your product.

Let it marinate.

Step #3: Identify potential point(s) of failure.

When things fail, people want to know why. Unless it’s their fault, then they typically want to insist it’s not failing, that the data is wrong/unclear/incomplete. Regardless it is generally healthier to know what went wrong in order to avoid those mistakes in the future, no matter what we may feel about it. There can be so many reasons for this:

1. You built the wrong product
2. You market your product to the wrong audience
3. You built a good product to a popular, well loved product
4. Your communication of your product sucks
5. Getting started with your product is too difficult
6. You don't ack or fix issues in the product so users don't stick with it
7. Your price is not right

etc etc etc
Be as explicit and all inclusive as you can; ruthlessly evaluate every potential issue. I recommend mind mapping this.

Advice #4: Decide.

Just like the first step of solving any problem is knowing whether you have a problem, the next step is deciding what to do about it. Questions you might ask yourself: Is this problem temporary? Do we want to fix it? If we did want to fix it, what could we do, how long would that take, and is it worth it to do so? What kind of $$ would that require?

If you should decide to burn the product to the ground, it will impact users. Even if you only have 5.

Advice #5: Clean up.

Once your decision is made, tell your users. Maybe there’s only five of them, but their work matters. That’s the most difficult part of the process - sitting at your keyboard and trying to figure out some eloquent way to say, We built something we were proud of but it wasn’t enough.
Then, complete all the necessary administrative things like JIRA mop ups, close out github issues, update the readme, then reassign engineers. Publish a retrospective, so that your failure can serve as a warning to others. Don’t give false hope.