Need someone to lead product management at your software company? I create software for people that create software and I'm looking for my next opportunity. Check out my resume and get in touch.

Measuring Feature success

17 Oct 2023

"You have launched a new feature in your product. How do you measure the success of the feature?"

If you don’t know the answer to this before you build features, you’re probably building the wrong things.

You’re building features to solve problems. If you don’t know what success looks like, how did you decide on that feature at all?

Starting with features instead of problems is a tragically common problem among product managers. They may be building capabilities for problems their customers don’t have. They may be building the wrong solution for their customer problem. Or they may be building small, insignificant things that don’t have large impacts on their business or customers.

You have a hypothesis that creating this feature will generate a certain outcome. Was that outcome created? Was the hypothesis correct? Trying to figure out after the fact a way to measure success means that you weren’t focused on the outcome when you were creating the feature. The shipping of the feature itself was viewed as the goal. This isn’t a good way to build products.

Product teams should be discussing product outcomes. They should review their releases to see if the feature worked. Did it create the expected outcome? Why or why not? What experiments could you run (iterations) to learn if there are better ways to solve that problem?

There are some signs that your product teams are feature-focused instead of outcome-focused.

Their OKRs, bonuses, and celebrations are mostly about whether they shipped instead of achieving an outcome for the customer or the business
Discussion of outcomes and KPIs from the last release doesn’t happen during the planning process
They don’t iterate on previous feature releases. Or if they do, the iterations are known in advance, instead of based on what they learned about what just shipped
People talk about what feature is needed to increase sales instead of what problems you aren’t solving for customers

Recently Written

Too Big To Fail (Apr 9): When a company piles resources on a new product idea, it doesn't have room to fail. That keeps it from succeeding.
Go small (Apr 4): The strengths of a large organization are the opposite of what makes innovation work. Starting something new requires that you start with a small team.
Start with a Belief (Apr 1): You can't use data to build products unless you start with a hypothesis.
Mastery doesn’t come from perfect planning (Dec 21): In a ceramics class, one group focused on a single perfect dish, while another made many with no quality focus. The result? A lesson in the value of practice over perfection.
The Dark Side of Input Metrics (Nov 27): Using input metrics in the wrong way can cause unexpected behaviors, stifled creativity, and micromanagement.
Reframe How You Think About Users of your Internal Platform (Nov 13): Changing from "Customers" to "Partners" will give you a better perspective on internal product development.
Measuring Feature success (Oct 17): You're building features to solve problems. If you don't know what success looks like, how did you decide on that feature at all?
How I use OKRs (Oct 13): A description of how I use OKRs to guide a team, written so I can send to future teams.