Measuring Feature success

"You have launched a new feature in your product. How do you measure the success of the feature?"

If you don’t know the answer to this before you build features, you’re probably building the wrong things.

You’re building features to solve problems. If you don’t know what success looks like, how did you decide on that feature at all?

Starting with features instead of problems is a tragically common problem among product managers. They may be building capabilities for problems their customers don’t have. They may be building the wrong solution for their customer problem. Or they may be building small, insignificant things that don’t have large impacts on their business or customers.

You have a hypothesis that creating this feature will generate a certain outcome. Was that outcome created? Was the hypothesis correct? Trying to figure out after the fact a way to measure success means that you weren’t focused on the outcome when you were creating the feature. The shipping of the feature itself was viewed as the goal. This isn’t a good way to build products.

Product teams should be discussing product outcomes. They should review their releases to see if the feature worked. Did it create the expected outcome? Why or why not? What experiments could you run (iterations) to learn if there are better ways to solve that problem?

There are some signs that your product teams are feature-focused instead of outcome-focused.

• Their OKRs, bonuses, and celebrations are mostly about whether they shipped instead of achieving an outcome for the customer or the business
• Discussion of outcomes and KPIs from the last release doesn’t happen during the planning process
• They don’t iterate on previous feature releases. Or if they do, the iterations are known in advance, instead of based on what they learned about what just shipped
• People talk about what feature is needed to increase sales instead of what problems you aren’t solving for customers