Start with a Belief

To build successful products using data and experiments, you must start with a hypothesis. Data can inform your decisions, but they cannot decide for you. Simply following the data wherever it leads will create an incoherent product. It will be lifeless and uninspiring.

Successful products begin with an insight. Data isn’t insight. Data by itself can lead you astray. It’s easy to drive a single metric up in a way that is harmful in the long term. Without an insight into how that number should move, chasing the data can lead to a quantitative success that’s a qualitative failure.

Start with the question you want to answer. Start with a belief about the world. Those beliefs can be formed by past research and experiments. Past data can reveal context or trends that become a hypothesis. The beliefs can also come from experience, domain expertise, observation, or even creative sparks.

Experiments and data can validate your hypothesis. You have an idea that if you do something you’ll change the way people behave. They’ll buy more products. They’ll use this feature more. They’ll stick around longer. They’ll invite their friends. The data tells you if you are right. It helps you add detail to your hypothesis. It tells you the idea was correct for some people but wrong for others.

Without a hypothesis to start from, data can’t provide insights. You don’t know what the data is telling you. Be data-informed, not data-driven.