There are a few terms used in product development that were borrowed from scientific research and even project management that have lost a bit of their precision in our discourse. In my experience, these terms are used incorrectly just about as often as they are used appropriately (even by thought leaders in our field). Some definitions and comparison/contrasting:
An assumption is something we expect to be true. I assume the sun will rise, for example. From a planning perspective, failure of an assumption to hold means our plan needs to be revisited. For example, our roadmap may make assumptions about development capacity over time. If we don’t get the development resources we assumed we would, the implication is that our roadmap and associated plans and commitments need to be revisited. It is highly advisable to make these assumptions transparent to decision-makers. BTW, we don't spend much time validating our assumptions: we simply identify them and move on. Validation is relevant to hypotheses.
A hypothesis is an assertion we make that has inherent uncertainty. We express it so that it can be validated or invalidated. Again, in the type of research we often do as PMs, we don’t typically test assumptions (we simply expect them to be true). We always test hypotheses (that’s their job!). Our UX designer might have a hypothesis that users would prefer a mobile experience to one in a desktop browser. We can do research and collect information to support or invalidate this hypothesis. We can rarely prove or disprove a hypothesis conclusively — we simply collect information convincing us it’s true or not. The data we collect to validate or invalidate a hypothesis is called evidence.
Evidence is information we consider in the context of a hypothesis. Evidence supports a hypothesis or it doesn’t. Think about this term’s common usage in criminal law. The police have a hypothesis that a criminal did something wrong but, to get a conviction, must collect evidence (fingerprints, for example) to support their hypothesis. We talk a lot about being data-driven in making product decisions, but I believe being evidence-based is more important, i.e., in many cases, we should educate ourselves sufficiently to generate hypotheses that guide our further research.
One of the ways we gather evidence is by performing experiments. We carefully design experiments to collect evidence, both objective and subjective, that can provide insight into the validity of our hypotheses. We must be careful as the design of experiments, their execution, and their results can be influenced by our cognitive biases, which cloud our judgment by oversimplifying our conclusions or making them more palatable.
Risks are adverse (bad) things that might happen. Folks often conflate assumptions and risks. We expect assumptions to be true while we are uncertain about risks. We often rate risks based on a function of their severity and likelihood of occurring. We focus first on mitigating or eliminating risks that are highly impactful and likely to occur. BTW, most assumptions carry inherent risk in that if they don’t hold, there could be adverse effects.
I have to admit that I’ve conflated these important concepts but now think I’ve managed to distinguish them and use them precisely.
Summary
We expect assumptions to be true so we rarely (if ever) test them.
Hypotheses are statements we make so that they can be tested (validated or invalidated).
Evidence is information collected to validate or invalidate a hypothesis
Risks are adverse things that might happen. Most assumptions carry some risk.
Much of this was originally a post on LinkedIn.
Comments