Hypothesis

A proposal to build something usually starts as "I think this would be good to have," and most of the room usually agrees. The trouble starts after it ships. Two months of work goes out, two weeks pass, and someone asks whether it worked — and that's where the conversation splits.

One person says the number looks a bit higher. Another says that's the ad campaign that ran the same week. Neither can prove it, because nobody wrote down beforehand what would have to move, and by how much, for this to count as working. So the team picks next quarter's feature the same way it picked this one, starting over with nothing learned from the last two months.

The more common version is an explanation assembled after the fact. If the metric rose, the feature did it; if it didn't, the launch was under-promoted or the timing was bad. Both stories were built after seeing the result, so neither helps with the next decision. A hypothesis inverts the order: write down what you expect and what would make you admit you were wrong, then read the result against that sentence.

Four pieces have to be in the sentence before it can be tested at all.

1. Who it applies to. Narrow the population. All users, people in their first week, teams on the free plan — the answer comes out completely differently depending on which.
2. The change. Say concretely what will be different. "Improve onboarding" leaves no way to know later what was actually done; "automatically create one example document right after signup" does.
3. The expected result. Name the metric and the movement in numbers — from 34 percent to 40 percent on the share of people who create a first document. Without the current value, there's no way to tell a rise from the way things already were, and a six-point gap is a different bet from a one-point gap.
4. The decision rule. Fix in advance when and under what conditions the result gets read: two weeks, or 1,000 people per group, whichever comes second. Skip this and the experiment ends on whatever day the number looks good.

Put the four together and you get one sentence. "For users in their first week, automatically creating an example document will raise first-document creation from 34 percent to 40 percent. We'll read it after two weeks with at least 1,000 people per group."

One more line belongs with it: what counts as being wrong. Writing down that anything under 40 percent means dropping the solution is what stops a 36 percent result from turning into an argument that it still went up, so it should stay.

The writer is usually the product manager who owns the problem, and never alone. If the earlier problem statement already pinned down whose problem is being solved, the hypothesis is the guess attached to a solution for it. Whoever works with the data supplies the current value, and the change gets written to a buildable size only by sitting with the designer and the engineer.

Four groups read it, and each reads a different part.

1. Designers and engineers. They read the change and scope what to build. A vague line here produces two people building different things.
2. Whoever analyzes the data. They read the expected result and the decision rule and prepare how the numbers will be split before anything launches. Settle this afterward and the criteria drift toward the result.
3. Whoever decides. They judge whether the guess is worth two months or two weeks. The more there is to lose from being wrong, the more the answer is to check it first with something cheap like a prototype.
4. The team a few months from now. By then nobody remembers the context of the decision. One surviving sentence keeps them from running the same attempt twice, and it also shows what was already ruled out when a similar idea comes back.

A hypothesis is therefore a shared document rather than a personal note. When the experiment ends, the result goes into the same file along with whether the guess held and what happens next.

"Can we write this up as a hypothesis? Right now I can't see what we're trying to find out."

A request to turn an argument for building into a sentence that can be tested.

"The hypothesis said 40 percent and we got 36. By our own rule, we drop it."

Don't move the decision rule to fit the result.

"There's no decision rule, so I don't know when this is supposed to end."

No agreed reading point, so the experiment stays open indefinitely.

When first-week drop-off came up at Pagedeck, Mia picked one line out of user research — "a blank document leaves me with no idea what to write" — and turned it into a hypothesis: automatically creating an example document would raise first-document creation from 34 percent to 40 percent.

Noa designed the example document screen and Theo shipped it behind an A/B test so only half the new users saw it. Two weeks later the number was 36 percent. The direction held but the bar didn't, and the team moved to a different opportunity instead of building more example documents. Without the sentence written in advance, the discussion would have stopped at "well, it went up."

It's easy to treat a hypothesis as something you're supposed to get right. Hitting the number looks like success and missing it looks like failure. The value is not in being right, though — it's in changing what the team does next. A wrong hypothesis saves the money and time that direction would have taken, which makes it a return on its own. Finding out in two weeks that a solution doesn't move anything costs far less than finding out after two quarters of building it. Aim to be right and the team starts picking only small changes that are certain to move, and nobody proposes the attempt that could be badly wrong and could also change a lot.

The second misreading is that a number is enough to make something a hypothesis. Add a figure to "this feature will raise return visits" and the shape looks right. But with no statement of what result would count as being wrong, the sentence can't be tested, because every outcome comes with an explanation attached. This bites hardest on the first of the four big risks, the question of whether people want the thing at all. With no condition that would disprove demand, a low number gets read as nobody knowing about it yet, and the feature survives another few quarters.