Confidence

Writing something as a number erases how strong the evidence behind it was. Two thousand people counted from logs and two thousand people guessed from a meeting sit in the same cell on the sheet in the same shape. The arithmetic can't tell them apart, so the ranking can't either.

Pagedeck ran into it. Mia asked where the reach figure came from on the item that had come out first, and the answer was that sales had heard it three times or so. The item directly below it was counted from logs. A verified number and a guessed one were being multiplied in the same slot to produce an ordering.

Confidence is the column that puts that strength into the arithmetic. Thin evidence lowers the score, and raising the ranking means fetching evidence rather than arguing harder.

Multiply the other values by a percentage fixed to the kind of evidence you have. The scale usually has three steps.

1. 100 percent — there is a measured number already, from logs or a previous experiment.
2. 80 percent — there is evidence but nothing directly measured, like a survey or several interviews.
3. 50 percent — it's a guess from one or two people's comments or from experience.

If something feels below 50 percent, it doesn't go on the sheet at all; the evidence gets checked first.

In the calculation, reach and impact are multiplied by this percentage and then divided by effort. Reach of 2,000 times impact of 2 times 80 percent, divided by three person-months, gives 1,067. With only a guess behind it, the same item at 50 percent drops to 667.

What this column does isn't limited to reordering. It makes producing evidence the cheapest way to raise a score. Half a day in the logs that turns 50 percent into 100 percent doubles the number, which puts that half day at the front of the queue. One caution: this is not a probability of success. It's the chance the estimates are right, and whether the feature will deliver is a separate question.

"Where does the reach number come from? Logs make it 100 percent, something sales heard makes it 50."

Setting the value by asking for the source instead of deleting the number.

"Leave this at 50 percent and I'll pull the logs next week and score it again."

Declining to decide now and creating one verification task instead.

"Getting to 100 percent would take an experiment, and that experiment is larger than the feature."

Noting the point where gathering more evidence costs more than it returns.

On Mia's sheet of 18 items, the one in first place fell to third once its confidence was set to 50 percent. It returned to first only after Theo spent half a day in the logs confirming that roughly that many users pass through the screen. A two-hour argument turned into a half-day check.