Experiment Backlog

Experiment ideas show up everywhere. One surfaces while you answer a support ticket, another while you stare at a metrics dashboard. The trouble was that they stayed scattered across Slack threads, meeting notes, and private memos. Nothing collected them, so what to run next got decided fresh in every meeting.

Those meetings always went the same way. Someone suggests cutting the signup form from five fields to two. Someone else suggests emailing users who haven't logged in for a month. With no shared basis for comparing the two, whoever sounded more certain walked away with the next two weeks. The losing idea was never written down anywhere, and two months later it came back around as if it were new.

An experiment backlog exists to pull that scatter onto one page. Collecting isn't really the point, though. Roughly thirty ideas pile up in a quarter and a team can actually run five or six, so what the list does is not hold things but visibly fold the other twenty-five.

Six fields go on each entry. An entry that is only a title can't be ranked, so it sinks to the bottom of the list and stays there.

1. The hypothesis, in one sentence. This is the growth hypothesis — what changes, which number moves, and in which direction. "Cutting the signup form from five fields to two raises signup completion from 24 percent to 30 percent" is one.
2. The reasoning. Why you expect that: drop-off concentrated in the third field, or twelve support tickets saying the same thing. Leave this blank and, when the number doesn't budge, there is nothing to check the result against.
3. One metric to move. The number that decides whether this worked. List two and you get no verdict on the day one rises and the other falls.
4. The expected effect. From what to what. Without it, nobody can say afterward whether the guess was right.
5. The effort. How many days to build and ship, split by person — half a day of design, two days of engineering.
6. The status. Candidate, picked for this cycle, running, finished, or folded.

Ranking compresses the first three into a single score. The most common method is ICE: rate impact, confidence, and ease from 1 to 10 each, then multiply. An idea at impact 8, confidence 4, ease 9 scores 288; one at impact 9, confidence 7, ease 2 scores 126. The bigger-looking idea loses, because three two-day experiments teach you more than one three-week experiment. The score is only good for sorting thirty items into three piles: run now, revisit, fold.

One person owns the list. At Pagedeck that's Ravi, the growth marketer, and both the reordering and the folding happen at his desk. A list with no owner stops getting opened within two weeks.

Adding an entry, on the other hand, is open to anyone. Sofia in sales adds what she heard on a call; Theo adds something odd he noticed in the logs. All that's required on the way in is a hypothesis and a reason — score and effort get filled in later. Raise the bar any higher and the ideas go back to living in Slack.

Effort estimates are the exception: the people who will build it give them. An item Ravi wrote down as two days is often six in Theo's reading, and that gap alone flips the order.

Three groups read the list, each looking for something different. The people executing look for what runs this cycle. Mia, the product manager, checks whether an entry overlaps with work already committed to the roadmap. Grace, the CEO, looks once a quarter, and not at individual entries but at two numbers: experiment velocity, how many experiments finished last quarter, and win rate, how many of those produced a clear improvement. If the first number is stuck at four, the shortage isn't ideas — it's the time it takes to get one live.

"Put that in the backlog. We've already got three committed this cycle."

Not a rejection, but a request to enter it into the next round of ranking.

"Highest score on the board, but it's three weeks of engineering, so it didn't make the cut."

A plain statement of the ranking rule: big expected effect still loses to big effort.

"We ran this back in April. The results are still written down."

The moment a repeat candidate gets checked against the experiment log and folded.

Pagedeck opens this list on day one of every growth sprint, the two-week cycle the team runs experiments in. They read the results of the two or three experiments from the last round first, then reorder the list against them. If the last experiment worked, confidence scores rise across every candidate resting on the same assumption; if it didn't, four or five candidates built on that assumption drop together.

Then they pick from the top. An item marked two days becomes six once someone notices the screen touches billing, and at that point they move straight to the next candidate.

The most common mistake is treating the list as an idea warehouse. Anything parked there feels like it will come back around someday, so the decision to fold keeps getting deferred. In reality five or six items run per quarter, which means an entry sitting there for six months isn't waiting — it's already been dropped. Without marking that difference the list passes a hundred entries, and past that nobody reads the bottom, so the good candidates parked down there get buried along with everything else.

Reading the score as the decision is nearly as common. It arrives as a number, so it looks like arithmetic settles the question. But confidence is a human judgment: Ravi gives a candidate a 7 and Theo gives the same candidate a 3. Follow the score literally and whoever rates confidence most generously always ends up on top. The ordering inside a pile is a human call. One more slot belongs outside the score entirely. A candidate that rewrites signup altogether carries low confidence and so sinks every time, and a team that always defers those ends up at a local maximum, polishing small gains while missing the larger jump.