Segment Analysis

An overall average is convenient in exactly the way that misleads. Pagedeck's free-to-paid conversion rate was 9 percent. Mia set a goal of moving it to 12, and then nothing followed from it — 9 percent doesn't say which screen to fix or who to talk to. A target set on a blended number never names whose behavior has to change.

Splitting by plan produced the answer. People who signed up on a team plan converted at 21 percent; people on an individual plan converted at 4. The 9 percent was a blend of the two, and not one actual user sat at 9 percent.

What an average hides isn't only the difference. It hides the work. Looking at 9 percent, the conversation turns to fixing the checkout screen. Looking at 21 against 4, it turns to what someone signing up as a team experiences differently. Segment analysis exists to split the number so that second question can be asked.

It doesn't start with splitting. It starts with writing down what decision you're trying to make. Without a decision the slices multiply and all you're left holding is a report.

1. Write the question in one sentence. Finding out why conversion fell and deciding which customers to focus on next quarter demand different axes.
2. Choose the axis to split on, from values already in your records. Plan, acquisition channel, signup period, device, company size are the common ones. The axis worth trying first is usually one that already decides something else, like the plan someone is on or the channel that brought them in. Splitting on something you don't record means the first task is to start recording it.
3. Check the size of each slice. A slice of thirty people has a rate that swings when one or two of them move. Before concluding anything, check the sample size — the minimum number of observations a conclusion needs.
4. Calculate the same metric for every slice and lay them out together, with the overall number on the same line. Seeing the whole at 9 and the slices at 21 and 4 in one row is what makes the size of the gap readable.
5. Dig into the slices with the largest gaps. Finding a gap is not the end of the analysis — treat it as where the question starts rather than as the answer. Why the team plan runs high gets answered by taking that slice alone through a funnel analysis to see where people drop, or by talking to the users in it.
6. Stop before stacking axes. Split by plan and channel together and you get twelve slices holding about twenty people each. Reach for a second axis only when one axis fails to answer the question.
7. Connect it to a decision. Write one line about which slice gets what.

Two situations fit. One is when an overall number moved and you don't know where the movement came from. It's also the cheapest analysis on the shelf, because it needs no new instrumentation, only a column you already store. The other is when limited resources have to be pointed at somebody, and here the technique becomes a way of ranking people: find the slice that converts well and churns slowly, and advertising and product direction can shift toward bringing in more people who resemble it.

Three limits are worth holding onto.

1. The axis you split on isn't necessarily the cause. People who signed up on a team plan may convert better because of the plan, or because anyone signing up as a team already had a clear need. Correlation, two values moving together, is a different claim from causation, one of them producing the other.
2. Keep splitting and you'll find the slice that says what you wanted. Cut twenty different ways and one of the cuts will show a striking gap by chance. Deciding the axes before you start is the cheap protection against this.
3. The whole and its slices can point in opposite directions. Option A can win in every slice while option B looks better once they're combined. It happens when slice sizes differ sharply, which is another reason to read the sizes alongside the rates. Where the two disagree, trust the slices and explain the blend.

"An overall 9 percent tells us nothing. Let's split it by plan."

A single average can't locate anything to work on.

"That slice is twenty-three people. Hard to talk about a rate in there."

The slice is small enough that chance moves it.

"Don't stack two axes yet. One first."

Confirm the large gaps before shattering the data into fragments.

Pagedeck now looks at its main metrics with two axes, plan and acquisition channel, on by default. After the episode above, Mia had the dashboard rebuilt to show the overall value and the per-slice values on one screen.

What the team actually did next wasn't to fix the checkout screen. They added a prompt encouraging individual signups to invite a colleague. Slicing further had shown that individual users who started using the product with someone else converted at three times the individual-plan average. That choice doesn't exist if all you're watching is 9 percent.

The common assumption is that finer slices are more accurate. Having once found the answer by splitting, splitting further looks like it should reveal more. But the smaller a slice, the more its rate swings on chance. Five conversions out of twenty-three is 22 percent, and one person either way makes it 17 or 26. Change a price or move an ad budget on that number and you've moved without evidence. A useful floor is to refuse to quote a percentage for any slice you couldn't fill a meeting room with. Read slice size first, and where it's small, either widen the period to collect more or hold the judgment on that slice.

The second is treating a visible gap as a cause found. Numbers as far apart as 21 and 4 read like an explanation. But the axis you split on is the criterion that sorted the outcomes, and the cause lives somewhere inside. When conversion differs by acquisition channel, whether the channel is good or the people arriving through it were already different isn't settled by splitting. Settling it takes an experiment with the conditions held level, or a conversation with users in that slice. Skip that step and you pour budget into a channel and watch nothing happen.