Path Analysis

Funnel analysis, which fixes the stages in advance and counts how many remain in each, tells you how far people followed the route you drew. It says nothing at all about where the ones who left that route ended up going.

Pagedeck's signup flow was like that. The funnel showed 58 percent dropping out at the create-a-document step, and what that number reported ended right there: how many people stopped. Mia assumed they had stopped for want of anything to write, and added document templates to the screen. Drop-off moved down to 55 percent.

Only after following the actual sequence of screens did the assumption turn out to be wrong. A third of the people who stopped at that step went straight to the pricing page, moved from there into the help center, and then left. They hadn't stopped because they had nothing to write; they had gone to check how far the free plan went. They left because they couldn't find that out anywhere.

Path analysis is how that movement becomes visible in the first place. Under analysis method, the ways of drawing meaning out of data, it belongs to the technique group, the one divided by how the data gets sliced.

It runs in five steps.

1. Set the starting point and direction. One direction begins at a given screen and looks at what came after; the other gathers the people who converted and works backward through what came before. Looking for a cause of drop-off runs forward, and looking for the common route into a conversion runs backward.
2. Tidy the screen names. If one screen is recorded as twelve different screens because of values appended to the address, no path is visible at all. The names have to be merged during event tracking, the work of recording what users do, for this analysis to be possible.
3. Cap the number of steps. Look at three to five. Unfold eight of them and the paths split hundreds of ways, and none of them reads as anything.
4. Keep only the top paths. Cut anything under one percent of the total, and record what share of the total the surviving top ten represent. When that share comes in under 20 percent, people are moving every which way, and the next move is splitting into groups rather than looking at everyone together.
5. Turn it into a hypothesis. Take one route that shows up frequently, write down a sentence saying why people moved that way, and check the sentence with segment analysis, which splits groups and compares them. Pagedeck's pricing-page route appeared twice as often in team accounts as in individual ones.

Three kinds of place fit: when you can't even begin to guess at the cause of drop-off, when you want to see by what route people find a newly shipped feature, and when checking whether the flow you drew matches the movement that actually happens. It's a tool for questions in the shape of diagnostic analysis, the kind that asks why something happened.

There are four limits.

1. The paths explode. With twenty screens, the possible combinations for a five-step path run into the millions. Even in real data it's common for the top ten paths to account for only around 20 percent of the total. The remaining 80 percent is made of individual paths each too small to read anything from.
2. A sequence doesn't report intent. Of two people who moved to the pricing page twice, one may have been considering payment and the other looking for how to cancel. Screen names on their own don't separate those two people.
3. It catches on session boundaries. Somebody who viewed the pricing page yesterday and signed up today is recorded as two sessions, one visit each from arrival to departure. Look at paths only within a single session and that person's whole decision process falls out of view.
4. It gives questions rather than answers. The output of this analysis is a plausible hypothesis rather than a confirmed cause, and the confirmation happens somewhere else.

"The funnel says 58 percent drop here. Can you look at where those people went next?"

Moving the question from a number over to a route.

"Paths are so split that the top ten are 14 percent of the total. Let's segment first."

Naming a state where the paths have exploded past the point of reading.

"Let's run it backward from people who converted. I want to see whether there's a screen they all pass through."

Proposing to reverse the direction of the analysis.

At Pagedeck, Ravi pulls the paths and Mia writes the hypotheses. After finding the pricing-page route, what the two of them did wasn't redesigning a screen but adding one line inside the signup flow stating the free plan's limits. Drop-off at the create-a-document step went from 58 percent down to 41.

The document templates added earlier stayed. Three percentage points was a real result, but had those three points been taken as grounds to push further in the same direction, the remaining 55 percent would have stayed exactly where it already was.

The most common reading treats this as the upgraded version of funnel analysis, since showing more of something looks like being better at it. But the two answer different questions. A funnel gives the percentage remaining at each predefined stage as a number; path analysis shows the terrain of the routes nobody defined. For checking weekly whether a goal is being met, the funnel is right, because fixed stages can be compared against last week. Path analysis results change shape from week to week, which makes them poor material for comparison. In practice the funnel finds where the loss is and the paths show what is happening at that particular spot.

The other frequent belief is that the route most people took is the optimal route, since the thick line gets drawn large on the screen. But a heavily traveled route may be the one we built rather than the one users chose. People crowding through the large button on the first screen is unremarkable, and that fact is no evidence at all that the route is a good one. Judging it means attaching what happened afterward to each route. The thickest path turning out to have the lowest conversion rate of them all does happen.