Cohort Analysis

Teams used to answer "is the product getting better?" with a single overall average — the share of all users who showed up at least once this month, held next to last month's share. That number stirs someone who signed up two years ago and someone who signed up yesterday into the same pot. When acquisition rises, newcomers make up more of the mix and the percentage falls. Stop advertising and only the regulars remain, so it climbs. The product hasn't changed; only the blend has.

Pagedeck got stuck in exactly that place once. The team revised the onboarding screens three times over two months, and the overall return rate moved from 34 percent to 33 percent. They were about to write the work off as useless when someone remembered that signups had doubled over the same period. The new arrivals hadn't had time to get comfortable with the product yet, so of course they dragged the average down.

Cohort analysis stops the blending by counting people according to when they arrived. Whoever joined in March belongs to the March group and nothing else; whoever joined in June belongs to June. Then the same moment in each group's own life — say, each one's third month — gets placed side by side. If March reads 40 percent and June reads 18 percent, something changed in between, and finding that something becomes the next job.

Decide what defines a group, lay those groups out along a time axis, then read the result in two directions. Five steps get you there.

1. Choose the grouping rule. Signup date is the common one, but the acquisition channel or the plan someone picked first works too. The rule sets the limit on which questions the table can answer.
2. Choose the width of each group. Weekly buckets show changes sooner but hold fewer people each; monthly buckets do the opposite.
3. Choose what counts. Whether "came back" means a login or means opening a document can move the numbers by a factor of two. Pagedeck counts opening or editing a document.
4. Choose the checkpoints. Week 1, week 2, week 4, week 8 — fix them in advance and measure every group at the same marks.
5. Lay it out as a table. Rows are arrival periods, columns are weeks since arrival.

Reading runs two ways. Across a row you see how one group thins out over time. If the March group goes 100, 62, 48, 41, 40 and then holds, that 40 percent is the share that found a reason to keep using the product. Down a column you see the same checkpoint across groups. A week-4 column reading 48, 44, 39 from top to bottom says that more recent arrivals are leaving faster.

Both directions are needed. A row alone can't tell you whether the shape is normal for this product or newly worse, and a column alone hides when in the lifecycle people slip away.

It fits changes whose effects take time to appear. Rework the onboarding, wait until the groups that arrived afterward have filled four weeks, then hold their week-4 numbers against the earlier ones. Comparing the quality of acquisition channels works the same way: put the week-8 figure for the group that came from ads next to the one that came from search and a cheap channel that leaks people quickly stops looking cheap.

Three limits are worth knowing. The first is size. In a group of 40 people, one person moves the figure by 2.5 percentage points, so most of the wobble is noise. Widen the buckets until each holds several hundred.

The second is unfinished rows. Someone who signed up last week has no week-4 number yet. Filling those blanks with zero, or letting them into an average, produces an illusion in which everything looks worse the more recent it is. Compare only the cells that are actually full.

The third is that the table never says what changed. It shows that the June group came out worse and stops there; whether the cause was the acquisition mix, a product release or the calendar is a separate search. People who sign up in late December being quiet in January may be a holiday effect rather than a product problem.

"Don't show me the blended average — cut it into cohorts."

A number mixing new and existing users can't support a decision, so split it by arrival date.

"The September cohort hasn't filled week 4 yet. Let's look again next week."

Keep groups with unfinished checkpoints out of the comparison.

"The ad cohort drops off a cliff at week 2 compared to the search cohort."

Comparing two channels at the same point and pointing out that one loses people faster.

Ravi put the weekly cohort table on screen at Pagedeck's Monday meeting. Starting with the group from the fourth week of August, the week-2 figure had fallen from 41 percent to 29 percent, and one new ad campaign had launched that same week. Measured by signups alone, the campaign had been a success.

Mia suggested regrouping by acquisition channel. Split that way, the group arriving from ads sat at 17 percent in week 2 while everyone else held at the usual 43 percent. The product hadn't gotten worse; the campaign was bringing in people the product didn't suit. Ravi rewrote the copy and narrowed the targeting.

Many people take a cohort to mean a signup-month group and nothing else, because the first example anyone sees is a table of monthly signups, and the shape gets mistaken for the definition. What actually defines a cohort is a shared trait, not a shared date. Group by the first plan purchased, or by whether someone arrived through a teammate's invitation, and the method still works. Believing only dates qualify throws away half the questions the table could answer.

The other misreading is that this is a retention technique specifically. The value inside each cell can be the share who returned, but it can just as easily be revenue per user or the free-to-paid conversion rate. Put the six-month cumulative spend of the March group beside the same point for the September group and the same frame answers a revenue question without changing shape.