Cohort Retention

Return rates used to be reported as a single figure: retention is 32 percent this month, 33 percent last month. The trouble was that whoever received the number had nowhere to go with it. Whether 32 percent was good or bad, and whether it was heading up or down, wasn't anywhere inside it.

Pagedeck's product meeting replayed the same scene for three months straight. Mia would put this month's retention on screen, someone would ask whether the onboarding change from last time had worked, and no answer came out. That one cell held people who joined in March next to people who joined yesterday, so nothing could be traced to anything.

Cohort retention spreads that single figure into many rows. People who arrived in the same window are grouped into a cohort, and every cohort keeps its own clock. Once the week-4 number for people who joined in the first week of September sits on a different row from the week-4 number for people who joined in the first week of June, comparing two cells vertically shows whether the product improved in between.

Fix one cohort's headcount as the denominator, then count how many of those people came back in each week and divide. Five steps.

1. Cut the cohort. The 500 people who signed up between September 1 and September 7 form one group, defined by a shared starting point.
2. Decide what returning means. Pagedeck counts opening or editing a document. Counting logins alone produces a much higher number for the same product.
3. Count the numerator for each week. Week 1 is the number who came back between day 8 and day 14 after signing up.
4. Keep the denominator at 500 the whole way. Re-counting it after dropping the people who left shrinks the base every week and flatters the result.
5. Stack the cohorts. Rows run September week 1, week 2, week 3 and so on, with weeks since arrival running to the right.

If 320 of those 500 came back in week 1, 225 in week 2 and 165 in week 4, the row reads 64, 45, 33. Tables like this always come out triangular, because the lower rows have had less time to pass and their right-hand cells are still empty. Filling blanks with zero, or letting them into an average, creates an illusion where everything recent looks worse, so only full cells get compared.

One more rule needs settling in advance: whether a resurrected user — someone who stayed away for eight weeks and then came back — counts in their original cohort's cell. Whichever way the team decides, write it next to the table and keep it. Change it midway and the rows on either side of the change stop being comparable.

Comparing your figure to another company's is close to useless, since a messenger people open several times a day and a tax filing service used once a year can't share a scale. Two readings inside your own table do the work instead.

Across a row, look for where the curve flattens. A row that goes 64, 45, 33, 31, 30 and then holds says that 30 percent found a reason to stay, and that every round of acquisition now adds to a layer that accumulates. A row that runs 45, 28, 14, 6, 2 to the floor says nobody found that reason, and spending more on acquisition at that point pours water into a leaking bucket.

Down a column, watch how the same week moves from older rows to newer ones. A week-4 column climbing 33, 35, 38 means recent arrivals are staying longer, so something done in between worked. Judgment runs this direction because any single row carries the circumstances of its own week. People who signed up during a holiday week show a low first week for reasons that have nothing to do with the product.

A flat curve also tells you what the ceiling on growth looks like. If 30 percent of each cohort stays, bringing in a thousand people a month adds roughly three hundred to the standing base, and the size the product settles at follows from that arithmetic rather than from any single month's signup count.

Numbers that jump right after a feature launch deserve the same caution. The flattering bump a new thing produces is called the novelty effect, and two weeks later the figure has usually returned to where it was. So the verdict waits until the next cohort has filled the same week.

"The September cohort is five points above June at week 4."

Comparing the same week across rows to claim that a change in between worked.

"This row has only filled through week 2."

Keep a cohort with too little history out of the decision.

"The curve flattens at week 8, so that's where the real customers start."

Pointing at the layer left standing once everyone who was going to leave has left.

Pagedeck once cut the guided setup before a user's first document from three steps to one. It shipped in the second week of September, and when Mia saw that week's cohort come in at 71 percent in week 1 against the usual 64, she wanted to announce it immediately. Theo asked her to wait for the next cohort.

Three weeks later the table had three more rows and week 1 held at 70, 72 and 69. The week-4 column had also risen from 33 to 39. A single row could not have been separated from luck; four rows read as a result.

Many people assume the contents of each cell are settled automatically. The table's shape is familiar enough that the counting rule looks like part of it. But cohort retention is a way of grouping people and stacking rows, and how each cell gets filled is a separate decision. It can follow N-day retention, counting only those who returned on exactly that day, or rolling retention, counting anyone who returned at any point after it. The second always reads higher, so mixing tables built both ways reports the same product with numbers twice apart.

Judging from one cohort's movement is the other common mistake. When a single row is in front of you it's tempting to conclude from it alone. That row carries its own week's acquisition mix, holidays and whatever campaign happened to be running. A team that pushes harder on whatever it did in the second week of September often finds, three months on, that the row still stands alone while nothing around it moved. The broader practice of grouping people by when they arrived and comparing across groups is called cohort analysis, and needing several rows is exactly why.