Analysis Method

For a long time, the most common data request inside a company was some version of "can you take a look at the numbers?" Nobody said which numbers, in what order, or what would count as an answer. Whoever received the request pulled whatever came to mind first.

Meetings ended up in a predictable place. Somebody would say paid conversions dropped this month. One person would come back with a table of monthly conversion counts. A second would bring a chart split by pricing plan. A third would collect the free-text reasons people gave when they cancelled. None of the three was wrong, and stacking all three together still produced no conclusion. When one question attracts three different pictures, the only conversation left is about which picture looks more convincing, which is a matter of taste rather than evidence.

It also never stayed done. Two months later the same question would arrive and everyone would start over in their own way, because the reasoning from last time lived in one person's head and left nothing for the next person to pick up.

The way out came from noticing that questions which feel new each time actually repeat in a handful of shapes. Where in the flow do people drop off? Which groups behave differently from each other? Do the people who show up once ever come back? Are these two numbers actually related, or do they just move together? Once somebody wrote down what to look at, and in what order, for each shape, those procedures acquired names. Once they had names, a single sentence — "let's cut this as a funnel first" — was enough for a room to agree on what would happen next. An analysis method is one of those named procedures, and the term covers the whole set of them.

Naming them changed who could do the work. Before, analysis was the private habit of whoever happened to be good at it. After, it became a sequence that somebody a month into the job could follow and get a defensible answer from.

The procedures alone are not enough. Judgment calls sit inside every step — whether to read the average or the median, whether a three percent gap in a sample of 200 people is a gap at all — so statistics belongs next to the methods rather than after them. Analysis also sits downstream of collecting events, moving and cleaning them, and agreeing on what to count. When those earlier steps are shaky, no amount of care at this step saves the conclusion.

The terms under analysis methods fall into four groups: what kind of question the analysis asks (nature), what shape the data gets cut into (technique), how to read the relationships that come out (interpretation), and what may already have gone wrong while the data was being gathered (bias).

Nature splits by the tense of the question, and it has three slots.

1. Descriptive analysis — summarizes what happened. Last month brought 4,120 signups and 210 of them converted to paid. It puts events that already occurred into a form somebody can read.
2. Diagnostic analysis — looks for why it happened. When conversion falls from 12 percent to 8 percent, this is the work of cutting the number apart until you can say which step and which group the drop came from.
3. Predictive analysis — estimates what comes next. It uses the last twelve months as raw material to sketch the coming month. Teams usually point it at decisions that can be walked back, like how much server capacity to reserve or how many support hires to plan for.

The order matters. Without knowing what happened you cannot look for a cause, and without a cause a forecast is arithmetic with no argument behind it. Nine out of ten analyses that run in a working week stay inside the first two slots. A forecast built on a number nobody has explained yet tends to be treated as fact by everyone who reads it later, which is why teams keep the three in sequence rather than jumping to the last one.

Technique splits by the shape the data gets cut into, and it has five slots.

1. Funnel analysis — lays out a defined sequence of steps and counts how many people remain at each one. If 1,000 people sign up, 420 create a document and 130 invite a teammate, the biggest single drop is the step worth working on.
2. Segment analysis — splits the population into groups and compares them. An overall conversion rate of 9 percent may turn out to be the midpoint between 21 percent on team plans and 4 percent on individual plans, in which case not a single user converts at 9 percent. The axis you cut on comes from values already in the data: plan, acquisition channel, signup month.
3. Retention analysis — tracks how often people who showed up once come back, laid out on a time axis. Users who joined in the same week get grouped together, and the share still active after one week, four weeks and twelve weeks is lined up side by side.
4. RFM analysis — sorts customers by three axes: how recently they bought, how often they buy, and how much they spend. Accounts that score high on all three get assigned to a person who reaches out directly, while an account whose last payment was a year ago goes into an automated email list.
5. Path analysis — follows the screens a user actually moved through. A funnel measures how well people followed the route you designed; path analysis shows how far they wandered off it. If a common pattern is arriving from search, bouncing twice between pricing and help, and leaving, the decision is happening somewhere outside the signup flow you drew.

Interpretation covers the rules for reading the numbers that come out, and it has four slots.

1. Correlation — the degree to which two values move together. The observation that accounts sending more invitations tend to stay longer lives here.
2. Causation — one thing actually producing the other. Moving together never gets you this far on its own.
3. Confounding variable — a hidden factor that affects both values and makes a relationship look real. Accounts that invite a lot and also stay a long time may be doing both because the team behind them is large.
4. Simpson's paradox — the case where A looks better in the combined numbers while B is better inside every single group. It is a standing warning about deciding from one merged figure.

Bias covers the cases where the raw material, not the conclusion, is what went wrong. It has two slots.

1. Survivorship bias — judging from the cases that lasted. Survey only the customers still using the product and the reasons the departed ones left never make it into the table at all.
2. Selection bias — a skew introduced by how the sample got picked. An in-app survey is answered by people who open the app often, so their answers cannot stand in for everyone.

The four groups get used together inside a single piece of work. Nature settles what is being asked, technique picks the cut that fits the question, and interpretation and bias check whether the result can be trusted as it stands. Skip the last two and a convincing chart turns into a bad decision. Two questions are worth asking before carrying a result out of the room: do the people behind this number represent everyone, and is the relationship it shows one you are allowed to call a cause?

"Let's cut this as a funnel first. Find out where people drop, then worry about why."

A request to pin down the location of the drop-off in numbers before anyone starts guessing at causes.

"Give me this by segment instead of the overall average. That number looks like two different groups mixed together."

A suspicion that one average sits between two populations and describes neither of them.

"This gets us as far as correlation. If you want to treat it as a cause, we need an experiment."

A brake applied to a proposal that wants to ship a feature on the strength of an observed relationship. The speaker is not saying the relationship is fake — only that the evidence available cannot carry the weight the proposal puts on it.

At Pagedeck, the meeting about a month of falling paid conversion ran in exactly that order. Mia started by reading the conversion counts for the last three months to establish which month bent. Theo then broke the path from signup to payment into steps and pulled the number of people remaining at each one, and only the step after document creation had visibly thinned — 31 percent three months ago, 19 percent now, while every other step moved within a point or two.

They did not stop there. The same table was cut again by pricing plan, and the drop turned out to belong entirely to individual plans while team plans held steady. Mia's last question was whether fixing that step would bring conversion back. Theo's answer was that the existing data could not say, so the two of them set up an experiment that showed half the new users a reworked screen and half the current one.

The most common mistake is starting from the method. There is a list of named techniques, and modern tools run most of them from a single button, so reaching for one first feels natural. But a method is a tool for answering a question, and with no question fixed in advance the output is unreadable even when it is correct. Reverse that order and the meeting gets a report that says "we ran the funnel and the retention curves and nothing stood out." Nothing stood out because nobody decided beforehand what standing out would look like. The thing to settle first is a single sentence of the form: if the answer comes back like this, we will do that. Decide that a heavy drop among individual-plan users at document creation means reworking that screen, and the method that gets you there picks itself.

The second mistake is believing that analysis delivers causes. One of the four groups is literally called diagnostic, and things that look like causes do surface regularly, which makes the belief easy to hold. What digging through a record of past events can produce, though, is the set of values that moved together. Knowing that accounts sending more invitations stayed longer does not tell you whether pushing people to send invitations makes them stay. Teams that intended to stick around may simply also have invited more. Skip that distinction and a company can spend six months building a feature on top of one co-moving number, then find itself unable to explain why the metric sits where it always did. The six months are gone, and nobody ever counts the work that did not get done instead. Analysis narrows the field to a few candidates. Which candidate is the real cause is settled by splitting the population and trying it.