Self-serve Analytics

When one or two people can see the data, every question in the company queues in front of them. The questions themselves are usually simple — how many of last week's ad arrivals converted to paid, and similar. Nobody is withholding anything; there is one queue and it has an order.

The problem is when the answer lands. Ask on Tuesday with six requests already ahead of you and the answer arrives Thursday. By then the ad budget has already moved. The answer was not wrong; it was late enough to be useless.

Letting everyone look however they like was no better. The same named metric starts producing different values per person. If Ravi's active user count and Mia's active user count disagree, the meeting stops being about what to do and becomes an argument about which number is right. An hour later the only conclusion is that somebody should check again.

Self-serve analytics splits the difference between those two failures by preparing things before opening the doors: written definitions of what gets counted, tables organized so a person can read them, and permissions specifying who can see how far. With those standing, everyone can look and still get the same value, and questions stop queueing.

Prepare definitions, a data model and permissions in advance, then open the dashboards and the query tools so people find answers themselves. Six things get prepared.

1. Collect metric definitions in one place. For each term like active users or conversion rate, write down what it counts, where the period is cut, and who is excluded. Without that document, no amount of dashboard work stops one name from meaning three values. The document is short — a paragraph per metric is usually enough — and it is the only thing that makes two people's numbers comparable.
2. Organize a data model. Exposing the raw tables the product writes to means nobody can use them: the column names are abbreviations and no units are recorded. Build a separate set of tables with readable names and units, and show only those. That translation layer is unglamorous, and it is the difference between a tool people use and a tool people merely have.
3. Divide permissions. Decide who can see which range, and mask columns that identify individuals from most people. Opening access and deciding what to withhold are one piece of work; without a rule for what to withhold, the range that stays open keeps shrinking.
4. Layer the surfaces. At the top sits a summary of the main metrics; next comes drill-down, clicking a value to split it by acquisition channel or plan; and where that still does not answer, asking directly in SQL. All three exist because people need different depths. With only the bottom layer most people cannot use it; with only the top, somebody spots an odd line and hands the rest back to an analyst. Most questions end at the second layer, which is where the button that splits by plan earns its place.
5. Attach guidance and documentation. A thirty-minute walkthrough, plus the answers to frequent questions built as surfaces in advance. Hand over a tool with no guidance and three weeks later nobody opens it.
6. Set up anomaly detection, which finds and flags values unlike the usual ones. It means nobody has to scan dashboards daily to avoid missing an unusual day.

The most frequent setting is the weekly meeting. When everybody brings their own numbers, the meeting starts at what to do rather than at confirming values.

Experiment results get checked the same way. Whoever opened the experiment reads the per-group values directly and decides that day, instead of asking and waiting two days.

It also shows up in customer-facing work. Sofia checking a specific account's recent usage before a call walks in knowing what that customer is not using. Previously that check entered a request queue and waited its turn.

The analyst's job changes too. Producing an answer per question shifts into maintaining definitions, the model and permissions. New events get folded into the model, and metrics whose definitions have drifted get straightened out. In substance that shift is usually a promotion, even when it does not look like one on a task list.

Certain work stays put. Questions about why something happened, questions requiring several tables joined, and questions where causation has to be separated out remain expert work. What the opened surfaces answer is mostly what happened; why belongs to whoever picks it up next.

"You can see that straight from the dashboard. There's a button to split it by plan."

Pointing out that the answer is reachable without a request.

"The definitions doc says how active users are counted. Let's align on that."

Keeping one name from pointing at different values.

"Self-serve won't get this. It needs three tables joined, so I'll need help."

A judgment that the question is outside what the opened tools reach.

At Pagedeck, Mia maintains the definitions document and Theo maintains the organized tables. There are only five summary surfaces, and in exchange the splitting dimensions are generously exposed. Columns that identify individuals are visible to two people.

The first attempt exposed the raw tables and failed. The column names were abbreviations, nobody could use them, and within two weeks the questions had queued back up in front of Theo. Only after six organized tables were built, with a description on every column, did people start opening them. Requests for a seventh now get weighed against how many people would actually open it.

The most common error is believing that handing over the tool makes it self-serve. Provisioning accounts is the only visible piece of work, which invites that reading. But opening a tool with no definitions and no organized tables means everyone arrives at the meeting holding a different value. That is worse than before: the state changed from answers arriving late to several answers arriving, and half the meeting goes into reconciling numbers. Opening access is the last step, and the three steps before it are the actual work.

The second error is assuming analysts become unnecessary. Requests visibly drop, so the assumption follows. But what drops is simple lookups, and in their place comes the work of maintaining definitions, correcting the model, and attaching new events. With nobody doing that, definitions scatter within about six months and the tables start disagreeing with the real data. At that point people stop trusting the surfaces and the questions concentrate on one person again. This environment is not built once; it holds only while somebody keeps working on it. Treating it as a project with an end date is how it decays.