Metric Definition

It's easy to assume that two numbers with the same name mean the same thing. The assumption usually falls apart in a meeting room.

At Pagedeck's monthly review, Mia reported last month's monthly active users (MAU) as 40,000. The next item on the agenda, Ravi read the same month as 32,000. Thirty minutes went into arguing about who was right, and it turned out both were, by their own rules. Mia counted any account that logged in at all; Ravi counted only accounts that opened or edited a document. Mia used a trailing 30 days; Ravi used the first through the last day of the calendar month. The 180 internal and test accounts were in Mia's number and not in Ravi's.

The cost of that meeting wasn't the thirty minutes. The decision on the table was whether to run the paid-conversion campaign this quarter, and once the reference number wobbled the decision slid to the following month.

A metric definition exists to move that argument in front of the meeting instead of into it. Write the counting rules down in advance and the meeting stops being about whether the value is right and becomes about what to do. Whether the rules are the best possible ones is a separate question and a later one. Everyone using the same rule comes first, even if a different rule would have been defensible.

A one-page metric definition usually carries six things.

1. The decision this number is for, in one line. Without a reason to count, there's no basis for judging the other five lines. For monthly active users it might read "sizes the audience for the paid-conversion campaign." Having that line written down means that when someone later proposes making the number a team target, you can check the proposal against the original purpose. This is where Goodhart's law — the rule that a measure stops measuring once it becomes a target — bites.
2. What counts as one. Whether the unit is an account, a person, or a device moves the value a lot. One human with a work account and a personal account is 2 by accounts and 1 by people.
3. Which action qualifies. Does logging in count, does opening one document count, does typing count? Pagedeck settled on accounts that opened or edited a document. Counting someone who clicked a notification email and closed the tab would make the number useless for sizing a campaign.
4. The window. Say whether it's the trailing 30 days or the calendar month. The two produce different values every month: a trailing window moves daily, and a calendar window swings a few percent on its own because February and March have different numbers of days.
5. What gets excluded. Internal staff accounts, test accounts, and traffic from automation tools are the usual three. Leave 180 such accounts in at a 45-person company and the early numbers run visibly higher than reality.
6. Where the value comes from. Name the table and the formula, and make every screen read from that one place. This is where a metric definition meets the single source of truth principle.

Below those six lines sits a change log: when the definition changed, why, and what the value was before and after. Editing a definition edits the past too, so the record matters as much as the rule.

The draft comes from whoever makes decisions with the number. At Pagedeck that's Mia, because she is the only one who can write line one. When a data specialist drafts first, the counting gets precise and the purpose stays blank.

Once there's a draft, Theo checks whether the data can actually be counted that way. This is usually where the definition takes a haircut. If the rule says "accounts that edited a document" but edit history is only retained for 30 days, that rule can't be run. When what you want to measure and what the logs hold don't match, either the definition bends to the logs, or the logging gets fixed first and the definition starts applying next quarter.

The readers are everyone who says the number out loud: whoever reads it off a dashboard, Grace putting figures into an investor deck, Ravi reporting campaign results. Which is why the document shouldn't be written in language only data specialists parse.

Changes get announced before they land. Dropping internal accounts from active users pulls last month's figure from 40,000 down to 38,000, and without a heads-up someone reports in the next meeting that growth has stalled.

"What's the definition on that number?"

Checking the counting rule before deciding whether to trust the value.

"We changed the definition, so last month's figure drops too."

A warning that the metric didn't fall — the rule moved.

"This definition can't answer that question."

The number is correct but isn't the right number for the decision being made.

While preparing the paid-conversion campaign, Pagedeck rewrote its active user definition. Mia drafted, Theo checked the logs, and Ravi confirmed it produced a list he could actually target. Once the three of them agreed, the value dropped from 40,000 to 31,000.

The day it dropped, Grace asked whether growth had stopped, and Mia opened the change log and put the before and after side by side. That's mostly what these documents do. They don't make numbers more accurate so much as they stop the same conversation from happening again.

The most common mistake is treating the definition as a technical document the data team handles on its own. It has formulas and table names in it, so it looks the part. But half of a definition is judgment rather than technique. Whether to exclude internal accounts, whether a login alone counts as active — those answers come from what the number is for, not from the data. Hand the whole job over and you get a number that is syntactically correct and useless for any decision, and six months later each team is quietly counting its own way again.

The other one is believing a definition can never change once set. It sounds responsible, since changing it breaks comparison with the past. But when the product changes, an unchanged rule starts measuring something else. Keep counting web sessions only after launching a mobile app and the number drifts further from actual usage every month. The rule isn't "never change it," it's "record the change." Put the date, the reason, and the before and after values in the same document and comparison with the past survives the edit.