Feature Adoption Rate

Ship a feature and someone will ask whether it worked, and there's rarely a good way to answer. Numbers that describe the whole product don't break out the contribution of one piece of it.

That's how the retrospective went at Pagedeck a month after comments shipped. Mia asked whether people were using it and nobody could say. The only number the team watched was weekly active users (WAU) — how many people opened the product in a given week — and it sat right where it had been the week before comments existed.

Two different stories explain a flat number like that. Maybe nobody is writing comments. Or maybe the people already coming in started writing comments and the headcount simply didn't move. Those two lead to completely different next steps, and a product-wide number can't tell them apart. So teams started measuring one feature at a time and asking only "what share of people have used this?" Because every feature gets its own value, a feature nobody touched shows up as such two weeks after launch instead of at the next quarterly review.

Take the number of users who used the feature at least once inside a chosen window and divide it by a chosen base of users. The division is trivial; three decisions determine what comes out of it.

1. What counts as "used" — opening the comment panel and actually leaving a comment are different events. Pagedeck counts only the latter. Count the opens and everyone who clicked around lands in the numerator, roughly doubling the number.
2. What to divide by — this swings the result more than anything else. Say 320 people left a comment. Divide by 8,000 signups and you get 4 percent; by the 2,400 who used the product that week, 13 percent; by the 900 who reached a document screen where the comment button lives, 36 percent. For one feature, 4 percent and 36 percent are both true.
3. How long to count — cumulative since launch, or a rolling four weeks. A cumulative number only ever goes up, so it can never report bad news. Seeing whether something is used now requires a closed window.

Each base answers a different question. Dividing by signups describes the feature's share of the whole customer base; dividing by active users describes its share among people currently using the product; dividing by people who reached the screen describes whether the feature itself was appealing. When the first two are low and the last is high, the feature is fine and the path to it is blocked. So teams keep all three side by side rather than picking one, and read where the drop happens.

Every feature has its own normal range. Saving a document should read above 95 percent; a bulk permission change used once a quarter can read 3 percent and be perfectly healthy. Lining one feature's number up against another's and calling it high or low is usually wrong. Three reference points work instead.

The first is the expectation written down before launch. Somebody argued for building this, and that argument contained a claim about who would use it and how much. If the claim was that 30 percent of people reaching a document screen would use it within two weeks and the result is 8 percent, the next meeting is about where the prediction broke rather than about the number itself.

The second is past values for comparable features. Write down the fourth-week number for the eight features shipped over the last year and you have a baseline of your own.

The third is the reach-based ratio. Among people who got to the screen, the share who used the feature judges the feature apart from how well it was promoted.

When the number comes in low, the cause is one of two things: people don't know it's there, or they know and don't want it. Changing the base separates them. A reach-based number of 40 percent against a whole-base number of 4 percent means the people who find it are happy, so the work sits in onboarding and in-product signposting. A reach-based number of 5 percent points at the feature itself.

A single use also isn't much to go on. An adoption rate of 40 percent with only 3 percent coming back a second time describes a button people pressed once out of curiosity. So a repeat-use rate sits next to this one. When both run high, the feature is producing power users — the people who use the product far more deeply and often than average.

"We're four weeks in at 6 percent. Should we rebase it on visitors to the document screen?"

Let's separate a weak feature from a feature nobody can find.

"Adoption went up but second uses aren't following."

We got people to try it and gave them no reason to come back.

"Don't judge this one by adoption. It was built for a handful of accounts on purpose."

Not every feature is meant for everyone, so measure this one some other way.

Pagedeck turns each new feature on for a slice of accounts first, using a feature flag — a switch that toggles a feature while the code stays deployed. Two weeks later Mia looks at adoption inside that slice and decides whether it goes out to everyone. Comments stalled at 11 percent on a reach basis in the first two weeks. Asking a handful of users why turned up the answer: the comment button only appeared after scrolling to the bottom of a document. The button moved up, the flag went back on, and the same slice came in at 34 percent.

A low number reads like a failed feature. The figure invites that reading. But it blends two things: how good the feature is and how visible it is. A good feature buried in a place nobody looks produces the same number as a bad one. Left unexamined, the team removes something that worked, builds a replacement in the same buried spot, and collects the same number again. The first move on a low number is changing the base to find where people fall off, not rewriting the feature.

The other mistake is making the number itself the goal. It sounds like a clear target, which is the appeal. The problem is how easy it is to move. Show every new user a popup that pushes them into the feature and adoption doubles inside a week. The set of people who actually need the feature hasn't changed, while the people who had to dismiss a popup are now slightly more annoyed, which shows up in other numbers. The question worth answering is whether the feature took work off someone's plate, and adoption is the first step toward it.