Indifferent Quality

With twenty candidates and room for five this quarter, deciding what to drop is harder than deciding what to build. The old approach was to rank by how often something was requested. But request volume tells you which group is loudest, and that isn't the same as telling you what will raise satisfaction.

Pagedeck once expanded theme colors to twelve options. Requests for it had been steady, and it took two weeks. Nothing changed after it shipped. The satisfaction survey score held flat, and three percent of users ever opened that settings screen. The requests kept arriving. Looking back at those two weeks, Mia had one question: could we have known before building it?

Each feature gets asked about in two directions. Ask how someone would feel if it existed and how they would feel if it didn't, separately, and when both answers land near "no strong feeling," it's indifferent quality.

Collecting that through a survey goes like this.

1. Write the feature as one sentence. Not "improve notifications" but "show only the notifications where I was mentioned."
2. Ask the present case and the absent case, with five choices each: I like it, I expect it, no strong feeling, I can live with it, I dislike it.
3. Read the type from the most common pairing per feature. A sample of around 100 usually separates them.

Feed in twenty candidates and seven or eight commonly land here. Half the value of the model is finding those seven or eight, because whatever gets removed returns time to the candidates that remain.

One condition applies when reading the result. If the top type and the second are a few percentage points apart, an average is hiding two user groups. A feature that administrators need and everyone else can take or leave reads as indifferent overall. Split the groups and read it again.

"This one came back as indifferent quality. I'm dropping it from the quarter."

Building it wouldn't move satisfaction, so it comes off the ranking.

"Seven of the twenty candidates were indifferent. We only have thirteen to look at."

The value of the study was in what it deleted, not in what it endorsed.

"Overall it's indifferent, but pull out the admins and it's attractive quality."

The average was covering a split in the user base.

Mia now runs the candidate list through this filter before planning a quarter. The theme color item Noa wanted came up in three studies afterward and read as indifferent in all three. The two of them kept it rather than deleting it, parked at the bottom of the list, because no reason to build it now is not the same as never needing it.