Recommendation

With twenty items, choosing isn't a problem. Show all of them and the user scans and picks. At 8,000 items the situation is different.

Opening the template gallery's usage data, Mia found the problem as a number: the top 20 of 8,000 templates accounted for 70 percent of all views. The remaining 7,980 might as well not have existed. Most of what had been built was failing to reach anyone, and nobody had noticed because the aggregate view numbers looked healthy.

A list and a search box don't solve this. Search works only when the user already knows what they're looking for; somebody unaware that a meeting-notes template exists never types those words. And once a list's ordering is fixed, only the first few entries keep getting seen.

So the ordering becomes per-user. Candidates get selected and surfaced based on what this person has done so far and what people like them have used. The ordering stops being one decision and becomes one decision per visitor.

Deciding the placement matters more than choosing the model. Within one product, where it sits turns it into a different task.

1. Decide the placement and the purpose. What appears on the gallery's first screen, what gets suggested right after a document is created, and what fills an empty state are three separate tasks. Write down the purpose too: raising clicks and getting unused features used call for different recommendations.
2. Collect signals. Some are deliberate, like a rating or a save; others accumulate on their own, like clicks and dwell time. The first kind is accurate but scarce and the second is plentiful but noisy, so the second usually drives the model and the first corrects it.
3. Choose an approach, from three.
4. Handle the cold start. A new user has no history to work from, and a newly uploaded item has no history of being used. Without a decided answer for both, the screen either goes empty or fills with nonsense.
5. Evaluate. Measuring whether it would have predicted past behavior runs alongside an A/B test measuring real effect. The two disagree often, and when they do the live experiment wins.
6. Layer rules on top. Exclude what the user already used, mix the results so five of the same kind don't stack up, and block anything that shouldn't be shown. The share these rules contribute to result quality is larger than people expect, and they are the cheapest part of the system to change.

The three approaches in step 3 divide like this. Popularity-based surfaces whatever has been used most recently — a few rules produce it, and it works for new users. Content-based finds items with similar attributes, so it can recommend something uploaded a day ago. Collaborative filtering pulls what people who behaved like you have used; it needs accumulated history but has the greatest power to surface something unexpected. Most teams start on popularity and add the others one at a time.

Two conditions: the candidate pool is larger than a user can scan, and behavioral history is accumulating. In a product with twenty items, a rule-defined ordering beats a model, and costs a day instead of a quarter.

Five limits come with it.

1. Cold start. New users and new items have no basis. Early in a product's life every user is new, which puts this problem front and center.
2. A self-confirming loop. Only recommended items get clicked, so the next round of training data tilts the same way. The model selects on the basis of what it already selected.
3. Popularity bias. Items already widely used get recommended more, and the 7,980 sink further. If the purpose was reviving the long tail, the result runs opposite to the purpose.
4. No available reason. Explaining why this item was chosen is hard, which also removes the thread to pull when a user gets a strange recommendation.
5. Evaluation mismatch. A model that predicts past behavior well frequently makes little difference to real users, because past behavior was shaped by the old ordering.

Some places don't suit it at all: items bought once, screens where regulation fixes the ordering, and search results where the user arrived with a clear intent. Vary search results per person and the same query returns something different each time, so people can't find again what they saw earlier, which reads as a broken search box rather than as personalization.

"Let's start by ordering on most-used templates. The model comes after we have history."

A proposal to build the placement rules-first.

"Recommendation clickthrough went up and paid conversion didn't move."

The proxy metric and the real metric diverged.

"New signups get an empty recommendation panel."

The cold start was never handled.

Pagedeck attached this task to the gallery's first screen. Theo wanted to build collaborative filtering first, and Mia argued for rules first. History was still thin, and more importantly the purpose was surfacing buried templates rather than raising clicks.

So the panel got half popularity-based and half recently uploaded, with one rule excluding what the user had already used. The top 20's share of views fell from 70 percent to 41. That came from placement and rules with no model involved, and collaborative filtering went back on the list for six months later, once there would be history worth learning from.

The most common reading is that a better model means better recommendations. It's classified as a machine learning task, so model performance looks like the deciding factor. What actually moves the result is where it was placed and which rules were layered on. One line excluding already-used items frequently raises clickthrough more than changing the model does, and so does moving the panel higher on the screen. Look at placement and rules before spending two months on model performance.

Equating it with personalization is also frequent. Both show different things to different people, so they appear to overlap. But recommendation is the task of selecting from candidates, while personalization is the design of fitting a whole screen to a person. Personalization contains recommendation, and it also contains changing currency by region and varying the first screen by job role. Attaching a recommendation model doesn't make a product personalized, and calling it that hides the parts still left undone.