Moderated Test

When a team disagrees about whether a screen works, the argument rarely resolves itself. Noa thinks the button on the invite screen is obvious and Mia thinks it disappears, and since both have looked at that screen a hundred times, there is no way to settle who is right. The decision drifts toward whoever is louder or more senior, and the answer arrives later as support tickets.

Surveys get tried as a substitute. Ask "how easy was this screen to use?" on a five-point scale, get an average of 3.8, and you still have no idea what to change. People don't accurately remember where they got stuck, and they tend to read getting stuck as their own failing, so they report less friction than they experienced.

A moderated test closes that gap by watching a person directly. Instead of asking for opinions, you give someone the real task, watch the moment they stall, and ask right then what they were looking for. What comes out is not an average of 3.8 but "four of five people looked for the invite button in the top right," and that sentence tells you what to change next.

Five steps. Preparation is half the work, and the time actually spent with a person is about an hour each.

1. Write down what you want to know, in one sentence. "Evaluate the invite screen" is not a question. "Does someone who doesn't know their colleague's email address finish an invitation?" is a question.
2. Recruit participants who match the conditions. This is usually the longest stretch. Finding five admins from 30-person teams on a paid plan takes about two weeks. Five people is the default, and if users split into two distinct types, you run five of each.
3. Write the session plan and rehearse it once on a colleague. This is where the task wording and your follow-up questions get fixed in advance, and where leading questions get caught. "This button is easy to find, isn't it?" becomes "please find this button," and what comes back is behavior rather than an opinion.
4. Run the session. Give the task, ask the participant to think out loud, and watch. When someone stalls, the urge to offer a hint is strong, but that silence is the core of the method. Wait thirty seconds, then ask what they are looking for. Record the screen, and write down what you observed without mixing in your interpretation of it.
5. Gather the five sessions and review them together. Pick out the stalls that repeated across people and rank them. What should exist at the end is not a report but a list of things to fix.

Use it when you need the reason behind the stall. Where someone got stuck is visible in a screen recording; why it happened only comes out if you ask in the moment. It runs against a prototype that hasn't been built yet, which is where it pays best — fixing a flow before anyone writes the code. Flows that only new users meet, like the minutes right after signup, are also hard to observe any other way.

Five people is enough because this is qualitative research, which captures why people behave as they do through their words and your observation, and leaves its results as sentences. When three people in a row stall at the same place, that is already enough grounds to fix it.

There are three limits.

1. It's expensive. An hour per person, plus the work of scheduling and paying incentives.
2. It produces no numbers. Four out of five people stalling does not mean 80 percent of users stall. If you need a rate, measure it separately with quantitative research, which needs dozens to hundreds of people.
3. The facilitator moves the result. Participants want to look competent in front of the person sitting there, and if the facilitator glances at one side of the screen, eyes follow. That is what the rehearsal is for.

So teams swap in neighboring methods depending on the situation. If you need completion rates rather than reasons and you need many people, an unmoderated test fits: participants work alone and leave a screen recording and their voice, so you can run ten in a day, but you can't ask anything at the point where they stalled. If you need an answer by tomorrow, a guerrilla test gets five minutes with whoever is in the café. Those people probably aren't real users, so it can't settle fine-grained questions — it cheaply catches the large, obvious ones.

"Four of five people looked for the invite button in the top right."

The argument about button placement just got settled by observation.

"We need the reason on this one, so let's go moderated."

Pick the method that lets you ask follow-ups over the one that only counts successes.

"Recruiting takes two weeks, so the test lands in the next sprint."

Finding people, not the method itself, sets the schedule.

At Pagedeck, Noa runs one before engineering starts whenever there's a new flow. The invite screen last quarter went that way. Four of five participants looked for the invite button in the top right while the actual button sat below the list, and after the third participant said "I thought it would be around here," nobody in the observation room argued anymore.

Mia and Theo sit in the observation room too. Anyone who watched all five sessions doesn't wait for a report; they cite the same moments when the team decides what to fix.

The most common misreading is that this is where you ask users what they'd prefer. You're sitting with a person for an hour, so it looks like an interview. But what the method collects is behavior, not opinion. People are poor at predicting their own actions — ask whether they'd use a button if it existed and most say yes, then don't. Steer the questions toward preference and an hour goes by without producing anything to fix.

The other is that five people can't be enough. If you're used to surveys and experiments with hundreds of respondents, that reaction makes sense. But the counting serves a different purpose here. You're looking for the places people stall, not for a rate, and serious stalls tend to repeat within the first few sessions. Raise the number to twenty and you mostly watch the same scene fifteen more times; fixing the screen and running it again is faster. That said, if you have to state which of two designs has the higher completion rate, five people cannot tell you.