Heuristic Evaluation

The standard way to find problems in an interface is to bring in people and watch them use it. The trouble is what that costs. Finding five participants who match the criteria takes two weeks, each session runs an hour and comes with an incentive payment, and reviewing the recordings afterward takes several more days.

Those expensive sessions kept filling up with obvious problems. Three of five participants would stall at the same error message, and the reason turned out to be that it said neither what went wrong nor how to fix it — just "Your request could not be processed." That's the kind of thing you can see by looking at the screen. It didn't need two weeks of recruiting to confirm.

The method Jakob Nielsen and Rolf Molich published in 1990 targets that waste. Instead of bringing in users, a few people who know the field sweep the interface first with a list of principles in hand and clear out the cheap problems. What separates it from simply saying "this looks awkward to me" is that the judgment rests on a published list rather than personal taste.

It runs in six steps. With no recruiting involved, preparation and all, two days is typical.

1. Pick the list of principles. Most teams use Nielsen's ten heuristics as written — a usability checklist made of items like whether system status is visible and whether users have a way out of a mistake.
2. Cut the scope down to a task. "The whole product" isn't a scope. Take one flow with a beginning and an end, such as signing up on the free plan and inviting three teammates.
3. Line up three or four evaluators and have each one go through alone. A single person catches roughly a third of the problems, and different people snag on different things, so the list only fills out when separate passes get merged.
4. Each evaluator sweeps the screens twice. The first pass is only for understanding the flow; the second one goes item by item with the principle list in hand. Go through once and you end up writing findings without knowing the overall structure, which mixes in things that aren't problems in the actual flow.
5. For every problem found, record three things: where it is, which of the ten principles it breaks, and how severe it is. Severity runs from 0 to 4, where 0 means it isn't a problem and 4 means it should block release.
6. Merge the lists. Collapse duplicate findings into one and settle disagreements about severity on the spot. What's left at the end isn't a report but a fix list ordered by severity.

Skip the severity step and the list loses its use. Forty findings in no particular order leave whoever receives them unable to tell where to start, so they postpone all of it.

This is the first method to reach for when the screens already exist but time and budget don't. Running it before scheduling a usability test — the session where people are given real tasks so you can watch where they get stuck — keeps that expensive hour pointed at the large problems instead of "I don't know what this button label means."

There are three limits worth knowing.

Problems experts find and places users actually get stuck don't always overlap. The principle list covers general conditions an interface should meet, so it won't catch someone stalling because they don't know a piece of domain vocabulary. If a button is labeled with a word only accountants use, it's just an unfamiliar word to the evaluator.

False positives are common. Findings come back that violate a principle but that nobody in practice trips over. Try to fix everything rated severity 1 or below and the effort goes in while usability — how easily and accurately people reach their goal — stays where it was.

The results depend on who evaluates. Three people who know usability and three people seeing the field for the first time produce different lists from the same screens.

"Let's run a heuristic evaluation before we book the user interviews, so we go in with the obvious stuff cleared."

Use the cheap method first and save the expensive sessions for what they're good at.

"I put that at severity 3. It's not release-blocking, but it should be picked up next sprint."

Naming a problem and deciding when it gets fixed in the same breath.

"I wrote down fifteen and only four of them overlap with Noa's list."

Confirming that separate passes have to be merged before anyone sees the whole picture.

Noa ran this on the invite flow during the week Pagedeck was reworking it. Two designers plus Mia each spent a day going through the invite screens, then came together with their lists. Merged, there were 31 findings, and only two of them appeared on all three lists — those two got severity 4. One was that nothing on screen indicated whether the invite email had been sent; the other was that a mistyped address couldn't be removed.

They didn't fix all 31. Only the nine rated 3 or above went into that sprint, and the rest stayed on the list. Two weeks later, when five users were brought in to test the same flow, not one of the nine came up. What came up instead was something nobody had predicted: people who don't know a teammate's email address give up on inviting altogether.

The most common mistake is treating it as a replacement for user testing. Both find problems in the same screens and both produce a similar-looking list, so the assumption is natural. But they find different kinds of problems. This method finds where the interface breaks a rule; user testing finds where a person stopped. The invite flow above is the difference. Substitute one for the other and the product ships with a flow that violates no principle and that nobody completes.

Running it solo is the other frequent mistake. No users are involved, so headcount seems beside the point. But one evaluator catches around a third of the issues, three get you to roughly 60 percent, and past five the additions taper off enough that adding people stops paying. Call a solo pass "inspected" and a list missing two out of every three problems gets treated as complete before moving on.