Usability Metric for User Experience

A standard for scoring usability already existed. The System Usability Scale (SUS) produces a value between 0 and 100 from ten items, and its average is widely cited as 68, which makes an outside comparison possible. Ten items cause no trouble when a participant is sitting in a research session working through them.

The trouble started when the questionnaire went into the product itself. Put a survey in the corner of a screen, ask ten items, and people close the panel on the third one. A survey asks the same question of many people to see the distribution of answers. Responses that stop partway can't be scored, so they get discarded.

Pagedeck hit exactly that wall. After shipping a rewritten document editor the team put a survey on screen, and 4 percent of the people who saw it finished. The sample never reached thirty, so any score it produced was useless for comparison against the next quarter. Measurement failed for one reason: the questionnaire was long.

UMUX is the short version aimed at that situation. It cuts the item count to four while keeping the same 0 to 100 scale as the longer instrument. Not inventing a new scale was the point — a score gathered quickly should be readable in the same place as the scores already collected.

Collect an answer to each of four items on a seven-point scale, convert the four answers to points, add them, then divide by the maximum and multiply by 100.

The items alternate between positive and negative wording.

1. This product's capabilities meet my requirements — positive.
2. Using this product is a frustrating experience — negative.
3. This product is easy to use — positive.
4. I have to spend too much time correcting things with this product — negative.

Answers run from strongly disagree (1) to strongly agree (7). Mixing the wording filters out people who answer every item the same direction without reading. A sheet with a 7 on both a positive and a negative item is discarded from analysis.

Conversion depends on the direction of the item.

1. Positive items: subtract 1 from the answer. An answer of 6 becomes 5 points.
2. Negative items: subtract the answer from 7. An answer of 2 becomes 5 points. Answering low on "frustrating" is the good signal, so the direction flips.
3. Add the four point values. Each item caps at 6, so the sum falls between 0 and 24.
4. Divide the sum by 24 and multiply by 100.

If someone answers 6, 2, 6, 3, the converted values are 5, 5, 5 and 4, summing to 19. Divide 19 by 24, multiply by 100, and the score is 79. Run that calculation per respondent; the average of those individual scores is the score for the study. Keeping individual scores lets you see the spread alongside the mean.

There is also a two-item version that asks only items 1 and 3 and converts them the same way. It shows up where even four items are too many for the space available on screen.

Somewhere near 68 is ordinary. The scale was designed to match the ten-item instrument, so the benchmark from that instrument carries over. Above 80 means usability is rarely what blocks anyone; below 50 means finishing the task at all is hard.

Two things come before the absolute value. The first is sample size. Fewer items means a single response moves the average further. A 72 from ten people cannot support the claim that the product got worse when the next round returns 65. Treat thirty as the floor, and gather more than fifty when the point is to compare across time.

The second is spread. An average of 68 where individual scores cluster between 62 and 74 is a completely different state from an average of 68 where half the responses are 90 and half are 45. The second pattern says the user base has split — long-time users report it as easy while people who joined this week are stuck. Chasing the average in that situation improves neither group.

Comparing a product against its own earlier scores is the main use. Comparing against a different product requires matching conditions. Whether respondents were asked after completing a specific task, or asked cold from a panel on screen, moves the number by several points either way.

"Response rate on the ten-item SUS was too low, so we switched to the four-item UMUX."

Questionnaire length was capping the sample, so the team moved to the short form.

"UMUX came back at 71, but the sample is twelve people. Too early to quote it."

There is a score, but the sample is too small for a comparison across time.

"The average held steady and the distribution split. New users dropped hard."

A difference between user groups that the single average was hiding.

Pagedeck settled on measuring once a quarter after the editor rewrite. Noa put the items in a quantitative research email rather than on screen. Quantitative research measures how much of something there is, in numbers. Asking on screen returns answers faster, but only people currently stuck tend to answer, which pulled the score toward the low end.

The first round came back at 64 and the second at 69. Mia asked whether a five-point rise was worth celebrating, and the answer was that with samples in the forties a gap that size shows up by chance. So the team agreed to read direction only until three rounds were in, and used the free-text answers to build the task list for an unmoderated test.

The most common mistake is reading the score as a statement about what to fix. When a number comes back low, the cause feels like it should be inside the number. But this questionnaire compresses an overall impression of usability into one value; it carries nothing about which step of which screen blocked anyone. All a 55 supports is the conclusion that a problem exists. Locating it takes sitting someone down or watching the recordings.

Treating it as a satisfaction score is the other frequent error. Both come from surveys and both land on a numeric scale, which invites putting them side by side. They ask about different things. Net promoter score (NPS) and satisfaction scores ask whether people like the product; this asks whether people find it easy to operate. A product that is hard to use but has no alternative scores low on usability and middling on satisfaction. Reporting the two as one figure erases that state.