Human in the Loop

Attaching a model to a feature got easy, and plenty of teams wired one in without ever deciding how far it should run unattended. It classified, it drafted, and removing the person looked like the obvious next move.

The trouble is in how models fail. A model states something false with exactly the confidence it uses for something true. Because of that — the failure mode called hallucination — a wrong result carries no marking, and in a setup that publishes directly, the error reaches the customer intact. A team auto-classifying support tickets reads a refund request as a how-to question, sends back a help article, and learns about it from the customer's public complaint.

Regulation arrived on top of that. The AI Act, which regulates uses of artificial intelligence by risk level, sets aside the uses that decide a person's opportunities and attaches heavier duties to them. Screening job applications and deciding loan approvals live there, and one of those duties is keeping a place for a person to check the result. Since it's hard to explain why a model decided what it decided, reviewing the output before it leaves is what remains.

Start by choosing which judgments a person sees. Review everything and the feature stops being worth building; choose nothing and everything ships unattended. Five steps.

1. Split the judgments. Two criteria do the work: whether a wrong result can be undone, and whether the result touches one person's rights or opportunities. A typo in a summary gets edited; a contract termination notice doesn't get taken back.
2. Have the model return its reasoning, not just its answer. Which sentence the judgment came from, how confident it is. Without that, the reviewer reads the source material from scratch and the work takes twice as long.
3. Build the review screen. Put the source and the model's proposal side by side, with a field for editing before sending and a way to send it back. A screen offering only approve and reject pushes fixable items into the reject pile.
4. Record decisions. Who approved what, when, and what they changed. This log is what regulators ask for, and it's also the input for the next step.
5. Feed the log back. Where reviewers edit often is where the model is weak, and a stretch with no rejections at all is a stretch that can ship unattended.

It fits judgments that are hard to reverse and land heavily on one person. Résumé screening, loan decisions, account suspensions, refund denials, anything that goes out under the company's name. It doesn't fit high-volume work with light consequences. Putting a person in front of every one of tens of thousands of daily tickets removes the reason for automating them.

There are three limits.

The first is review that exists in form only. This gets worse as the model gets better. Someone who has watched it be right ninety-nine times approves the hundredth without reading it. Once the approval rate climbs past 99 percent, the first thing to question is whether anyone is actually looking.

The second is the bottleneck. Adding a person stretches handling time from minutes to hours, and the reviewer's salary becomes an operating cost. With nobody reviewing at night or on weekends, work piles up for those hours.

The third is that responsibility collapses onto an individual. When something goes wrong, the log holds the name of whoever pressed approve. Leave that name there while giving the person neither time to judge nor authority to overturn, and the review step stops reducing risk and starts transferring it.

All three shrink with design. Send up only the high-risk items and the ones the model wasn't confident about instead of reviewing everything, track rejection and edit rates as metrics, and give reviewers both the power to overturn and the minutes it takes.

"Put a human in the loop on this one. There's no undoing it if it goes out wrong."

A proposal to separate the judgments that can ship unattended from the ones that can't.

"A 99 percent approval rate means nobody is reviewing, just clicking."

The review step is still there and is no longer catching anything.

"If this feature gets classified high-risk, the review step is a legal requirement."

What the feature is used for turns this design from a choice into an obligation.

When Pagedeck added automatic replies to support tickets, Mia split the judgments in two. Questions about how a feature works get an article sent automatically; anything involving refunds or cancellations waits for Sofia to approve. Getting the first kind wrong costs one follow-up question, while the second moves money and contracts.

The same conversation came up when Grace looked at a résumé-screening tool for internal hiring, except there was nothing to decide. Sorting candidates by opportunity meant the review step and the decision log were both mandatory, and AI bias — skewed training data turning into discriminatory output — had to be checked alongside them.

The most common belief is that having a person look makes the system safe. The review step appears as a line item in compliance documents, so it gets treated as a box to tick. But this design reduces risk only while the reviewer is genuinely catching wrong results. Show someone an answer with no reasoning attached and ask for 600 a day, and the position becomes a rubber stamp. Left that way, an incident produces a record showing a process existed and nobody caught anything, and the individual named in that record absorbs the blame.

The other misreading treats a human in the loop as giving up on automation. Seen that way there's no point attaching a model at all, and teams swing toward full automation or toward not using a model at all. What actually changes is where a person's work starts: at a draft rather than a blank page. Narrow the obviously-filterable range first with something like a guardrail, a safety mechanism that keeps model output from straying, and reviewers spend their time only on the judgment calls. Model transparency — disclosing that a model was involved and what its answer rests on — pulls in the same direction, since a reviewer needs some signal about where to be suspicious for the check to be more than a formality.