Test Coverage

How much testing existed was, for a long time, something people only discussed in words. "Enough" meant something different to each person, and whoever asked had no way to verify the answer. Often the person answering didn't know either. Which parts of a codebase several people have touched over six months carry tests is not the kind of question memory answers.

Then a refund calculation shipped wrong. The function that failed had no tests at all, while the function directly beside it in the same file had eight. It wasn't that testing was insufficient so much as that nobody knew where the gap was, and the difference between those two functions was invisible until somebody opened the file.

So machines started counting the executed lines. The real use of this number isn't proving that testing went well; it's finding the lines nobody has run even once. With that list in hand, deciding where to start becomes a lookup rather than a judgment call. Nobody has to be persuaded that a gap exists.

Record which lines execute while the tests run, then divide by the total.

1. Count the lines of code under measurement. At 1,000 lines, that's the denominator.
2. Mark the lines that execute during one full test run.
3. Divide. If 720 executed, that's 72 percent.

Know first that several units are in use. Counting by line is the most common; counting by branch is stricter. Test only the true case of one conditional and line coverage reads 100 percent while branch coverage reads 50. Nobody has checked what happens on the false side, and the first number conceals that. So when quoting a figure, name the unit along with it. A team comparing a line number against a branch number is comparing nothing.

Deciding what to exclude from the denominator is part of the calculation too. Leave generated code, configuration files, and third-party libraries in and the number drifts away from the real state. In a repository carrying 3,000 lines of generated code, removing just that moves coverage by 20 percentage points. Whatever gets excluded should be listed somewhere, because the exclusions are as load-bearing as the count.

Measurement doesn't happen by hand; it attaches to CI/CD and reports automatically. How the threshold gets set is where approaches split. Put a threshold on the overall figure and old code keeps everyone from passing for a while; put it only on changed code and the parts being newly written fill in first. The second is the practical one. When setting a threshold for the first time, pick a number slightly below where you are so the build passes on day one — a threshold nothing can pass gets switched off within a day.

Read the accompanying list rather than the number. The tool marks unexecuted lines per file, and if that list shows gaps in payment, permission checks, or anything that writes data, those come first regardless of the percentage. Which file holds how many uncovered lines is what decides next week's work. The percentage is a summary of that list and nothing more.

There's no absolute threshold. The low end, though, is unambiguously risky. In a repository at 30 percent, there's no way to know what else breaks when you change one function, so every change carries manual clicking-around time with it. That time is invisible on any schedule and it is paid every single release.

The figures used in practice are 70 to 80 percent overall and 80 percent on changed code. Those are the targets because the effort climbs steeply above them. The remaining 20 percent is mostly error handling and failure paths, where constructing the situation takes most of the time. Pushing from 80 to 95 often costs more than the first 80 did.

Different thresholds per area work better: aim for 95 percent on payment calculation and permission checks, and leave code that renders values on a screen at 50. Concentrate the number where being wrong is hard to undo. Without writing that split down, the same argument recurs in every review.

Why a high number isn't safety belongs in the standard too. Executed and verified are different things. A test that calls a function without checking the result still raises coverage. And problems that appear only when two requests arrive at once, or only when bad data has already accumulated, aren't caught by this number at all. So treat it as a floor rather than a passing grade. Something below it is definitely a problem; something above it is merely not that problem.

"Coverage is 82 percent and there are 40 uncovered lines in payment."

Where the gaps are matters before the ratio does.

"Instead of an overall threshold, I'll put 80 percent on changed code only."

Avoid a state where old code stops everyone from passing.

"This test calls the function and never checks the result. It only moves the number."

Separating execution from verification.

Pagedeck came in at 41 percent when it first wired this up. Theo pulled the uncovered-lines list and found more than half of it was code rendering values on screen, while payment and permissions were mostly covered. The number looked alarming; the actual risk was smaller than it suggested.

After that, no target was set for the overall figure and only the 80 percent threshold on changed code stayed. Six months later the overall number was 58 percent, and rollbacks after deployment had dropped from three a month to one. Coverage itself wasn't the cause. What actually changed was the habit of writing a unit test alongside every new piece of code, and the number was the window onto whether that habit held. Remove the window and the habit lasts about a quarter.

The most common reading is that this is a quality metric. High looks well-made, so the assumption comes easily. But the only question this value answers is whether the code ran. A test that never checks its result, a test carrying a wrong expectation, and a flaky test that passes some days and fails others all raise coverage equally. Use it as a quality metric and you get a state where the number climbs while the real risk sits untouched.

The other frequent belief is that setting it as a target makes teams fill it in. Thresholds do move behavior, so it reads that way. But turn a measuring instrument into a goal and it stops measuring. A team handed 90 percent starts with the places that are easy to test, and easy places are usually the ones where being wrong costs little. The time it takes to test one payment function properly covers twenty lines of rendering code, so demanding the number alone gets the rendering filled first.