Regression Test

Breakage usually does not appear where the edit was made. It appears somewhere nobody touched. After that pattern repeats a few times, a team needs a check that asks a different question from "does the new feature work" — namely, did anything that was left alone quietly stop working.

Pagedeck ran into it directly. Theo reworked the document permission logic and removed a step from sharing a team document externally. The part he changed worked fine. But opening a document through a share link started bouncing people to the login screen, and the team found out three days later from a support ticket. Nobody had clicked that path again. Theo tested what he changed, and the share link route belonged to a feature shipped a year earlier that nobody thought of as related to permissions.

Checking everything by hand sounds like the obvious fix, and scale rules it out. Walking a product with 200 features end to end takes two days, and two days cannot be spent on every release. People doing that walk also skip the boring parts. So the things that must keep working got written down as a list, and a machine runs the list in the same order every time.

Start by pinning down what "already worked" means. Five steps.

1. Write down the flows that must not break. Signup, login, creating a document, opening a share link, payment — the ones a customer notices within minutes. The list is the scope of the test, so a feature nobody wrote down goes unchecked no matter how important it is. Exporting to PDF is the kind of thing that gets left off: a tenth of customers use it, and that tenth uses it every day.
2. Move the list into automated tests, layered. Hundreds of unit tests — fast checks that feed one function a value and compare the result — cover individual calculations, while end-to-end tests, which drive a real browser the way a person would, stay limited to a few dozen flows like signup and payment. Put 500 of those in when each takes ten seconds and a full pass costs two hours.
3. Decide when each part runs. The usual split is three: the fast subset before a change gets merged, the full set before a release, and the slowest material once a night. Without that split, developers sit waiting on results instead of working.
4. When something goes red, sort out the cause before anything else. A genuine break has to be separated from a flaky test — one that passes some days and fails others with no code change at all. Leave the flaky ones alone and the team starts rerunning red builds and moving on, and from that moment the set has stopped functioning as a warning.
5. For each new bug, add a test that reproduces it. That is how the same failure is kept from happening twice, and it makes the list grow along the shape of real incidents rather than someone's guesses.

The value is highest on a product that releases often. Ship two or three times a week and there is no time for a manual pass, so the machine walks the same list instead. Old code where nobody still knows what is wired to what is another good case, and so is any area about to be restructured — this set is the instrument that shows the screens did not change.

Three limits.

1. What is not on the list does not get looked at. Test coverage is the share of code the tests actually executed: 720 of 1,000 lines is 72 percent. Even at a high number, failures in the other 280 lines pass straight through.
2. Once it is slow, nobody runs it. At 40 minutes a pass, developers stop waiting for the result and move on; at two hours, they look for ways to skip it. That is why speed here is a trust problem rather than a performance problem.
3. It costs upkeep. Change a screen's structure or wording and the tests that leaned on it break with it. Every release picks up a little work fixing tests while the product itself is fine, and if nobody counts that cost the team quietly starts deleting tests. Logging the hours spent repairing tests separately helps, because the number is then visible when someone argues the suite is not worth keeping.

"This change touches permissions, so let's run the full regression set before it goes out."

The blast radius is wide enough that the fast subset is not enough.

"Two payment cases are red in regression. Check whether they were red yesterday too."

Find out whether this is new breakage or something that was already unstable.

"Fix the bug, and then add it to the regression set as well."

Asking for the step that keeps the same failure from coming back.

After the share link incident, Theo added that flow to the list. Opening a document through a link while logged out became one end-to-end test, and the permission calculation underneath it got twelve unit tests. The same class of mistake has been caught before release ever since. Those twelve unit tests finish in under a second, which is why they sit in the fast subset rather than the nightly run.

Mia reads this set's result first in the release meeting. All green and the release proceeds; a single red and nothing ships until someone has explained what it is. The rule exists because one exception removes the standard entirely.

The most common mistake is filing regression tests alongside unit tests and end-to-end tests as a third kind of test. The name sounds like it belongs on that shelf. But it describes a purpose, not a scope. Rerun yesterday's 300 unit tests after today's change and those 300 are today's regression tests. Teams that miss this go off to build a separate regression suite, end up with two overlapping sets covering the same ground, and double the upkeep for nothing. The two copies also drift: a rule corrected in one suite and not the other leaves them disagreeing about what correct behavior is.

The second is reading all-green as safe. The result is consistent enough that the green starts to feel like a statement about the whole product. What it guarantees is that the items on the list still work. Features that are not on the list, features built this week, and cases on the list running under different conditions all sit outside it. Read green as proof of quality and the habit becomes shipping new work unverified — and incidents keep arriving from outside the list.