Postmortem

Relief arrives first when an incident ends. The people who were awake at 3 a.m. need to sleep, and the work they postponed is waiting for them in the morning. So what usually remains in the aftermath is a single line reading "resolved."

That is what happened the night Theo fixed an outage caused by database connections running past their limit. Finding the cause took an hour, and several things got learned in that hour: which metric makes this situation visible, which command drops the stuck connections, why the alert came late. All of it stayed in Theo's head.

Three months later the same cause brought the service down again, and Theo was on vacation. Whoever held the pager spent the same hour from scratch, because nothing already worked out the first time could be retrieved. The problem wasn't two outages; it was never having saved the learning even once.

The name comes from medicine, where it describes opening a body to establish a cause of death. In software the purpose is somewhat different — the document exists so the same thing doesn't happen next time, not to explain something already lost. That difference shows up in what gets written: the sharpest postmortems spend more space on what will change than on what broke.

One document covers one incident. The items vary by team, though seven of them are the ones whose absence makes the document useless.

1. Summary. What happened, from when to when, in five lines or fewer. Busy readers read only this.
2. Impact. Written in numbers rather than impressions. Not "many customers were affected" but "for 42 minutes, 18 percent of document save requests failed, affecting 230 paying teams."
3. Timeline. Times and facts in sequence. Record the moment the problem started and the moment the team noticed separately — the gap between those two is exactly how late detection was.
4. Trigger. What set this off. Usually a deploy, a configuration change, a traffic increase, or an outage at an external service.
5. Underlying cause. Why the trigger turned into an outage. The deploy itself was fine, but the connection ceiling was set too low — the condition one level beneath.
6. Assessment of the response. What helped quickly and what consumed time, both. The dashboard that scoped the problem in three minutes goes in, and so does the twenty-minute wait because only one person knew the recovery command.
7. Prevention items. Actions with an owner and a date. A sentence with no owner gets done by nobody.

Items 3 through 5 are where teams most often blur together, and keeping them separate is what makes the document usable. A trigger is visible within minutes; an underlying cause usually takes the hour. Item 7 decides the value of the whole document. The first six can be written well and still come to nothing if the last one reads "improve monitoring." Write it as "alert when connections pass 80 percent — Theo, this week." An alert is the mechanism that notifies whoever is responsible once a defined threshold is crossed.

The writer is one of the people who worked the response, usually whoever led it, but never alone. The timeline only comes out correct when every participant fills in what they did and when. Memory reorders events within hours, and two people who worked the same outage will disagree about sequence by the next morning.

Sooner is better. Many teams draft within two days of recovery and finalize within a week. Delay means rereading chat logs, and it means the prevention items miss their chance to enter the next quarter's plan.

Four kinds of people read it.

1. The team itself — the people who will actually do the prevention items.
2. Other teams — anyone running the same structure is waiting on the same outage.
3. Whoever holds the pager later. On-call is the rotation of being reachable for incidents during a defined window; meeting a similar symptom with a document already written saves that hour.
4. Customers and leadership. Some organizations publish a separate summary covering cause and remediation, and enterprise customers ask for it contractually.

For any of this to be read, a promise not to blame people has to come first. Blameless postmortem is that promise, and without it the timeline fills with defensive sentences. Turn the document into a search for whoever clicked wrong and nobody writes down what they did in the next incident.

"I'll write the postmortem — everyone who was on the response, please fill in the timeline."

Get each person's part down before the memories scatter.

"The action items have no owners. Nothing here will be different next month."

The prevention list exists only as prose.

"This is a bug, not an incident. It doesn't need a postmortem."

No service impact, so there is nothing to record this way.

Pagedeck writes these only for high-severity incidents. Drafting the first one, the item that took Theo longest was the timeline. The chat log held the response, but not what anyone had been looking at when they made each call. From then on, one person writes times and judgments down during the response itself.

What Grace asked about after reading it wasn't the cause but the detection time. Thirty-eight minutes passed between the problem starting and the team noticing, and customer tickets arrived inside that window. Knowing before customers tell us became the largest item of that round, and the alert threshold that came out of it has fired twice since.

The most common reading treats it as a search for who is responsible. There's a field for cause, so it looks like a name belongs in it. But writing a name stops the document. A record that names whoever clicked the wrong thing, while leaving the screen that allowed the click untouched, does nothing to stop the next person doing the same. What belongs there is the condition — that production data could be deleted with no confirmation step. Fix the condition and the question of who clicked stops mattering.

Believing it applies only to large outages is also frequent. Plenty of teams do set a severity threshold and write only above it, so the belief is half right. Setting that threshold too high creates its own problem, though: an outage resolved in five minutes can repeat six times across three months without ever being recorded. Some teams add a rule that three repeats of the same symptom earn a document regardless of size.