Blameless postmortems: how to run one that actually changes something

Every team says they do postmortems. Most run one of two broken versions: the trial, where the meeting quietly becomes about who caused it, or the ritual, where a document gets written, filed, and never read. Both produce the same result — the next incident looks a lot like the last one.

The working version has a specific shape. It is blameless, it is built on a real timeline, and it ends with action items that have owners and dates. This is how to run it.

Blameless means the analysis assumes people acted reasonably on the information they had at the time, and asks what made the wrong action look right. The engineer who ran the migration against production believed they were pointed at staging — the interesting question is why the tooling let that belief survive.

The reason this matters is data quality, not kindness. The moment a postmortem becomes a trial, people start editing the timeline to protect themselves, and you lose the only accurate record of what happened. A team that punishes honesty in postmortems is choosing to run its next incident on fiction.

Keep the document short enough that people will actually read it. Six sections cover everything that matters: an impact summary (who was affected, for how long, what it cost — one paragraph, numbers where you have them); a timestamped timeline taken from the incident channel, not from memory; and contributing causes, derived by asking "why" past the first answer until you reach process or system — "human error" is where analysis starts, never where it ends.

Then two short honesty sections — what went well (detection that worked, a runbook that helped; this is signal too) and what went poorly (where minutes were lost: the alert nobody saw, the dashboard that lied, the escalation that stalled) — and finally action items, each with one owner and one date. No owner means no item.

The five whys technique is simple: keep asking why until the answer is a process or a system. Service went down — why? Deploy shipped a bad config. Why? Config was not validated. Why? Validation only runs on the app repo. Why? Nobody owns config tooling. That last answer is fixable in a way that "be more careful" never is.

Two failure modes to avoid: stopping early, which yields a person to blame instead of a cause to fix, and forcing a single root cause when real incidents are usually several contributing causes lining up. Write down all of them; fix the cheapest ones first.

The graveyard of postmortem programs is the action item list. Items written in a document nobody reopens will not get done, and a team that watches its postmortem items evaporate learns that the whole exercise is theatre.

The fix is to stop treating them as special: action items go into the same queue as all other work, where aging is visible and ownership is tracked like any ticket. Review open postmortem items in the same weekly review as the rest of the queue. When an item is genuinely not worth doing, close it explicitly and say why — silent expiry is what kills trust.

Every P1 and P2 gets a postmortem within a week, while the timeline is fresh and before the details blur. For recurring small incidents, run one on the pattern rather than each occurrence — five instances of the same alert storm are one postmortem, not five.

If tickets, alerts, and the incident channel already live in one system, the timeline mostly writes itself — FlowTux keeps the full trail on the ticket, including what auto-triage did and when, so the postmortem starts from a record instead of an archaeology dig.