Write runbooks from resolved incidents, not from imagination

Every team agrees runbooks are good, schedules a runbook-writing sprint, produces eleven documents for failures someone imagined, and then handles the next real incident from memory anyway. The problem is the direction of writing: runbooks written speculatively document what might break; the queue already knows what does break, and how it got fixed, because someone typed the fix into a ticket thread at 2 a.m.

The working method inverts the sprint: no runbook is written from scratch, and every qualifying incident produces one. The first draft already exists — it is the resolution thread. The job is fifteen minutes of promotion, not an afternoon of authorship.

Not everything deserves a runbook. The filter is recurrence times reasoning: write one when the same failure has appeared twice — the second occurrence is the proof it recurs — or when the fix required non-obvious steps that the next responder would have to rediscover. A one-off with an obvious fix needs neither; a monthly certificate dance absolutely does.

The trigger should be mechanical, attached to closing the ticket: when a P1/P2 closes, or a ticket closes as a duplicate of a previous incident, the closer answers one question — "would the next person benefit from the steps you just took?" Yes spawns a fifteen-minute runbook task, prefilled with the ticket link. Deciding at close time matters because the fix is still in working memory; two weeks later the same runbook costs five times as much to write and loses the details that make it trustworthy.

The resolution thread has the steps; the runbook adds what the thread assumes. Preconditions and access — the dashboards, permissions, and VPN the responder happened to have. Expected results per step — the thread says what was typed, the runbook must say what you should see before continuing, because that is what tells the next responder they are on the happy path. Verification — the check that says the system is healthy, as a command or observation, not a feeling. Rollback — what to do if a step makes things worse. And the escalation boundary: the conditions under which this runbook does not apply and who gets the handoff.

The last two are what separate a runbook from a forum answer. A responder following steps that stop working needs to know how to get back to safe ground and when to stop following, and neither is in the thread, because the person who wrote it knew those things implicitly.

A runbook wrong about one step costs more than no runbook, because the responder trusts it past the point where they would have double-checked their own reasoning. Scheduled annual reviews are theatre; the honest review trigger is use. Every time a runbook is used, the last line of the incident is one question: did every step work as written? A "no" spawns the correction task on the spot, while the discrepancy is known.

Stamp every runbook with owner, last-used date, and source-incident link. A runbook that has not been used in a year and whose source system has since been rearchitected is a candidate for deletion, not preservation — to the next responder, and to any AI retrieving it, a stale runbook is misinformation with authority.

A runbook with explicit steps, expected results, and a verification check is one step from executable. That is the quiet payoff of writing them properly: the well-trodden, reversible, verifiable procedures that make good runbooks are the same four tests that qualify actions for an AI allow-list. The runbook becomes the specification; the AI becomes the executor; the verification section becomes the automated check before the ticket closes.

This is the shift-left ladder in miniature: the fix moves from the specialist’s head, to a document any responder can follow, to an action the system takes itself — with the runbook’s escalation boundary becoming the AI’s handoff condition. FlowTux runs this loop on the queue itself: resolved tickets are retrievable context for the next occurrence, and allow-listed fixes execute with the trail on the ticket. The runbook you promote today is automation you do not have to specify next quarter.