Most organisations that suffer a serious incident produce a lessons-learned document. Far fewer change anything. The document gets written because governance requires it; the changes do not land because nobody built the machinery to make them land. This runbook is about that machinery.
It applies to any incident big enough to have had an incident lead: a ransomware event, a BEC with money moved, a serious outage with a security dimension.
Two reviews, not one
The single biggest improvement to most review processes is splitting the review in two.
- The hot debrief: within 48 hours, thirty minutes. The responders, still tired, while memory is raw. No slides, no report. Three questions on a whiteboard: what actually happened, what made the response harder than it should have been, what nearly went very wrong. The scribe captures everything verbatim. The details that matter most, the workaround someone improvised at 2am, the contact number that was wrong, the eight hours lost waiting for a decision, evaporate within a week. The hot debrief exists to catch them before they do.
- The full review: two to four weeks later, ninety minutes. Far enough out that the timeline is complete, close enough that people still care. This is where causes get analysed and fixes get decided. Invite the responders, the incident lead, the business owners affected, and whoever owns the budget that fixes will need. That last invitation is the one that usually gets forgotten, and it is why fixes stall.
Before the full review: build the timeline
The review meeting is only as good as the timeline it works from. Someone, usually the incident lead, assembles it beforehand from the incident log, chat exports, ticket history and alert timestamps. Every entry gets a time, a fact and a source.
Mark four moments explicitly, because the intervals between them are where the learning lives:
- When the compromise actually began, best current estimate.
- When it became detectable, meaning evidence existed in some log or alert.
- When it was detected by a human who understood something was wrong.
- When response actions actually began.
The gap between detectable and detected is your monitoring finding. The gap between detected and responding is your process finding. Most reviews only discuss what happened after response began, which is precisely the part that matters least for prevention.
Circulate the timeline a few days ahead with one instruction: correct it, do not defend it.
Running the room
- State the blame position and mean it. The review examines systems, processes and decisions in the context they were made, not the people who made them. The question is never “why did you click it” but “why did clicking it give the attacker the domain”. If leadership cannot hold that line, the review will still happen, but it will be theatre: everyone protecting themselves, nothing true said, nothing learned.
- Walk the timeline, stopping at the branch points. At each decision, ask what information the decider had at that moment, what the options looked like then, and what would have made the better path easier. Hindsight makes every decision look obvious; the useful question is what the decision looked like from inside.
- Ask what went right, and why, seriously. Somewhere in the incident, someone did something that stopped it being worse: an unusually paranoid backup habit, a fast decision, a good relationship with the bank. These are controls you did not know you had. Name them, then make them official so they survive that person’s resignation.
- Separate causes into three piles as you go. Technical (the VPN had no MFA), process (nobody knew who could authorise disconnection), and resourcing (one person held all the knowledge and was on leave). The piles get different fixes, different owners and different budgets, and collapsing them into one list is how resourcing problems get relabelled as technical debt and quietly buried.
- Cap the actions. Every finding is real, but twenty actions produce the same result as zero. Force-rank and take the top five to eight. A smaller list that closes beats a comprehensive list that does not. Unfunded findings go on the risk register by name, not into a footnote.
The report: two pages, written for two audiences
Page one is for leadership: what happened in plain English, business impact stated plainly, what is changing, what it costs, and what risk remains and is being accepted. That last sentence is the one boards most need and least often get.
Page two is for practitioners: the timeline, the four marked moments and their intervals, indicators, and the technical detail the next responder will want. If the incident holds a lesson that would help peers, this page is the seed of an anonymised write-up worth sharing with the community.
Keep the incident log and evidence exports with the report. Eighteen months later, insurers, auditors or a regulator may want the record, and reconstructed memory will not do.
Follow-through: the part that is the actual point
This is where nearly everyone fails, so build the machinery deliberately:
- Every action has one named owner and a date. Not a team, a person. Actions assigned to “IT” belong to nobody.
- The action list lives in the system where work already happens, the ticket queue or project board, not in the PDF. Reports do not nag; backlogs do.
- Review the actions monthly in an existing meeting until the list closes. No new meeting; a standing agenda item in one that already runs.
- Set a six-month verification, where someone checks not whether actions were marked done, but whether they took: is the MFA actually on every account, does the new payment verification step actually happen when finance is busy.
- Feed the results into the next tabletop. Exercise the fixed weakness. If the fix holds under exercise pressure, it is real. This closes the loop between responding, learning and preparing, which is the whole game.
One-page checklist
Hot debrief within 48 hours, full review at two to four weeks. Build the timeline first; mark began, detectable, detected, responding, and study the gaps. Blameless in substance, not just in name. Walk the branch points asking what deciders knew at the time. Capture what went right and institutionalise it. Sort causes into technical, process and resourcing. Cap actions at five to eight with named owners and dates. Two-page report: leadership page and practitioner page. Actions into the live backlog, monthly check in an existing meeting, six-month verification, and exercise the fix in the next tabletop.
Practitioner Runbooks are working documents, not standards. Spotted something that does not match how it works in your environment? Email hello@incidentresponse.com and we will improve it, credited to you.