Overview
Nobody reads a long document in the middle of an outage, so runbooks need to be short, precise and rehearsed. We prepare a separate runbook for the 10 failures you are most likely to face, covering the symptoms, the checks, the actions and when to escalate. You also receive a severity scale, an on-call schedule and a blameless template for post-incident reviews. We finish by running a practice incident with your team, so gaps show up in rehearsal rather than during a real outage.
What you receive
- Runbooks for the 10 failure scenarios you are most likely to hit
- Defined severity levels, each paired with its expected response
- On-call schedule built inside the paging and scheduling tool you pick
- Contact names, phone numbers and decision points for each escalation route
- Pre-written messages for customers and for your own staff during outages
- A no-blame template for reviewing incidents, plus guidance on running reviews
- One tabletop practice exercise facilitated with your team
How it runs
-
Step - 01
Identify
We review past incidents and your architecture to find probable failures.
-
Step - 02
Draft
Runbooks are written together with the people on your team who handle outages.
-
Step - 03
Configure
We configure the rota, escalation chain and alert routes in your tools.
-
Step - 04
Rehearse
A tabletop exercise tests the runbooks and we close the gaps found.
Timeline: 10 working days, counted from the day we receive what we need from you.
Is it a fit?
A good match for
- Teams where one person ends up handling every incident
- Companies bound by uptime commitments yet lacking a formal on-call rota
- Young startups that discovered during a major outage they had no process
What we need from you
- Logins for whatever alerting and monitoring systems you use
- Sessions with whichever engineers currently fix things when they break
- Any notes on past incidents, even informal ones
Why it helps
- Incident response no longer hinges on one person picking up
- Customer messaging during outages is prepared ahead of time
- Reviews after an outage lead to improvements, not finger-pointing
Questions
Not in this service. It sets up the process for your own team. If you want our engineers on call, that is covered by our monthly managed support retainer.
Probably not. Runbooks are only useful when alerts trigger them, so we would suggest setting up monitoring and alerting first, then adding these runbooks once alerts are reaching people.
After any incident where a runbook was used, plus a review every quarter. We include a simple process your team can follow to keep them current.