Mental Model: Error Budget¶
Category: Operational Reasoning Origin: Google Site Reliability Engineering, formalized in the SRE Book (2016); attributed to Ben Treynor Sloss and the original Google SRE team One-liner: Your SLO defines how much unreliability you are allowed — spend that budget intentionally on risk, and when it runs out, stop taking risk.
The Model¶
The error budget model converts a reliability target into a concrete, spendable resource. If your service has a 99.9% availability SLO, that means the service is allowed to be unavailable for 0.1% of the time. Over a 30-day month, 0.1% equals 43.8 minutes. That 43.8 minutes is your error budget. It is not a punishment threshold — it is a risk allocation. You can spend it on planned downtime, risky deployments, infrastructure migrations, or experimental features. Or you can save it. But you only get 43.8 minutes. When it is gone, further risk-taking is prohibited until the next measurement window opens.
This framing resolves a recurring organizational conflict: product teams want to ship fast, and SRE or operations teams want to maintain reliability. Without error budgets, these goals fight for policy space in every release planning meeting. With error budgets, the conflict is replaced by a shared currency. If reliability is high and budget is unspent, the product team has room to take risks — the operations team has no grounds to block releases. If reliability is low and budget is depleted, operations has grounds to freeze releases based on a number, not a feeling. Both sides agreed to the SLO upfront, so the freeze is a contract fulfillment, not a political power play.
The error budget also shifts the incentive structure for SRE teams. Without it, SRE is incentivized toward maximal caution — reliability is always good, so more is always better. But reliability above the SLO is waste: you are spending engineering resources on headroom no one agreed they needed. An SRE team managing to an error budget can say, "We have 30 minutes of budget remaining this month. Let's burn some of it on the infrastructure upgrade we've been deferring." This is rational risk management, not recklessness.
The measurement window matters. A 30-day rolling window is common, but some teams use 28-day windows (aligned to four weeks) or 90-day windows (for less frequent, larger releases). Shorter windows create more responsive feedback — you feel the impact of a bad week quickly. Longer windows allow more smoothing — a bad day does not immediately freeze your release pipeline. The choice depends on your release cadence and organizational tolerance for volatility.
Error budgets require good SLIs (Service Level Indicators) to be meaningful. If you cannot measure your availability accurately, you cannot know your budget status. SLI → SLO → error budget is the dependency chain. Bad measurement produces false budget readings, which produce bad risk decisions. The quality of your observability infrastructure is therefore a prerequisite for this model, not an optional enhancement.
Visual¶
SLO MATH
────────────────────────────────────────────────────────────
SLO | Error Rate | Monthly Budget | Daily Budget
─────────────|───────────-|----------------|─────────────
99.0% | 1.0% | 7h 18m | 14m 24s
99.5% | 0.5% | 3h 39m | 7m 12s
99.9% | 0.1% | 43m 48s | 1m 26s
99.95% | 0.05% | 21m 54s | 43s
99.99% | 0.01% | 4m 22s | 8.6s
99.999% | 0.001% | 26s | 0.86s
ERROR BUDGET CONSUMPTION OVER A MONTH
────────────────────────────────────────────────────────────
Budget: 43m 48s (99.9% SLO, 30-day window)
Week 1: Normal operation Budget remaining: 43m 48s
Week 2: Risky deploy → 8m outage Budget remaining: 35m 48s
Week 3: Infrastructure migration → 20m Budget remaining: 15m 48s
Week 4: Canary rollout goes bad → 18m Budget remaining: -2m 12s ← DEPLETED
↓
RELEASE FREEZE
(until window resets)
DECISION FRAMEWORK
────────────────────────────────────────────────────────────
Budget Status
│
┌─────────┼──────────┐
│ │ │
>50% 10-50% <10%
(full) (caution) (depleted)
│ │ │
Risky Standard Freeze
deploys releases new risk;
OK; OK; reliability
consider monitor work only
stretch closely
goals
When to Reach for This¶
- When product and engineering teams are in conflict over release frequency versus reliability — error budgets transform the debate from politics to math
- When deciding whether a proposed change is too risky to ship this week — check the budget against the estimated blast radius
- When an SRE team is being asked to approve every deploy manually — error budgets enable self-service: if budget is available, the team can deploy without SRE review
- When setting SLOs for a new service — the conversation about what error budget you can afford forces alignment between reliability requirements and cost of engineering effort
- When a service has been consistently at 99.99% against a 99.9% SLO — use the budget conversation to justify reducing headroom and redirecting the effort elsewhere
When NOT to Use This¶
- Do not apply error budgets to situations where any downtime is genuinely unacceptable (emergency services, patient monitoring, financial transaction settlement) — these require hard availability targets and the budget model's "spend it" framing is dangerous
- Do not treat error budget exhaustion as a punitive event; if the budget ran out because of incidents outside the team's control, the SLO or window may need adjustment
- Do not use this model without reliable SLI measurement in place — a budget based on bad data produces bad decisions; fix observability before enforcing budget policy
- Avoid gaming the budget by measuring availability in ways that exclude real customer impact (e.g., measuring server uptime, not request success rate) — the SLI must reflect what the user actually experiences
Applied Examples¶
Example 1: Risk-based release decision for a database migration¶
Your service has a 99.9% monthly availability SLO. It is the 20th of the month. You have used 12 minutes of your 43m 48s budget — you are in good shape. A database schema migration is ready. The estimated maintenance window is 15 minutes. Should you ship it?
Without the error budget model: this becomes a judgment call, and judgment calls become political. The cautious voice wins by default.
With the error budget model: 43m 48s budget - 12m used = 31m 48s remaining. The migration needs 15 minutes. If it goes perfectly, you have 16m 48s left for the rest of the month. If it needs a rollback (estimated 10 additional minutes), you would be at 6m 48s — tight, but not depleted. Decision: ship it, with rollback plan rehearsed, and communicate the planned maintenance window in advance so the downtime is expected.
Example 2: Freezing releases after a cascade of incidents¶
A new microservice had a rough November: two incidents totaling 52 minutes of downtime against a 43m 48s monthly budget. Budget is depleted by November 15th. The product team wants to ship three new features before end of month.
Error budget model response: no new features ship for the remainder of November. The team spends the remaining two weeks on reliability work — fixing the alert that fired 8 minutes after the outage started, adding a circuit breaker between this service and its dependency, and writing the missing runbook. December begins with a full budget and a more reliable system. The product team accepted this contract when they signed off on the 99.9% SLO, so this is not an ops team veto — it is a contractual enforcement of a shared agreement.
The Junior vs Senior Gap¶
| Junior | Senior |
|---|---|
| Thinks of reliability as binary: either up or down | Thinks of reliability as a rate — and a spendable rate at that |
| Treats every outage as equally catastrophic | Knows that a 2-minute outage in a month with 40 minutes of budget remaining is fine |
| Argues for higher SLOs because "more reliable is always better" | Questions whether a 99.99% SLO is justified given the engineering cost and marginal customer benefit |
| Cannot answer "is it safe to ship this week?" with data | Checks error budget status before any release with estimated downtime risk |
| Sets SLOs without specifying what SLI is being measured | Knows that SLO is meaningless without a precise SLI definition tied to user-visible behavior |
| Treats error budget exhaustion as a failure to be hidden | Reports budget depletion to stakeholders as a scheduled event with a clear reliability improvement plan |
Connections¶
- Complements: OODA Loop (error budget status is an input to the Orient phase during incident response — knowing you are at 80% budget consumed changes what risks you take in the Act phase)
- Complements: Toil vs Automation ROI (when error budget is consistently depleted, the ROI calculation for automation investments shifts dramatically — the cost of toil now includes lost shipping velocity, making the automation investment easier to justify)
- Tensions: Feature velocity culture (product teams optimizing for shipping speed will feel the budget freeze as a constraint; managing this tension requires SLO buy-in from product leadership before the first depletion event, not after)
- Topic Packs: sre, observability, alerting-rules