SRE: establishing SLOs as contractual capability boundaries

Research Question

How do Site Reliability Engineering (SRE) practices establish what a system can safely do, expressed as a contractual boundary rather than an observed average: specifically, how are Service Level Objectives (SLOs) set, and what evidence justifies the chosen threshold?

Findings

Executive Summary

SRE establishes SLOs as contractual capability boundaries through a combined evidence-gathering, three-party stakeholder-negotiation, and error-budget-enforcement process, not through the threshold number alone. User-impact evidence is the specified primary input for threshold selection, with historical telemetry as the practical first-pass proxy when user-impact data is unavailable. The error budget converts the SLO threshold into a measurable resource consumed by failures, and exhaustion of that resource triggers prescribed organisational responses that give the threshold its contractual force. SLOs adopted without an enforced error budget policy become passive reporting metrics, the "SLO without teeth" failure mode the SRE Workbook explicitly identifies as a documented adoption risk.

Key Findings

1. SRE defines the SLO threshold as the reliability level below which users are likely to start complaining or stop using the service, grounding the contractual boundary in user-impact evidence rather than in technical capability assertion alone.

2. The SRE SLO-setting process requires three-party stakeholder agreement across product manager, development team, and SRE team as a procedural safeguard ensuring the threshold reflects both user needs and production achievability before it becomes operative.

3. The error budget (100% minus the SLO target) converts the threshold into a quantified resource: when it is exhausted over a four-week window, all releases other than highest-priority or security fixes are halted until the service returns within its SLO.

4. The Google SRE Workbook explicitly advises against basing SLO thresholds solely on current observed performance because this can commit a service to unnecessarily strict targets; user-impact evidence is the specified primary input, with telemetry as the practical first-pass proxy.

5. Google SRE frames the SLO target as both a minimum and a maximum: consistently exceeding the SLO wastes engineering capacity, establishing the threshold as a specific agreed capability range rather than an aspirational upper bound.

6. The SRE model distinguishes the contractual SLO boundary from the observed operational average through the error budget measurement loop: the SLO is the target, the Service Level Indicator (SLI) measurement is the observed value, and the error budget tracks the accumulated difference with mandated organisational responses.

7. SLOs adopted without an enforced error budget policy become passive reporting metrics (the "SLO without teeth" failure mode) because compliance becomes another key performance indicator rather than a decision-making tool with prescribed consequences.

8. Evernote set its initial SLO at 99.95% through negotiation with customer support and product teams supplemented by available uptime telemetry, then iterated as evidence accumulated, illustrating that at least one practitioner implementation began with negotiation rather than rigorous user-impact studies.

9. The SRE book documents the Chubby planned-outage case as empirical evidence that the absence of an explicit SLO threshold allows users to develop availability expectations that exceed what the service has committed to, creating fragile dependencies on unguaranteed reliability levels.

10. Google SRE applies an economic constraint to SLO threshold selection: the marginal cost of each additional nine of availability must be justified against marginal revenue or user-value gain, providing a quantitative ceiling on justifiable threshold stringency.

11. The error budget policy specifies escalation to the CTO for disputes about error budget calculation or required actions, demonstrating that the contractual force of the SLO boundary is backed by an explicit organisational authority structure rather than by engineering convention alone.

Assumptions

• Assumption: Google SRE documentation is representative of established SRE practice across the industry, not only within Google. Justification: The SRE book and SRE Workbook are the canonical primary references cited by organisations adopting SRE practices; the Evernote and Home Depot case studies in the SRE Workbook provide cross-organisation corroboration from organisations of different sizes and sectors. [source: sre.google

• Assumption: The Hidalgo (2020) "Implementing Service Level Objectives" book provides guidance consistent with Google SRE sources on evidence-based threshold selection. Justification: Hidalgo was a co-author on the SRE Workbook chapter on implementing SLOs; the O'Reilly book is described as extending the same methodology with statistical analysis tools. [source: www.oreilly.com

Analysis

The SRE model provides a specific answer to how a threshold becomes a contractual boundary: through the combination of evidence-based threshold selection and an enforced error budget policy; neither component alone is sufficient. A well-evidenced threshold without an enforced error budget policy produces an "SLO without teeth," and an enforced error budget policy with an inadequately evidenced threshold produces enforcement of an arbitrary number, which can result in heroic efforts to meet an overly aggressive target or a degraded product if the target is too lax.

The error budget loop is the structural mechanism that distinguishes a contractual boundary from an observed average. An observed average is computed from past measurements and carries no enforcement consequences; an SLO threshold carries them through the error budget policy, making the distinction structural rather than merely semantic: the same reliability number can be a passive average or an enforced boundary depending on whether the error budget policy exists and is acted upon.

The three-party stakeholder agreement requirement ensures that user-impact evidence reaches the threshold-setting process. The product manager's agreement criterion is explicitly that the threshold must be "good enough for users." Without this criterion, a threshold could be set based on technical achievability or administrative convenience rather than on user requirements.

The tension between the advice to avoid basing SLOs on current performance and the practical starting point of current telemetry is resolved through an iterative refinement process: start with an achievable threshold, observe whether it correlates with user satisfaction, and tighten iteratively. Organisational commitment to acting on the error budget policy at each iteration is a necessary precondition for this refinement process to produce an accurate, user-grounded threshold over time.

The economic constraint framework for threshold selection provides a quantitative ceiling on SLO stringency: a threshold set above what the business can economically justify defending is structurally aspirational rather than contractual, because the organisation lacks the incentive to enforce it consistently.

Risks, Gaps, and Uncertainties

• No controlled before-after studies comparing SLO threshold-setting methods (evidence-based vs. negotiated vs. telemetry-only) and their subsequent reliability, user satisfaction, or engineering efficiency outcomes are cited in the Google SRE sources. Effectiveness claims rest on documented practice and case studies rather than controlled measurement.

• The SRE model assumes organisations are willing and able to enforce the error budget policy. [assumption; justification: The SRE Workbook presents the error budget policy as the intended mechanism but does not report empirical data on adoption rates; source: sre.google The Workbook acknowledges that SLOs remain passive reporting metrics without this enforcement but provides no empirical data on the prevalence of the "SLO without teeth" failure mode in practice.

• The Evernote case study is a single vendor account from a consumer application; it may not represent SLO-setting practice in regulated, safety-critical, or infrastructure-service contexts.

• The economic framework for availability target selection is presented with one illustrative example and no empirical validation of the cost function shape across service types.

Open Questions

• What is the empirical distribution of SLO-setting methods across industry, and do evidence-based methods produce measurably better outcomes than negotiated or telemetry-only approaches? (Potential backlog item.)

• In practice, how often do organisations adopt the SLO threshold without implementing an enforced error budget policy, and what measurable reliability outcomes result from the "SLO without teeth" state?

• How should SLO thresholds be set for services with no prior production history, where neither telemetry nor historical user-impact data is available?

sources

• [ ] Beyer et al. (2016) Site Reliability Engineering: How Google Runs Production Systems: primary Google SRE reference: SLO definition, error budgets, and threshold selection
• [ ] Beyer et al. (2018) The Site Reliability Workbook: Practical Ways to Implement SRE: practical SLO setting guidance and worked examples
• [ ] Hidalgo (2020) Implementing Service Level Objectives: practitioner guide to evidence-based SLO threshold selection
• [ ] DORA State of DevOps Reports: empirical data on SLO adoption and associated reliability outcomes