SRE Best Practices Checklist
Strong reliability comes from many small habits practiced consistently. This checklist collects the most important habits in one place. Use it for self-review, team audits, and planning improvements for the next quarter.
How to Use the Checklist
Score each item as Done, Partly Done, or Not Done. Pick the three weakest items and turn them into projects. Review the list every quarter. A short, steady cycle of improvement beats a large one-time cleanup.
[Score the List] → [Pick Top 3 Gaps] → [Plan and Fix] → [Re-Score Next Quarter]
↑ │
└─────────────────────────────────────────────────────────┘
Objectives and Error Budgets
- Every user-facing service has at least one SLI and one SLO.
- SLOs reflect what users care about, such as speed and success rate.
- Targets stay below 100 percent on purpose.
- An error budget policy explains what happens when the budget runs out.
- Product and engineering leaders have read and agreed to the policy.
Monitoring and Alerting
- Dashboards show latency, traffic, errors, and saturation for each service.
- Alerts fire on user-facing symptoms, with a person ready to act.
- Every alert links to a runbook.
- The team reviews noisy alerts monthly and removes or tunes them.
- Synthetic checks cover the most important user journeys.
- Logs use a structured format with correlation identifiers.
Incident Handling
- A severity scale exists, and everyone knows it.
- Incident roles are defined: commander, operations, communications, scribe.
- Status updates follow a fixed rhythm.
- Every serious incident gets a blameless postmortem within a week.
- Action items have owners and dates, and the team tracks them to completion.
Change and Release Safety
- Every change passes automated tests in a pipeline.
- Releases roll out in stages, such as canary before full.
- Rollback takes minutes and the team rehearses it.
- Feature flags protect risky features, and old flags get removed.
- Infrastructure changes live in version-controlled code.
Capacity and Performance
- Load tests confirm the expected peak with a safety margin.
- Forecasts cover the next six to twelve months.
- Autoscaling has sensible minimum and maximum limits.
- The team watches saturation and acts before it reaches 80 percent.
Data Protection and Recovery
- Backups follow the 3-2-1 rule.
- Restore drills run at least every quarter.
- RPO and RTO exist for each critical service.
- Recovery runbooks exist and the team has practiced them.
Automation and Toil
- The team measures time spent on repetitive manual work.
- Toil stays below half of the working week.
- Repeated tasks become scripts, tools, and fully automatic jobs.
People and Culture
- On-call rotations include enough people to keep the load fair.
- New members shadow before taking a full rotation.
- Engineers share lessons through short write-ups and talks.
- Leaders praise reliability work, not only feature launches.
Sample Scorecard
| Area | Score (0 to 5) | Top Gap | Owner |
|---|---|---|---|
| Objectives | 4 | Budget policy lacks leadership sign-off | SRE Lead |
| Monitoring | 3 | Too many noisy alerts | On-Call Team |
| Incidents | 3 | Slow postmortems | Incident Manager |
| Releases | 4 | Rollback untested | Release Team |
| Recovery | 2 | No restore drill this year | Data Team |
Review Rhythm
| Frequency | Activity |
|---|---|
| Daily | Scan alerts and open incidents |
| Weekly | Review pages, toil, and error budget burn |
| Monthly | Tune alerts and review dashboards |
| Quarterly | Run restore drills, refresh targets, re-score this checklist |
| Yearly | Review architecture risks and capacity plans |
Common Mistakes
- Trying to fix every gap at once.
- Treating the checklist as a one-time audit.
- Scoring generously to look good.
Key Takeaways
Reliability grows through steady habits across objectives, monitoring, incidents, releases, capacity, recovery, automation, and culture. Score honestly, fix the biggest gaps, and repeat.
