SRE Service Mesh
Modern applications split into many small services that call each other all day. Each call can fail, slow down, or expose data. A service mesh moves common safety features out of application code and into a shared network layer.
What Is a Service Mesh?
A service mesh is a layer of small proxies that handle communication between services. Picture a company mailroom. Employees hand envelopes to the mailroom, and staff handle addresses, tracking, retries, and locked bags. Employees concentrate on their work while the mailroom handles delivery. Proxies play the mailroom role for service calls.
How It Works
Each service instance runs next to a proxy, often called a sidecar. All traffic passes through the proxy. A control plane sends policies to every proxy.
[Control Plane: policies, certificates]
/ \
[Service A]↔[Proxy] ══ encrypted ══ [Proxy]↔[Service B]
The proxies form the data plane. Service code speaks plain requests, and proxies add the extra features.
Features Teams Gain
| Feature | Benefit |
|---|---|
| Retries and timeouts | Consistent handling of slow or failed calls |
| Circuit breaking | Stops calls to a failing service so it can recover |
| Mutual TLS | Encrypts traffic and proves the identity of both sides |
| Traffic splitting | Sends a chosen share of calls to a new version |
| Observability | Automatic metrics and traces for every call |
| Access policies | Defines which service may call which |
Circuit Breaking Explained
An electrical circuit breaker cuts power when current becomes dangerous. A software circuit breaker stops sending requests to a service that keeps failing. After a pause, it lets a few test requests through. Success closes the circuit again.
[Closed: normal] → many failures → [Open: block calls]
↑ ↓ wait period
└──── test calls succeed ← [Half-Open: try a few]
Traffic Splitting Example
A team releases version 2 of a recommendation service. A mesh rule sends 90 percent of calls to version 1 and 10 percent to version 2. Dashboards show the same error rate for both. The team moves to 50 percent and later to 100 percent. A single rule edit reverses the change.
[Frontend] → [Mesh Rule] ─ 90% → [Recommendations v1]
└ 10% → [Recommendations v2]
Danger: Retry Storms
Retries help with brief glitches and hurt during real overload. Ten services that each retry three times can multiply traffic by a thousand. Limit retries, add small random delays, and set a retry budget, such as no more than 10 percent extra traffic.
When a Mesh Fits
- Dozens of services with many teams
- Strict security rules that require encryption between services
- Need for uniform telemetry across languages
- Frequent canary releases
When a Mesh May Not Fit
- A handful of services that a library can handle
- Small teams without time to operate extra infrastructure
- Very strict latency limits, since each proxy adds a small delay
Costs to Plan For
A mesh consumes CPU and memory on every pod. The control plane needs upgrades, monitoring, and expert attention. A broken mesh can break every service at once, so teams roll out changes gradually and keep a fast way to bypass the mesh.
Popular Options
Istio and Linkerd rank among the best-known open-source meshes. Cloud providers offer managed versions. Evaluate each against team skill, feature needs, and operating cost.
Common Mistakes
- Adopting a mesh to follow a trend, with no clear problem to solve.
- Stacking retries at the mesh and in the code.
- Skipping resource planning for proxies.
- Rolling out to every service in a single week.
Key Takeaways
A service mesh adds security, resilience, and visibility to service-to-service calls without changing application code. Adopt it for real needs, control retries, and operate it with care.