Microservices Load Balancing
A popular service runs as many identical copies, called instances. Customers should not choose among them. A load balancer sits in front, accepts every request, and hands each one to a suitable instance. Work spreads evenly, failures stay hidden, and capacity grows by adding instances.
Layman Explanation
A supermarket opens several checkout lanes. A staff member at the entrance points each shopper to a lane with a short line. Shoppers get served faster, and a closed lane causes no trouble. The staff member is the load balancer, and the lanes are the instances.
Diagram: Basic Flow
+--> Instance 1 (10.0.0.11)
Clients --> Load |
Balancer --+--> Instance 2 (10.0.0.12)
|
+--> Instance 3 (10.0.0.13)
Common Algorithms
Round Robin
Requests rotate through the instances in order: 1, 2, 3, 1, 2, 3. Simple and fair when every request costs about the same.
Weighted Round Robin
Stronger machines receive a larger share. An instance with weight 3 gets three requests for every one that a weight 1 instance gets.
Least Connections
The balancer picks the instance with the fewest active requests. Long-running requests no longer pile up on one unlucky machine.
IP Hash
The balancer computes a value from the client address and always sends that client to the same instance. Useful when an instance keeps temporary data about the client.
Random with Two Choices
The balancer picks two instances at random and sends the request to the less busy one. Large systems like this method for its low overhead and good spread.
Where Balancing Happens
Server-Side Load Balancing
A dedicated component, such as a hardware appliance, cloud balancer, or software proxy, sits between callers and instances. Callers know one address only.
Client-Side Load Balancing
The calling service asks the service registry for the list of instances and picks one itself. No middle hop exists, and each caller carries the selection logic.
Order service --asks--> Registry: "Where is Payment?" Registry --replies--> [ 10.0.0.21, 10.0.0.22, 10.0.0.23 ] Order service picks 10.0.0.22 and calls it directly
Layer 4 vs Layer 7
| Type | Sees | Strength |
|---|---|---|
| Layer 4 | IP address and port | Very fast and simple |
| Layer 7 | URL, headers, cookies | Smart routing by path or user type |
A Layer 7 balancer can send /images to one group of instances and /orders to another.
Working with Health Checks
The balancer calls each instance's readiness endpoint. Unready instances leave the rotation automatically. Recovered instances rejoin. Customers keep getting answers throughout.
Sticky Sessions
Sticky sessions pin a user to one instance. They help when the instance stores session data in memory, though they weaken even distribution and complicate failures. Storing session data in a shared cache lets any instance serve any request and removes the need for stickiness.
Good Practices
- Keep services stateless so any instance can answer.
- Run at least two balancer nodes to avoid a single point of failure.
- Set connection timeouts and a sensible retry limit.
- Watch instance-level metrics to spot uneven load.
Global Load Balancing
Large products run in several regions. A global balancer uses DNS or anycast routing to send each visitor to the closest healthy region. A user in Mumbai reaches a nearby data center, and a regional outage shifts traffic to the next closest region automatically.
Key Takeaways
Load balancers spread traffic, hide failures, and make scaling invisible to callers. Match the algorithm to your traffic pattern and pair the balancer with health checks.