DevOps Load Balancing and Reverse Proxy

A load balancer spreads incoming traffic across several servers. A reverse proxy sits in front of servers and forwards user requests to them. Many tools, such as Nginx, HAProxy, and cloud load balancers, perform both jobs together.

The Restaurant Host Example

Picture a busy restaurant with five waiters. A host stands at the door and sends each new guest group to a waiter with free time. Guests never choose a waiter themselves, and no waiter handles all the tables. The host is the load balancer, and the waiters are the servers.

The Basic Layout

                         +--> [ Server A ]
 Users --> Load Balancer |--> [ Server B ]
                         +--> [ Server C ]

Users see one address. The load balancer hides how many servers stand behind it.

Why Teams Use Load Balancers

• Capacity: Many small servers handle more traffic than one big server.
• Availability: The balancer stops sending traffic to a failed server.
• Maintenance: Engineers remove one server for updates while the others serve users.
• Growth: Teams add servers when traffic rises.

Balancing Methods

Health Checks

A health check is a small test the balancer runs against every server. The balancer might request the page /health every five seconds. A server that fails several checks leaves the pool. The server returns after it passes again. Health checks give the balancer its self-healing ability.

Reverse Proxy Jobs

A reverse proxy does more than forward traffic.

• SSL termination: The proxy handles HTTPS encryption, so backend servers stay simple.
• Caching: The proxy stores common responses and serves them quickly.
• Compression: The proxy shrinks responses to save bandwidth.
• Routing: The proxy sends /api to one group of servers and /blog to another.
• Protection: The proxy hides internal server addresses and limits abusive traffic.

Forward Proxy Compared

 Forward proxy:   Users  --> [ Proxy ] --> Internet
                  (proxy represents the users)

 Reverse proxy:   Internet --> [ Proxy ] --> Servers
                  (proxy represents the servers)

Example: Nginx Configuration

upstream backend {
    least_conn;
    server 10.0.0.11:8080;
    server 10.0.0.12:8080;
    server 10.0.0.13:8080;
}

server {
    listen 80;
    server_name shop.example.com;

    location / {
        proxy_pass http://backend;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
    }
}

The upstream block lists three backend servers and picks the least busy one. The proxy_pass line forwards each request to that group. The headers pass the original host name and visitor address to the backend.

Layers of Balancing

• Layer 4: The balancer reads only network details such as IP and port. This method runs fast.
• Layer 7: The balancer reads web details such as URL paths and headers. This method allows smarter routing.

Session Persistence

Some applications remember a user on one server, such as items in a shopping cart. A sticky session sends the same user to the same server through a cookie or an IP hash. Sticky sessions break when that server fails and the cart disappears. A better design stores sessions in a shared store such as Redis, so any server can serve any user.

Rate Limiting and Web Protection

A proxy can slow down visitors who send too many requests. Rate limiting protects servers from overload and from abusive scripts.

limit_req_zone $binary_remote_addr zone=perip:10m rate=10r/s;

server {
    location /api/ {
        limit_req zone=perip burst=20;
        proxy_pass http://backend;
    }
}

The setting allows ten requests per second for each visitor address with a short burst. A web application firewall (WAF) adds rules that block known attack patterns, such as SQL injection.

Global Load Balancing

A global load balancer spreads users across data centers in different regions. DNS-based routing answers each visitor with the address of the nearest healthy region.

                    +--> Region Asia   [ Load Balancer ] --> servers
 User --> DNS -------+--> Region Europe [ Load Balancer ] --> servers
 (nearest healthy)   +--> Region US     [ Load Balancer ] --> servers

Users get faster pages, and a regional outage sends traffic to the remaining regions.

A Reliable Balancer Pair

A single balancer is a single point of failure. Teams run two balancers that share a floating IP address. The active balancer holds the address. The standby balancer watches it with a protocol such as VRRP, which tools like Keepalived implement. A failure moves the address to the standby within seconds.

 Floating IP 203.0.113.10
      |
 [ Active LB ]  <--heartbeat-->  [ Standby LB ]

Cloud Load Balancers

Managed balancers scale on their own and need no server patching. Self-managed tools such as Nginx and HAProxy give finer control and run anywhere.

Key Points

• Load balancers share traffic and remove failed servers.
• Reverse proxies add security, caching, and routing.
• Round robin and least connections cover most needs.
• Redundant balancers keep the entry point reliable.