System Design Core Metrics

Every system design starts with numbers. Metrics describe how fast, how busy, and how dependable a system must be. Three core metrics guide most design choices: latency, throughput, and availability.

Latency

Latency is the time one request needs to reach the server and bring back a response. Engineers measure it in milliseconds. A shopper who taps "Add to Cart" waits for latency to finish before the screen updates.

Low latency feels instant. High latency feels sluggish and pushes users away. Long network distance, slow database queries, and heavy calculations all raise latency.

Layman example

Latency equals the waiting time between placing an order at a restaurant and receiving the first dish.

Throughput

Throughput counts how many requests a system completes in a fixed period. Teams write it as requests per second (RPS) or transactions per second (TPS). A ticket site that serves 5,000 requests every second has a throughput of 5,000 RPS.

Layman example

Throughput equals the number of guests the restaurant serves in one hour. A kitchen with four cooks serves more guests than a kitchen with one cook.

Highway Diagram

A highway shows how latency and throughput differ. Car speed controls the travel time. The number of lanes controls how many cars pass each hour.

Start ==================================> Finish
        Travel time for ONE car = LATENCY

1 lane  : [Car][Car]                  = 100 cars per hour
4 lanes : [Car][Car][Car][Car] x 4    = 400 cars per hour
                                        (THROUGHPUT)

A system can have low latency and low throughput at the same time. A single fast cashier serves each customer quickly, yet the line still grows during a rush.

Availability

Availability shows the percentage of time a system works correctly. Engineers describe it with "nines". More nines mean less downtime.

Each extra nine costs more money and effort. A school website can live with 99.9%. A hospital system or payment service aims much higher.

Percentiles

Averages hide slow requests. Suppose 99 requests finish in 100 ms and one request needs 10 seconds. The average looks fine, yet one user suffers.

Percentiles fix this problem. The p50 value shows the typical request. The p95 value shows the time that 95 out of 100 requests beat. The p99 value reveals the slowest 1 percent. Strong teams track p95 and p99 to protect every user.

Other Useful Metrics

• Error rate: the share of requests that fail.
• Response size: the amount of data sent back to the user.
• Resource usage: the CPU, memory, and disk a server consumes.
• Durability: the chance that stored data survives failures.

Trade-offs Between Metrics

Metrics compete with each other. Adding checks to protect data raises latency. Packing more requests into one batch raises throughput and delays each single request. Extra servers raise availability and raise cost.

Good designers pick targets before building. A chat app may demand latency under 200 ms. A video upload service may accept slower responses and demand high durability. Clear targets turn vague wishes into measurable goals.

Quick Summary

• Latency measures speed for one request.
• Throughput measures volume over time.
• Availability measures uptime.
• Percentiles expose hidden slowness.