What a load balancer does
A load balancer is a network service placed in front of backend servers. Clients connect to the load balancer rather than choosing an application server directly. The load balancer then selects an eligible backend and forwards the work.
This indirection creates one stable entry point while allowing the backend fleet to grow, shrink, or lose unhealthy members. It does not create backend capacity by itself. It makes existing capacity reachable and controllable.
A service that accepts incoming connections or requests and forwards each one to an eligible backend.
The load balancer endpoint that accepts traffic on a protocol and port, such as HTTPS on port 443.
A server instance that can receive forwarded work from the load balancer.
A named or logical group of backends eligible for a particular class of traffic.
A condition and action that decide which backend pool receives traffic.
A periodic probe used to decide whether a backend should remain eligible for new traffic.
Continued service after a component fails, using healthy capacity that remains available.
A connection currently open on a backend. Duration determines how long it contributes to current work.
A relative value that controls how much traffic one backend receives compared with others.
A common proxy term for the backend service or pool to which traffic is forwarded.
The forwarding path
A valid path has three stages: traffic arrives, the load balancer makes a routing decision, and a healthy backend handles the work. A server drawn on the canvas but not connected to the load balancer contributes no usable capacity.
A load balancer distributes traffic among instances that already exist. An autoscaler changes the number of instances. Many systems use both, but they solve different problems.
Choose what the load balancer needs to understand
Layer 4 and Layer 7 refer to layers in the network model. A Layer 4 load balancer routes transport connections using addresses and ports. A Layer 7 load balancer understands an application protocol such as HTTP and can inspect request details.
Routes using network and transport information. It is fast and protocol-agnostic, but cannot choose a backend from an HTTP path such as /media.
Understands application protocols such as HTTP. It can route by hostname, URL path, method, or headers and apply request-level policies.
| Question | Layer 4 | Layer 7 |
|---|---|---|
| What can it inspect? | IP, port, TCP or UDP connection | HTTP host, path, method, headers |
Can it route /api and /media differently? | No | Yes |
| Typical use | General TCP services, very high throughput | HTTP APIs, host and path routing, request policies |
Choose the signal that represents backend work
A routing algorithm is the rule used to select a backend. The correct algorithm depends on whether servers are equal, whether request cost varies, and whether current connection count is a useful signal.
Equal request count. Best for similar servers and similar request cost.
Configured traffic ratio. Best when backend capacities differ predictably.
Current open connection count. Best when connection durations differ.
Cycles through eligible backends. It assumes each request costs roughly the same and each backend has roughly the same capacity.
Cycles according to configured weights. It is deterministic and appropriate when backend capacities differ in known proportions.
Chooses the backend with the fewest active connections. It reacts to long-lived requests that remain open while short requests finish.
Route in proportion to backend capacity
Round Robin sends equal request counts. That is unsafe for a mixed fleet because a Small server and a Large server receive the same traffic despite having different limits. Weighted Round Robin lets the routing share mirror capacity.
Weights 22, 43, 80, and 80 sum to 225. The load balancer normalizes them into traffic shares. They do not need to sum to 100.
Stop routing before spare capacity can help
A failed backend must be removed from the eligible pool. Health checks perform this control-loop function by probing each backend and changing its routing state when failures cross a threshold.
Health detection and failover capacity are separate
Health checks prevent requests from reaching a failed server. They do not create replacement capacity. After removal, the remaining servers must still handle the specified failure traffic without crossing their utilization limit.
What makes a useful health check
- Representative
Probe a path that proves the service can perform meaningful work, not only that a process exists.
- Cheap
The probe runs frequently, so it must not create significant application or database load.
- Fast enough
Detection must be quick enough for the recovery objective without reacting to one harmless transient error.
- Independent
Avoid a probe that declares every backend unhealthy because one optional shared dependency is degraded.
Request count is not current work
Round Robin equalizes arrivals, not active work. When all requests finish in similar time, that distinction may not matter. When some requests remain open far longer, one backend can accumulate more active work than another even after receiving the same number of requests.
Many requests complete quickly, so each connection disappears almost immediately.
Only a few exports arrive each second, but they remain open long enough to dominate active connection count.
Least Connections uses open connection count as a live approximation of backend work. It does not replace capacity planning. The fleet still needs enough RPS capacity, and connection count is only useful when it correlates with resource usage.
Route each workload to dedicated capacity
A Layer 7 routing rule can inspect the URL path and choose a backend pool. This allows one domain to serve multiple workloads while keeping their compute capacity independent.
Pool isolation
A backend is dedicated only when it belongs to one pool. Connecting the same application server to two route pools creates shared capacity and allows one route to affect the other. Isolation requires separate membership, separate sizing, and separate utilization checks.
The section exercise represents each path rule as a separate Layer 7 load-balancer node because canvas nodes do not carry editable route labels. In production, one Layer 7 proxy commonly owns several path rules that select several backend pools.
A repeatable interview method
Separate topology, algorithm, health, and capacity decisions. A correct answer should explain what traffic is being routed, which signal drives selection, which backends are eligible, and what happens when one fails.
- 1
Identify the traffic unit
State whether the input is requests, connections, jobs, bytes, or sessions. Keep units attached to every calculation.
- 2
Choose Layer 4 or Layer 7
Use Layer 7 when routing depends on HTTP host, path, method, or headers. Use Layer 4 when transport-level forwarding is sufficient.
- 3
Define backend pools
List which servers may handle each traffic class. Decide whether capacity is shared or isolated.
- 4
Choose the routing algorithm
Use Round Robin for a uniform fleet, Weighted Round Robin for known capacity differences, and Least Connections for varying connection duration.
- 5
Calculate normal capacity
Verify aggregate throughput and every backend's utilization under the selected traffic split.
- 6
Define health and eligibility
Specify how failed backends are detected and removed from rotation.
- 7
Calculate failure capacity
Remove the failed backend from the math and verify that the remaining fleet serves the required failover load.
- 8
Check isolation and cost
Verify that no backend is unintentionally shared and that every load balancer and server fits the monthly budget.
Do not trust aggregate capacity
A bad traffic split can overload one backend while total fleet capacity looks sufficient.
Do not ignore duration
Equal arrival counts do not imply equal active work when requests have different lifetimes.
Do not fake isolation
A backend shared by two pools is still a shared failure and capacity domain.
Apply each traffic-routing concept
Each lab isolates one routing decision. Build mode asks you to construct the system from a specification. Fix mode gives you a concrete routing failure and asks for the smallest effective repair.
Uneven Fleet
Use Weighted Round Robin to distribute traffic across backends with different capacities.
The Dead Server Still Gets Traffic
Enable health checks and prove the remaining fleet can carry failover traffic.
Long Requests Break Round Robin
Use Least Connections when active connection duration matters more than arrival count.
You are ready when you can justify the routing layer, algorithm, health behavior, backend membership, failure capacity, and cost without relying on the diagram alone.