load balancer

A load balancer is a component that sits in front of a pool of servers and spreads incoming requests across them, so no single machine carries all the traffic and the service keeps answering when one of them fails. It’s what makes running several copies of an application worthwhile, which is why one shows up wherever a web app or a microservice runs on more than one instance.

Say a Flask app runs on three application servers behind a load balancer, and each server reports its own hostname at a /whoami endpoint. Four requests to the same public URL come back from three different machines:

Shell 
$ for _ in 1 2 3 4; do curl -s https://example.com/whoami; done
served by web-01
served by web-02
served by web-03
served by web-01

The client used one address and never learned that three machines answered. That indirection is the whole point, and everything else a load balancer does builds on it.

How It Shows Up in Practice

A Python developer usually meets a load balancer as the thing their app sits behind rather than something they write. Gunicorn or Uvicorn serves the app on a private port, and nginx, HAProxy, or a managed service such as AWS Elastic Load Balancing takes the public traffic and forwards it on.

The balancer also runs health checks, watching each backend server and pulling failing ones out of rotation until they recover. Some implementations probe a dedicated endpoint on a schedule, and others simply count failed requests and stop sending traffic for a while.

That behavior is what lets a rolling deployment swap instances a few at a time without dropping requests, and what a canary release uses to point a small slice of traffic at a new version.

Two constraints follow for the application itself. Consecutive requests from one user can land on different instances, so session state belongs in a shared store rather than in process memory, which is the stateless-processes rule from the twelve-factor app. Logs for a single user journey are also scattered across machines, so observability tooling has to stitch them back together.

Common Variations

Load balancers are usually described by how deep into the request they look, using the layer numbers of the OSI networking model. A layer 4 balancer routes on TCP connection data alone, such as source and destination address and port, which is fast and works for any protocol.

A layer 7 balancer reads the HTTP request, so it can route by hostname, path, or header, sending /api/ to one pool and everything else to another. AWS splits these into its Network Load Balancer and its Application Load Balancer.

The rule that picks which backend gets the next request is the balancing algorithm. Round robin, the default in both nginx and an Application Load Balancer target group, cycles through the pool in order. Least connections hands the request to whichever backend has the fewest requests still in progress, which holds up better when response times vary widely.

Hashing on the client IP address sends a given client back to the same backend each time, an arrangement called sticky sessions that the twelve-factor guidance advises against relying on. Most implementations also accept a per-server weight, so a larger machine takes a bigger share.

Switch between those four rules below and watch where a stream of requests lands in a pool where web-03 answers about three times slower than its peers, with each color standing for a different client IP address:

Interactive diagram — enable JavaScript to view.
Securely Deploy a Django App With Gunicorn, Nginx, & HTTPS

Tutorial

Securely Deploy a Django App With Gunicorn, Nginx, & HTTPS

Ready to take your Django app beyond development? Learn how to securely deploy your Django web app in production over HTTPS with Gunicorn and Nginx. Along the way, you'll explore how HTTP headers can fortify your app's security.

intermediate django

For additional information on related topics, take a look at the following resources:

Have a question about this? Mentor AI can show you examples, compare related terms, and point you to tutorials.

  • Ask Mentor AI
  • Pick a question and Mentor AI takes it from here.
By Martin Breuss • Updated Oct. 22, 2026
Software Engineering Glossary Share Feedback