Why Rate-Limit at the Load Balancer

Application-level rate limiting catches abuse after the request has already consumed backend resources -- TCP connection, TLS handshake, request parsing. HAProxy stick-tables let you shed load at the edge before any of that happens.

The Docker Compose Lab 

# docker-compose.yml version: '3.8' services: haproxy: image: haproxy:2.9 ports: - "80:80" - "8404:8404" # stats volumes: - ./haproxy.cfg:/usr/local/etc/haproxy/haproxy.cfg:ro depends_on: - backend1 - backend2 - backend3 backend1: build: ./backend environment: - BACKEND_ID=1 backend2: build: ./backend environment: - BACKEND_ID=2 backend3: build: ./backend environment: - BACKEND_ID=3

The backend is a minimal Go HTTP server:

// backend/main.go package main import ( "fmt" "net/http" "os" "time" ) func main() { id := os.Getenv("BACKEND_ID") http.HandleFunc("/api/data", func(w http.ResponseWriter, r *http.Request) { // Simulate 20ms of work time.Sleep(20 * time.Millisecond) w.Header().Set("Content-Type", "application/json") fmt.Fprintf(w, `{"backend":"%s","status":"ok"}`, id) }) http.HandleFunc("/healthz", func(w http.ResponseWriter, r *http.Request) { w.WriteHeader(200) fmt.Fprint(w, "ok") }) http.ListenAndServe(":8080", nil) }

HAProxy Config with Stick-Table Rate Limiting 

# haproxy.cfg global log stdout format raw local0 defaults mode http log global timeout connect 5s timeout client 30s timeout server 30s option httplog frontend http_front bind *:80 # Stick-table: track request rate per source IP # 100k entries, 30s TTL, track HTTP request rate over 10s window stick-table type ip size 100k expire 30s store http_req_rate(10s) # Track the client http-request track-sc0 src # Deny if more than 50 requests in 10s (5 req/s sustained) http-request deny deny_status 429 if { sc_http_req_rate(0) gt 50 } # Add rate limit headers for observability http-response set-header X-RateLimit-Remaining %[sc0_http_req_rate(10s),sub(50)] if { sc_http_req_rate(0) gt 0 } default_backend app_servers backend app_servers balance leastconn option httpchk GET /healthz http-check expect status 200 server backend1 backend1:8080 check inter 3s fall 3 rise 2 server backend2 backend2:8080 check inter 3s fall 3 rise 2 server backend3 backend3:8080 check inter 3s fall 3 rise 2 frontend stats bind *:8404 stats enable stats uri /stats stats refresh 5s

The key section is the stick-table. type ip size 100k creates an in-memory table keyed by source IP. store http_req_rate(10s) tracks a sliding window of request rates. The http-request deny line checks if the rate exceeds 50 requests per 10 seconds and returns 429 before the request ever reaches a backend.

Benchmarking with wrk

Start the lab and run wrk at increasing concurrency:

docker compose up -d # Baseline: 10 concurrent connections wrk -t4 -c10 -d30s http://localhost/api/data # Expected: all 200s, ~20ms avg latency # Over limit: 100 concurrent connections wrk -t4 -c100 -d30s http://localhost/api/data # Expected: mix of 200s and 429s # Heavy load: 500 concurrent connections wrk -t4 -c500 -d30s -L http://localhost/api/data # Expected: mostly 429s, but p99 latency stays low # because HAProxy rejects before forwarding

The -L flag prints the full latency distribution. Compare p50, p99, and p999 across the three runs. Without stick-table rate limiting, the 500-connection test would show p99 latency climbing as backends queue requests. With it, p99 stays flat because excess requests never reach backends.

Checking the Stick-Table Live

HAProxy exposes stick-table contents via the stats socket or the stats page at :8404/stats. You can also query it from the command line:

# Show all tracked IPs and their request rates docker compose exec haproxy \ socat stdio /var/run/haproxy/admin.sock <<< "show table http_front"

This outputs each tracked IP with its current request rate, which is useful for debugging why a specific client is getting denied.

Key Differences from Application-Level Rate Limiting

  1. No backend resources consumed: HAProxy denies at the connection level before proxying
  2. Shared state by default: one HAProxy instance sees all traffic, no Redis needed for single-LB deployments
  3. No Retry-After header: stick-tables do not natively calculate when the window resets, so clients get a bare 429. Add application-level rate limiting behind this for richer error responses.
  4. Coarser granularity: stick-tables work on IP, not on user identity or API key. Use them as the first layer, with app-level per-user limits behind them.

This two-tier approach matches what Day 19 recommended: coarse limits at the edge, fine-grained limits in the application.