A service can work perfectly in production for months and still fail under a sudden traffic spike.
CPU usage climbs. Memory consumption grows. Request latency increases. Goroutines accumulate. Database connections become exhausted. Eventually, even lightweight endpoints start timing out.
The first instinct is often to add more workers, increase connection pool sizes, or scale horizontally.
Sometimes that helps. But there's a deeper problem: the system is accepting work faster than it can process it, and nothing is controlling the difference.
This is where backpressure comes in.
Backpressure is not simply rate limiting, and it isn't just a worker pool with a fixed number of goroutines. It's a way to make a system respond to its actual processing capacity instead of pretending that capacity is unlimited.
In this article, we'll explore how to implement backpressure in Go, how unbounded concurrency creates cascading failures, and how to design services that remain predictable when demand exceeds capacity.
1. The Problem: Go Makes Unbounded Concurrency Easy
One of Go's strengths is how inexpensive it is to start a goroutine.
Consider a typical HTTP handler:
func (s *Server) Handle(w http.ResponseWriter, r *http.Request) { go s.process(r.Context()) w.WriteHeader(http.StatusAccepted) } It looks efficient. The handler returns immediately, and processing happens in the background.
But what happens when 5,000 requests arrive in a few seconds?
Each request creates another goroutine. If processing depends on a slow database, a third-party API, or a downstream service experiencing problems, those goroutines may remain alive while waiting for work to complete.
Now imagine that each task retains several megabytes of data, holds a database connection, or waits on a network operation.
The consequences compound:
- Memory usage increases.
- More goroutines compete for CPU and scheduler time.
- Downstream connection pools become saturated.
- Requests wait longer.
- Timeouts trigger retries.
- Retries create even more work.
The service can enter a feedback loop where overload causes slower processing, and slower processing causes more overload.
The problem isn't that goroutines are expensive in isolation.
The problem is allowing the number of outstanding operations to grow without a deliberate limit.
2. Understand the Difference Between Concurrency and Capacity
Concurrency controls how much work can be in progress simultaneously. Throughput describes how much work the system completes over time.
They are related, but they aren't interchangeable.
Suppose a downstream service processes 200 requests per second, and each request takes an average of 100 milliseconds.
Using Little's Law:
[
L = \lambda W
]
Where:
- (L) is the average number of requests in the system.
- (\lambda) is the throughput.
- (W) is the average time spent in the system.
At 200 requests per second and 100 milliseconds per request:
[
L = 200 \times 0.1 = 20
]
Approximately 20 concurrent requests are needed to sustain that throughput under those assumptions.
Increasing concurrency to 2,000 doesn't automatically increase throughput to 20,000 requests per second. If the downstream service remains limited to 200 requests per second, the additional concurrency mostly creates more waiting work.
In practice, latency changes as utilization increases, so this is a starting model rather than a capacity guarantee.
A useful principle follows:
Concurrency should be sized around the resources doing the work, not around the number of requests arriving.
3. Start With a Bounded Worker Pool
A worker pool is one of the most straightforward ways to bound concurrency in Go.
Instead of creating one goroutine for every task, create a fixed number of workers and give them a bounded queue.
The following is a production-ready implementation with lifecycle management, cancellation, error reporting, and graceful shutdown:
package pool import ( "context" "errors" "fmt" "log/slog" "sync" "time" ) // Task is a unit of work the pool executes. type Task func(ctx context.Context) error // Result holds the outcome of a single task execution. type Result struct { Err error Duration time.Duration } // Config holds pool configuration. type Config struct { Workers int QueueSize int // OnError is called after every task that returns a non-nil error. // It is called from the worker goroutine; keep it non-blocking. OnError func(err error) } // Pool is a bounded worker pool with graceful shutdown support. type Pool struct { jobs chan Task results chan Result wg sync.WaitGroup cfg Config once sync.Once } // New creates a new Pool and starts its workers. // Call Shutdown to drain in-flight work and stop all workers. func New(cfg Config) (*Pool, error) { if cfg.Workers <= 0 { return nil, errors.New("pool: workers must be positive") } if cfg.QueueSize < 0 { return nil, errors.New("pool: queue size must be non-negative") } p := &Pool{ jobs: make(chan Task, cfg.QueueSize), results: make(chan Result, cfg.QueueSize), cfg: cfg, } for range cfg.Workers { p.wg.Add(1) go p.worker() } return p, nil } func (p *Pool) worker() { defer p.wg.Done() for task := range p.jobs { start := time.Now() err := task(context.Background()) elapsed := time.Since(start) if err != nil && p.cfg.OnError != nil { p.cfg.OnError(err) } // Non-blocking send; drop result if nobody is reading. select { case p.results <- Result{Err: err, Duration: elapsed}: default: } } } // ErrPoolFull is returned by Submit when the queue is at capacity. var ErrPoolFull = errors.New("pool: queue is full") // Submit enqueues a task without blocking. // Returns ErrPoolFull when the queue is at capacity. func (p *Pool) Submit(task Task) error { select { case p.jobs <- task: return nil default: return ErrPoolFull } } // SubmitWait enqueues a task, blocking until space is available or ctx is done. func (p *Pool) SubmitWait(ctx context.Context, task Task) error { select { case p.jobs <- task: return nil case <-ctx.Done(): return fmt.Errorf("pool: submission canceled: %w", ctx.Err()) } } // Results returns a read-only channel of task results. func (p *Pool) Results() <-chan Result { return p.results } // Shutdown closes the job queue and waits for all workers to finish. func (p *Pool) Shutdown() { p.once.Do(func() { close(p.jobs) p.wg.Wait() close(p.results) }) } // --- Usage example --- func ExamplePool() { cfg := Config{ Workers: 10, QueueSize: 100, OnError: func(err error) { slog.Error("task failed", "error", err) }, } pool, err := New(cfg) if err != nil { panic(err) } defer pool.Shutdown() ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second) defer cancel() for i := range 50 { i := i if err := pool.SubmitWait(ctx, func(ctx context.Context) error { slog.Info("processing item", "id", i) time.Sleep(10 * time.Millisecond) return nil }); err != nil { slog.Warn("submission failed", "error", err) break } } } Two important limits are now explicit:
- The number of workers bounds concurrent execution.
- The channel capacity bounds queued tasks.
If the pool has 20 workers and a queue capacity of 100, it can hold at most 120 tasks in progress or waiting — a meaningful and measurable resource boundary. An unbounded queue, by contrast, can continue accepting work until the process runs out of memory.
What happens when the queue fills?
A bounded queue doesn't eliminate overload. It forces the system to decide what overload means.
Possible policies include:
- Block the producer until capacity becomes available.
- Reject the task immediately.
- Wait for a limited amount of time and then reject.
- Drop or coalesce work when the task is safe to discard.
- Persist the work durably and let a separate consumer process it.
Each policy expresses a different contract with the caller.
For an HTTP API, blocking indefinitely is usually a poor default. For a durable job system, immediate rejection might be inappropriate if the job must eventually run.
Backpressure starts by making that choice explicit.
4. A Context-Aware Concurrency Limit
Not every service needs a full worker pool. Sometimes, all you need is a limit on how many expensive operations can execute simultaneously.
A buffered channel can act as a semaphore:
package semaphore import ( "context" "errors" "fmt" ) // ErrLimitReached is returned by TryAcquire when all permits are taken. var ErrLimitReached = errors.New("semaphore: limit reached") // Semaphore limits the number of concurrent operations. type Semaphore struct { permits chan struct{} } // New creates a Semaphore with the given concurrency limit. func New(limit int) (*Semaphore, error) { if limit <= 0 { return nil, fmt.Errorf("semaphore: limit must be positive, got %d", limit) } return &Semaphore{ permits: make(chan struct{}, limit), }, nil } // Acquire blocks until a permit is available or ctx is canceled. func (s *Semaphore) Acquire(ctx context.Context) error { select { case s.permits <- struct{}{}: return nil case <-ctx.Done(): return fmt.Errorf("semaphore: acquire canceled: %w", ctx.Err()) } } // TryAcquire takes a permit without blocking. // Returns ErrLimitReached when all permits are occupied. func (s *Semaphore) TryAcquire() error { select { case s.permits <- struct{}{}: return nil default: return ErrLimitReached } } // Release returns a permit to the pool. // Panics if called more times than Acquire. func (s *Semaphore) Release() { select { case <-s.permits: default: panic("semaphore: Release called without a matching Acquire") } } // Available returns the number of currently free permits. func (s *Semaphore) Available() int { return cap(s.permits) - len(s.permits) } // Active returns the number of permits currently held. func (s *Semaphore) Active() int { return len(s.permits) } Usage in a service, with an explicit acquisition deadline:
func (s *Service) Process(ctx context.Context, input Input) error { // Apply a short acquisition deadline so callers don't wait indefinitely. acquireCtx, cancel := context.WithTimeout(ctx, 500*time.Millisecond) defer cancel() if err := s.sem.Acquire(acquireCtx); err != nil { return fmt.Errorf("service overloaded: %w", err) } defer s.sem.Release() return s.repo.Save(ctx, input) } If the limit is 30, no more than 30 callers can hold the semaphore at the same time. When all permits are occupied, new callers wait until capacity becomes available or their contexts are canceled.
This is better than creating unlimited concurrent operations, but it introduces another question: how long should a request be allowed to wait for capacity?
If requests spend several seconds waiting for a semaphore before the actual operation begins, the service may technically remain within its concurrency limit while still delivering terrible latency.
The caller's deadline should cover the entire operation, including the wait for capacity.
Note: A semaphore limits concurrency, not arrival rate. If a downstream service has a strict requests-per-second quota, you'll need rate limiting in addition.
5. Waiting Is Not Always Backpressure
Consider an API that receives 10,000 requests while its downstream service can handle only 500 concurrent operations.
A semaphore with a capacity of 500 prevents more than 500 operations from running simultaneously.
But if every incoming request waits for a permit, you may still accumulate:
- Thousands of HTTP requests waiting for capacity.
- Occupied server resources.
- Large amounts of retained request data.
- Expired deadlines and requests that no longer matter.
You've bounded execution, but you haven't necessarily bounded the amount of work waiting to execute.
This distinction matters.
A concurrency limit protects active work. A bounded admission policy protects the system from accumulating too much pending work.
For many HTTP services, rejecting excess work is safer than letting the waiting population grow indefinitely.
Let's implement a non-blocking admission limiter with built-in metrics:
package admission import ( "errors" "net/http" "sync/atomic" ) // ErrOverloaded is returned when the service is at capacity. var ErrOverloaded = errors.New("service overloaded") // Limiter enforces a non-blocking concurrency cap. type Limiter struct { slots chan struct{} accepted atomic.Int64 rejected atomic.Int64 } // NewLimiter creates a Limiter with the given concurrency limit. func NewLimiter(limit int) (*Limiter, error) { if limit <= 0 { return nil, errors.New("admission: limit must be positive") } return &Limiter{ slots: make(chan struct{}, limit), }, nil } // TryAcquire takes a slot without blocking. // Returns ErrOverloaded when the limit is reached. func (l *Limiter) TryAcquire() error { select { case l.slots <- struct{}{}: l.accepted.Add(1) return nil default: l.rejected.Add(1) return ErrOverloaded } } // Release returns a slot to the pool. func (l *Limiter) Release() { <-l.slots } // Stats returns current admission counters. func (l *Limiter) Stats() (active, accepted, rejected int64) { return int64(len(l.slots)), l.accepted.Load(), l.rejected.Load() } HTTP handler integration:
func (s *Server) Handle(w http.ResponseWriter, r *http.Request) { if err := s.limiter.TryAcquire(); err != nil { // Hint the client when to retry. Not a guarantee. w.Header().Set("Retry-After", "1") http.Error( w, "service temporarily overloaded", http.StatusServiceUnavailable, ) return } defer s.limiter.Release() if err := s.process(r.Context()); err != nil { http.Error(w, "processing failed", http.StatusInternalServerError) return } w.WriteHeader(http.StatusNoContent) } This version allows at most the configured number of active operations and doesn't build an application-level waiting queue.
Note on status codes: Use
503 Service Unavailablefor temporary capacity exhaustion; prefer429 Too Many Requestswhen enforcing a client-specific quota. A202 Acceptedresponse should only be returned after the system has actually accepted responsibility for the task.
6. Cancellation Must Reach the Actual Work
A request can be canceled for many reasons:
- The client disconnects.
- The request deadline expires.
- An upstream service gives up waiting.
- The system shuts down.
- The operation is no longer useful.
If the underlying work ignores cancellation, the service continues consuming resources for a result nobody needs.
Always propagate the request context to operations that support cancellation:
func (s *OrderService) PlaceOrder(ctx context.Context, order Order) error { // Respect the caller's deadline for permit acquisition. acquireCtx, cancel := context.WithTimeout(ctx, 200*time.Millisecond) defer cancel() if err := s.sem.Acquire(acquireCtx); err != nil { return fmt.Errorf("order service at capacity: %w", err) } defer s.sem.Release() // Database call: ctx cancellation will abort the query. if err := s.saveOrder(ctx, order); err != nil { return fmt.Errorf("save order: %w", err) } // Outbound HTTP: create request with the same context. req, err := http.NewRequestWithContext(ctx, http.MethodPost, "https://fulfillment.internal/orders", nil) if err != nil { return err } resp, err := s.httpClient.Do(req) if err != nil && !errors.Is(err, context.Canceled) { slog.Warn("notify fulfillment failed", "order_id", order.ID, "error", err) } if resp != nil { resp.Body.Close() } return nil } Context cancellation is cooperative. For long-running work, check cancellation at meaningful boundaries:
func processLargeDataset(ctx context.Context, records []Record) error { for i, record := range records { // Check cancellation every 100 records. if i%100 == 0 { Source：https://dev.to/serifcolakel/backpressure-in-go-keeping-services-stable-under-load-22jkRelated posts
Where macOS keeps Contacts, Calendar, Reminders, Safari tabs and Voice Memos, and how to read them from a backup
Build an Incremental ETL Pipeline in Python Using PostgreSQL Watermarks
Your Stripe inbox prevents duplicate updates. What about duplicate emails?
CloudNativePG on k3s: PostgreSQL That Survives Node Failures
The One-Story Resume Test: Building a Flagship Project Section That Still Passes ATS
