Preventing Runaway LLM Bills: Building a Resilient AI Worker Queue with Redis, BullMQ, and Model Fallbacks

Building autonomous agents or AI-driven background tasks is exciting—until a bad retry loop or unexpected 429 rate-limit cascades into a $2,000 API bill overnight.

In production, naive API calls with standard try/catch fail for three predictable reasons:

Cascading Rate Limits: Hitting OpenAI's TPM (Tokens Per Minute) limit causes concurrent workers to retry simultaneously, intensifying rate limits.
Zombie Jobs & Runaway Budgets: Recursive agent loops consume tokens indefinitely without a global kill switch.
Provider Downtime: When an upstream provider experiences elevated latencies or 503 outages, the entire application grinds to a halt.

In this tutorial, we will construct a production-ready AI Worker Queue in TypeScript using BullMQ and Redis, featuring:

Pre-execution budget kill-switches (blocking jobs before tokens are spent).
Multi-model fallback routing (automatically routing from OpenAI to Anthropic or Groq on 429/500 errors).
Atomic token consumption accounting.

🏗️ Architecture Overview

flowchart TD A[Incoming Task Request] --> B[BullMQ Job Queue] B --> C{Daily Budget Pre-Check} C -- Exceeds Spend Ceiling --> D[Abort / Kill Switch Triggered] C -- Within Budget Limit --> E[Resilient Multi-Provider Router] E -- Primary 429/Timeout --> F[Fallback Provider: Groq / Anthropic] E -- Primary Success --> G[Worker Output Delivered] F --> G G --> H[Atomic Redis Token Increment]

🛑 Step 1: Redis-Backed Budget Kill Switch

Before an expensive prompt is dispatched to an LLM provider, we pre-calculate estimated tokens and verify against an atomic Redis counter:

export class BudgetManager { private redis: Redis; private keyPrefix = "ai_budget:daily:"; public async canExecute(estimatedTokens: number): Promise<{ allowed: boolean; currentUsage: number; reason?: string }> { const today = new Date().toISOString().split("T")[0]; const key = `${this.keyPrefix}${today}`; const current = parseInt((await this.redis.get(key)) || "0", 10); if (current + estimatedTokens > MAX_DAILY_TOKEN_BUDGET) { return { allowed: false, currentUsage: current, reason: `Daily token limit breached: ${current} + ${estimatedTokens} > ${MAX_DAILY_TOKEN_BUDGET}`, }; } return { allowed: true, currentUsage: current }; } public async recordUsage(tokens: number): Promise<number> { const today = new Date().toISOString().split("T")[0]; const key = `${this.keyPrefix}${today}`; const total = await this.redis.incrby(key, tokens); await this.redis.expire(key, 172800); // 48h TTL return total; } }

🔄 Step 2: Resilient Model Fallback Router

If your primary provider returns a 429 Too Many Requests or 500 Internal Error, you shouldn't crash the queue. Instead, fall back to secondary high-availability endpoints seamlessly:

export class ResilientModelRouter { constructor(private providers: IModelProvider[]) {} public async executeWithFailover(req: ModelRequest): Promise<ModelResponse> { for (const provider of this.providers) { try { return await provider.execute(req); } catch (err: any) { console.warn(`[Failover] Provider ${provider.name} failed: ${err.message}. Trying next fallback...`); } } throw new Error(`All fallback providers exhausted for request [${req.id}]`); } }

⚡ Step 3: BullMQ Worker Orchestration

We bind the budget check and failover router directly into a BullMQ worker process with jittered exponential retries:

const worker = new Worker<ModelRequest, ModelResponse>( "resilient-ai-jobs", async (job: Job<ModelRequest>) => { const estimatedTokens = Math.ceil((job.data.prompt.length / 4)) + 500; // 1. Budget kill switch check const budgetStatus = await budgetManager.canExecute(estimatedTokens); if (!budgetStatus.allowed) { throw new Error(`[BUDGET_EXCEEDED] ${budgetStatus.reason}`); } // 2. Execute with automatic provider failover const response = await router.executeWithFailover(job.data); // 3. Atomically account for actual token consumption await budgetManager.recordUsage(response.tokensUsed); return response; }, { connection: redisConnection, concurrency: 5 } );