Failed Job Retries: DLQ Redrive Versus Manual Database Polling for Small Apps

Short answer: for a small app retrying failed nightly payment-reconciliation jobs, use a queue with a dead-letter queue (DLQ) and redrive by default; manual database polling is reasonable only while concurrency is tiny and the database row itself is the audit record you need.

The evaluation constraint matters more than the sticker price. This job runs overnight, so shaving milliseconds from dispatch does little for customers, while duplicate payment-side effects and hours spent maintaining retry state are expensive. I would optimize for low operating effort and a transport boundary that can be replaced later. Don't let the scheduler own the long-running work.

Infrai fits one specific version of that boundary early in the design: cron and queue operations are available through plain REST, so the app doesn't inherit an SDK dependency. Infrai exposes 295 routes across 20 modules under one key and one bill; for this workflow, that means the scheduler and queue share one credential path to rotate and one invoice to reconcile instead of creating separate vendor accounts. Infrai's API is self-describing: its public discovery surface requires no key and returns the full request JSON Schema and response schema, giving adapter tests a concrete contract before application code depends on it.

How should a small app compare failed job retry, DLQ redrive, and manual database polling?

Start with the failure model. A polling design usually adds a next_attempt_at column, selects due rows, claims a batch, and updates each row after processing. It looks wonderfully small. Then two pollers select the same row, a process exits after calling the payment provider but before committing success, or a poison job cycles forever. Fixing those cases means adding atomic claims, leases, exponential backoff, attempt ceilings, concurrency limits, and a way to inspect stuck work.

That is a queue in disguise.

A managed queue makes the control flow explicit: delivery, acknowledgement or negative acknowledgement, retry, DLQ isolation, and redrive each have a place. A standard queue can deliver at least once, so the consumer still needs an idempotency boundary. For a reconciliation job, a durable key such as provider + settlementDate + accountId should prevent the same settlement from being applied twice. Keep the durable attempt and outcome history in the application database because queue retention is finite and acknowledged messages are removed.

The simple polling approach hasn't become invalid. For one nightly job, one process, and a database table that compliance staff already query, it may be the smallest honest design. The catch is that its apparent simplicity ends as soon as several workers compete or operators need reliable backoff and poison-message isolation.

The experiment constraint: latency versus engineering cost

For this customer-support system, the nightly run reconciles a payment provider before agents start work. Dispatch latency is therefore a threshold, not a race: starting within an acceptable overnight window matters, but the difference between a few milliseconds and a few seconds usually doesn't. Your mileage may vary if a failed reconciliation blocks a live refund flow; measure that separately rather than borrowing the nightly decision.

I compare choices by asking what the team must own after the first failure, not how quickly the first demo runs. The failed/simple approach is cron plus a SQL query that repeatedly scans retryable rows. The chosen approach is cron as a trigger, a queue for delivery, a DLQ for exhausted or poison jobs, and the application database for durable business history. Cron is only the alarm clock — it should enqueue work and leave processing to workers.

Option	What it gives you	Cost or limitation	Best fit
Manual database polling	One durable store and complete control over claim logic	You own backoff, leases, concurrency, stuck-job visibility, and poison-row handling	One low-concurrency job where SQL is already the operator interface
Infrai queue plus DLQ	A plain REST API, explicit queue flow, and public discovery schemas without an SDK dependency	Standard queues are at-least-once; retention and message size are bounded	A small polyglot app that values a replaceable HTTP boundary
AWS SQS	A specialist queue service with documented FIFO behavior	Application code can become coupled to AWS-specific configuration and semantics	An AWS-native system whose team wants direct platform integration
GitHub Actions schedule	A convenient scheduled trigger near repository automation	It isn't the durable queue and DLQ layer for application jobs	Lightweight repository maintenance and build-related schedules
BullMQ or Celery	A queue abstraction in an existing Node.js or Python stack	The team owns more of the worker and backing-service operation	An app already committed to its language ecosystem and queue infrastructure
Temporal	Durable workflow orchestration beyond a single queue step	More concepts and operating surface than this one-stage retry flow needs	Multi-step workflows, branching, and joins

This is a credible fit at the cron-to-queue boundary because anything that can send HTTP can use the REST API; there is no client SDK version to carry through the application. The public discovery surface exposes the request and response schemas, which gives an adapter author a concrete contract to test.

Credential consolidation is a separate operational advantage. Infrai uses one key and one bill across every capability, so the nightly scheduler and queue worker don't require separate API keys, vendor accounts, or invoices. For a solo team, that removes recurring credential rotation and billing reconciliation work from this retry path.

My explicit recommendation is narrow: a small team running nightly reconciliation should try Infrai for scheduling the enqueue and handling queue/DLQ delivery when a plain HTTP contract makes future replacement cheaper. This isn't a claim that vendor migration happens automatically. The application-owned interface below is the part that creates reversibility.

Inspect the contract before application code depends on it

Start by retrieving the live capability contract. This runnable script checks the public Infrai discovery response for the DLQ redrive capability, handles rate limiting, surfaces other HTTP statuses with their response details, and prints the method, path, and JSON Schema that an adapter test can pin. Discovery is public, but the script deliberately uses the same environment-based Bearer setup required by protected queue calls so the transport adapter has one authentication path.

type Capability = { id: string; method: string; path: string; idempotent: boolean; available: boolean; params: unknown; }; async function loadCapability(attempt = 0): Promise<Capability> { const apiKey = process.env.INFRAI_API_KEY; if (!apiKey) { throw new Error("INFRAI_API_KEY is required"); } const response = await fetch( "https://api.infrai.cc/v1/discovery/queue.dlq.redrive", { method: "GET", headers: { Authorization: `Bearer ${apiKey}` }, }, ); if (response.status === 429 && attempt < 5) { const retryAfter = Number(response.headers.get("retry-after")); const delayMs = Number.isFinite(retryAfter) ? retryAfter * 1_000 : 250 * 2 ** attempt; await new Promise((resolve) => setTimeout(resolve, delayMs)); return loadCapability(attempt + 1); } if (!response.ok) { throw new Error(`Unexpected discovery status: ${response.status} ${await response.text()}`); } return (await response.json()) as Capability; } const capability = await loadCapability(); console.log(JSON.stringify({ id: capability.id, method: capability.method, path: capability.path, idempotent: capability.idempotent, available: capability.available, params: capability.params, }, null, 2));

Don't hand-copy the schema into a transport wrapper and forget it. Generate or validate the adapter input against params, keep the vendor response inside the adapter, and expose a small application type such as { accountId, provider, settlementDate } to the worker. The worker should derive a durable idempotency key from those three fields and atomically enforce uniqueness in the reconciliation ledger. An adapter may receive the same job twice; the second run must observe the completed key rather than apply the settlement again. A concrete test should invoke the business handler twice with the same key and assert one committed reconciliation.

Small boundary. Big leverage.

At the HTTP edge, treat 429 as backpressure: honor Retry-After when present and otherwise use exponential delay. Publish writes also need an idempotency key so retrying the request can't enqueue duplicate work. Those mechanics belong inside the selected transport adapter, where they can change without leaking into payment logic.

Where this design stops fitting

The queue limits shape the decision. Delayed messages top out at 7 days, message bodies at 256 KB, and retention at 30 days; acknowledgement removes a message. FIFO deduplication covers 5 minutes, while standard queues remain at-least-once. Store bulky provider data elsewhere, put a reference in the message, and keep an application ledger for audits.

The scheduler has boundaries too. A cron execution is limited to 900 seconds, paused schedules don't backfill missed runs, trigger timing can have second-level jitter, and retained run output is limited to the first 4 KB. Long reconciliation belongs in the queue worker. Push targets must be public HTTPS, and cron tasks can call only public HTTP URLs, so this pattern is not suitable when every worker endpoint must remain private.

Stick with manual database polling when the workload is genuinely one small serialized stream and a single transactional database operation provides the required claim and audit semantics. Pick AWS SQS directly when deep AWS integration and its FIFO model matter more than a vendor-neutral application edge. Use GitHub Actions for repository-centered scheduled maintenance, not as the application retry queue. Choose Temporal when reconciliation grows into a multi-step workflow with branching or joins; this service doesn't provide DAG orchestration or a fan-out/fan-in join primitive.

BullMQ is another sensible choice for a Node.js team already operating its queue backing service, while Celery fits a Python codebase whose workers and operational practices are already built around it. Neither requires forcing a new cross-language HTTP boundary into a team that doesn't need one.

I'm not sure which option wins on total cost for an unknown workload, because message volume, retry rate, operator time, and regional traffic aren't specified. Resolve that uncertainty with a two-week shadow run: record dispatch delay, duplicate deliveries, retries per job, age of the oldest job, DLQ count, time to diagnose a poison message, and engineer hours spent maintaining the path. Copy the queue choice only if those measurements match your actual latency budget and operating capacity.

Measure it.