Google Cloud Pub/Sub and AWS SQS Dead Letter Recovery — 4 Edtech Failure Classes

Short answer: Google Cloud Pub/Sub dead-letter topics, AWS SQS DLQs, and Azure Service Bus dead-letter subqueues can all handle retries; choose the one whose recovery evidence and replay controls fit the cleanup job.

Choose a dead-letter design by how an operator will recover a failed cleanup, not by which queue has the shorter setup. For an education platform, the useful question is: can someone identify the bad enrollment record, replay only safe work, and prove that the web request was never part of the recovery path?

The request is already gone.

No magic.

A practical default is a scheduled worker that writes an idempotent job envelope, retries transient failures with bounded delay, and moves poison messages to a separately monitored dead-letter destination. The queue product matters after that. Its redrive rules, message metadata, retention behavior, and permissions decide how much detective work remains at 02:00.

The comparison below uses three common patterns: Google Cloud Pub/Sub dead-letter topics, Amazon SQS dead-letter queues, and Azure Service Bus dead-letter subqueues. They solve a similar failure problem, but they expose different operational handles.

Recovery question	Pub/Sub dead-letter topic	SQS DLQ	Service Bus dead-letter subqueue
What is isolated?	An undeliverable message is forwarded to a dead-letter topic after repeated delivery attempts	A message is moved after its receive count exceeds the redrive threshold	A message is placed in the entity's dead-letter subqueue
Main retry control	Subscription retry policy and maximum delivery attempts	Visibility timeout and `maxReceiveCount`	Lock duration, delivery count, and dead-letter reason data
Operator's first task	Check forwarding permissions and inspect the wrapped message	Inspect receive count, retention, and redrive policy	Read the dead-letter reason and description before deciding to resubmit
Best fit	Topic-based fanout with subscription-level recovery	Queue-shaped work with explicit redrive ownership	Workflows where failure reason metadata is part of triage

The matrix is a starting point, not a ranking. The right choice is the one whose recovery evidence matches the failure you expect.

Should Google Cloud Pub/Sub or AWS SQS own dead-letter recovery?

Consider a nightly cleanup in an edtech system. It removes expired temporary exports and marks orphaned upload records. A web request starts the run, writes one message per account shard, and returns. The worker may run for hours after the request has disappeared.

That separation changes the failure contract. A timeout at the learning portal is not proof that cleanup failed. A worker crash is not permission to run the same database mutation twice. A poison message should become inspectable work, not an invisible retry loop.

I model each message with an operation identifier, a scope, and a reason for the cleanup. The identifier is stable across retries. The consumer records it with the mutation, so a replay can answer “already applied?” before changing data again. This is more important than the name of the dead-letter feature.

type CleanupJob = { operationId: string; accountId: string; shard: number; requestedAt: string; schemaVersion: 1; }; type FailureClass = | "transient_dependency" | "invalid_scope" | "permission" | "unknown"; function shouldRetry(failure: FailureClass): boolean { return failure === "transient_dependency"; }

The queue should carry enough context to find the job. It should not carry a second, competing source of truth for the account's state. Keep the payload boring. Config bloat is a liability during recovery.

Which differences affect the operator, not the brochure?

Pub/Sub uses a dead-letter topic attached to a subscription. Google documents a maximum delivery-attempt setting in the range of 5 to 100, and the subscription can also use a retry policy with minimum and maximum backoff. Forwarding requires the relevant service agent permissions. That last detail is easy to miss: a valid retry policy can still produce a confusing recovery path if the service identity cannot publish to the dead-letter topic or acknowledge the original message.

The forwarded message also needs deliberate inspection. Pub/Sub documents that the forwarded message contains the original message and attributes that help identify the source subscription. Your replay worker should treat that envelope as an input format, not assume the dead-letter payload is byte-for-byte the original event.

SQS makes the boundary explicit with a redrive policy. maxReceiveCount controls how many receives are allowed before SQS moves the message to the dead-letter queue. Visibility timeout is a separate control: it determines when an un-deleted message becomes available again. Mixing those two numbers is a common operational mistake. A long visibility timeout can delay a retry; a low receive threshold can send a temporarily slow job to the DLQ before its dependency recovers.

AWS also documents retention behavior that affects recovery windows. For a standard queue, the message's original enqueue timestamp is retained when it moves to a DLQ, so the DLQ retention period must be chosen with that lifecycle in mind. FIFO queues have different timestamp behavior. This is not trivia if an education provider investigates a failed term-end cleanup days later.

Service Bus uses a dead-letter subqueue associated with the queue or subscription. It records dead-letter reason and description properties, and its delivery count and lock behavior are separate parts of the retry design. That makes it useful when a human needs to distinguish an expired lock from an application-level rejection, provided the consumer writes meaningful descriptions and does not flatten every exception into “failed.”

These are different operational shapes. A topic-based system makes fanout and subscription-specific recovery natural. A queue redrive policy is direct for one work stream. A dead-letter subqueue keeps failure context close to the original entity. None eliminates the need for an idempotent consumer.

Each option has a boundary. Pub/Sub is a poor fit if the team cannot operate subscription-level permissions and wrapped-message inspection. SQS is a poor fit if the workflow needs fanout but the team has no clear owner for each queue's redrive policy. Service Bus is a poor fit if its lock and dead-letter metadata will be discarded by a generic adapter. Those are trade-offs, not feature defects.

How should the cleanup worker classify a failure?

Dead-lettering every exception is lazy and expensive in engineering time. Retrying every exception is worse. The worker needs a small classification table that the on-call person can understand without opening the application source.

Failure	Retry action	Dead-letter action
Database connection reset	Retry with bounded backoff	Escalate only after the retry budget
Account identifier fails validation	Do not retry	Preserve payload and validation detail
Permission denied for one tenant	Do not blindly retry	Route to an access owner with the tenant scope
Worker loses its lease	Let the message become visible again	Avoid a second mutation by using the operation identifier

The exact number of attempts belongs to the dependency's recovery time and the cleanup's urgency. I would rather document “six attempts over 25 minutes, then operator review” than paste a large attempt count into infrastructure and call it resilience. The number is a policy. It needs an owner.

Start small. A four-message failure drill tells you more than another page of queue configuration: break the database connection, send an invalid account, revoke a tenant permission, and stop a worker while it holds a lease. Record which signal the operator sees first and which replay guard prevents a duplicate mutation.

The poison-message path should expose at least these fields in logs and metrics: operation identifier, account or tenant identifier, shard, first-seen time, delivery or receive count when available, failure class, and last error code. Do not put student data into a dead-letter payload merely because it makes a dashboard convenient.

What should a replay command guarantee?

A replay is a production mutation with a smaller blast radius. It deserves a dry-run mode, an allowlist of operation identifiers, and a limit on how many messages can be released in one invocation. The command should report why each message was selected and where the result was written.

type ReplayRequest = { operationIds: string[]; dryRun: boolean; maxMessages: number; }; type ReplayResult = { operationId: string; action: "released" | "skipped" | "already_applied"; detail: string; }; async function replay( request: ReplayRequest, load: (id: string) => Promise<CleanupJob | undefined>, apply: (job: CleanupJob) => Promise<void>, ): Promise<ReplayResult[]> { const selected = request.operationIds.slice(0, request.maxMessages); const results: ReplayResult[] = []; for (const operationId of selected) { const job = await load(operationId); if (!job) { results.push({ operationId, action: "skipped", detail: "not found" }); continue; } if (request.dryRun) { results.push({ operationId, action: "released", detail: "dry run" }); continue; } await apply(job); results.push({ operationId, action: "released", detail: "submitted" }); } return results; }

The example deliberately leaves queue-specific release calls behind an interface. That keeps the replay policy testable and prevents a provider SDK from becoming the business logic. In real code, I would also make apply consult the idempotency store before the mutation and write the result transactionally with the state change where the database supports it.

When is the runner-up the better fit?

A Pub/Sub design is a reasonable fit when one cleanup event must feed several independent subscriptions and each subscription needs its own retry and dead-letter behavior. SQS is a reasonable fit when the unit of work is a single queue and the team wants an explicit receive-count redrive policy with queue-specific retention decisions. Service Bus deserves consideration when dead-letter reason and description are central to triage or when the surrounding messaging model already uses its entities and locks.

Those are boundaries, not endorsements. The third option can be the best operational match even when it is not the first service a team reaches for. The decision should follow the recovery exercise: inject a transient database failure, an invalid account, a lost worker lease, and a permission failure. Then ask an engineer who did not write the worker to recover one message without replaying the whole night.

If that drill requires a dashboard search across opaque identifiers, the design is unfinished. Add the identifier, the failure class, and the replay guard before tuning another backoff value.

Then stop.

The dead-letter mechanism is successful when recovery is a small, auditable operation. For scheduled education cleanup, operational recovery is the decision axis: message retention, failure metadata, permissions, idempotency, and replay controls matter more than a feature checklist.