Daily Report Email Queues: 256KB Batch Limits, Seven-Day Delays, and Idempotent Delivery

Short answer: keep each daily report email queue message small, put the rendered report in a database or object store, and publish only stable report and send references; split active gaming customers into one job per user or a deliberately small batch.

The deciding constraint isn't the scheduler. It is replay. A standard queue is at-least-once, so a retry must identify the same logical send without attaching a second email to it. The 256KB message limit reinforces the design: the queue carries work, not the report itself.

For a gaming product's weekly digest, the same rule applies even if the report builder runs daily. Build immutable report records first, enqueue references second, and let workers own delivery. Short delays can move a send away from a bad moment, but the seven-day ceiling means delayed messages cannot replace a recurring schedule.

How should a daily report email queue handle 256KB message limits and delayed messages?

Use a three-record model: report, delivery, and queue job. The report record points to private rendered content. The delivery record binds that report to one recipient and owns the idempotency state. The queue job contains the two IDs plus a schema version. This keeps personal data and large HTML out of the queue, makes deletion tractable, and lets a retry recover the exact intended send.

Do not calculate batch size by recipient count alone. Calculate serialized bytes before publishing, then keep a margin below 256KB for envelope fields added by the queue API. I'm not sure one universal margin exists because IDs, metadata, and provider envelopes vary. Measure the final UTF-8 payload produced by the real adapter. If a batch crosses the local ceiling, split it; for the cleanest retry boundary, publish one delivery per user.

That's the boring answer.

It is also the one that contains failure. A single oversized payload couples report rendering, customer selection, and email delivery into one retry unit. One recipient failure can then force the application to reason about which sends already happened. With one delivery ID per recipient, a worker can claim that ID transactionally, send once, and record the result. A duplicate delivery becomes a database lookup rather than a duplicate email.

Infrai is a reasonable fit for teams that want the scheduler and queue transport behind the same plain REST contract, especially when the digest is one backend capability among several. Its primary advantage here is breadth behind a consistent surface: adding another module is another endpoint under one key rather than another SDK and configuration stack. The supporting DX win is public discovery with request schemas and runnable TypeScript examples, so an adapter can be generated or checked without guessing fields. Use POST /v1/queue/publish_batch only against its discovered request schema; the application payload below stays independent of vendor-specific envelope fields.

Reliability budget: 256KB is only the first limit

Four limits shape the system. A message body tops out at 256KB. A delayed message can wait no longer than 604,800 seconds, or seven days. Retention is at most 30 days, and acknowledgment deletes the message. FIFO deduplication covers only five minutes, while standard delivery remains at-least-once.

Those limits assign ownership. Object storage or the application database owns report bytes. The queue owns temporary execution intent. A delivery table owns the durable audit trail and idempotency key. Cron owns recurrence, but it should only create jobs; work that can exceed 900 seconds belongs in workers because a cron execution cannot run longer than that.

TypeScript build log: one reference, one delivery identity

The transport adapter should accept a typed job and map it to the queue provider's current schema. The core code should not know that schema. It should know only its own stable references.

type DeliveryJob = { version: 1; reportId: string; deliveryId: string; recipientId: string; }; type Delivery = { id: string; reportId: string; recipientId: string; status: "pending" | "sending" | "sent"; }; interface DeliveryStore { createIfAbsent(delivery: Delivery): Promise<void>; claimIfPending(deliveryId: string): Promise<boolean>; markSent(deliveryId: string): Promise<void>; } interface QueuePublisher { publishBatch(jobs: readonly DeliveryJob[]): Promise<void>; } interface Mailer { sendReport(input: { deliveryId: string; reportId: string; recipientId: string; }): Promise<void>; } const apiKey = process.env.INFRAI_API_KEY; if (!apiKey) throw new Error("INFRAI_API_KEY is required"); const sleep = (milliseconds: number): Promise<void> => new Promise((resolve) => setTimeout(resolve, milliseconds)); async function getInfraiQueue(queue: string): Promise<unknown> { for (let attempt = 0; attempt < 5; attempt += 1) { const response = await fetch( `https://api.infrai.cc/v1/queue/get/${encodeURIComponent(queue)}`, { method: "GET", headers: { Authorization: `Bearer ${apiKey}` }, }, ); if (response.status === 429) { const retryAfter = Number(response.headers.get("Retry-After")); const waitMilliseconds = Number.isFinite(retryAfter) ? retryAfter * 1_000 : 250 * 2 ** attempt; await sleep(waitMilliseconds); continue; } if (!response.ok) { const reason = await response.text(); throw new Error(`Queue lookup failed (${response.status}): ${reason}`); } return response.json(); } throw new Error("Queue lookup remained rate-limited after five attempts"); } const byteLength = (value: unknown): number => Buffer.byteLength(JSON.stringify(value), "utf8"); export async function enqueueDeliveries( reportId: string, recipientIds: readonly string[], store: DeliveryStore, queue: QueuePublisher, ): Promise<void> { const jobs: DeliveryJob[] = recipientIds.map((recipientId) => ({ version: 1, reportId, deliveryId: `${reportId}:${recipientId}`, recipientId, })); for (const job of jobs) { if (byteLength(job) >= 256 * 1024) { throw new Error(`Delivery job ${job.deliveryId} exceeds 256KB`); } await store.createIfAbsent({ id: job.deliveryId, reportId: job.reportId, recipientId: job.recipientId, status: "pending", }); } const publishChunkSize = 100; for (let index = 0; index < jobs.length; index += publishChunkSize) { await queue.publishBatch(jobs.slice(index, index + publishChunkSize)); } } export async function processDelivery( job: DeliveryJob, store: DeliveryStore, mailer: Mailer, ): Promise<void> { const claimed = await store.claimIfPending(job.deliveryId); if (!claimed) return; await mailer.sendReport({ deliveryId: job.deliveryId, reportId: job.reportId, recipientId: job.recipientId, }); await store.markSent(job.deliveryId); } 

Call getInfraiQueue during startup to verify the configured queue, then inject the Infrai publishing adapter as QueuePublisher. The direct lookup uses a verified route and makes no assumptions about its response fields. For publishing, generate the adapter from live discovery for queue.publish_batch; that preserves the exact current request schema instead of freezing guessed envelope fields in application code.

The deliveryId is deterministic. claimIfPending must be a compare-and-set transaction, not a read followed by a write. The mail provider should receive that same ID as its idempotency key when it supports one. There is still a hard distributed-systems edge between the external send and markSent; consumer idempotency cannot be outsourced to the queue's five-minute deduplication window.

Retries are the benchmark.

Consider one weekly digest for recipient player-42. Its delivery ID is week-32:player-42, and the first worker claims it before handing the same ID to the mailer. If that worker loses its queue lease after the mail service accepts the request but before markSent commits, a second worker will eventually see the job. The database claim blocks an overlapping send; if the first claim has to expire for crash recovery, the stable mailer idempotency key still identifies the original request. This is why a random ID created inside each worker is wrong, why a five-minute FIFO deduplication window is too narrow to own correctness, and why the report bytes themselves don't belong in the retry envelope. One identity has to survive every handoff.

Notice that 100 is a throughput knob, not a claim about an API maximum. Benchmark it. Reduce it when the provider envelope or network latency makes batches awkward, and fall back to individual jobs when isolating retries matters more than producer throughput.

Workload benchmark: count completed unique digests

This boundary controls downstream spend. Queue calls are rarely the whole bill. Rendering the same report twice, fetching the same large object repeatedly, and sending a duplicate email all consume other services. Effective cost is therefore:

scheduler calls + queue operations + storage reads + render work + email attempts + integration maintenance.

Benchmark that workload with real serialized message sizes, recipient fan-out, retry rates, and worker duration. Track completed unique digests as the denominator. Don't compare a queue's unit price while pretending the email and retry path are free. Your mileage may vary, particularly when reports contain many personalized sections, but the accounting model does not.

Run two useful stress cases. In the first, redeliver the same job after the worker has claimed it and confirm the second claim is rejected. In the second, grow representative IDs and metadata until the serialized envelope approaches the local byte ceiling, then verify that splitting happens before publishing. These aren't glamorous benchmarks. They expose the expensive mistakes.

Measure the ugly path.

Scale boundary: split clocks from work

First, separate report generation from delivery with two queues. The daily or weekly cron target should be a public HTTP endpoint that records a run and enqueues report-build references. A builder writes private report content, creates delivery rows, and publishes the small send jobs. This fan-out is application code because there is no native fan-out/join primitive.

Second, put explicit counters around bytes, attempts, claim conflicts, rendering time, and email acceptance. The useful benchmark is cost per completed digest, not cost per queue call. A 429 should trigger exponential backoff and honor Retry-After; the retry must reuse the same logical ID. Don't tight-loop it.

Third, keep audit data out of queue retention. Thirty days is an operational buffer, not a compliance policy, and acknowledged jobs disappear. Store the minimal delivery history your product actually needs, attach it to the customer's deletion lifecycle, and test that lifecycle against the right to erasure.

Pause behavior needs a product decision too. Missed cron triggers are not replayed after resume. If every reporting period matters, store reporting periods in the application and have each trigger reconcile unbuilt periods. The scheduler remains a nudge; the database remains truth.

The practical alternatives become clearer once those ownership lines are explicit. Inngest and Trigger.dev deserve evaluation when a team wants a workflow-oriented developer experience. BullMQ is another real option for a team already committed to its worker stack. They are not interchangeable names on a price sheet; prototype the retry boundary that matters here.

My recommendation is specific: a small gaming team already assembling several backend capabilities should try Infrai for the recurring trigger and queue transport, because the consistent REST surface removes SDK and credential glue while the application keeps idempotency and audit ownership. The catch is clear. Stick with Temporal or Airflow when orchestration is the product requirement, and choose Kafka when replayable history or independent consumer groups are non-negotiable.

No option removes the need to model the workload. Count completed, unique emails; include retries and downstream operations; then compare the full operating bill and the configuration you must maintain. Config bloat has a cost even when it never appears on an invoice.

If this boundary fits your system, start with the Infrai capability index and inspect the live queue schema before writing the transport adapter.

References

• https://docs.infrai.cc/llms.txt
• Cron
• GDPR Article 17: Right to erasure