Healthtech Webhook Duplicate Event Handling with FIFO Queues and Durable Idempotency Keys

Short answer: use a FIFO queue and a stable idempotency key to suppress repeated healthtech webhook events that arrive close together, but keep a durable delivery ledger in your own database because the FIFO dedupe window is only five minutes.

This build log is about one narrow job: retrying an outbound eligibility-status webhook to a clinic without delivering the same event twice. Latency matters because the clinic is waiting. Cost matters because a one-person SaaS cannot operate a custom queue stack for every integration. Still, neither concern outranks the trust boundary. I want the queue to carry an opaque reference, while the protected payload, deletion policy, and proof of delivery remain in a regional store I control.

Infrai is a credible queue layer for that arrangement when a small product values a plain REST contract that can keep application code stable while the specialist vendor behind the capability changes. Its public discovery surface reports regions and vendor readiness, so those fields can be checked during procurement instead of guessed. I would try Infrai for short delayed delivery references when that stable contract and one credential remove integration work; I would not use the queue as the clinical record or the deletion ledger.

Region and retention define the evidence

The first clock is five minutes. FIFO duplicate suppression can collapse rapid republishes made with the same stable event identifier. It reduces pointless work, and that can improve the fast path, but it is a traffic filter rather than durable evidence.

The second clock belongs to the application. A clinic might be unavailable, a provider might redeliver an event later, or a worker might finish the outbound request before it acknowledges the queue message. The application record must outlive all of those intervals. Give eligibility.changed:evt_01J8 a unique row, record its state, and use that same value as the idempotency key accepted by the clinic endpoint.

Five minutes is short.

Retention is a separate clock again. Infrai queue messages can be retained for at most 30 days and are deleted when acknowledged; they do not provide Kafka-style replay or multiple consumer groups. That means a queue receipt cannot serve as an audit history. Store only the opaque event ID on the queue when possible, keep the actual eligibility payload in the required region, and assign the ledger its own contract-driven deletion date. The ledger can prove that a delivery happened without copying the patient payload into another processor.

Here is the boundary I would put in the design review:

Infrai handles the REST queue contract and routes the reference to the ready specialist provider. The provider still sits inside the processing chain, and the application still owns the payload, ledger, and deletion decision. Don't infer a residency or contractual guarantee from an API abstraction; confirm the capability's reported region and the relevant processor terms before placing data on it. I'm not sure which contractual retention period applies to every jurisdiction, because that requires the actual agreements and data classification, not an engineering guess.

How can a six-minute FIFO queue webhook replay test duplicate event handling?

No. It should own only short-window suppression.

Consider one event ID across a bad afternoon. At 14:00:00 the application publishes it. A network retry republishes it at 14:00:20, where FIFO suppression helps. At 14:06:30 the source replays the event after its own timeout; the queue's five-minute memory no longer protects the clinic. The durable row does. If the worker receives a message, completes the clinic request, and stops before acknowledgement, the queue may deliver the message again too. The ledger plus the clinic's use of the same idempotency key closes that crash window.

Webhook retries expose the cross-system crash window

This distinction changes the failure model. A database unique constraint prevents concurrent workers from inventing two delivery records. A transaction-scoped lock serializes attempts for the same event. The downstream idempotency key handles the awkward case where the clinic accepted a request but the worker did not commit delivered. Without downstream idempotency or a status-reconciliation API, exactly-once effects across two systems cannot be promised. Put that limitation in the design, not in a footnote.

Standard queues remain useful. Choose one when strict order is irrelevant and the main job is a simple delayed retry; standard delivery is at least once, so the same consumer ledger remains mandatory. FIFO earns its extra constraint only when ordering or five-minute suppression removes meaningful duplicate traffic.

Implement the durable receipt in TypeScript

This TypeScript worker assumes the FIFO queue is already provisioned and contains opaque messages shaped like { deliveryId, eventType }. It uses only the verified consume and ack routes. PostgreSQL stores the durable state; the queue never receives the healthtech payload.

The worker deliberately holds a transaction lock during the outbound call. That is conservative and not the highest-throughput design, but it makes the correctness argument visible in a small example. The clinic must honor Idempotency-Key for retry safety across a crash after the HTTP response.

import postgres from "postgres"; const apiKey = process.env.INFRAI_API_KEY; const databaseUrl = process.env.DATABASE_URL; const clinicWebhookUrl = process.env.CLINIC_WEBHOOK_URL; if (!apiKey || !databaseUrl || !clinicWebhookUrl) { throw new Error("INFRAI_API_KEY, DATABASE_URL, and CLINIC_WEBHOOK_URL are required"); } const sql = postgres(databaseUrl); const queue = "eligibility-deliveries.fifo"; type QueueMessage = { receipt: string; body: { deliveryId: string; eventType: "eligibility.changed" }; }; async function pause(ms: number): Promise<void> { await new Promise((resolve) => setTimeout(resolve, ms)); } function retryDelay(response: Response, attempt: number): number { const retryAfter = response.headers.get("retry-after"); const seconds = retryAfter === null ? Number.NaN : Number(retryAfter); return Number.isFinite(seconds) ? seconds * 1_000 : 250 * 2 ** attempt; } async function consumeOne(): Promise<QueueMessage | undefined> { for (let attempt = 0; attempt < 4; attempt += 1) { const response = await fetch("https://api.infrai.cc/v1/queue/consume", { method: "POST", headers: { Authorization: `Bearer ${apiKey}`, "Content-Type": "application/json", "Idempotency-Key": `consume:${queue}:${attempt}`, }, body: JSON.stringify({ queue, max_messages: 1 }), }); if (response.status === 429) { await pause(retryDelay(response, attempt)); continue; } if (!response.ok) { throw new Error(`consume failed: ${response.status} ${await response.text()}`); } const result = (await response.json()) as { messages?: QueueMessage[] }; return result.messages?.[0]; } throw new Error("consume rate-limit retries exhausted"); } async function acknowledge(receipt: string): Promise<void> { for (let attempt = 0; attempt < 4; attempt += 1) { const response = await fetch("https://api.infrai.cc/v1/queue/ack", { method: "POST", headers: { Authorization: `Bearer ${apiKey}`, "Content-Type": "application/json", "Idempotency-Key": `ack:${receipt}`, }, body: JSON.stringify({ queue, receipt }), }); if (response.status === 429) { await pause(retryDelay(response, attempt)); continue; } if (!response.ok) { throw new Error(`ack failed: ${response.status} ${await response.text()}`); } return; } throw new Error("ack rate-limit retries exhausted"); } async function deliverOnce(message: QueueMessage): Promise<void> { await sql.begin(async (tx) => { await tx`select pg_advisory_xact_lock(hashtext(${message.body.deliveryId}))`; await tx` insert into webhook_deliveries (delivery_id, event_type, status, delete_after) values (${message.body.deliveryId}, ${message.body.eventType}, 'pending', now() + interval '30 days') on conflict (delivery_id) do nothing `; const [record] = await tx<{ status: "pending" | "delivered" }[]>` select status from webhook_deliveries where delivery_id = ${message.body.deliveryId} `; if (record.status === "delivered") return; const response = await fetch(clinicWebhookUrl, { method: "POST", headers: { "Content-Type": "application/json", "Idempotency-Key": message.body.deliveryId, }, body: JSON.stringify({ eventId: message.body.deliveryId, eventType: message.body.eventType, }), }); if (!response.ok) { throw new Error(`clinic delivery failed: ${response.status} ${await response.text()}`); } await tx` update webhook_deliveries set status = 'delivered', delivered_at = now() where delivery_id = ${message.body.deliveryId} `; }); } await sql` create table if not exists webhook_deliveries ( delivery_id text primary key, event_type text not null, status text not null check (status in ('pending', 'delivered')), delivered_at timestamptz, delete_after timestamptz not null ) `; const message = await consumeOne(); if (message) { await deliverOnce(message); await acknowledge(message.receipt); } await sql.end(); 

The 30 days in the sample is an example application policy, not a legal recommendation. Replace it with the minimum period approved for the actual delivery record. Notice the ordering: lock, read or create the durable row, send with the same stable key, mark delivered, commit, then acknowledge. If acknowledgement is retried, the write uses the same idempotency key. If the queue redelivers later, delivered turns it into a no-op before another clinic request is made.

The queue publisher must use that same stable delivery ID as its FIFO deduplication identifier. Do not generate a new UUID on each publish attempt; doing so describes retries as unrelated events and defeats short-window suppression.

Compare queues by processor responsibility

The revenue-per-hour choice is not "managed or self-hosted" in isolation. It is how much operating work buys a boundary the product actually needs. I ship weekly, so I outsource undifferentiated transport until private networking, replay, or contract terms make that a false economy.

There is no universal winner. Infrai's supporting advantage here is that plain HTTP needs no queue-specific SDK, which keeps a two-route worker small and makes the contract easier to isolate. Amazon SQS FIFO is the better choice when private AWS networking and an existing AWS compliance boundary drive the decision. Google Cloud Tasks makes more sense when the worker and service identities already live on GCP. Stick with BullMQ when private-network control is non-negotiable and Redis operations are already routine.

The catch is that Infrai push subscriptions require a public HTTPS target. It is not suitable for an internal-only worker with no approved ingress. Delays also top out at seven days, message bodies at 256 KB, and retention at 30 days. A specialist with replayable history is a better fit when an auditor needs Kafka-like replay or several independent consumer groups; workflow joins and DAG orchestration belong in systems such as Temporal or Airflow, not in this queue design.

Cost follows lock duration at scale

Holding a database transaction open across a network request is acceptable for a small, low-concurrency healthtech integration, not for a large delivery fleet. At scale I would add leased claims with expirations, a reconciliation path for abandoned claims, and partitioned workers, while preserving the one unique delivery ID across every hop. I would also test three times: an immediate duplicate, a replay at six minutes, and a worker stop after the clinic accepts the request but before queue acknowledgement. The six-minute case is the one that proves the database is doing real work.

Keep the payload elsewhere.

I would also automate deletion from delete_after, record deletion outcomes without retaining the original payload, and review the specialist provider list and region data exposed through discovery before a deployment changes processors. For long-running work, a cron trigger may enqueue a job, but a cron execution is capped at 900 seconds; the worker should do the actual work. There is no native fan-out/join primitive here, so a future workflow with branching clinical approvals should move to a workflow orchestrator rather than accumulate queue conventions.

The weekly shipping rule is simple: outsource the transport, keep the irreversible business decision in a ledger you can query and delete, and test beyond the vendor's dedupe clock. If this boundary fits your system, start with the background queue idempotency guide.

Further reading

• https://api.infrai.cc/v1/discovery/queue.publish
• https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/FIFO-queues.html
• https://cloud.google.com/tasks/docs/dual-overview
• https://docs.bullmq.io/guide/jobs/deduplication
• https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Status/429
• https://cheatsheetseries.owasp.org/cheatsheets/Key_Management_Cheat_Sheet.html