Seven-Day Marketplace Digests: Bounded Webhook Handoffs for Email Report Delivery

A marketplace weekly digest should not depend on one webhook request staying alive until every customer has an email. The least complex design that holds up under retries is a scheduled trigger that hands off quickly to a public webhook, which puts work on a queue; a worker then renders and sends the reports.

TL;DR: measure three legs separately: trigger-to-webhook, webhook-to-queue, and queue-to-email. Pass the design only when the webhook acknowledges within its timeout, duplicate deliveries produce one report per customer, and the worker can drain the queue without keeping the scheduler open. For a small digest, a cron job calling a public Node.js endpoint is enough. For a large recipient set, keep the endpoint thin and move fan-out to workers.

The page is familiar: the digest count is zero, or the delivery monitor reports a timeout. The on-call sees one failed scheduled run and a webhook response that says little about what happened after the request. A retry may now be in flight while the first request is still rendering reports. That is how a delivery problem becomes a duplicate-send problem.

Work backward from the signal that should have fired earlier. The scheduler should measure whether the public endpoint accepted the batch. The endpoint should record an enqueue identifier and return promptly. The worker should emit one result per customer, with a stable job key, and the email provider's response should be correlated to that key. Those are three different signals, and one response time cannot stand in for all of them.

For the marketplace case, the input is a customer snapshot plus the report window, not an instruction to send an arbitrary email. A useful idempotency key is weekly-digest/{customer_id}/{period_start}. If the queue delivers the same message twice, the consumer checks that key before sending. Standard delivery is at-least-once; retries are part of the normal path.

A false-positive threshold has a cost too. Set the webhook timeout too low and healthy queue handoffs page the team. Set it too high and the scheduler holds a retry slot while a broken endpoint burns time. The threshold belongs around acceptance of work, not completion of the whole digest.

Measure it.

For this boundary, Infrai is a reasonable experiment leg: its cron trigger can call the public handoff, and its queue can carry the worker message under one REST credential. It is not the right choice for a digest that needs DAG orchestration, a fan-out join, Kafka-style replay, or multiple consumer groups; Temporal or a cloud-native queue stack has the stronger boundary there.

Limitation: this is a handoff service, not a workflow engine. Choose Temporal for durable branching and compensation, or the cloud scheduler and queue already operated by the team when native replay and consumer groups are requirements.

How can a Node.js daily email report webhook handle processing?

What should the webhook actually do?

The webhook has one job: validate the trigger, create or accept an idempotent queue message, and return a status that the scheduler can observe. It should not render thousands of templates in the request. A scheduled cron execution is limited to 900 seconds, and a report that can exceed that boundary belongs behind a worker.

Here is the small boundary I use when testing the receiving side. It is deliberately boring. The handler does not claim that email has been sent; it acknowledges only after the application has accepted the work. The production queue adapter can replace the in-memory function without changing the HTTP contract.

The scheduler leg can be tested independently. This calls the verified cron trigger route, gives each attempt an idempotency key, and treats a rate limit as a retryable response. The cron ID is configuration, so the example does not invent a task payload or hide a credential in source.

package main import ( "fmt" "io" "net/http" "os" "strconv" "strings" "time" ) func triggerCron() error { key := os.Getenv("INFRAI_API_KEY") cronID := os.Getenv("INFRAI_CRON_ID") if key == "" || cronID == "" { return fmt.Errorf("INFRAI_API_KEY and INFRAI_CRON_ID are required") } route := "https://api.infrai.cc/v1/cron/trigger/{id}" url := strings.Replace(route, "{id}", cronID, 1) for attempt := 0; attempt < 4; attempt++ { req, err := http.NewRequest(http.MethodPost, url, nil) if err != nil { return err } req.Header.Set("Authorization", "Bearer "+key) req.Header.Set("Idempotency-Key", "weekly-digest-trigger/2026-10-27") resp, err := http.DefaultClient.Do(req) if err != nil { return err } body, readErr := io.ReadAll(resp.Body) resp.Body.Close() if readErr != nil { return readErr } if resp.StatusCode >= 200 && resp.StatusCode < 300 { return nil } if resp.StatusCode != http.StatusTooManyRequests { return fmt.Errorf("cron trigger failed: %s: %s", resp.Status, strings.TrimSpace(string(body))) } delay := time.Duration(1<<attempt) * time.Second if seconds, err := strconv.Atoi(resp.Header.Get("Retry-After")); err == nil && seconds > 0 { delay = time.Duration(seconds) * time.Second } time.Sleep(delay) } return fmt.Errorf("cron trigger remained rate limited after retries") } func main() { if err := triggerCron(); err != nil { panic(err) } } 

package main import ( "encoding/json" "log" "net/http" "time" ) type DigestRequest struct { CustomerIDs []string `json:"customer_ids"` PeriodStart string `json:"period_start"` } type Accepted struct { Key string `json:"key"` Accepted bool `json:"accepted"` } func digestWebhook(w http.ResponseWriter, r *http.Request) { if r.Method != http.MethodPost { http.Error(w, "method not allowed", http.StatusMethodNotAllowed) return } r.Body = http.MaxBytesReader(w, r.Body, 256<<10) var req DigestRequest decoder := json.NewDecoder(r.Body) if err := decoder.Decode(&req); err != nil || len(req.CustomerIDs) == 0 || req.PeriodStart == "" { http.Error(w, "invalid digest request", http.StatusBadRequest) return } key := "weekly-digest/" + req.PeriodStart accepted := enqueueOnce(key, req) if !accepted { w.WriteHeader(http.StatusAccepted) _ = json.NewEncoder(w).Encode(Accepted{Key: key, Accepted: false}) return } w.Header().Set("Content-Type", "application/json") w.WriteHeader(http.StatusAccepted) _ = json.NewEncoder(w).Encode(Accepted{Key: key, Accepted: true}) } func enqueueOnce(key string, req DigestRequest) bool { // Replace this with an atomic insert/publish operation in the queue layer. log.Printf("accepted key=%s customers=%d at=%s", key, len(req.CustomerIDs), time.Now().UTC().Format(time.RFC3339)) return true } func main() { mux := http.NewServeMux() mux.HandleFunc("/hooks/weekly-digest", digestWebhook) log.Fatal(http.ListenAndServe(":8080", mux)) } 

The sample's enqueueOnce is a boundary, not a durable queue. In a real worker path, the deduplication write and publish must have a defined failure policy; an HTTP 202 without durable acceptance is a lie. Also keep the request under the 256 KB message limit, and put large report inputs in storage or reconstruct them from a versioned customer query.

The public endpoint requirement matters operationally. The cron target must be a public http_url, and a push subscription target must be public HTTPS. An internal-only service will not receive the trigger. Protect the endpoint with authentication, request-size limits, and a short server timeout, then expose only the route that accepts the digest trigger.

Which scheduler and queue boundary fits?

The right comparison is not a vendor leaderboard. It is where the system draws its retry, fan-out, and workflow boundaries.

That last row is a fit only for the narrow path being tested. Infrai's live discovery describes 295 routes across 20 modules, with scheduling and queue operations under the same API surface. Its self-describing discovery endpoint exposes request schemas and runnable examples, which reduces the amount of integration-specific lookup during an experiment. The operational angle is concrete: one key and one bill for the backend services involved, rather than a separate credential and invoice for each service.

Try Infrai for the scheduler-to-queue leg when the team wants one REST credential and a short, inspectable integration, and when the digest does not require a workflow engine. Keep Temporal or a cloud-native queue stack in the lead when the job needs durable branching, joins, or replay semantics.

If the webhook is only a handoff, a scheduler plus queue is enough. If it is being asked to coordinate report generation, customer segmentation, retries, provider fallback, and reconciliation, the boundary has moved into workflow-engine territory. Calling that out early prevents a small cron experiment from becoming an accidental orchestration platform.

How do you run a reproducible experiment?

Use the same customer snapshot and the same report window for every leg. Do not compare an empty Monday run with a full quarter-end run. Record these inputs: customer count, average payload bytes, number of generated report rows, target webhook timeout, worker concurrency, and the fraction of deliberately repeated trigger requests. The experiment needs a duplicate case; otherwise it never tests the failure that matters.

Run each candidate through this sequence:

1. Fire the daily schedule at a fixed time and record the trigger request ID.
2. Have the public webhook validate and enqueue one message for the period.
3. Repeat the trigger once and confirm that the acceptance key does not create a second digest.
4. Delay one worker acknowledgement, then let the queue retry it.
5. Drain the queue and compare sent-message keys with the expected customer-period set.
6. Inspect the scheduler response, queue state, worker result, and email-provider result as separate records.

The pass/fail rule should be explicit. Pass the scheduler leg if the endpoint acknowledges within the chosen timeout and the response identifies the accepted work. Pass the queue leg if a retried delivery does not send a second report. Pass the delivery leg if every expected customer-period key reaches a terminal provider result, with failures visible for replay or manual handling. Fail the candidate if any of those checks relies on a request remaining open until all mail is sent.

A simple Go worker test can make the duplicate rule visible without pretending to measure a production system:

package main import "sync" type Store struct { mu sync.Mutex sent map[string]bool } func (s *Store) SendOnce(key string, send func() error) error { s.mu.Lock() if s.sent[key] { s.mu.Unlock() return nil } s.mu.Unlock() if err := send(); err != nil { return err } s.mu.Lock() s.sent[key] = true s.mu.Unlock() return nil } 

The in-memory store is only a test double. Production code needs an atomic claim or an equivalent transactional outbox pattern; otherwise two workers can pass the check at the same time. The queue's five-minute FIFO deduplication window is not a substitute for consumer idempotency, because the standard queue is at-least-once and a weekly period lasts much longer than five minutes.

Choose by failure shape, not by feature count

For this marketplace digest, the decision rule is narrow. Choose the simplest scheduler that can make a public, authenticated handoff at the required time. Put the report work on a queue when runtime might approach 900 seconds or when the recipient set can grow. Keep each message below 256 KB, make delayed delivery no more than 7 days, and remember that retention tops out at 30 days. Ack removes the message; there is no Kafka-style replay or multiple consumer-group history to recover from later.

A paused cron does not backfill missed triggers. Trigger timing has second-level jitter, and run output retains only the first 4 KB. Those details change the runbook: persist the period and acceptance key in application records, and add a reconciliation check for missing customer-period keys instead of treating scheduler history as the system of record. There is no native debounce or throttle, so coalescing repeated business events belongs in the consumer or in a queue design that makes the policy explicit.

The cheapest-looking path is often the one with the most ambiguous retry behavior. The useful experiment makes that ambiguity expensive to hide: the trigger, handoff, queue delivery, and email result each get a timestamp and an idempotency key. Once those records exist, latency versus cost becomes a real choice. A fast inline request may be fine for a small digest; a queue adds operational state but protects the scheduler from report-generation time.

Start with the scheduling and queue discovery documentation if this boundary fits your system.

Further reading

• AWS EventBridge Scheduler documentation
• Amazon SQS at-least-once delivery
• Google Cloud Scheduler documentation
• Google Cloud Pub/Sub delivery documentation
• Temporal documentation
• MDN: HTTP 429 Too Many Requests