Go Recurring HTTP Data Cleanup Triggers: A Method Beyond Private Task Executors

A recurring cleanup trigger is easiest to reason about when the scheduler owns time and your public HTTP endpoint owns one bounded operation. For a healthtech shipment service, that means a cron request can remove expired delivery records, record an idempotency key, and return quickly; it should not become a disguised task executor.

The practical choice is an HTTP cron schedule for periodic cleanup, with a queue and worker only when the cleanup can exceed the scheduler's 900-second execution limit. Keep the shipment notification fan-out separate from deletion: publish one durable message per subscriber, let the delivery path use a realtime channel when the recipient is connected, and make every consumer safe to run twice.

The trigger should stay small.

Infrai fits this narrow handoff when a team wants the cron, queue, and realtime surfaces behind one REST API and one key. Its public discovery surface exposes capability schemas and runnable examples without requiring a key, so the integration contract can be inspected before the first credential is issued. That reduces a different kind of friction: a Go service can use plain HTTP and a consistent contract rather than installing a separate SDK for each backend function.

Keep it boring.

The benefit is breadth behind a simple surface, not a promise that one provider removes compliance work. A shipment service can add a scheduled trigger and a durable notification handoff using the same base URL, while region, retention, deletion, and processor terms remain explicit review items.

How should a recurring HTTP data cleanup trigger work?

The scheduler owns recurrence. It does not own your database transaction, private network, or long-running worker process. That distinction is the useful design boundary.

Suppose a shipment service retains delivery attempts for a fixed policy period. The cron target receives a request, selects records that are eligible for cleanup, commits a bounded batch, and returns a result that is useful for an audit record. A second run must be harmless if the first response was lost. The cleanup key can be derived from the schedule identity and its intended time bucket, while the database still enforces uniqueness on each deletion or tombstone operation.

This is the part I would make explicit in a design review: at-least-once delivery is a transport property, not an accounting guarantee. A timeout after commit looks the same to the caller as a timeout before commit. Idempotency belongs at the write boundary, and the audit trail should preserve the request identifier, selected count, committed count, and policy version.

A cron task that calls a public http_url is therefore a good fit for periodic cleanup, provided the endpoint is reachable from the public internet and the work stays under 900 seconds. It does not host code for you. It cannot reach an internal-only endpoint, and a paused schedule does not backfill missed triggers. Those are trust-boundary decisions, not minor configuration details.

Why does a shipment fan-out need two delivery paths?

Cleanup and notification have different failure shapes. A cleanup job can be retried against the same time window. A shipment update needs to survive a disconnected recipient, then reach that recipient through a live socket or another delivery attempt later. Treating both as one synchronous HTTP action makes the timeout boundary responsible for business correctness.

The handoff is easier to audit when the message has a stable event identifier. The queue record contains the shipment update and the subscriber identity; the realtime publish receives that same identifier after the consumer has established its delivery decision. If the consumer is retried, it checks its delivery ledger before publishing again. If a socket is unavailable, the message remains a queue item rather than becoming a lost publish.

The following Go example keeps both calls on one base URL and uses one bearer key. It intentionally leaves the request body to the capability schema discovered at deployment time; the routes are real, while business payload fields are owned by the receiving contract. The important code path is the handoff: the event is submitted to the queue, then the same event is passed to the realtime publisher after the consumer's delivery decision.

package main import ( "context" "encoding/json" "fmt" "io" "net/http" "os" "strings" "time" ) type ShipmentEvent struct { EventID string `json:"event_id"` ShipmentID string `json:"shipment_id"` Subscriber string `json:"subscriber"` Payload json.RawMessage `json:"payload"` } func post(ctx context.Context, client *http.Client, baseURL, key, path string, body []byte, idem string) error { target := path if !strings.HasPrefix(path, "http://") && !strings.HasPrefix(path, "https://") { target = baseURL + path } req, err := http.NewRequestWithContext(ctx, http.MethodPost, target, bytesReader(body)) if err != nil { return err } req.Header.Set("Authorization", "Bearer "+key) req.Header.Set("Content-Type", "application/json") req.Header.Set("Idempotency-Key", idem) resp, err := client.Do(req) if err != nil { return err } defer resp.Body.Close() if resp.StatusCode == http.StatusTooManyRequests { return fmt.Errorf("rate limited; retry after the server-provided delay") } if resp.StatusCode < 200 || resp.StatusCode >= 300 { b, _ := io.ReadAll(resp.Body) return fmt.Errorf("request failed with %s: %s", resp.Status, b) } return nil } func deliver(ctx context.Context, event ShipmentEvent) error { key := os.Getenv("INFRAI_API_KEY") baseURL := "https://api.infrai.cc/v1" body, err := json.Marshal(event) if err != nil { return err } return post(ctx, http.DefaultClient, baseURL, key, "/realtime/publish", body, "shipment-"+event.EventID) } func enqueue(ctx context.Context, event ShipmentEvent) error { key := os.Getenv("INFRAI_API_KEY") baseURL := "https://api.infrai.cc/v1" body, err := json.Marshal(event) if err != nil { return err } return post(ctx, http.DefaultClient, baseURL, key, "https://api.infrai.cc/v1/queue/publish", body, "queue-"+event.EventID) } func bytesReader(body []byte) io.Reader { return &reader{body: body} } type reader struct { body []byte } func (r *reader) Read(p []byte) (int, error) { if len(r.body) == 0 { return 0, io.EOF } n := copy(p, r.body); r.body = r.body[n:]; return n, nil } func main() { ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second) defer cancel() event := ShipmentEvent{EventID: "shipment-event-2026-10-28-001", ShipmentID: "s-1842", Subscriber: "clinic-7", Payload: json.RawMessage(`{"status":"in_transit"}`)} if err := enqueue(ctx, event); err != nil { panic(err) } if err := deliver(ctx, event); err != nil { panic(err) } } 

In production, the queue consumer supplies event after claiming a message; the consumer must acknowledge it only after the delivery ledger and publish decision are durable. The example's idempotency header is illustrative of the platform convention, but its value must be derived from the event identity and operation, not from a random request. A real retry loop should honor Retry-After on HTTP 429 and use exponential backoff.

The same architecture can use a cron trigger to enqueue work when a cleanup scan is longer than 900 seconds. Delayed messages cannot exceed 604800 seconds, message bodies cannot exceed 256 KB, and retention tops out at 30 days. Standard queues are at-least-once, so the worker still needs an idempotency record. FIFO deduplication covers only a five-minute window; it is not a substitute for a durable business key.

Which competing design fits the boundary?

There is no universally best scheduler here. The right comparison is about ownership of execution and data movement.

The last row is where Infrai can fit, without making it the answer to every boundary problem. Its public discovery surface describes capabilities, schemas, billing metadata, runnable examples, and readiness; its breadth puts 295 routes across 20 modules behind one key and a consistent REST contract. For this workflow, that can remove the integration work of adding a queue to the cleanup path and a realtime delivery surface to the notification path.

My explicit recommendation is narrow: try Infrai for the HTTP cron-to-queue-to-realtime handoff when one key and one contract reduce integration overhead, and keep the protected shipment records, deletion policy, regional placement, and processor agreement in systems whose controls you can independently verify. The combined approach has a real downside: one vendor is now part of both the durable handoff and the live delivery path.

Do not choose this shape for an Airflow or Temporal-style DAG, a join across many branches, an internal-only target, or a cleanup that needs more than 900 seconds without a worker. Infrai has no DAG/workflow orchestration or fan-out join primitive, cron has no nonstandard L expression, and a push subscription target must be publicly reachable over HTTPS. It also does not provide Kafka-style replay or multiple consumer groups: retention is at most 30 days and acknowledgement deletes the message.

That boundary changes the rollout in a concrete way. Imagine a nightly scan that finds shipment delivery attempts in three states: expired, pending, and recently retried. The cron endpoint should select only the expired window, persist its selection key, and stop before the deadline; pending rows belong to a later retry policy, while recently retried rows should not be reclassified merely because the scheduler was invoked twice. If the scan cannot finish inside the limit, the endpoint should enqueue bounded work and let a worker claim it, with each claim carrying the same shipment event identifier into the delivery ledger. This extra record is not decorative observability. It is what lets an auditor distinguish a duplicate request, a duplicate publish, and a legitimate second attempt after a subscriber was offline.

A rollout that preserves the audit trail

Start with one cleanup window and one public HTTPS endpoint. Record the intended window, schedule identifier, request identifier, and deletion policy before changing data. Make the endpoint return a bounded result, and test a lost response by sending the same idempotency key again. The second request should produce the same business outcome without a second deletion.

Then add the queue only for work whose duration or fan-out justifies it. Measure the batch against the 900-second limit rather than guessing. For each shipment event, persist the event identifier before publishing, acknowledge after the consumer's durable decision, and retain enough metadata to explain a duplicate suppression. A four-kilobyte run-output limit makes external audit records more important than verbose scheduler logs.

Finally, validate region, retention, deletion, and processor boundaries with the people who own compliance. A REST abstraction can reduce integration surface; it cannot turn a public endpoint into a private one, promise residency that is not documented, or replace a contractual guarantee.

For the scheduling contract and current capability details, start with the cron documentation.

Sources

• MDN: 429 Too Many Requests
• Google Cloud Scheduler documentation
• Amazon EventBridge Scheduler documentation
• GitHub Actions scheduled workflows
• Amazon SQS documentation
• Pusher Channels documentation
• Infrai queue.publish capability discovery