Short answer: upload the original image from the browser straight to private object storage, finalize it in the backend, and put a small job on a Node.js queue for thumbnail generation. Use a storage notification or webhook only as a delivery signal. The backend database remains the authority for tenant access and processing state.

That split keeps large bytes away from the application process. It also gives every retry a clear boundary. The browser moves bytes. The API grants a narrowly scoped upload. A worker reads a finalized original and writes deterministic thumbnail keys. A notification wakes the worker; it does not grant permission.

I choose this shape for a gaming media pipeline where players upload large screenshots or clips and the UI needs a quick preview. I've learned to draw the trust boundary before choosing a queue library. Otherwise a perfectly good resize function ends up deciding who may read an image.

What should a browser upload prove before a Node.js worker generates thumbnails?

The upload starts with an authenticated request for an upload intent. The backend derives a key from the account and asset IDs, records the expected content type and size, and returns a short-lived signed upload operation. The browser sends the original image directly to object storage. XMLHttpRequest can expose upload progress events for the progress bar; the application server never has to proxy the image bytes (see the MDN reference below).

The browser then calls a finalize operation. Finalize is important because an object appearing in storage is not the same thing as an authorized, complete asset. In one database transaction, the backend checks the tenant, expected metadata, and an idempotency token, then marks the asset ready for processing. A notification that arrives before that commit is simply deferred or ignored by the worker.

Here is the part I keep copyable. The queue interface is deliberately generic, so the access checks stay visible instead of hiding inside a vendor SDK.

type ThumbnailJob = { tenantId: string; assetId: string; originalKey: string; widths: number[]; }; async function processThumbnailJob(job: ThumbnailJob) { const asset = await db.assets.find(job.assetId); if (!asset || asset.tenantId !== job.tenantId || asset.status !== "ready") { return; // Duplicate or early delivery is a no-op. } const original = await objects.get(job.originalKey); for (const width of job.widths) { const thumbnail = await resizeToWidth(original, width); const key = `tenants/${job.tenantId}/assets/${job.assetId}/thumb-${width}.webp`; await objects.put(key, thumbnail, { contentType: "image/webp" }); } await db.assets.markThumbnailsReady(job.assetId, job.widths); } 

Notice what the job does not contain: a bearer token, a public URL, or an arbitrary bucket path. The object key is only a routing hint. The database check decides access. I log the tenant ID, asset ID, event ID, and attempt number; I don't log signed URLs or image bytes.

Small boundary. Big payoff.

How do object-storage webhooks, notifications, and a Node.js queue fit together?

A webhook is an HTTP delivery attempt to your receiver. A native storage notification may deliver to a queue directly. Either way, delivery is a signal that must be persisted and deduplicated. RFC 9110 is useful here: a successful HTTP response reports handling of the request, not completion of downstream thumbnail work.

The receiver should verify the message signature when one is provided, insert the event ID into an inbox table, acknowledge quickly, and enqueue only new events. The queue then supplies backpressure when a game launch produces a burst of uploads.

type StorageEvent = { id: string; tenantId: string; assetId: string; originalKey: string; }; async function acceptStorageEvent(event: StorageEvent) { const inserted = await db.inbox.insertIfNew(event.id, event); if (!inserted) return; await thumbnailQueue.add("generate", { tenantId: event.tenantId, assetId: event.assetId, originalKey: event.originalKey, widths: [320, 640, 1280], }, { jobId: event.id }); } 

The queue must tolerate duplicates and retries. A deterministic thumbnail key makes a retry idempotent. If the storage system supports conditional writes, use them when replacing an existing derivative would be dangerous. Mark the asset ready only after every required width exists; a partial set should be observable as partial, not reported as success.

This is where metrics help. Track upload-finalize latency, notification-to-worker latency, retry count, abandoned uploads, and the age of the oldest queued job. Alert on a growing queue and on a high ratio of finalized originals with no completed thumbnail set. Those signals tell you whether the problem is delivery, authorization, or image processing.

The access-control trade-off is bigger than the transfer speed

Keep originals private. A signed upload operation delegates one transfer; it does not replace the application authorization check. On download, authenticate the player, verify ownership of the asset row, and mint a short-lived signed read operation. Never let a client submit an arbitrary prefix and treat that as an export. Prefixes are naming conventions, not policy.

The before-and-after mental model is simple:

Before After
API receives every image byte Browser sends bytes directly to object storage
Upload completion implies processing Finalize transaction creates an explicit ready state
Notification triggers trusted work Notification is deduplicated input to a tenant-checked job
Thumbnail names vary per retry Keys are deterministic and safe to replay

There is a real cost. The service now owns an upload-intent record, an inbox, queue retries, dead-letter review, and cleanup for abandoned originals. I'm not sure every small internal tool needs those moving parts. If files are tiny, traffic is low, and one trusted operator owns the catalog, a synchronous multipart endpoint can be easier to test and operate. Stick with the simpler path when your recovery drill cannot exercise the queue and cleanup states.

What should I test before calling the image pipeline finished?

Test the gaps, not only the happy path. Upload an original, delay finalize, then deliver the same storage event twice. Revoke the user session before the worker runs. Submit an event whose tenant ID does not match the asset row. Kill a worker after writing one thumbnail and replay the job. In a useful replay script, I give each event a fixed ID, advance a fake clock between upload and finalize, and record the database row after every worker attempt. I then inspect the object keys, not just the queue result: there should be no key outside the tenant prefix, no second derivative with a random suffix, and no success state while one requested width is missing. Finally, I run the same sequence with a stale signed upload operation and with a duplicate finalize request. The expected result is one authorized derivative set, no cross-tenant reads, and a visible retry or partial state when work is interrupted.

The browser should also handle a rejected or expired signed operation without exposing credentials. A fresh upload intent can be requested after the user is still authorized. Keep retry delays bounded, and record the reason for each retry so a temporary delivery pause is distinguishable from a bad object key.

The catch is delivery simplicity versus access control. Direct browser uploads reduce proxy bandwidth and make progress reporting pleasant, but they add state that must be cleaned up. A worker queue absorbs bursts, but it cannot make an unsafe object key safe. Choose this architecture when the original files are large and thumbnail work is asynchronous; choose a smaller synchronous design when those conditions do not hold.

That decision is easy to revisit. Keep the upload intent, finalize record, event inbox, and thumbnail status as separate records, and you can change the queue or resize implementation without changing the browser contract. If the team cannot name who owns each record during an outage, the design is still too implicit.

References