Should background workers ship with the web app or deploy separately? Let's argue about it

Most apps with background jobs start the same way: the web app and the workers live in the same codebase, built into the same image, deployed together. The worker is just the app started with a different command.

At some point, someone suggests splitting them: separate deploys, maybe separate services, maybe separate teams. The argument that follows is one of the more interesting ones in backend architecture, because both sides are right about different things.

Here's my attempt at a fair summary. Then tell me where you land.

The case for deploying together

One version of the truth. When the web app enqueues a job, the worker that picks it up runs the same code. The job's payload format, the database schema and the business logic all match. A whole class of bugs, "the worker didn't understand the new job format", mostly disappears.

Simpler pipelines. One build, one artifact, one deploy. Rollbacks are one action. Fewer moving parts to monitor.

Shared code without ceremony. Models, validation and helpers are just imports. No shared library to version and publish.

Easier local development. One repo, one setup, everything runs together.

The case for deploying separately

Different reliability needs. A web request takes milliseconds; a job might run for twenty minutes. Restarting workers on every web deploy means interrupting long jobs constantly, or making every deploy wait for them to finish.

Different scaling. Web traffic and job volume rarely peak at the same time. Separate deployments make it natural to scale them independently.

Smaller blast radius. A bad change to a report-generation job shouldn't require redeploying the checkout page, and a web deploy shouldn't risk the payment retry worker.

Release cadence. Web code often ships many times a day. Workers that process money, emails or data imports may deserve a more careful pace.

The part both camps underestimate: jobs in flight

Whichever you choose, there's a moment that matters more than the architecture diagram: the deploy, with jobs still in the queue.

Consider a job enqueued by the old code and processed by the new code, or vice versa. This happens in both setups:

Deploying together, rolling deploys mean old and new versions run side by side for a while. A job enqueued by a new web instance can be picked up by an old worker.
Deploying separately, the gap is longer and more explicit. The web app might be days ahead of the workers.

And for scheduled jobs, the gap can be enormous. A job scheduled today to run in 30 days will be executed by whatever code is deployed in 30 days. Its payload was written by today's code.

So the real requirement, in both camps, is the same:

Job payloads must be backwards and forwards compatible, at least for a while. Add fields; don't rename or remove them in the same release that starts using the new ones.
Version your payloads when changes are unavoidable, and make workers handle every version that might still be in the queue.
Long-delayed jobs should carry minimal data: an ID to look up, rather than a snapshot of state that may be stale or in an old format by the time it runs.

{ "type": "send_renewal_reminder", "version": 2, "subscription_id": "sub_123" }

Deploying together reduces the window. It doesn't remove it.

Graceful worker restarts matter more than the topology

The other underestimated piece: what happens to a job that's running when a worker is told to stop? Workers need to stop taking new jobs, finish or safely abandon the current one within a timeout, and leave anything unfinished to be retried. If that isn't in place, deploy frequency, together or separate, translates directly into interrupted jobs.

What about database migrations?

Migrations sharpen the whole debate. If the web app and workers share a database, a schema change has to work with every version of both that might be running at once. The usual answer, expand first and contract later, applies in both camps: add the new column, deploy code that writes both, backfill, switch reads, and only then remove the old column in a later release. Separate deploys make that sequence longer, but also more visible, which some teams find is an advantage rather than a cost.

Where I land

For most small and medium teams: same repo, same image, separately scaled and separately restarted processes. Build once, deploy the same artifact to both, but don't force every web deploy to bounce long-running workers at the same moment. Treat job payloads as a public contract regardless.

Split into truly separate services when there's a real reason: different teams, very different scaling, or reliability requirements that justify their own release process. "It feels cleaner" isn't quite enough.

But I can see the opposite case clearly. If your workers handle money and your web app ships twenty times a day, a slower, separate release process for workers isn't bureaucracy. It's caution in the right place.

Questions to settle it for your team

Do any jobs run longer than your typical deploy takes?
Are job payloads versioned, and how long can a job sit in the queue?
Do web and job load peak together?
Who gets paged when a worker deploy goes wrong, and is it the same people?

Long-delayed jobs, whose payloads outlive several deploys, are the reason we think so much about this at delay.run: the code that schedules a job and the code that runs it are rarely the same version.