Django Tasks with Due: commit the work, recover after a crash

Your Django request saves a document and schedules some background work. The database commits, the process dies before publishing the task, and the document sits there forever waiting for a result.

That gap is easy to miss:

with transaction.atomic(): revision = Revision.objects.create(text=text) transaction.on_commit(lambda: publish_to_broker(revision.pk)) # After the commit, the process might die before this callback runs. 

Waiting for the commit solves the worker-reading-uncommitted-data problem. It doesn't make a database commit and a broker publish atomic. You need somewhere durable to remember the work before the process disappears.

Django's Tasks framework gives you @task, .enqueue(), and task results. A backend supplies storage and execution. Due can be that backend: it stores each invocation in PostgreSQL and uses Celery workers to execute it. Recovery finds committed work even when its broker notification is lost.

This tutorial builds a small word-count report for an immutable document revision. The same pattern fits thumbnail generation, rebuilding derived data, and other work that can safely run again.

Disclosure: this article was prepared and its example verified with OpenAI Codex for the Due project at Gigaverse. The comparison below explains where the project fits; it isn't an independent benchmark.

What Django users gain

The useful part is what happens around the task function:

1. Commit the application change and the task together. With Due, enqueue inside the same transaction.atomic() block and default PostgreSQL database. A rollback removes both. Worker notification happens after commit.
2. Recover a lost notification. PostgreSQL holds the invocation. Beat schedules recovery, which can dispatch work whose original notification never arrived.
3. Recover after a worker dies. Once a claim expires, eligible replay-safe work can be attempted again, within the configured retry budget.
4. Protect the task result from an obsolete worker. Each execution owns a claim token. An older execution cannot overwrite the result after losing that ownership. This protection covers Due's settlement, not arbitrary writes inside your function.
5. Keep Django's task interface. You still use task decorators, result IDs, delayed execution, named queues, and sync or async APIs. Results and attempt history live in PostgreSQL; you don't need a Celery result backend.

The tradeoff is additional infrastructure and policy. Due still needs Celery, a broker, and Beat. This integration requires Python 3.14 and Django 6.0.8+ in the 6.x series. Due is currently alpha. The integration first shipped in django-due 0.8.0; there is no separate django-due-tasks installation.

Build a task that is safe to repeat

We'll assume an existing Django project named project, configured with PostgreSQL as DATABASES["default"]. The web process and workers must use that same database. The commands below use a Unix-like development environment and Redis at localhost:6379.

Install the package and create an app:

python -m pip install 'django-due[redis]==0.8.0' python manage.py startapp reports 

Create a model for a document revision:

# reports/models.py from django.db import models class Revision(models.Model): text = models.TextField() word_count = models.PositiveIntegerField(null=True) 

Treat text as immutable after creation: editing a document creates a new revision. The model alone doesn't enforce that rule, so enforce it in your application's write paths.

Now define the task:

# reports/tasks.py from django.tasks import task from .models import Revision @task def count_words(revision_id: int) -> int: revision = Revision.objects.get(pk=revision_id) count = len(revision.text.split()) Revision.objects.filter(pk=revision_id).update(word_count=count) return count 

If the worker dies after the update but before storing the task result, another attempt writes the same count. It doesn't increment a counter or create a second report row. That makes this operation safe to repeat as long as the source text stays immutable.

For an external API, you need that provider's idempotency mechanism or another application-level guarantee. Adding a function to a configuration list cannot make charging a card twice safe.

Configure the backend and recovery

Add these apps to your existing INSTALLED_APPS, and add the settings below. If reports is already listed, replace that entry with its AppConfig rather than registering it twice.

# project/settings.py (additions) INSTALLED_APPS += [ "due.django", "due.django.tasks", "reports.apps.ReportsConfig", ] USE_TZ = True CELERY_BROKER_URL = "redis://localhost:6379/0" TASKS = { "default": { "BACKEND": "due.django.tasks.backend.DueBackend", "QUEUES": ["default"], "OPTIONS": { "replay_safe_tasks": ["reports.tasks.count_words"], }, }, } 

The allowlist is an explicit declaration that you've checked the function's replay behavior. Keep task import paths available while queued work and retained results still refer to them.

Register the runtime during Django startup:

# reports/apps.py from django.apps import AppConfig class ReportsConfig(AppConfig): name = "reports" def ready(self): from due.django.runtime.registry import configure_durable_work from due.django.tasks.runtime import django_tasks_registration configure_durable_work([ django_tasks_registration( recovery_queue="default", cadence_seconds=5, ), ]) 

Five seconds makes recovery easy to observe in this demo. The normal default is 30 seconds; tune the cadence for your latency and database-load requirements. If you already use native Due capabilities, include them in this same manifest. Call configure_durable_work() once with the complete set.

Configure Celery to load Django's settings:

# project/celery.py import os from celery import Celery os.environ.setdefault("DJANGO_SETTINGS_MODULE", "project.settings") app = Celery("project") app.config_from_object("django.conf:settings", namespace="CELERY") app.autodiscover_tasks() 

Make the Celery application available during normal Django startup too:

# project/__init__.py from .celery import app as celery_app __all__ = ("celery_app",) 

Create the tables:

python manage.py makemigrations reports python manage.py migrate python manage.py check 

With PostgreSQL and Redis running, start these in separate terminals:

celery -A project.celery:app worker -Q default --loglevel=INFO 

celery -A project.celery:app beat --loglevel=INFO 

Keep Beat running even if immediate tasks appear to work without it. Recovery and delayed dispatch depend on it. Run one Beat scheduler for this schedule, and ensure workers consume every queue you configure.

Save the revision and enqueue in one transaction

Put the transaction in a small service function:

# reports/services.py from django.db import transaction from .models import Revision from .tasks import count_words def create_revision(text: str) -> tuple[int, str]: with transaction.atomic(): revision = Revision.objects.create(text=text) result = count_words.enqueue(revision.pk) return revision.pk, result.id 

For this backend, call .enqueue() inside the transaction. Wrapping it in transaction.on_commit() would move the task insert outside the transaction and reopen the gap we're trying to close. This is backend-specific advice, not a rule for every Django Tasks implementation.

Open python manage.py shell and submit some work:

from reports.services import create_revision revision_id, result_id = create_revision("Django tasks can recover") print(revision_id, result_id) 

After the worker runs, fetch the result using that saved ID:

from django.tasks import TaskResultStatus from reports.models import Revision from reports.tasks import count_words result = count_words.get_result(result_id) result.refresh() if result.status == TaskResultStatus.SUCCESSFUL: print(result.return_value) # 4  print(Revision.objects.get(pk=revision_id).word_count) # 4 

Enqueue returns a result snapshot, not a completed calculation. Keep the result ID in your application if you need to retrieve it from another request. Don't wait in your HTTP request for the worker to finish.

Prove that a lost notification recovers

Here's a small local test worth trying before you need it. Due exposes hold_nudges() to suppress immediate worker notifications while leaving the database admission intact.

With the worker and Beat still running, use the Django shell:

import time from django.tasks import TaskResultStatus from due.django.testing import hold_nudges from reports.services import create_revision from reports.tasks import count_words with hold_nudges(): revision_id, result_id = create_revision("A lost notification still recovers") result = count_words.get_result(result_id) deadline = time.monotonic() + 60 while not result.is_finished and time.monotonic() < deadline: time.sleep(0.5) result.refresh() assert result.status == TaskResultStatus.SUCCESSFUL, result.status assert result.return_value == 5 

The immediate notification never went out. Beat's recovery pass found the committed work and dispatched it. The test's 60-second deadline is a local check, not a service-level guarantee.

If it times out, check Beat, the worker's queue, the database connection, and broker connectivity. Recovery still needs working infrastructure to make progress.

Delays, async code, and the limits you should know

Use the same Django API for a delayed invocation:

from datetime import timedelta from django.utils import timezone later = count_words.using( run_after=timezone.now() + timedelta(minutes=10), ).enqueue(revision_id) 

run_after is the earliest eligible start, not an exact execution time. The recovery cadence and queue load affect when it actually runs.

Coroutine task bodies are supported. Async callers can use await count_words.aenqueue(revision_id) and await result.arefresh(). For an async view that must save data and enqueue atomically, put both operations in one synchronous transactional function and call that function through sync_to_async.

A few details matter:

• Every enqueue is a new invocation. Identical arguments don't deduplicate requests. Replay safety handles repeated attempts of work; business-level deduplication is still your responsibility.
• Your function might run more than once. The claim token protects Due's result settlement. It doesn't stop an obsolete worker from making an external API call or protect arbitrary task-body database writes.
• Retries are bounded. Defaults allow three additional attempts, a 60-second retry delay, and a 300-second lease. Default worker limits are 50 seconds soft and 60 seconds hard. For longer work, configure soft < hard < lease.
• Results have a retention policy. Successful invocations are pruned after seven days by default; failed invocations remain for diagnosis. Copy business records you need to keep into your own models.
• Payloads must be JSON-compatible. Pass small identifiers instead of model objects, credentials, or large files. Nonzero task priority and the older django_tasks backport are unsupported.

Getting the replay behavior right is the bit that's easy to overlook. An allowlist is a useful review point, but it doesn't do that thinking for you. Take a seperate look at every irreversible side effect before enabling retries.

When should you choose Due over the alternatives?

Choose Due when your background work is becoming part of the business state of your Django application. Publishing the current document, keeping a catalog in sync, completing a provider order, and cleaning up abandoned outputs all need more than knowing whether a function returned.

You can start with the Django Tasks backend in this tutorial. As requirements grow, move the operations that need stronger semantics into native Due capabilities, alongside your ordinary tasks. They share the work-table runtime, registration, recovery, and operational tools. You can make that move one operation at a time.

The word-count task gets durable admission and recovery immediately. Here is how that same application can gain more.

1. Your report now has a “current published version”

Suppose revision A starts rendering a PDF. The user edits the document, revision B finishes, then A's slower worker returns. If both tasks assign document.published_key, the old report might become current again.

This is a strong reason to use Due's native ReplaySafeEffect. Separate the external work from the product update:

perform_effect: write an immutable output for the captured revision finalize_success: publish its pointer, under Due's ownership + revision checks 

Due guards the local completion callback. An obsolete worker cannot publish its old pointer through that boundary. Retries of one target can also reuse provider_idempotency_key() when the provider honors it.

The gain: you can let workers overlap without building another version-check and completion protocol around every integration. Use immutable output identities so an old worker cannot overwrite the new file's bytes. The generic @task adapter protects its result row; this product-level protection comes from moving the pointer update into native settlement. The document publication example shows the complete split.

2. The report also updates search or an external catalog

Now the external destination has a mutable record. A late write of A might overwrite B at the provider even though Django correctly rejects A's local completion.

Use Projection to express the current desired value and retain the evidence needed to repair an obsolete write. Recovery can write the current value again. The catalog example exercises exactly this ordering: B finishes, A lands late, and recovery restores B.

The gain: synchronization becomes a maintained obligation. You reuse the same revision and recovery machinery for search entries, catalog records, or other derived views instead of writing a reconciliation loop for each one. This provides eventual repair under your provider and recovery policy; provider-side conditional writes are still preferable when available.

3. A timeout might mean the provider already accepted the order

Suppose publishing a report can also order a physical copy. The printer accepts the order, but the response disappears. Repeating the call might order a second copy.

Use ObservedExternalOperation when the provider offers a correlated receipt or lookup. Due records the attempt before the call, retains an uncertain outcome as UNKNOWN, and can perform read-only observations. Once positive evidence arrives, local completion can recover from that evidence without submitting another order.

The gain: you can handle “we don't know yet” explicitly and keep it visible to operators. A retry budget or expired lease alone never establishes that the first order didn't happen. The print-order example demonstrates this with an independently committing simulated provider. Prefer provider-enforced idempotency where it solves the problem; if reliable evidence never arrives, an uncertain operation can still require a human decision.

4. Completing one obligation creates another

After publishing, you might owe indexing, a follow-up export, or deletion of an obsolete object. A worker can die between recording success and scheduling that next step.

Native settlement can update the product and admit required follow-up work in the same transaction. Dependent wakes let existing work reread changed facts. Discovery can derive missing work from an explicitly configured selector over durable product facts. IndependentCompensation lets you record cleanup intent before an external call, so cleanup can survive the worker that created the output.

The gain: required follow-ups and cleanup have durable identities and recovery paths. You avoid spreading “remember to enqueue this too” callbacks across views, signals, and workers. For deadlines that move or become unnecessary, ScheduledAction adds explicit supersession and cancellation semantics; the task adapter's run_after alone is only delayed invocation.

These are additional native capability choices, not features that appear automatically on every decorated task.

5. You need to prove the recovery works in your application

There is another adoption benefit: Due integrates with pytest-obligation through django-due[testing].

You bind a production capability to application examples: its real producer, provider boundary, and an independent observation of the business result. The capability's mode and policies determine which proofs are required. The suite exercises relevant failure histories through the actual admission, execution, and recovery paths.

That lets you check questions such as:

• Does a dropped notification still produce the report users can read?
• If the effect succeeds but settlement fails, does replay leave one logical result?
• Can an old revision replace the current published pointer?
• Does an uncertain external operation stay blocked from blind resend?
• Does cleanup remain owed after its producer stops?
• Does one broken identity prevent unrelated work from progressing?

The gain: reusable failure tests grow with the semantics you adopt. You supply the application's meaning of success instead of copying the same crash and stale-worker scenarios into each integration. The conformance guide and executable enrollment show how. These proofs require real application bindings; they cannot certify a provider guarantee you have only assumed.

Where the adoption pays off

For a team already operating Django, PostgreSQL, and Celery, this is a practical growth path: start with a task; adopt stronger semantics only where a business requirement earns them. Your Django rows can remain the authority for product decisions, and you can test each operation's failure contract as it grows.

django-tasks-db is a good smaller deployment when an ORM-backed queue and its worker are enough. RQ, Huey, and Celery remain good choices for ordinary jobs. Their existing retry and delivery features are useful. The reason to choose Due is the amount of application coordination you can stop rebuilding as those jobs acquire changing intent, uncertain outcomes, and required follow-ups.

You still pay for concrete work models, provider policies, and recovery infrastructure. Due is alpha, and this article makes no performance comparison or exactly-once execution claim. Its advantage is strongest when several operations need these semantics and can share their implementation and tests.

The setup guide covers the task backend, and the capability guide is the next step when one of the problems above appears.