Deployment Rollback Gets Complicated When the Database Has Moved On

“Just roll it back.”

A beautiful sentence. Four words. Very relaxing.

Then someone asks whether the previous application still understands the new database schema, and suddenly everyone's tea is going cold.

Rolling back an application release is not necessarily the same as restoring the whole system to an earlier state. Writes, migrations, jobs and external effects may have continued after deployment.

Let's work through an illustrative Django application. No customer incident or measured outcome is being claimed here.

The innocent-looking change

Suppose a customer table has a full_name column. A new release introduces separate given_name and family_name fields.

A direct replacement sounds neat:

1. Remove full_name.
2. Add the new fields.
3. Deploy the new application.

But an older application instance may still read full_name. A worker might also be processing jobs created by the previous version.

Now your deployment is a compatibility problem, not just a file replacement.

And real names do not always split cleanly into two fields. The product assumption needs review before the migration gets anywhere near a database.

Define a compatibility window

One approach is to separate expansion from removal:

Stage	Database shape	Application behaviour
Expand	Keep the old field; add new fields	Existing release still operates
Transition	Both representations exist	New code uses a defined synchronisation and fallback strategy
Verify	Reconciliation and behaviour checks pass	All active services and jobs support the new representation
Contract	Remove the old field when safe	Returning to the old release may no longer be supported

This is a proposed design pattern, not a promise that every field migration can be handled this way.

The transition stage deserves most of the thinking. What happens if one write succeeds and another fails? Which field is authoritative? How will old and new application instances coexist? Can a backfill be resumed without corrupting later updates?

“Write both” is a starting thought, not a complete consistency design.

A Django migration isn't automatically reversible

Framework support is useful, but it does not recover information that was discarded.

Django supports migration dependencies and reverse operations, while some operations or custom data migrations can be irreversible. Review what your specific migration does rather than assuming a reverse command makes the business change safe. Django migration documentation

Before release, I would want the team to answer:

• Can the previous application version run against the expanded schema?
• Does the backfill preserve the original values needed for recovery?
• Can old workers consume new jobs, or do queues need version-aware handling?
• Which step ends the compatibility window?

That last question is the one that tends to hide behind “we'll tidy it later.” Later has an impressive backlog.

Separate the rollback decisions

There are several different actions people may mean:

Application rollback: route traffic to a previous application artefact.

Configuration rollback: restore previously reviewed settings.

Migration reversal: execute a specific reverse schema or data operation.

Database recovery: restore data from a backup or recovery point.

These actions have different consequences. Database recovery can lose legitimate writes made after the recovery point and may disagree with external systems that already acted on those writes.

A payment provider is unlikely to say, “Ah, you rolled back your database? No worries, we'll forget that transaction too.”

Rehearse a journey, not just a deployment command

For a controlled staging exercise:

1. Start with the previous release and synthetic data.
2. Apply the expansion step.
3. Exercise old and new application behaviour under the planned coexistence model.
4. Create representative new records and queued jobs.
5. Return application traffic to the earlier version.
6. Check reads, writes, authorisation and job processing.
7. Record exactly which actions are safe and which require a different recovery procedure.

Use acceptance criteria defined before the exercise. “No obvious errors” is a mood, not a test case.

Google's discussion of reliable product launches is useful background on planning and managing launch risk. The specific compatibility matrix above is an illustrative design to adapt to your own system.

The useful release question

Instead of asking only, “How do we deploy?” ask:

“After each step, which earlier state can we safely return to—and what evidence supports that?”

That gives the Technical PM a real coordination task: align application engineers, database owners, operations and product decision-makers around the same recovery options.

Which dependency makes rollback hardest in your environment: schema, background jobs, configuration, third-party APIs or data written during the release?

Draft prepared with AI support. The migration scenario and release plan are illustrative, not a claimed production implementation.

#DevOps #Architecture #Database #Django