Bikramjeet Singh

Posted on Jul 2, 2021

Soft Deletes in Django

#python #django

What is Soft Deletion?

Sometimes, you want to delete something from your database without actually deleting it. You want to preserve it, in case you ever decide to restore it in the future, or to use it for some other purpose, such as analytics … but it should still appear as if it were actually deleted, not show up in unexpected places in your application. This is called the 'soft delete' pattern.

Instead of actually deleting a record from your database, which is irreversible, you simply 'mark' it as deleted, usually in another column in your table. Now the challenge remains, how to prevent soft-deleted records from 'leaking' into your application.

In this article, we will learn how to implement soft deletion in Django by taking the example of a simple note-taking app backend.

Soft Delete Model

We will start by creating a base SoftDeleteModel that can be inherited by the rest of our models.

class SoftDeleteModel(models.Model):

    is_deleted = models.BooleanField(default=False)

    def soft_delete(self):
        self.is_deleted = True
        self.save()

    def restore(self):
        self.is_deleted = False
        self.save()

    class Meta:
        abstract = True

Note that we have marked this model as abstract = True. This means that Django will not create a database table for it.

Now, we can create our models as subclasses of SoftDeleteModel to grant them the ability to be soft-deleted. Let's take the example of a Note model.

class Note(SoftDeleteModel):

    user = models.ForeignKey(User, on_delete=models.CASCADE, related_name="notes")
    title = models.CharField(max_length=32)
    content = models.CharField(max_length=255)

We can query our database filtering by is_deleted in order to exclude records that have been soft-deleted.

Note.objects.filter(is_deleted=False)

Trying It Out

Let's try playing around a bit with the code we've written so far. First, open up the Django shell by typing python manage.py shell in your terminal.

Import the models required:

from django.contrib.auth.models import User
from tutorialapp.models import Note

Since each note is foreign-keyed to a user, our first step is to create a User object:

john = User.objects.create_user('john', 'lennon@thebeatles.com', 'johnpassword')

Now we can create a couple of notes:

my_note = Note.objects.create(user=john, title="Strawberry Fields", content="Strawberry Fields Forever")
another_note = Note.objects.create(user=john, title="Here Comes The Sun", content="It's All Right")

You are now ready to soft delete and restore notes:

my_note.soft_delete()
my_note.restore()

You can query for all notes, whether they have been soft deleted or not:

Note.objects.all()

You can also filter only for notes that have not been soft deleted:

Note.objects.filter(is_deleted=False)

Soft Delete Manager

While our code is functionally correct, the disadvantage is that we will have to remember to filter by is_deleted=False each time we write a query.

We can improve upon this behaviour by creating a custom model manager to apply the filter automatically, behind the scenes. If you've used Django in the past, you might be familiar with statements that look like this: MyModel.objects.all(). The .objects part in the statement is the manager. Managers act as the 'bridge' between your Django code and the database. They control the database operations performed on the tables that they 'manage'.

Our new custom manager can be defined as:

class SoftDeleteManager(models.Manager):

    def get_queryset(self):
        return super().get_queryset().filter(is_deleted=False)

We will need to add the new manager to our SoftDeleteModel base class:

class SoftDeleteModel(models.Model):

    is_deleted = models.BooleanField(default=False)
    objects = models.Manager()
    undeleted_objects = SoftDeleteManager()

    def soft_delete(self):
        self.is_deleted = True
        self.save()

    def restore(self):
        self.is_deleted = False
        self.save()

    class Meta:
        abstract = True

Note that, since we have added a custom manager to our class, we are required to explicitly add the default objects manager as well.

Then, we can simply rewrite our query as,

Note.undeleted_objects.all()

to get a QuerySet of undeleted notes.

We can still use

Note.objects.all()

to get the full list of notes, including those that have been soft-deleted.

Handling Foreign Key Relationships

Now, what if you have multiple users, and you want to fetch all the notes belonging to a specific user? The naive approach is to simply write a query filtering against the user:

Note.objects.filter(user=john, is_deleted=False)

However, a more elegant and readable solution is to make use of the reverse relationships Django provides for this purpose.

john.notes.all()

Try soft deleting some of your notes and running this query. Do you notice something unusual about the results?

We find that the resultant QuerySet contains records that we had soft deleted. This is because Django is using the default objects manager to perform the reverse lookup, which, as you may recall, does not filter out soft deleted records.

How can we force Django to use our custom SoftDeleteManager to perform reverse lookups? We can simply replace the default objects manager in our SoftDeleteModel:

class SoftDeleteModel(models.Model):

    is_deleted = models.BooleanField(default=False)
    objects = SoftDeleteManager()
    all_objects = models.Manager()

    def soft_delete(self):
        self.is_deleted = True
        self.save()

    def restore(self):
        self.is_deleted = False
        self.save()

    class Meta:
        abstract = True

Now, the objects manager will automatically filter out soft-deleted objects when querying our database, ensuring they never leak into our application under any circumstances! If we want to, we can still include soft deleted objects in our queries by making use of the all_objects manager.

Note.all_objects.all()

Storing More Information

We've already got a pretty solid soft deletion framework in our Django app, but we can make one final improvement. Knowing whether a record is soft deleted or not is useful, but another piece of information that would be nice to know is when the record was soft deleted. For this, we can add a new a attribute deleted_at to our SoftDeleteModel:

deleted_at = models.DateTimeField(null=True, default=None)

We can also update our soft_delete and restore methods as follows:

def soft_delete(self):
    self.deleted_at = timezone.now()
    self.save()

def restore(self):
    self.deleted_at = None
    self.save()

For undeleted records, the value of deleted_at will be null, while for soft deleted records, it will contain the date and time at which it was deleted.

The addition of the new deleted_at attribute makes our previously created is_deleted attribute redundant, because we can simply perform a null-check on deleted_at to find out whether the record is soft deleted or not.

Our rewritten SoftDeleteModel now looks like this:

class SoftDeleteModel(models.Model):

    deleted_at = models.DateTimeField(null=True, default=None)
    objects = SoftDeleteManager()
    all_objects = models.Manager()

    def soft_delete(self):
        self.deleted_at = timezone.now()
        self.save()

    def restore(self):
        self.deleted_at = None
        self.save()

    class Meta:
        abstract = True

And our rewritten SoftDeleteManager looks like this:

class SoftDeleteManager(models.Manager):

    def get_queryset(self):
        return super().get_queryset().filter(deleted_at__isnull=True)

In addition to our previous capabilities, we can now also see the exact date and time at which our record was soft deleted:

my_note.deleted_at

In Conclusion

Soft deletion is a powerful software pattern that comes with several advantages, including better data preservation and restoration, history tracking, and faster recovery from failures.

At the same time, it should be used with care. Sensitive and personal data including payment-related information should always be hard-deleted. Users should always have the option to have their data permanently deleted, if they wish. Several jurisdictions around the world have information privacy and data protection laws that include the 'right to be forgotten', such as the European Union's GDPR. It might also make sense to periodically delete or archive data that is very old, to avoid eating up excess database storage space.

If you would like to view the complete source code for the example used in this tutorial, it is available on GitHub.

Image by Makalu from Pixabay

References and Further Reading

How to use managers in Django
What Are Soft Deletes, and How Are They Implemented?
Soft Deletion in Django

Top comments (6)

Joost Helberg • Jul 4 '21

Nice article and an important subject. A more sophisticated pattern is to implement a valid timestamp-range to allow a certain object to be valid for some timestamps and not valid for others. Implementation is similar to the above, with different operators and a lot more functionality.

Bikramjeet Singh • Jul 6 '21

Thanks Joost! The valid timestamp range pattern is an interesting extension of the soft deletion functionality. I suppose it will give us the capability to view the complete history if the object has been deleted and restored multiple times, am I right?