How to soft delete many to many relation with Django

Question

In my Django project, all entities deleted by the user must be soft deleted by setting the current datetime to deleted_at property. My model looks like this: Trip <-> TripDestination <-> Destination (many to many relation). In other words, a Trip can have multiple destinations.

When I delete a Trip, the SoftDeleteManager filters out all the deleted trip. However, if I request all the destinations of a trip (using get_object_or_404(Trip, pk = id)), I also get the deleted ones (i.e. TripDestination models with deleted_at == null OR deleted_at != null). I really don't understand why since all my models inherit from LifeTimeTracking and are using the SoftDeleteManager.

Can someone please help me to understand why the SoftDeleteManager isn't working for n:m relation?

class SoftDeleteManager(models.Manager):
    def get_query_set(self):
        query_set = super(SoftDeleteManager, self).get_query_set()
        return query_set.filter(deleted_at__isnull = True)

class LifeTimeTrackingModel(models.Model):
    created_at = models.DateTimeField(auto_now_add = True)
    updated_at = models.DateTimeField(auto_now = True)
    deleted_at = models.DateTimeField(null = True)

    objects = SoftDeleteManager()
    all_objects = models.Manager()

    class Meta:
        abstract = True

class Destination(LifeTimeTrackingModel):
    city_name = models.CharField(max_length = 45)

class Trip(LifeTimeTrackingModel):
    name = models.CharField(max_length = 250)
    destinations = models.ManyToManyField(Destination, through = 'TripDestination')

class TripDestination(LifeTimeTrackingModel):
    trip = models.ForeignKey(Trip)
    destination = models.ForeignKey(Destination)

Resolution I filed the bug 17746 in Django Bug DB. Thanks to Caspar for his help on this.

Answer 1

It looks like this behaviour comes from the ManyToManyField choosing to use its own manager, which the Related objects reference mentions, because when I try making up some of my own instances & try soft-deleting them using your model code (via the manage.py shell) everything works as intended.

Unfortunately it doesn't mention how you can override the model manager. I spent about 15 minutes searching through the ManyToManyField source but haven't tracked down where it instantiates its manager (looking in django/db/models/fields/related.py).

To get the behaviour you are after, you should specify use_for_related_fields = True on your SoftDeleteManager class as specified by the documentation on controlling automatic managers:

class SoftDeleteManager(models.Manager):
    use_for_related_fields = True

    def get_query_set(self):
        query_set = super(SoftDeleteManager, self).get_query_set()
        return query_set.filter(deleted_at__isnull = True)

This works as expected: I'm able to define a Trip with 2 Destinations, each through a TripDestination, and if I set a Destination's deleted_at value to datetime.datetime.now() then that Destination no longer appears in the list given by mytrip.destinations.all(), which is what you are after near as I can tell.

However, the docs also specifically say do not filter the query set by overriding get_query_set() on a manager used for related fields, so if you run into problems later, bear this in mind as a possible cause.