Django-taggit prefetch_related

Question

I'm building a basic time logging app right now and I have a todo model that uses django-taggit. My Todo model looks like this:

class Todo(models.Model):
    project = models.ForeignKey(Project)
    description = models.CharField(max_length=300)
    is_done = models.BooleanField(default=False)
    billable = models.BooleanField(default=True)
    date_completed = models.DateTimeField(blank=True, null=True)
    completed_by = models.ForeignKey(User, blank=True, null=True)
    tags = TaggableManager()

    def __unicode__(self):
        return self.description

I'm trying to get a list of unique tags for all the Todos in a project and I have managed to get this to work using a set comprehension, however for every Todo in the project I have to query the database to get the tags. My set comprehension is:

unique_tags = { tag.name.lower() for todo in project.todo_set.all() for tag in todo.tags.all() }

This works just fine, however for every todo in the project it runs a separate query to grab all the tags. I was wondering if there is any way I can do something similar to prefetch_related in order to avoid these duplicate queries:

unique_tags = { tag.name.lower() for todo in project.todo_set.all().prefetch_related('tags') for tag in todo.tags.all() }

Running the previous code gives me the error:

'tags' does not resolve to a item that supports prefetching - this is an invalid parameter to prefetch_related().

I did see that someone asked a very similar question here: Optimize django query to pull foreign key and django-taggit relationship however it doesn't look like it ever got a definite answer. I was hoping someone could help me out. Thanks!

Answer 1

Slightly hackish soution:

ct = ContentType.objects.get_for_model(Todo)
todo_pks = [each.pk for each in project.todo_set.all()]
tagged_items = TaggedItem.objects.filter(content_type=ct, object_id__in=todo_pks)   #only one db query
unique_tags = set([each.tag for each in tagged_items])

Explanation

I say it is hackish because we had to use TaggedItem and ContentType which taggit uses internally.

Taggit doesn't provide any method for your particular use case. The reason is because it is generic. The intention for taggit is that any instance of any model can be tagged. So, it makes use of ContentType and GenericForeignKey for that.

The models used internally in taggit are Tag and TaggedItem. Model Tag only contains the string representation of the tag. TaggedItem is the model which is used to associate these tags with any object. Since the tags should be associatable with any object, TaggedItem uses model ContentType.

The apis provided by taggit like tags.all(), tags.add() etc internally make use of TaggedItem and filters on this model to give you the tags for a particular instance.

Since, your requirement is to get all the tags for a particular list of objects we had to make use of the internal classes used by taggit.

Answer 2

Taggit now supports prefetch_related directly on tag fields (in version 0.11.0 and later, released 2013-11-25).

This feature was introduced in this pull request. In the test case for it, notice that after prefetching tags using .prefetch_related('tags'), there are 0 additional queries for listing the tags.