Django multiple annotate slows down the query

Question

I have been debugging with django debug_toolbar, if I use more than one annotate in query then it takes a lot of time for Django to fetch the query results.

class Project_First(models.Model):
   project_first_results_M2M = models.ManyToManyField(Project_First_Results)

class Project_Second(models.Model):
   project_second_results_M2M = models.ManyToManyField(Project_Second_Results)

class Project(models.Model):
    project_first_M2M = models.ManyToManyField(Project_First)
    project_second_M2M = models.ManyToManyField(Project_Second)

• I m trying to count all the objects present in project_first_results_M2M of all the project_first_M2M objects.

ie, let's suppose, project_first_M2M has 3 objects of Project_First and I want to count all the total project_first_results_M2M objects present in all 3 of them.

Project.objects.all().annotate(first_res_count=Count('project_first_M2M__project_first_results_M2M',distinct=True))

• Above Query works fine and would take 80ms to fetch the results. But problem occurs when i try to add an additional annotate to query.

Project.objects.all().annotate(first_res_count=Count('project_first_M2M__project_first_results_M2M',distinct=True)).annotate(second_res_count=Count('project_second_M2M__project_second_results_M2M',distinct=True))

• This would take almost 4000ms to fetch the results.

Both project_second_M2M and project_first_M2M contains the same fields and the same number of objects. I even tried in vice-versa condition to the above query and query slows down only when I add additional annotate.

• Is there any fast and alternate solution to achieve the same in much efficient way? maybe with raw sql queries.
• I want to count all the objects of project_first_results_M2M of all project_first_M2M objects within each Project object and similarly for project_second_results_M2M

Answer 1

Probably you can use prefetch related:

Project.objects.prefetch_related('project_first_M2M__project_first_results_M2M', 'project_second_M2M__project_second_results_M2M').annotate(first_res_count=Count('project_first_M2M__project_first_results_M2M',distinct=True)).annotate(second_res_count=Count('project_second_M2M__project_second_results_M2M',distinct=True))