Caching queryset choices for ModelChoiceField or ModelMultipleChoiceField in a Django form

Question

When using ModelChoiceField or ModelMultipleChoiceField in a Django form, is there a way to pass in a cached set of choices? Currently, if I specify the choices via the queryset parameter, it results in a database hit.

I'd like to cache these choices using memcached and prevent unnecessary hits to the database when displaying a form with such a field.

Answer 1

The reason that ModelChoiceField in particular creates a hit when generating choices - regardless of whether the QuerySet has been populated previously - lies in this line

for obj in self.queryset.all(): 

in django.forms.models.ModelChoiceIterator. As the Django documentation on caching of QuerySets highlights,

callable attributes cause DB lookups every time.

So I'd prefer to just use

for obj in self.queryset:

even though I'm not 100% sure about all implications of this (I do know I do not have big plans with the queryset afterwards, so I think I'm fine without the copy .all() creates). I'm tempted to change this in the source code, but since I'm going to forget about it at the next install (and it's bad style to begin with) I ended up writing my custom ModelChoiceField:

class MyModelChoiceIterator(forms.models.ModelChoiceIterator):
    """note that only line with # *** in it is actually changed"""
    def __init__(self, field):
        forms.models.ModelChoiceIterator.__init__(self, field)

    def __iter__(self):
        if self.field.empty_label is not None:
            yield (u"", self.field.empty_label)
        if self.field.cache_choices:
            if self.field.choice_cache is None:
                self.field.choice_cache = [
                    self.choice(obj) for obj in self.queryset.all()
                ]
            for choice in self.field.choice_cache:
                yield choice
        else:
            for obj in self.queryset: # ***
                yield self.choice(obj)


class MyModelChoiceField(forms.ModelChoiceField):
    """only purpose of this class is to call another ModelChoiceIterator"""
    def __init__(*args, **kwargs):
        forms.ModelChoiceField.__init__(*args, **kwargs)

    def _get_choices(self):
        if hasattr(self, '_choices'):
            return self._choices

        return MyModelChoiceIterator(self)

    choices = property(_get_choices, forms.ModelChoiceField._set_choices)

This does not solve the general problem of database caching, but since you're asking about ModelChoiceField in particular and that's exactly what got me thinking about that caching in the first place, thought this might help.

Answer 2

You can override "all" method in QuerySet something like

from django.db import models
class AllMethodCachingQueryset(models.query.QuerySet):
    def all(self, get_from_cache=True):
        if get_from_cache:
            return self
        else:
            return self._clone()


class AllMethodCachingManager(models.Manager):
    def get_query_set(self):
        return AllMethodCachingQueryset(self.model, using=self._db)


class YourModel(models.Model):
    foo = models.ForeignKey(AnotherModel)

    cache_all_method = AllMethodCachingManager()

And then change queryset of field before form using (for exmple when you use formsets)

form_class.base_fields['foo'].queryset = YourModel.cache_all_method.all()