Reload choices dynamically when using MultipleChoiceFilter

Question

I am trying to construct a MultipleChoiceFilter where the choices are the set of possible dates that exist on a related model (DatedResource).

Here is what I am working with so far...

resource_date = filters.MultipleChoiceFilter(
    field_name='dated_resource__date',
    choices=[
        (d, d.strftime('%Y-%m-%d')) for d in
        sorted(resource_models.DatedResource.objects.all().values_list('date', flat=True).distinct())
    ],
    label="Resource Date"
)

When this is displayed in a html view...

This works fine at first, however if I create new DatedResource objects with new distinct date values I need to re-launch my webserver in order for them to get picked up as a valid choice in this filter. I believe this is because the choices list is evaluated once when the webserver starts up, not every time my page loads.

Is there any way to get around this? Maybe through some creative use of a ModelMultipleChoiceFilter?

Thanks!

Edit: I tried some simple ModelMultipleChoice usage, but hitting some issues.

resource_date = filters.ModelMultipleChoiceFilter(
    field_name='dated_resource__date',
    queryset=resource_models.DatedResource.objects.all().values_list('date', flat=True).order_by('date').distinct(),
    label="Resource Date"
)

The HTML form is showing up just fine, however the choices are not accepted values to the filter. I get "2019-04-03" is not a valid value. validation errors, I am assuming because this filter is expecting datetime.date objects. I thought about using the coerce parameter, however those are not accepted in ModelMultipleChoice filters.

Per dirkgroten's comment, I tried to use what was suggested in the linked question. This ends up being something like

resource_date = filters.ModelMultipleChoiceFilter(
    field_name='dated_resource__date',
    to_field_name='date',
    queryset=resource_models.DatedResource.objects.all(),
    label="Resource Date"
)

This also isnt what I want, as the HTML now form is now a) displaying the str representation of each DatedResource, instead of the DatedResource.date field and b) they are not unique (ex if I have two DatedResource objects with the same date, both of their str representations appear in the list. This also isnt sustainable because I have 200k+ DatedResources, and the page hangs when attempting to load them all (as compared to the values_list filter, which is able to pull all distinct dates out in seconds.

Answer 1

One of the easy solutions will be overriding the __init__() method of the filterset class.

from django_filters import filters, filterset


class FooFilter(filterset.FilterSet):
    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)
        try:
            self.filters['user'].extra['choices'] = [(d, d.strftime('%Y-%m-%d')) for d in sorted(
                resource_models.DatedResource.objects.all().values_list('date', flat=True).distinct())]
        except (KeyError, AttributeError):
            pass

    resource_date = filters.MultipleChoiceFilter(field_name='dated_resource__date', choices=[], label="Resource Date")

NOTE: provide choices=[] in your field definition of filterset class

---

Results

I tested and verified this solution with following dependencies
1. Python 3.6
2. Django 2.1
3. DRF 3.8.2
4. django-filter 2.0.0

I used following code to reproduce the behaviour

# models.py
from django.db import models


class Musician(models.Model):
    name = models.CharField(max_length=50)

    def __str__(self):
        return f'{self.name}'


class Album(models.Model):
    artist = models.ForeignKey(Musician, on_delete=models.CASCADE)
    name = models.CharField(max_length=100)
    release_date = models.DateField()

    def __str__(self):
        return f'{self.name} : {self.artist}'


# serializers.py
from rest_framework import serializers


class AlbumSerializer(serializers.ModelSerializer):
    artist = serializers.StringRelatedField()

    class Meta:
        fields = '__all__'
        model = Album


# filters.py
from django_filters import rest_framework as filters


class AlbumFilter(filters.FilterSet):
    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)
        self.filters['release_date'].extra['choices'] = self.get_album_filter_choices()

    def get_album_filter_choices(self):
        release_date_list = Album.objects.values_list('release_date', flat=True).distinct()
        return [(date, date) for date in release_date_list]

    release_date = filters.MultipleChoiceFilter(choices=[])

    class Meta:
        model = Album
        fields = ('release_date',)


# views.py
from rest_framework.viewsets import ModelViewSet
from django_filters import rest_framework as filters


class AlbumViewset(ModelViewSet):
    serializer_class = AlbumSerializer
    queryset = Album.objects.all()
    filter_backends = (filters.DjangoFilterBackend,)
    filter_class = AlbumFilter

Here I've used the django-filter with DRF.

Now, I populated some data through Django Admin console. After that, the album api become as below,

and I got the release_date as


Then, I added new entry through Django admin -- (Screenshot) and I refresh the DRF API endpoint and the possible choices became as below,

Answer 2

I have looked into your problem and I have following suggestions

The Problem

You have got the problem right. Choices for your MultipleChoiceFilter are calculated statically whenever you run server.Thats why they don't get updated dynamically whenever you insert new instance in DatedResource.

To get it working correctly, you have to provide choices dynamically to MultipleChoiceFilter. I searched in documentation but did not find anything regarding this. So here is my solution.

The solution

You have to extend MultipleChoiceFilter and create your own filter class. I have created this and here it is.

from typing import Callable
from django_filters.conf import settings
import django_filters


class LazyMultipleChoiceFilter(django_filters.MultipleChoiceFilter):
    def get_field_choices(self):
        choices = self.extra.get('choices', [])
        if isinstance(choices, Callable):
            choices = choices()
        return choices

    @property
    def field(self):
        if not hasattr(self, '_field'):
            field_kwargs = self.extra.copy()

            if settings.DISABLE_HELP_TEXT:
                field_kwargs.pop('help_text', None)

            field_kwargs.update(choices=self.get_field_choices())

            self._field = self.field_class(label=self.label, **field_kwargs)
        return self._field

Now you can use this class as replacement and pass choices as lambda function like this.

resource_date = LazyMultipleChoiceFilter(
    field_name='dated_resource__date',
    choices=lambda: [
        (d, d.strftime('%Y-%m-%d')) for d in
        sorted(resource_models.DatedResource.objects.all().values_list('date', flat=True).distinct())
    ],
    label="Resource Date"
)

Whenever instance of filter will be created choices will be updated dynamically. You can also pass choices statically (without lambda function) to this field if want default behavior.