Django filter with OR condition using dict argument

Question

I have a function on my Django app where I perform some Queryset actions and set it's result to Memcache. Since it is a function it has to be of general usage. So in order to make it reusable I pass a dict as parameter for filter and exclude actions. This is the function:

def cached_query(key, model, my_filter=None, exclude=None, order_by=None, sliced=50):
    """
    :param key: string used as key reference to store on Memcached
    :param model: model reference on which 'filter' will be called
    :param my_filter: dictionary containing the filter parameters (eg.: {'title': 'foo', 'category': 'bar'}
    :param sliced: integer limit of results from the query. The lower the better, since for some reason Django Memcached
        won't store thousands of entries in memory
    :param exclude: dictionary containing the exclude parameters (eg.: {'title': 'foo', 'category': 'bar'}
    :param order_by: tuple containing the list of fields upon which the model will be ordered.
    :return: list of models. Not a QuerySet, since it was sliced.
    """
    result = cache.get(key, None)
    if not result:
        if my_filter:
            result = model.objects.filter(**my_filter)
        if exclude:
            result = result.exclude(**exclude)
        if order_by:
            result = result.order_by(*order_by)
        else:
            result = model.objects.all()
        result = result[:sliced]
        cache.set(key, result, cache_timeout)
    return result

It works pretty fine if I filter the queryset with a simple dict like {'title': 'foo', 'name': 'bar'}. However that won't be always the case. I need to perform filters using the django.db.models.Q utility for more complex queries which require OR condition.

So, how can I pass these parameters as a dictionary on a filter. Is there any approach for this?

Answer 1

You can use the bitwise | operator.

my_filter = Q()

# Or the Q object with the ones remaining in the list
my_or_filters = {'some_field__gte':3.5, 'another_field':'Dick Perch'}

for item in my_or_filters:
    my_filter |= Q(**{item:my_or_filters[item]})

model.objects.filter(my_filter)
# unpacks to model.objects.filter(Q(some_field__gte=3.5) | Q(another_field='Dick Perch'))

With this in mind, you may want to load all your queries stored in my_filter into Q objects. Then, you could join all non-OR queries via the same method above w/ the bitwise &: my_filter &= ...

Answer 2

You can restructure your dictionary into a list of single key-value dictionaries and use unpacking on each dict inside the Q expression like so:

from functools import reduce
import operator

from django.db.models import Q

# your dict is my_filter
q = model.objects.filter(reduce(operator.or_, 
                                (Q(**d) for d in [dict([i]) for i in my_filter.items()])))

reduce on or_ joins the Q expressions on an OR.

You could also use a generator expression where you have the list of dicts:

q = model.objects.filter(reduce(operator.or_, 
                                (Q(**d) for d in (dict([i]) for i in my_filter.items()))))

Answer 3

Based on @Moses Koledoye's answer I could solve the issue. This is how my function looks like now:

 cached_query(key, model, my_filter=None, or_filter={}, exclude=None, order_by=None, sliced=50):
    """
    :param key: string used as key reference to store on Memcached
    :param model: model reference on which 'filter' will be called
    :param my_filter: dictionary containing the filter parameters (eg.: {'title': 'foo', 'category': 'bar'}
    :param or_filter: dictionary containing the filter parameters (eg.: {'title': 'foo', 'category': 'bar'}
    :param sliced: integer limit of results from the query. The lower the better, since for some reason Django Memcached
        won't store thousands of entries in memory
    :param exclude: dictionary containing the exclude parameters (eg.: {'title': 'foo', 'category': 'bar'}
    :param order_by: tuple containing the list of fields upon which the model will be ordered.
    :return: list of models. Not a QuerySet, since it was sliced.
    """

    result = cache.get(key, None)
    if not result:
        result = model.objects.all()
        if my_filter:
            result = model.objects.filter(**my_filter)
        if or_filter:
            reduced_filter = reduce(operator.or_, (Q(**d) for d in [dict([i]) for i in or_filter.items()]))
            result = result.filter(reduced_filter)
        if exclude:
            result = result.exclude(**exclude)
        if order_by:
            result = result.order_by(*order_by)
        result = result[:sliced]
        cache.set(key, result, cache_timeout)
    return result