How to mock datetime.date.today() method in Django 1.5.5 tests

Question

I have a set of tests that rely on mocking out dates using the python mock library, and the @mock.patch decorator, together with the date mocking code sample found here. Using this, we have a FakeDate class:

class FakeDate(original_date):
    "A fake replacement for datetime.date that can be mocked for testing."
    def __new__(cls, *args, **kwargs):
        return original_date.__new__(original_date, *args, **kwargs)

And in our tests we have:

from datetime import date as real_date

@mock.patch('datetime.date', FakeDate)
def test_mondays_since_date(self):

    FakeDate.today = classmethod(lambda cls: real_date(2014, 1, 1))  # A Wednesday
    self.assertNotEqual(datetime.date.today(), real_date.today())
    self.assertEqual(datetime.date.today().year, 2014)
    # and so on..

Everything has been working up till I upgraded Django from 1.4.8 to 1.5.5. Unfortunately now the mock dates are causing tests to fail, but only on model save operations. The stack trace is as follows:

File "/site-packages/django/db/models/base.py", line 546, in save
  force_update=force_update, update_fields=update_fields)
File "/site-packages/django/db/models/base.py", line 650, in save_base
  result = manager._insert([self], fields=fields, return_id=update_pk, using=using, raw=raw)
File "/site-packages/django/db/models/manager.py", line 215, in _insert
  return insert_query(self.model, objs, fields, **kwargs)
File "/site-packages/django/db/models/query.py", line 1675, in insert_query
  return query.get_compiler(using=using).execute_sql(return_id)
File "/site-packages/django/db/models/sql/compiler.py", line 942, in execute_sql
  for sql, params in self.as_sql():
File "/site-packages/django/db/models/sql/compiler.py", line 900, in as_sql
  for obj in self.query.objs
File "/site-packages/django/db/models/fields/__init__.py", line 304, in get_db_prep_save
  prepared=False)
File "/site-packages/django/db/models/fields/__init__.py", line 738, in get_db_prep_value
  value = self.get_prep_value(value)
File "/site-packages/django/db/models/fields/__init__.py", line 733, in get_prep_value
  return self.to_python(value)
File "/site-packages/django/db/models/fields/__init__.py", line 697, in to_python
  parsed = parse_date(value)
File "/site-packages/django/utils/dateparse.py", line 36, in parse_date
  match = date_re.match(value)
TypeError: expected string or buffer

I've pdb'd my way into the Django source, and the problem seems to be here (in django/db/models/fields/init.py:

def to_python(self, value):
    if value is None:
        return value
    if isinstance(value, datetime.datetime):
        if settings.USE_TZ and timezone.is_aware(value):
            # Convert aware datetimes to the default time zone
            # before casting them to dates (#17742).
            default_timezone = timezone.get_default_timezone()
            value = timezone.make_naive(value, default_timezone)
        return value.date()
    if isinstance(value, datetime.date):  # <-- This is the problem!
        return value

    try:
        parsed = parse_date(value)

The type equality expression is failing, hence the call to parse_date which actually raises the error. (i.e. the value, which is the date returned from my FakeDate.today() expression is not being seen as a standard lib datetime.date object.)

So, I know where the problem lies, but what can I do to get around it? Mocking dates is critical to our application testing.

[EDIT 1: comparison of earlier version of Django]

To compare the above expression that fails in Django 1.5.5, the following is 1.4.8 (which does not fail):

def to_python(self, value):
    if value is None:
        return value
    if isinstance(value, datetime.datetime):
        return value.date()
    if isinstance(value, datetime.date):
        return value

    value = smart_str(value)

    try:
        parsed = parse_date(value)

i.e. they are the same. So why is one passing and the other failing - is this related to changes in the testrunner?

[EDIT 2: More debugging]

Digging a little further into the discrepancy:

> /site-packages/django/db/models/fields/__init__.py(685)to_python()
    684         import ipdb; ipdb.set_trace()
--> 685         if value is None:
    686             return value
ipdb> value
datetime.date(2012, 12, 7)
ipdb> isinstance(value, datetime.date)
False
ipdb> type(value)
<type 'datetime.date'>
ipdb> type(datetime.date)
<type 'type'>
ipdb> datetime.date
<class 'testutils.FakeDate'>
ipdb> datetime.datetime
<type 'datetime.datetime'>

[EDIT 3: located the problem]

I have discovered the discrepancy between the 1.4 and 1.5 branches, and it's not in the test runner. The key is the value = smart_str(value) line in the 1.4 branch. This is called before parse_date, and it converts our FakeDate into a string repr, which is parseable (e.g. '2012-05-09'). This is not called in the 1.5 version, which bombs.

This is the sequence within the 1.4.x branch:

# value = FakeDate(2012, 12, 31)
# code fails the isinstance(datetime.date) test
value = smart_str(value)
# value is now a string '2012-12-31'
parsed = parse_date(value)
# inside parse_date we try a regex match
match = date_re.match(value)
# because we have called smart_str, this now parses as a date

The sequence within the 1.5.x branch does not include the smart_str conversion, and so the regex match fails, as the value argument in this case is a FakeDate object and not a string.

[EDIT 5: Bug submitted to Django]

I have submitted a bug to the Django issue tracker (https://code.djangoproject.com/ticket/21523) for this.

Answer 1

My investigations into this are in the question as a set of edits, but the long and short of this is that changes to the to_python method between 1.4 and 1.5 mean that anything that is neither a valid datetime.date nor datetime.datetime must be a string in order to pass through.

It looks (without too much further digging) as if the django.utils.encoding.smart_str method was removed in 1.5, and although it's been replaced by smart_text, this never made it into the to_python method.

I have raised a ticket in the django Trac instance, https://code.djangoproject.com/ticket/21523

I have also created a patch for this issue - but obviously this may never make it in (and the patch is for 1.5.x, which is already out-of-date, so I really wouldn't count on this making it in).

[EDIT 1: a solution!]

There is a solution ;-) - I have done a write up here - http://tech.yunojuno.com/mocking-dates-with-django - the key is overriding the FakeDate instancecheck method, so that when comparing a real datetime.date to a FakeDate you get True. I have put together a gist with some sample FakeDate classes and associated tests for reference - https://gist.github.com/hugorodgerbrown/7750432