Fixtures in Django testing with South / Selenium

Question

I am trying to run Selenium tests on a Django project (1.5.4), which uses South. I think South is conflicting with my tests when I try to inject initial data with fixtures, but I'm not sure why; I appreciate any help.

According to the Django documentation, fixtures are supposed to be loaded after the first syncdb and then all migrations are applied.

Question 1) Does this take into account South migrations?? Do I need to run those separately somehow?

The error I get when I run my tests makes it seem like my South migrations are still present in the test database after the first test...but I thought each test has its own database (and migrations / fixtures) created? The first test passes / fails, but each subsequent test raises this IntegrityError:

IntegrityError: Problem installing fixture '<PROJECT_PATH>/fixtures/toy_course.json': Could not load contenttypes.ContentType(pk=8): (1062, "Duplicate entry 'south-migrationhistory' for key 'app_label'")

This South documentation and SO question seem to indicate that I need to override some type of forwards method in order to get fixtures working, but I'm not entirely sure how to apply that to a testing situation instead of production (or if that is the solution I need).

Question 2) Am I supposed to override forwards in my test setup? Where would I do it?

My relevant test code:

from django.conf import settings

from selenium import webdriver

from functional_tests.test import SeleniumTestCase

class Resources(SeleniumTestCase):
    fixtures = ['toy_course.json']

    def setUp(self):      
        self.browser = webdriver.Chrome(settings.SELENIUM_WEBDRIVER)
        self.browser.implicitly_wait(3)

    def tearDown(self):
        self.browser.quit()

    def test_main_page_renders_correctly(self):
        """
        User sees a properly formatted main page
        """
        self.open('/RDB/')

        h3_headers = self.browser.find_elements_by_tag_name('h3')
        self.assertIn(
                'Complete List of Resources',
                [header.text for header in h3_headers])

        self.assertTrue(self.check_exists_by_id('main_table'))
        self.assertTrue(self.check_exists_by_id('searchDiv'))

        self.assertTrue(self.check_exists_by_class_name('tablesorter'))

Thanks!

---

UPDATE

So per Alex's suggestion below and this South doc, I added this line to my settings.py:

SOUTH_TESTS_MIGRATE = False

But I am now getting 8 of 8 Errors (before I was getting 1 pass/fail on the first test, and then 7 Errors). The full error for a single test is below:

======================================================================
ERROR: test_table_sorts_on_click (functional_tests.tests.main_resources.Resources)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/<PATH TO VIRTUAL ENV>/virtual_environments/relate/lib/python2.7/site-packages/django/test/testcases.py", line 259, in __call__
self._pre_setup()
  File "/<PATH TO VIRTUAL ENV>/virtual_environments/relate/lib/python2.7/site-packages/django/test/testcases.py", line 479, in _pre_setup
self._fixture_setup()
  File "/<PATH TO VIRTUAL ENV>/virtual_environments/relate/lib/python2.7/site-packages/django/test/testcases.py", line 518, in _fixture_setup
**{'verbosity': 0, 'database': db_name, 'skip_validation': True})
  File "/<PATH TO VIRTUAL ENV>/virtual_environments/relate/lib/python2.7/site-packages/django/core/management/__init__.py", line 161, in call_command
return klass.execute(*args, **defaults)
  File "/<PATH TO VIRTUAL ENV>/virtual_environments/relate/lib/python2.7/site-packages/django/core/management/base.py", line 255, in execute
output = self.handle(*args, **options)
  File "/<PATH TO VIRTUAL ENV>/virtual_environments/relate/lib/python2.7/site-packages/django/core/management/commands/loaddata.py", line 193, in handle
obj.save(using=using)
  File "/<PATH TO VIRTUAL ENV>/virtual_environments/relate/lib/python2.7/site-packages/django/core/serializers/base.py", line 165, in save
models.Model.save_base(self.object, using=using, raw=True)
  File "/<PATH TO VIRTUAL ENV>/virtual_environments/relate/lib/python2.7/site-packages/django/db/models/base.py", line 626, in save_base
rows = manager.using(using).filter(pk=pk_val)._update(values)
  File "/<PATH TO VIRTUAL ENV>/virtual_environments/relate/lib/python2.7/site-packages/django/db/models/query.py", line 605, in _update
return query.get_compiler(self.db).execute_sql(None)
  File "/<PATH TO VIRTUAL ENV>/virtual_environments/relate/lib/python2.7/site-packages/django/db/models/sql/compiler.py", line 1014, in execute_sql
cursor = super(SQLUpdateCompiler, self).execute_sql(result_type)
  File "/<PATH TO VIRTUAL ENV>/virtual_environments/relate/lib/python2.7/site-packages/django/db/models/sql/compiler.py", line 840, in execute_sql
cursor.execute(sql, params)
  File "/<PATH TO VIRTUAL ENV>/virtual_environments/relate/lib/python2.7/site-packages/django/db/backends/mysql/base.py", line 122, in execute
six.reraise(utils.IntegrityError, utils.IntegrityError(*tuple(e.args)), sys.exc_info()[2])
  File "/<PATH TO VIRTUAL ENV>/virtual_environments/relate/lib/python2.7/site-packages/django/db/backends/mysql/base.py", line 120, in execute
return self.cursor.execute(query, args)
  File "/<PATH TO VIRTUAL ENV>/virtual_environments/relate/lib/python2.7/site-packages/MySQLdb/cursors.py", line 201, in execute
self.errorhandler(self, exc, value)
  File "/<PATH TO VIRTUAL ENV>/virtual_environments/relate/lib/python2.7/site-packages/MySQLdb/connections.py", line 36, in defaulterrorhandler
raise errorclass, errorvalue
IntegrityError: Problem installing fixture '/<PATH TO PROJECT>/RDB/fixtures/toy_course.json': Could not load contenttypes.ContentType(pk=8): (1062, "Duplicate entry 'south-migrationhistory' for key 'app_label'")

The command I ran:

$ python manage.py test functional_tests

I'm not quite sure if I made the problem better, worse, or the same, but I seem to be more in-line with the documentation...

Thanks!

---

UPDATE #2 -- with solution

So a couple of other pages that helped me figure it out (in addition to Alex's pointer to the South doc). First, this person had a similar issue, and solved it using the SOUTH_TESTS_MIGRATE = False statement. So half my solution was to include that.

The second half of my solution was to fix my fixture document. I was dumping everything into my fixture with:

$ python manage.py datadump > RDB/fixtures/toy-course.json

This is, apparently, a bad way to do fixtures it with South--because it also dumps the South migration tables into the fixture. The post above shows the blogger using app-specific fixtures (which are also talked about in this SO post), and that was the key to getting my fixtures to work. The Django docs on fixtures do show the optional parameters to dump just an app, but I didn't know ignoring them would cause South to conflict. So the second half of my solution was to create my fixture to be app-specific:

$ python manage.py datadump RDB > RDB/fixtures/toy-course.json

And my tests now run fine (slow, but probably a different issue)!

Answer 1

Your test database is created using South migrations by default. Set SOUTH_TESTS_MIGRATE = False in your settings.py, quote from docs:

If this is False, South’s test runner integration will make the test database be created using syncdb, rather than via migrations (the default).