Selenium / Django gives Foreign Key error

Question

I am trying to run functional tests using Selenium for a Django project. On both Firefox and Chrome I am getting a weird Foreign Key error when I try to test that a superuser can change a normal user's status to staff (I assume this is all verified via Django's internal testing, but thought it would be good practice to include it in my app's testing since my user scenarios depend on the functionality). It almost looks like Django doesn't like Selenium saving anything to the database? This one error trickles down to my other tests, too, so it seems like something breaks behind the scenes--like Selenium loses its database connection. Does anyone know why this happens?? I'm willing to dump the test in favor of assuming that the functionality works, but would love to fix this and use the test.

One possible symptom is that in Chrome, between between the outputs of "checked the box" and "clicked the save button", I get a broken pipe (does not happen in Firefox).

It seems to break around here, whenever Selenium clicks the 'save' button:

# Form looks properly rendered, now click the 'Staff status'
# checkbox and submit it
isStaffCheckbox = self.browser.find_element_by_id('id_is_staff')
isStaffCheckbox.click()               
print 'checked the box'
# Save the form
saveBtn = self.browser.find_element_by_css_selector('input[value="Save"]')
saveBtn.click()
print 'clicked the save button'

The error message I get is:

Traceback (most recent call last):
File "~/virtual_environments/VideoSearch/lib/python2.7/site-packages/django/test/testcases.py", line 268, in __call__
    self._post_teardown()
File "~/virtual_environments/VideoSearch/lib/python2.7/site-packages/django/test/testcases.py", line 533, in _post_teardown
    self._fixture_teardown()
File "~/virtual_environments/VideoSearch/lib/python2.7/site-packages/django/test/testcases.py", line 553, in _fixture_teardown
    skip_validation=True, reset_sequences=False)
File "~/virtual_environments/VideoSearch/lib/python2.7/site-packages/django/core/management/__init__.py", line 161, in call_command
    return klass.execute(*args, **defaults)
File "~/virtual_environments/VideoSearch/lib/python2.7/site-packages/django/core/management/base.py", line 255, in execute
    output = self.handle(*args, **options)
File "~/virtual_environments/VideoSearch/lib/python2.7/site-packages/django/core/management/base.py", line 385, in handle
    return self.handle_noargs(**options)
File "~/virtual_environments/VideoSearch/lib/python2.7/site-packages/django/core/management/commands/flush.py", line 82, in handle_noargs
    emit_post_sync_signal(set(all_models), verbosity, interactive, db)
File "~/virtual_environments/VideoSearch/lib/python2.7/site-packages/django/core/management/sql.py", line 195, in emit_post_sync_signal
    interactive=interactive, db=db)
File "~/virtual_environments/VideoSearch/lib/python2.7/site-packages/django/dispatch/dispatcher.py", line 170, in send
    response = receiver(signal=self, sender=sender, **named)
File "~/virtual_environments/VideoSearch/lib/python2.7/site-packages/django/contrib/auth/management/__init__.py", line 96, in create_permissions
    auth_app.Permission.objects.using(db).bulk_create(perms)
File "~/virtual_environments/VideoSearch/lib/python2.7/site-packages/django/db/models/query.py", line 444, in bulk_create
    self._batched_insert(objs_without_pk, fields, batch_size)
File "~/virtual_environments/VideoSearch/lib/python2.7/site-packages/django/db/models/query.py", line 902, in _batched_insert
    using=self.db)
File "~/virtual_environments/VideoSearch/lib/python2.7/site-packages/django/db/models/manager.py", line 215, in _insert
    return insert_query(self.model, objs, fields, **kwargs)
File "~/virtual_environments/VideoSearch/lib/python2.7/site-packages/django/db/models/query.py", line 1661, in insert_query
    return query.get_compiler(using=using).execute_sql(return_id)
File "~/virtual_environments/VideoSearch/lib/python2.7/site-packages/django/db/models/sql/compiler.py", line 937, in execute_sql
    cursor.execute(sql, params)
File "~/virtual_environments/VideoSearch/lib/python2.7/site-packages/django/db/backends/mysql/base.py", line 122, in execute
    six.reraise(utils.IntegrityError, utils.IntegrityError(*tuple(e.args)), sys.exc_info()[2])
File "~/virtual_environments/VideoSearch/lib/python2.7/site-packages/django/db/backends/mysql/base.py", line 120, in execute
    return self.cursor.execute(query, args)
File "~/virtual_environments/VideoSearch/lib/python2.7/site-packages/MySQL_python-1.2.4b4-py2.7-macosx-10.8-intel.egg/MySQLdb/cursors.py", line 202, in execute
    self.errorhandler(self, exc, value)
File "~/virtual_environments/VideoSearch/lib/python2.7/site-packages/MySQL_python-1.2.4b4-py2.7-macosx-10.8-intel.egg/MySQLdb/connections.py", line 36, in defaulterrorhandler
    raise errorclass, errorvalue
IntegrityError: (1452, 'Cannot add or update a child row: a foreign key constraint fails (`test_videos2002`.`auth_permission`, CONSTRAINT `content_type_id_refs_id_d043b34a` FOREIGN KEY (`content_type_id`) REFERENCES `django_content_type` (`id`))')

===== Updated with code

• Also note that test_admin_can_make_a_user_staff causes my other tests to Error out with the same error--but they are okay when I take out the "save" command from that test.
• I guess two things I really don't understand are: 1) Why this happens with the built-in Django admin view (thought it should just work), and 2) Why one test error cascades through to my other tests? I thought they were independent.

Thanks for looking at this!

From my functional_tests.test.py (borrowed from https://github.com/lincolnloop/django-selenium-intro/tree/master/selenium_intro):

from django.test import LiveServerTestCase

class SeleniumTestCase(LiveServerTestCase):
    """
    A base test case for selenium, providing hepler methods for generating
    clients and logging in profiles.
    """

    def open(self, url):
        self.browser.get("%s%s" % (self.live_server_url, url))

From my admin_django.py (set of test cases)

from functional_tests.test import SeleniumTestCase

from selenium.webdriver.common.keys import Keys
from django.core.urlresolvers import reverse
from django.contrib.auth.models import User
from django.test import LiveServerTestCase
from django.conf import settings
from selenium import webdriver


class AdminDjango(SeleniumTestCase):
    def setUp(self):
        User.objects.create_superuser(username='vcb',
                                      password='rock5!',
                                      email='[email protected]')
        User.objects.create_user(username='teaching',
                                 password='assistant',
                                 email='[email protected]')
#        self.browser = webdriver.Chrome(settings.SELENIUM_WEBDRIVER)
        self.browser = webdriver.Firefox()
        self.browser.implicitly_wait(3)
        self.browser.set_page_load_timeout(10)

    def tearDown(self):
        self.browser.quit()

    def check_for_links(self, link_text):
        """
        Helper function to check links on a page for certain text
        """
        links = self.browser.find_elements_by_tag_name('a')
        self.assertTrue(link_text, [link.text for link in links])

    def admin_logs_in(self):
        """
        Helper function that logs the admin user into the page
        """
        username_field = self.browser.find_element_by_name('username')
        username_field.send_keys('vcb')

        password_field = self.browser.find_element_by_name('password')
        password_field.send_keys('rock5!')
        password_field.send_keys(Keys.RETURN)

    def admin_log_in_complete(self):
        """
        Includes navigation to the admin page
        """
        self.open('/admin/')
        self.admin_logs_in()

    def test_admin_can_login(self):
        """
        Admin user can log into the Django admin interface
        """
        self.open('/admin/')
        body = self.browser.find_element_by_tag_name('body')
        self.assertIn('VCB Administration', body.text)

        self.admin_logs_in()

        # her username and password are accepted, and she is taken to
        # the Site Administration page
        body = self.browser.find_element_by_tag_name('body')
        self.assertIn('Site administration', body.text)

    def test_admin_page_renders_properly(self):
        """
        The admin page should have at least two fields:
         - Users
         - Classes
        Admins may have to add staff status to users, and they may have to
        adjust the information for a class
        """
        self.admin_log_in_complete()

        self.check_for_links('Users')
        self.check_for_links('Groups')
        self.check_for_links('Classess')

    def test_admin_can_make_a_user_staff(self):
        """
        Admin users can add staff status to existing users
        """
        self.admin_log_in_complete()
        pageLinks = self.browser.find_elements_by_tag_name('a')

        for link in pageLinks:
            if link.text == 'Users':
                userLink = link

        userLink.click()

        headers = self.browser.find_elements_by_tag_name('h1')
        self.assertTrue('Select user to change', 
                [header.text for header in headers])

        users =     self.browser.find_elements_by_xpath('//table[@id="result_list"]/tbody/tr/th/a')

        self.fail('Finish writing the test!')
#        rowCount = 1
#        for user in users:
#            xpath = '//table[@id="result_list"]/tbody/tr[' + str(rowCount) + ']/td[5]/img'
#            # check first that this user is not a staff
#            staffIcon = self.browser.find_element_by_xpath(xpath)
#            isStaff = staffIcon.get_attribute('alt')
#            
#            if isStaff == 'false':
#                user.click()
#                userHeaders = self.browser.find_elements_by_tag_name('h1')
#                self.assertTrue('Change user', 
#                        [userHeader.text for userHeader in userHeaders])
#            
#                # Are the right fields present in the user's form?
#                formHeaders = self.browser.find_elements_by_tag_name('h2')
#                self.assertTrue('Personal info', 
#                        [formHeader.text for formHeader in formHeaders])
#                self.assertTrue('Permissions', 
#                        [formHeader.text for formHeader in formHeaders])
#                self.assertTrue('Important dates', 
#                        [formHeader.text for formHeader in formHeaders])
#            
#                # Form looks properly rendered, now click the 'Staff status'
#                # checkbox and submit it
#                isStaffCheckbox = self.browser.find_element_by_id('id_is_staff')
#                isStaffCheckbox.click()
#                print 'checked the box'
#                # Save the form
#                saveBtn = self.browser.find_element_by_css_selector('input[value="Save"]')
#                saveBtn.click()
#                print 'clicked the save button'
#                # Returns you to the admin page
#                messageBox = self.browser.find_element_by_class_name('info')
#                self.assertIn('successfully', messageBox.text)
#                
#                # Check that staff status changed
#                staffIcon = self.browser.find_element_by_xpath('//table[@id="result_list"]/tbody/tr[' + str(rowCount) + ']/td[5]/img')
#                isStaff = staffIcon.get_attribute('alt')
#                self.assertTrue(isStaff)
#                print 'should now be staff'
#            rowCount += 1


    def test_admin_can_change_a_class_obj_bank_id(self):
        """
        Admin users can change a class's objective bank id
        """
        self.fail('Finish writing the test!')

    def test_logging_out_redirects_to_login_page(self):
        """
        Logging out of the admin page should redirect to the main page
        """
        self.admin_log_in_complete()
        logOut = self.browser.find_element_by_link_text('Log out')
        logOut.click()
        body = self.browser.find_element_by_tag_name('body')
        self.assertIn('VCB Administration', body.text)

Answer 1

As I mentioned in the comments, this seems to be a documented bug in Django. Here and here are the official bug reports. As reported in the second link, one workaround is to swap the order of django.contrib.auth and django.contrib.contenttypes in INSTALLED_APPS, as such:

What originally is this:

INSTALLED_APPS = (
    'django.contrib.auth',
    'django.contrib.contenttypes',
    'django.contrib.sessions',
    'django.contrib.sites',
    'django.contrib.admin',
    'testapp'
)

Should become:

INSTALLED_APPS = (
    'django.contrib.contenttypes',
    'django.contrib.auth',
    'django.contrib.sessions',
    'django.contrib.sites',
    'django.contrib.admin',
    'testapp'
)