Qunit test alternates between pass and fail on page refresh

Question

I have a two tests that are causing side effects with each other. I understand why as I am replacing a jQuery built-in function that is being called internally in the second test. However what I don't understand is why the test alternately passes and fails.

This question is similar However, I am not doing anything directly on the qunit-fixture div.

Here are my tests

test('always passing test', function() { // Always passes
    var panelId = '#PanelMyTab';

    var event = {};
    var ui = {
        tab: {
            name: 'MyTab',
        },
        panel: panelId,
    };

    $('<div id="' + panelId + '">')
        .append('<a href="#" class="export">Test</a>')
        .append('<a href="#" class="showForm">Show Form</a>')
        .appendTo('#qunit-fixture');

    jQuery.fn.on = function(event, callback) {
        ok(this.selector == panelId + ' .export', 'Setting export click event');
        equal(callback, tickets.search.getReport, 'Callback being set');
    };

    loadTab(event, ui);
});

test('alternately passing and failing', function() { // Alternates between passing and failing on page refresh
    expect(5);

    var testUrl = 'test';
    $('<div class="ui-tabs-panel">')
        .append('<a href="'+ testUrl + '" id="getReport">Get Report</a>')
        .append('<form action="notest" target="" class="ticketSearch"></form>')
        .appendTo('#qunit-fixture');

    // Setup form mocking
    $('form.ticketSearch').submit(function() {
        var urlPattern = new RegExp(testUrl + '$');
        ok(urlPattern.test($(this).prop('action')), 'Form action set to link href');
        equal($(this).prop('target'), '_blank', 'Open form on a new page');
    });

    var event = {
        target: 'a#getReport',
    };

    var result = getReport(event);

    var form = $('form.ticketSearch');

    ok(/notest$/.test($(form).prop('action')), 'Making sure action is not replaced');
    equal($(form).prop('target'), '', 'Making sure that target is not replaced');

    ok(false === result, 'click event returns false to not refresh page');
});

The tests will start off passing but when I refresh they will alternate between passing and failing.

Why is this happening? Even adding GET parameters to the url result in the same behavior on the page.

In the failing cases, the test is failing because internal jQuery is calling .on() when the submit() handler is set. But why isn't the test always failing in that case? What is the browser doing that a state is being retained during page refresh?

Update:

Here is the code that is being tested:

var tickets = function() {
    var self = {
        loadTab: function(event, ui) {
            $(panel).find('.export').button().on('click', this.getReport);
        },

        search: {
            getReport: function(event) {
                var button = event.target;
                var form = $(button).closest('div.ui-tabs-panel').find('form.ticketSearch').clone(true);

                $(form).prop('action', $(button).prop('href'));
                $(form).prop('target', '_blank');

                $(form).submit();

                return false;
            }
        }
    };

    return self;
}();

Answer 1

I've modified @Ben's fiddle to include your code with both of your tests. I modified some of your code to make it run correctly. When you hit the run button all of the tests will pass. When you hit the run button again, the second test ("alternately passing and failing") will fail -- this is basically simulating your original issue.

The issue is your first test ("always passing test") alters the global state by replacing the jQuery.fn.on function with an overridden one. Because of this, when the tests are run in order, the second test ("alternately passing and failing") uses the incorrect overridden jQuery.fn.on function and fails. Each unit test should return the global state back to its pre-test state so that other tests can run based on the same assumptions.

The reason why it's alternating between pass and fail is that under the hood QUnit always runs failed tests first (it remembers this somehow via cookie or local storage, I'm not exactly sure). When it runs the failed tests first, the second test runs before the first one; as a result, the second test gets jQuery's native on function and works. When you run it a third time, the tests will run in their "original" order and the second test will use the overridden on function and fail.

Here's the working fiddle. I've add the fix to "un-override" the on function after the test by caching the original var jQueryOn = jQuery.fn.on; function and resetting it at the end of the test via: jQuery.fn.on = jQueryOn;. You can probably better implement this using QUnit's module teardown() method instead.

You can check out https://github.com/jquery/qunit/issues/74 for more info.