How to test JavaScript minification output

Question

We recently upgraded to a newer build of a JavaScript minification library.

After a significant amount of quality assurance work by the testing team, it was discovered that the new version of our minifier had an issue that changed the intention and meaning behind a block of code.

(Life lesson: don't upgrade JS minifiers unless you are really convinced you need the new version.)

The minifier is used for client side JavaScript code with a heavy emphasis on DOM related activity, not nearly as much "business logic".

A simplified example of what was broken by the minifier upgrade:

function process(count)
{
     var value = ""; 
     value += count; //1. Two consecutive += statements
     value += count;
     count++;        //2. Some other statement
     return value;   //3. Return
}

Was minified incorrectly to the following:

function process(n){var t="";return t+n+n,n++,t}

While we could write some unit tests to catch some of the issues potentially, given that the JavaScript is heavy on DOM interactions (data input, etc.), it's very difficult to test thoroughly without user testing (non-automated). We'd pondered using a JS to AST library like Esprima, but given the nature of the changes that could be done to the minified code, it would produce far too many false positives.

We also considered trying to write representative tests, but that seems like a never-ending task (and likely to miss cases).

FYI: This is a very sophisticated web application with several hundred thousand lines of JavaScript code.

We're looking for a methodology for testing the minification process short of "just test everything again, thoroughly, and repeat." We'd like to apply a bit more rigor/science to the process.

Ideally, we could try multiple minifiers without fear of each breaking our code in new subtle ways if we had a better scientific method for testing.

Update:

One idea we had was to:

1. take minification with old version
2. beautify it
3. minify with new version,
4. beautify, and
5. visually diff.

It did seem like a good idea, however the differences were so common that the diff tool flagged nearly every line as being different.

Answer 1

Have you considered a unit test framework, such as QUnitjs ? It would be quite a bit of work to write the unit tests, but in the end you would have a repeatable test procedure.