Karma don’t fail if tests have errors

Question

When a test can’t be run by Karma due to an error (like a syntax error), Karma complains with a warning like that :

INFO [karma]: Karma v0.12.31 server started at http://localhost:9876/
INFO [launcher]: Starting browser PhantomJS
INFO [PhantomJS 1.9.8 (Linux)]: Connected on socket WT-AsaXZq4odkQe2DgZJ with id 10861626
WARN [web-server]: 404: /undefinedhttp%3A%2F%2Flocalhost%3A9876%2Fbase%2Ftests%2Funit%2Fjs%2Fspecs%2Fcore%2FDeviceSpec.js%3F1e346ab1ad6e6e9240be6a6d5effaaa8f0dd96d8/9/%22ReferenceError%3A%20Can't%20find%20variable%3A%20SYNTAXERROR%22

The last line unescaped is : "/undefinedhttp://localhost:9876/base/tests/unit/js/specs/core/DeviceSpec.js?1e346ab1ad6e6e9240be6a6d5effaaa8f0dd96d8/9/"ReferenceError: Can't find variable: SYNTAXERROR""

Which is right since that I inserted SYNTAXERROR somewhere in my test for the purpose of that question.

So the test is ignored but, since other tests are OK, Karma returns that tests have passed :

 PhantomJS 1.9.8 (Linux): Executed 41 of 41 SUCCESS (0.354 secs / 0.016 secs)
➜ echo $?
0

I want Karma to fail if it can‘t execute all tests (I don’t think of any reason not to !).

I haven’t found any configuration variable to act that way.

I found the line where Karma print the warning above but I don’t understand why it’s handled as a 404...

Any idea to make Karma fail when it can’t execute all tests ?

Answer 1

I’m using a script which fails if WARN are displayed by karma :

#!/bin/bash
set -eo pipefail

mkdir -p tests/unit/js/dist || true
./node_modules/.bin/karma start tests/unit/js/karma.conf.js --port 9875 | tee tests/unit/js/dist/results.txt

if grep 'WARN' tests/unit/js/dist/results.txt; then
    exit 1
else
    exit 0
fi