Test failures: false positive or false negative

Question

If my unit test fails, but the underlying code actually works, would this be called a false positive, or a false negative?

(Here is the context. I run a bunch of tests in parallel and some tests fail. I run them sequentially and they all pass. All the code actually works, there is just an issue with the parallel test runner.)

My question is just with the nomenclature though. I've found examples of people calling it either. After reading Wikipedia, I would've thought it is a false positive, but notably Andrew Ng in his machine learning class said:

If the test passes, even if the code is broken, that is a false positive. If the test fails, when the code is NOT broken, that is a false negative.

Answer 1

I believe it's the other way around than what the other answers are saying.

At least that's how false positives and false negatives are defined in the "XUnit Test Patterns: Refactoring Test Code" book by Gerard Meszaros.

An easy way to understand this is to think of it like in medicine where the tests are "testing for diseases".

If you have the disease you are "disease-positive".

In our software world, you can think it like this:

A POSITIVE test, means you are POSITIVE to a bug i.e. Test fails because you are "NullReferenceExceptionitis-positive"

A NEGATIVE test, means you are NEGATIVE to a bug i.e. Test passes because you are "StackOverflu-negative"

So, keep in mind bug = disease and:

A FALSE positive (to a bug) means your code is falsely accused to have a bug. (Your code has no bugs, yet test fails)

A FALSE negative (to a bug) means your code is falsely declared to have no bugs. (Your code has bugs, yet test passes)

A TRUE positive means your code is rightfully(truly) accused to have bugs. (Your code has bugs, test fails)

A TRUE negative means your code is rightfully(truly) declared to have no bugs (Your code has no bugs, test passes)

I hope this helps.

References: http://xunitpatterns.com/false%20positive.html

And from that book:

If we are having problems with Buggy Tests or Production Bugs, we can reduce the risk of false negatives (tests that pass when they shouldn’t)

also:

false negative

A situation in which a test passes even though the system under test (SUT) is not working properly. Such a test is said to give a false-negative indication or a “false pass.” See also: false positive.

false positive

A situation in which a test fails even though the system under test (SUT) is working properly