TDD: why might it be wrong to let app code know it is being tested, not run?

Question

In this thread, Brian (the only answerer) says "Your code should be written in such a fashion that it is testing-agnostic"

The single comment says "Your code should definitely not branch on a global "am I being tested flag".".

But neither gives reasons, and I would really like to hear some rational thoughts on the matter. It would be immensely easy (particularly given the fact that a lot of tests have package-private access to the app classes) to reach into a given app class and set a boolean to say "this is a test, not a run".

All sorts of things which I find myself jumping through hoops (injected mocked private fields, etc.) to achieve could become easier to accomplish.

It's also obvious that if you took this too far it could be disastrous... but as one tool among many in the software testing armoury why does the concept meet with such opprobrium?

Answer to Mick Mnemonic:

A trivial example of how this might help would be if you're actually creating a new class instance in the middle of a method and assigning it to a private field: private field mocks won't help in that case because you are replacing the private field. But actually creating a real object might be very costly: you might want to replace it with a lightweight version when testing.

I encountered such a situation yesterday, in fact... and my solution was to create a new package-private method called createXXX()... so I could mock it. But this in turn goes against the dictum "thou shalt not create methods just to suit your tests"!

Answer 1

Think of the big Volkswagen scandal. A system which behaves differently under test than under production load isn't really tested. That is: it is really two systems, the production system and the test system - and the only one of these which is tested is the test system. The production system, being different, is not tested. Every difference in behavior you introduce between the two systems is a testing vulnerability.