How do I write (test) code that will not be optimized by the compiler/JIT?

Question

I don't really know much about the internals of compiler and JIT optimizations, but I usually try to use "common sense" to guess what could be optimized and what couldn't. So there I was writing a simple unit test method today:

@Test  // [Test] in C#
public void testDefaultConstructor() {
    new MyObject();
}

This method is actually all I need. It checks that the default constructor exists and runs without exceptions.

But then I started to think about the effect of compiler/JIT optimizations. Could the compiler/JIT optimize this method by eliminating the new MyObject(); statement completely? Of course, it would need to determine that the call graph does not have side effects to other objects, which is the typical case for a normal constructor that simply initializes the internal state of the object.

I presume that only the JIT would be allowed to perform such an optimization. This probably means that it's not something I should worry about, because the test method is being performed only once. Are my assumptions correct?

Nevertheless, I'm trying to think about the general subject. When I thought about how to prevent this method from being optimized, I thought I may assertTrue(new MyObject().toString() != null), but this is very dependent on the actual implementation of the toString() method, and even then, the JIT can determine that toString() method always returns a non-null string (e.g. if actually Object.toString() is being called), and thus optimize the whole branch. So this way wouldn't work.

I know that in C# I can use [MethodImpl(MethodImplOptions.NoOptimization)], but this is not what I'm actually looking for. I'm hoping to find a (language-independent) way of making sure that some specific part(s) of my code will actually run as I expect, without the JIT interfering in this process.

Additionally, are there any typical optimization cases I should be aware of when creating my unit tests?

Thanks a lot!

Answer 1

Every I/O is a side effect, so you can just put

Object obj = new MyObject();
System.out.println(obj.toString());

and you're fine.

Answer 2

Don't worry about it. It's not allowed to ever optimize anything that can make a difference to your system (except for speed). If you new an object, code gets called, memory gets allocated, it HAS to work.

If you had it protected by an if(false), where false is a final, it could be optimized out of the system completely, then it could detect that the method doesn't do anything and optimize IT out (in theory).

Edit: by the way, it can also be smart enough to determine that this method:

newIfTrue(boolean b) {
    if(b)
        new ThisClass();
}

will always do nothing if b is false, and eventually figure out that at one point in your code B is always false and compile this routine out of that code completely.

This is where the JIT can do stuff that's virtually impossible in any non-managed language.

Answer 3

I think if you are worried about it getting optimized away, you may be doing a bit of testing overkill.

In a static language, I tend to think of the compiler as a test. If it passes compilation, that means that certain things are there (like methods). If you don't have another test that exercises your default constructor (which will prove it wont throw exceptions), you may want to think about why you are writing that default constructor in the first place (YAGNI and all that).

I know there are people that don't agree with me, but I feel like this sort of thing is just something that will bloat out your number of tests for no useful reason, even looking at it through TDD goggles.

Answer 4

The JIT is only allowed to perform operations that do not affect the guaranteed semantics of the language. Theoretically, it could remove the allocation and call to the MyObject constructor if it can guarantee that the call has no side effects and can never throw an exception (not counting OutOfMemoryError).

In other words, if the JIT optimizes the call out of your test, then your test would have passed anyway.

PS: Note that this applies because you are doing functionality testing as opposed to performance testing. In performance testing, it's important to make sure the JIT does not optimize away the operation you are measuring, else your results become useless.

Answer 5

It seems that in C# I could do this:

[Test]
public void testDefaultConstructor() {
    GC.KeepAlive(new MyObject());
}

AFAIU, the GC.KeepAlive method will not be inlined by the JIT, so the code will be guaranteed to work as expected. However, I don't know a similar construct in Java.

Answer 6

Think about it this way:

Lets assume that compiler can determine that the call graph doesn't have any side effects(I don't think it is possible, I vaguely remember something about P=NP from my CS courses). It will optimize any method that doesn't have side effects. Since most tests don't have and shouldn't have any side effects then compiler can optimize them all away.