How do I write a scala unit test that ensures compliation fails?

Question

Is there any way to write something like a "unit test" which makes sure some code does not compile?

Why would I want such a thing? Two reasons.

1) Check the type safety of my API. I'd like a way to make sure if someone passes in a bad value, you get a compiler error, not just a runtime error. Obviously, I can just run the compiler and check for the error, but having it formalized in a unit test is good for avoiding a regression & also for documentation.

Eg., consider this test. There is some commented out code which I had used to check type-safety: https://github.com/squito/boxwood/blob/master/core/src/test/scala/com/quantifind/boxwood/EnumUnionTest.scala#L42 (lines 42 & 48 -- on line 34 I call a different API which has a runtime exception, which I can check)

It actually took me a while to get the type-safety right, so those were important checks. Now if I go and modify the underlying implementation, I can't just run my test suite -- I've got to also remember to uncomment those lines and check for a compiler error.

2) Testing error handling of macros. If a macro has some bad input, it should result in a compiler error. Same issues here, same desire to have it in a easy-to-run test-suite.

I use ScalaTest, but I'm happy to here a solution with any unit-testing framework.

Answer 1

As I note in a comment above, Shapeless 2.0 (not yet released but currently available as a milestone) has a very nice implementation of the functionality you're looking for, based on a solution by Stefan Zeiger. I've added a demo to your project here (note that I've had to update to Scala 2.10, since this solution uses a macro). It works like this:

import shapeless.test.illTyped

//this version won't even compile
illTyped("getIdx(C.Ooga)")

//We can have multiple enum unions exist side by side
import Union_B_C._
B.values().foreach {b => Union_B_C.getIdx(b) should be (b.ordinal())}
C.values().foreach {c => Union_B_C.getIdx(c) should be (c.ordinal() + 2)}

//Though A exists in some union type, Union_B_C still doesn't know about it,
// so this won't compile
illTyped("""
  A.values().foreach {a => Union_B_C.getIdx(a) should be (a.ordinal())}
""")

If we were to change the code in the second call to illTyped to something that will compile:

B.values().foreach {a => Union_B_C.getIdx(a) should be (a.ordinal())}

We'd get the following compilation error:

[error] .../EnumUnionTest.scala:56: Type-checking succeeded unexpectedly.
[error] Expected some error.
[error]     illTyped("""
[error]             ^
[error] one error found
[error] (core/test:compile) Compilation failed

If you'd prefer a failed test, you could pretty easily adapt the implementation in Shapeless. See Miles's answer to my previous question for some addition discussion.

Answer 2

Scalatest can also do this.

Checking that a snippet of code does not compile

Often when creating libraries you may wish to ensure that certain arrangements of code that represent potential “user errors” do not compile, so that your library is more error resistant. ScalaTest Matchers trait includes the following syntax for that purpose:

"val a: String = 1" shouldNot compile

If you want to ensure that a snippet of code does not compile because of a type error (as opposed to a syntax error), use:

"val a: String = 1" shouldNot typeCheck

Note that the shouldNot typeCheck syntax will only succeed if the given snippet of code does not compile because of a type error. A syntax error will still result on a thrown TestFailedException.

If you want to state that a snippet of code does compile, you can make that more obvious with:

"val a: Int = 1" should compile

Although the previous three constructs are implemented with macros that determine at compile time whether the snippet of code represented by the string does or does not compile, errors are reported as test failures at runtime.