How do I expect failure in a unit test?

Question

We're writing unit tests for our code in Clojure using clojure.test.

Some of our tests ignore the API and purposely break the code, in order to serve as documentation for latent deficiencies in the code.

However, we want to distinguish between failures of these tests, and failures of normal tests.

We haven't seen any suggestions in the clojure.test documentation -- only (is (thrown? ...)), which of course doesn't do what we need.

Any suggestions? Basically, we're looking for something like (is (not <condition>)), except that the test framework should record an expected failure -- something like this.

Answer 1

I have made tests throw an exception when they 'fail' like this, and then used thrown? to test if the exception arrives as expected. There very well may exist a more elegant way, but this gets the job done.

Answer 2

As @andy said you can rebind report function.

(defmacro should-fail [body]
  `(let [report-type# (atom nil)]
     (binding [clojure.test/report #(reset! report-type# (:type %))]
       ~body)
     (testing "should fail"
       (is (= @report-type# :fail )))))

And use this macro like this:

(should-fail (is (= 1 2)))

That will be successful passing test.