Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to get ScalaTest correctly reporting tests results when using scalacheck with Propspec and PropertyCheck?

I'd like to test my scala program using property based testing with scalacheck. I wrote :

class MyProperties extends PropSpec with PropertyChecks {
    property("My property") {
        val myProperty: org.scalacheck.Prop = new MyProperty
        // some code I need to set myProperty
        myProperty.check
    }
}

But this seems wrong since when I run this class using ScalaTest, I get in the Console :

Run starting. Expected test count is: 1
MyProperties:

! Falsified after 51 passed tests.
> ARG_0: myGeneratedArgument
- My property
Run completed in 1 second, 623 milliseconds.
Total number of tests run: 1
Suites: completed 1, aborted 0
Tests: succeeded 1, failed 0, canceled 0, ignored 0, pending 0
All tests passed.

So the problem is: my property is falsified, but the test passes !?! Does someone see what's wrong with my code ?

Thanks...

EDIT: I tried to call myProperty instead of myProperty.check, but this is not much better, cause this way, generators seem to be ignored (only one test is launched instead of a hundred).

like image 225
ygu Avatar asked Nov 18 '13 10:11

ygu


1 Answers

Eventually, I found a way of writing my test which is taken into account by Scalatest. I used Checkers Trait instead of PropertyChecks:

class MyProperties extends PropSpec with Checkers {
    property("My property") {
        val myProperty: org.scalacheck.Prop = new MyProperty
        // some code I need to set myProperty
        Checkers.check(myProperty)
    }
}

I'm not sure this is the best way of writing it, but I get what I wanted. Locally :

*** FAILED ***
  GeneratorDrivenPropertyCheckFailedException was thrown during property evaluation.
   (MyProperties.scala:175)
    Falsified after 0 successful property evaluations.
    Location: (MyProperties.scala:175)
    Occurred when passed generated values (
      arg0 = myGeneratedArgument
    )

and finally :

Run completed in 4 seconds, 514 milliseconds.
Total number of tests run: 1
Suites: completed 1, aborted 0
Tests: succeeded 0, failed 1, canceled 0, ignored 0, pending 0
*** 1 TESTS FAILED ***

If someone could evaluate this proposition, I would be pleased ^^

like image 106
ygu Avatar answered Oct 11 '22 13:10

ygu