I have to restart IDE to run tests

Question

Weird bug with Android Studio 2.2 when running a single test directly in IDE - I'll either have "No tests found" or run my previously compiled test.

Example:

@Test
fun testyTest() {
}

I'll create this test, and get the "No tests found". So I restart, and test is found! Obviously, the above test passes because it's empty. Next, I want my test to fail. So I manually throw an exception and add one line of code, below.

@Test
fun testyTest() {
    throw RuntimeException()
}

Now I recompile, and this test passes too, because IntelliJ is for some reason has cached the previous test and reruns that. To get this test to properly fail - I must restart the IDE.

Anyone know what's going on? Note - I'm using Kotlin pretty heavily in this project.

EDIT 2: This is a JVM test (non-android, src/test/java) in an android library project.

EDIT: here's a full test class

package com.example.zak

import org.junit.Before
import org.junit.Test


class ExampleTest {

    @Before
    fun setUp() {

    }

    @Test
    fun testyTest() {
        //throw RuntimeException()
    }
}

Answer 1

Found the fix...the issue only occurs when there's a variant selection conflict. Removing the conflict fixed the issue.

Answer 2

Check if you're have all these dependencies:

dependencies {
  compile "org.jetbrains.kotlin:kotlin-stdlib:$kotlin_version"
  testCompile 'junit:junit:4.11'
  testCompile "org.jetbrains.kotlin:kotlin-test-junit:$kotlin_version"
}

Also as you're using Kotlin please disable entirely Instant Run in File -> Preferences.

EDIT: You're talking about tests in library - do modules and project has these dependencies and proper configuration.

It looks like after Gradle rebuilding, test works fine.