Suspending function test with MockWebServer

Question

I'm testing api that returns result using suspending function with MockWebServer, but it does not work with runBlockingTest, testCoroutineDispatcher, testCorounieScope unless a launch builder is used, why?

abstract class AbstractPostApiTest {

    internal lateinit var mockWebServer: MockWebServer

    private val responseAsString by lazy {
        getResourceAsText(RESPONSE_JSON_PATH)
    }

    @BeforeEach
    open fun setUp() {
        mockWebServer = MockWebServer()
        println("AbstractPostApiTest setUp() $mockWebServer")
    }


    @AfterEach
    open fun tearDown() {
        mockWebServer.shutdown()
    }

    companion object {
        const val RESPONSE_JSON_PATH = "posts.json"
    }


    @Throws(IOException::class)
    fun enqueueResponse(
        code: Int = 200,
        headers: Map<String, String>? = null
    ): MockResponse {

        // Define mock response
        val mockResponse = MockResponse()
        // Set response code
        mockResponse.setResponseCode(code)

        // Set headers
        headers?.let {
            for ((key, value) in it) {
                mockResponse.addHeader(key, value)
            }
        }

        // Set body
        mockWebServer.enqueue(
            mockResponse.setBody(responseAsString)
        )

        return mockResponse
    }


}


class PostApiTest : AbstractPostApiTest() {

    private lateinit var postApi: PostApiCoroutines

    private val testCoroutineDispatcher = TestCoroutineDispatcher()

    private val testCoroutineScope = TestCoroutineScope(testCoroutineDispatcher)

    @BeforeEach
    override fun setUp() {
        super.setUp()

        val okHttpClient = OkHttpClient
            .Builder()
            .build()

        postApi = Retrofit.Builder()
            .baseUrl(mockWebServer.url("/"))
            .addConverterFactory(GsonConverterFactory.create())
            .client(okHttpClient)
            .build()
            .create(PostApiCoroutines::class.java)


        Dispatchers.setMain(testCoroutineDispatcher)

    }

    @AfterEach
    override fun tearDown() {
        super.tearDown()

        Dispatchers.resetMain()
        try {
            testCoroutineScope.cleanupTestCoroutines()
        } catch (exception: Exception) {
            exception.printStackTrace()
        }
    }

    @Test
    fun `Given we have a valid request, should be done to correct url`() =
        testCoroutineScope.runBlockingTest {

            // GIVEN
            enqueueResponse(200, RESPONSE_JSON_PATH)

            // WHEN
              postApi.getPostsResponse()

            advanceUntilIdle()

            val request = mockWebServer.takeRequest()

            // THEN
            Truth.assertThat(request.path).isEqualTo("/posts")

        }
}

Results error: java.lang.IllegalStateException: This job has not completed yet

This test does not work if launch builder is used, and if launch builder is used it does not require testCoroutineDispatcher or testCoroutineScope, what's the reason for this? Normally suspending functions pass without being in another scope even with runBlockingTest

 @Test
    fun `Given we have a valid request, should be done to correct url`() =
        runBlockingTest {

            // GIVEN
            enqueueResponse(200, RESPONSE_JSON_PATH)

            // WHEN
            launch {
                postApi.getPosts()
            }

            val request = mockWebServer.takeRequest()

            // THEN
            Truth.assertThat(request.path).isEqualTo("/posts")

        }

The one above works.

Also the test below pass some of the time.

@Test fun Given api return 200, should have list of posts() = testCoroutineScope.runBlockingTest {

    // GIVEN
    enqueueResponse(200)

    // WHEN
    var posts: List<Post> = emptyList()
    launch {
        posts = postApi.getPosts()
    }

    advanceUntilIdle()

    // THEN
    Truth.assertThat(posts).isNotNull()
    Truth.assertThat(posts.size).isEqualTo(100)

}

I tried many combinations invoking posts = postApi.getPosts() without launch, using async, putting enqueueResponse(200) inside async async { enqueueResponse(200) }.await() but tests failed, sometimes it pass sometimes it does not some with each combination.

Answer 1

There is a bug with runBlockTest not waiting for other threads/jobs to complete before completing the coroutine that the test is running in.

I tried using runBlocking with success (I use the awesome port of Hamcrest to Kotlin Hamkrest)

fun `run test` = runBlocking {
  mockWebServer.enqueue(MockResponse().setResponseCode(200).setBody(""))

  // make HTTP call 

  val result = mockWebServer.takeRequest(2000L, TimeUnit.MILLISECONDS)

  assertThat(result != null, equalTo(true))
}

There's a few things to note here:

1. The use of thread blocking calls should never be called without a timeout. Always better to fail with nothing, then to block a thread forever.

2. The use of runBlocking might be considered by some to be no no. However this blog post outlines the different method of running concurrent code, and the different use cases for them. We normally want to use runBlockingTest or (TestCoroutineDispatcher.runBlockingTest) so that our test code and app code are synchronised. By using the same Dispatcher we can make sure that the jobs all finish, etc. TestCoroutineDispatcher also has that handy "clock" feature to make delays disappear. However when testing the HTTP layer of the application, and where there is a mock server running on a separate thread we have a synchronisation point being takeRequest. So we can happily use runBlocking to allow us to use coroutines and a mock server running on a different thread work together with no problems.