launch long-running task then immediately send HTTP response

Question

Using ktor HTTP server, I would like to launch a long-running task and immediately return a message to the calling client. The task is self-sufficient, it's capable of updating its status in a db, and a separate HTTP call returns its status (i.e. for a progress bar).

What I cannot seem to do is just launch the task in the background and respond. All my attempts at responding wait for the long-running task to complete. I have experimented with many configurations of runBlocking and coroutineScope but none are working for me.

// ktor route
get("/launchlongtask") {
    val text: String = (myFunction(call.request.queryParameters["loops"]!!.toInt()))
    println("myFunction returned")
    call.respondText(text)
}

// in reality, this function is complex... the caller (route) is not able to 
// determine the response string, it must be done here
suspend fun myFunction(loops : Int) : String {
    runBlocking {
        launch {
            // long-running task, I want to launch it and move on
            (1..loops).forEach {
                println("this is loop $it")
                delay(2000L)
                // updates status in db here
            }
        }
        println("returning")
        // this string must be calculated in this function (or a sub-function)
        return@runBlocking "we just launched $loops loops" 
    }
    return "never get here" // actually we do get here in a coroutineScope
}

output:

returning
this is loop 1
this is loop 2
this is loop 3
this is loop 4
myFunction returned

expected:

returning
myFunction returned
(response sent)
this is loop 1
this is loop 2
this is loop 3
this is loop 4

Answer 1

inspired by this answer by Lucas Milotich, I utilized CoroutineScope(Job()) and it seems to work:

suspend fun myFunction(loops : Int) : String {
    CoroutineScope(Job()).launch { 
        // long-running task, I want to launch it and move on
        (1..loops).forEach {
            println("this is loop $it")
            delay(2000L)
            // updates status in db here
        }
    }
    println("returning")
    return "we just launched $loops loops" 
}

not sure if this is resource-efficient, or the preferred way to go, but I don't see a whole lot of other documentation on the topic.

Answer 2

Just to explain the issue with the code in your question, the problem is using runBlocking. This is meant as the bridge between the synchronous world and the async world of coroutines and

"the name of runBlocking means that the thread that runs it ... gets blocked for the duration of the call, until all the coroutines inside runBlocking { ... } complete their execution."

(from the Coroutine docs).

So in your first example, myFunction won't complete until your coroutine containing loop completes.

The correct approach is what you do in your answer, using CoroutineScope to launch your long-running task. One thing to point out is that you are just passing in a Job() as the CoroutineContext parameter to the CoroutineScope constructor. The CoroutineContext contains multiple things; Job, CoroutineDispatcher, CoroutineExceptionHandler... In this case, because you don't specifiy a CoroutineDispatcher it will use CoroutineDispatcher.Default. This is intended for CPU-intensive tasks and will be limited to "the number of CPU cores (with a minimum of 2)". This may or may not be want you want. An alternative is CoroutineDispatcher.IO - which has a default of 64 threads.