WebFlux WebClient loading entire file into direct buffer memory during multipart upload

Question

I'm building an app with Spring Boot 2.1.6.RELEASE & Webflux

We have an endpoint that we can upload files using multipart and are using WebClient for our uploads.

Our upload client code looks like this

@Component
class UploadClient(
    private val client: WebClient,
) {
    suspend fun upload(filePath: String) =
        client.post()
              .uri("/upload")
              .contentType(MULTIPART_FORM_DATA)   
              .body(BodyInserters.fromMultipartData(generateMultipartBody(filePath)))
              .retrieve()
              .bodyToMono(UploadResult::class.java)
              .awaitFirst()

    private fun generateMultipartBody(filePath: String): MultiValueMap<String, HttpEntity<*>> {
        val builder = MultipartBodyBuilder()
        builder.part("file", FileSystemResource(filePath))
        return builder.build()
    }
}

However when we upload a large file, (1.6gb) we are seeing that this entire file is loaded into direct memory:

As the file is uploaded, the memory is released, then when the next file is uploaded you can see the spike in memory again.

For contrast I tried replacing WebClient with https://github.com/AsyncHttpClient/async-http-client and the memory usage is much lower, ~60mb per upload

I don't really want to pull in another http client dependency when the WebFlux client is fine for all of our other uses.

Answer 1

This was a bug in Spring Framework - https://github.com/spring-projects/spring-framework/issues/23518