Webflux + Netty NIO performance decrease ~30 times compared to traditional IO

Question

We are having problems in network transfer while using Spring Boot 2.0, Webflux 5.0.7 and Netty 4.1.25. We want to transfer 100000 items serialized as JSON (roughly 10Mb of network traffic) to 1 client.

The performance of network transfer dramatically differ between NIO and traditional IO. The test results are below:

Start reading 100000 from server in 5 iterations
Avg HTTP 283 ms
Avg stream 8130 ms

At the moment the number of requests per second is not an issue, but the network transfer speed is. We have read that in terms of network speed NIO may be about 30% slower but 1/30x is an overkill.

While sampling on client and serverside we have observed that the reason is mostly on the serverside implementation. From the screanshot below it is visible that most of the time server spends in methods select() and doWrite().

The endpoint code itself:

@RestController
@RequestMapping(produces = {APPLICATION_JSON_VALUE, APPLICATION_STREAM_JSON_VALUE})
@Validated
public class StreamingController {

    @GetMapping("/instruments/{eodDate}")
    public Flux<TestItem> getInstruments(
            @PathVariable @DateTimeFormat(pattern = "yyyy-MM-dd") LocalDate eodDate,
            @RequestParam(required = false) Instant asOfTimestamp) {
        //Generate test data in memory
        List<TestItem> collect = IntStream.range(0, 100000)
             .mapToObj(i -> new TestItem.Builder().build())
             .collect(Collectors.toList());
        return Flux.fromIterable(collect);
    }
}

We are using Spring Boot configuration for Netty and we are suspecting that by default Netty is misconfigured. We are looking for your help. I will add any other details by your request.

Upd: The goal is to read whole response in batches to avoid putting all response into memory, because the expected data volumes are huge (couple of Gb). It is acceptable to consume batch of data on the client side instead of one element.

Answer 1

You're not actually testing NIO vs. IO. Spring WebFlux applications are always using non-blocking IO at the server level (with Netty, Undertow, or any Servlet 3.1+ async IO compliant server).

In this case, you're comparing:

1. serving an "application/json" payload in one shot with Spring WebFlux
2. serving a stream "application/stream+json" response with Spring WebFlux

In the first case, Spring WebFlux is producing a response body in a reactive manner but leaves the buffering and the flushing decisions to the server itself. Writing to the network has a cost, a buffering a but and writing bigger chunks is efficient.

In the second case, you're asking Spring WebFlux to write and flush for every element of the Flux. This is useful when clients are listening to a (possibly infinite) stream of events and there might be some time between two different events. That approach consumes more resources and explains the performance difference.

So this benchmark is not showing IO vs. NIO, but streaming vs. non-streaming.

If you want fine-grained control over response writing/flushing, you can drop down to the level of ServerHttpResponse and use writeAndFlushWith(Flux<Flux<DataBuffer>>), but this is quite low level since you're dealing with DataBuffer instances directly.

An alternative would be to create intermediate JSON objects that hold lists of TestItem, like:

public Flux<TestItemBatch> batch() {
    Flux<TestItem> items= //...;
    Flux<List<TestItem>> itemsLists = items.buffer(100);
    return itemsLists.map(list -> new TestItemBatch(list));
}