Increase or remove content length restrictions in Spring-Boot's embedded Netty

Question

I'm putting a Spring Cloud Gateway in front of some existing microservices. It mostly works, but I have a websocket (SockJS) connection which (apparently) transfers huge amounts of data.

Turns out that Netty apparently has a content length maximum -- when I trip that limit in my SockJS route, I get this error:

2018-06-22 16:47:58.740 ERROR 11164 --- [ctor-http-nio-5] r.ipc.netty.channel.ContextHandler       : Error cannot be forwarded to user-facing Mono

io.netty.handler.codec.TooLongFrameException: content length exceeded 65536 bytes.
    at io.netty.handler.codec.MessageAggregator.handleOversizedMessage(MessageAggregator.java:399) [netty-codec-4.1.24.Final.jar:4.1.24.Final]
    at io.netty.handler.codec.MessageAggregator.invokeHandleOversizedMessage(MessageAggregator.java:383) [netty-codec-4.1.24.Final.jar:4.1.24.Final]
    at io.netty.handler.codec.MessageAggregator.decode(MessageAggregator.java:277) [netty-codec-4.1.24.Final.jar:4.1.24.Final]
    at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) [netty-codec-4.1.24.Final.jar:4.1.24.Final]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.24.Final.jar:4.1.24.Final]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.24.Final.jar:4.1.24.Final]
    at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.24.Final.jar:4.1.24.Final]
    at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:310) [netty-codec-4.1.24.Final.jar:4.1.24.Final]
    at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:284) [netty-codec-4.1.24.Final.jar:4.1.24.Final]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.24.Final.jar:4.1.24.Final]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.24.Final.jar:4.1.24.Final]
    at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.24.Final.jar:4.1.24.Final]
    at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1434) [netty-transport-4.1.24.Final.jar:4.1.24.Final]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.24.Final.jar:4.1.24.Final]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.24.Final.jar:4.1.24.Final]
    at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:965) [netty-transport-4.1.24.Final.jar:4.1.24.Final]
    at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) [netty-transport-4.1.24.Final.jar:4.1.24.Final]
    at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:645) [netty-transport-4.1.24.Final.jar:4.1.24.Final]
    at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:580) [netty-transport-4.1.24.Final.jar:4.1.24.Final]
    at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:497) [netty-transport-4.1.24.Final.jar:4.1.24.Final]
    at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:459) [netty-transport-4.1.24.Final.jar:4.1.24.Final]
    at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:884) [netty-common-4.1.24.Final.jar:4.1.24.Final]
    at java.lang.Thread.run(Thread.java:748) [na:1.8.0_161]

How do I configure the embedded Netty in spring-boot to not have a content length limit? I have no control over the length of the message being returned in the response.

The route in the gateway looks like this:

- id: check_status_sockjs
  uri: http://foo.example.com:8080
  predicates:
    - Path=/foo/status/**
  filters:
    - RewritePath=/foo/status/(?<segment>.*), /status/$\{segment}

The service at the other end that I'm routing the sockJS request to is a Spring-Boot app as well, deployed as a war in a Tomcat 8.5 server. When I interact directly with the backing service, I do not run into any issues with this content length stuff -- it's only when I try to route through the embedded Netty in my gateway app.

I also successfully send and receive smaller messages across this route -- it's only when I hit this apparent length limit does it explode.

My dependencies in the gateway are this:

• spring-cloud-starter-gateway (2.0.0.RELEASE)
• spring-boot-starter-webflux (2.0.2.RELEASE)
• spring-boot-starter-actuator (2.0.2.RELEASE)

Answer 1

How do I configure the embedded Netty in spring-boot to not have a content length limit?

It is currently not possible. See reactor-netty #223 and Spring Framework #16228.

---

Analysis:

Turns out that Netty apparently has a content length maximum

The limitation comes from reactor-netty, not directly from Netty.

The limit comes from reactor. ... .WebsocketInbound, a Java interface which will aggregate up to 65,536 bytes per frame.

I could find only one class that implements WebsocketInbound: HttpServerWSOperations. This class doesn't change the default method WebsocketInbound.aggregateFrames so the limit of 65k bytes remains.

HttpServerWSOperations is used by the final method HttpServerOperations.withWebsocketSupport and it is instantiated directly, so you can not change the implementation.

Answer 2

It is now possible using the following boms:

dependencyManagement {
  imports {
    mavenBom("org.springframework.boot:spring-boot-dependencies:2.2.2.RELEASE")
    mavenBom("org.springframework.cloud:spring-cloud-dependencies:Hoxton.RELEASE") 
  }
}

And then in your gateway application config you can set:

spring.cloud.gateway.httpclient.websocket.max-frame-payload-length: <bytes_new_limit>

I was struggling with this for a while, my spring boot was on '2.1.10.RELEASE'. They fixed it in the new version of webflux that comes with those managed dependencies.