Spring boot cloud gateway closes Websocket connection when package size is larger

Some data is being sent to the back end that is used for user notification and vise versa. This is done by using websockets. Functionality vise there is no problem, BUT when the package or frame is larger in my case 501kb the websocket gets closed.

This websocket connection is routed throug a Spring boot cloud gateway (The 4.1.2 starter for spring-cloud-starter was used). When I go around the gateway things are working even with a larger package or frame. But for some reason the gateway does not let it go through if its bigger data.

What i've tried is setting the "max-frame-payload-length" that is increasing it in hopes that it would work. (The property being spring.cloud.gateway.httpclient.webs.max-frame-payload-lenght) But regardless of how big of a number I set it does not help. On the other side if I set it to a single digit number like 5 the websocket fails to even connect for lack of memory I'm not sure what else I could try.

Edit:

After setting everything to debug level logging in the gateway I've come to the exception io.netty.handler.codec.http.websocketx.CorruptedWebSocketFrameException: Max frame length of 65536 has been exceeded. Now the strugle is increasing that number

Edit 2:

I've traced the problem to the class WebSocketDecoderConfig which is inside of package io.netty.handler.codec.http.websocketx; Some parts of the code use the defined limit that is in the application.yml. But the decoder that throws an exception uses the default limit defined in this class. Which is 65536. And I can't seem to override the decoder limit or alter this configuration through spring boot. OR im doing something wrong

If by any chance someone comes across this question.

Problem is solved. If you have no custom code the property spring.cloud.gateway.httpclient.webs.max-frame-payload-lenght is your answer.

If you have custom code look for classes such as WebSocketService and HandshakeWebSocketService.

What was the problem in my case was that we had an implementation of WebSocketService which internally had instanced a HandshakeWebSocketService without any arguments. What this does is create a default handshake startegy based on what you have under the "hood". This can be tomcat, netty, netty2 and some other options.

This meant that I could set what ever I want in the application.yml it meant nothing.

So what I did is as follows:

public OurImplementationOfWebSocketService(HttpClientProperties httpClientProperties) {
    this.httpClientProperties = httpClientProperties;
    this.handshakeWebSocketService = getHandshakeWebSocketService();
}

private HandshakeWebSocketService getHandshakeWebSocketService() {
    if (isReactorNettyPresent()) {
        return new HandshakeWebSocketService(
                new ReactorNettyRequestUpgradeStrategy(
                        WebsocketServerSpec.builder()
                                .maxFramePayloadLength(
                                        getMaxFramePayloadLengthFromSpringConfiguration()
                                )
                )
        );
    }

    return new HandshakeWebSocketService();
}

private Integer getMaxFramePayloadLengthFromSpringConfiguration() {
    return this.httpClientProperties.getWebsocket()
            .getMaxFramePayloadLength();
}

private boolean isReactorNettyPresent() {
    ClassLoader classLoader = WebSocketServiceWebviewerHealthcheckBeforeHandshake.class.getClassLoader();
    return ClassUtils.isPresent("reactor.netty.http.server.HttpServerResponse", classLoader);
}

I've copied the code and emited specific code to our implementation so if there is a bracket missing, sorry.

But the point here is I inject the properties that are defined by default. Check the same way the class HandshakeWebSocketService checks to see which implementation is under as in netty or something else. And if it is netty I create a netty strategy with the defined max frame length from the application.yml. We dont really plan on changing that any time soon so I went with just adding it in when its netty under the "hood".

So this way thing are still configurable the spring way. So the next person doesnt lose 3 days debuging everything like I did. When the frame size gets changed next.

Hope this helps someone at some point