[SOLVED] Java spring boot server sending response all at once instead of streaming depending on how it's run

Java spring boot server sending response all at once instead of streaming depending on how it's run

I am trying to stream a response from a java server. The response is sent to a Vue client. When the server is run in VS Code, it works beautifully. However, when I run the server using java -server -jar xxx.jar, it seems to be streaming on the server side logs but the client receives it all as one batch. It seems to be a configuration issue with the Java server or a problem with the .jar. It's not a code issue as the exact same code is running. Please enlighten me.

The server log shows the response streaming like below.

OpenAiController - =====>> Thread Message Delta:  approved
OpenAiController - =====>> Thread Message Delta:  solely
OpenAiController - =====>> Thread Message Delta:  for
OpenAiController - =====>> Thread Message Delta:  the
OpenAiController - =====>> Thread Message Delta:  treatment
OpenAiController - =====>> Thread Message Delta:  of
OpenAiController - =====>> Thread Message Delta:  fa
OpenAiController - =====>> Thread Message Delta: ec
OpenAiController - =====>> Thread Message Delta: al
OpenAiController - =====>> Thread Message Delta: oma
OpenAiController - =====>> Thread Message Delta: ,
OpenAiController - =====>> Thread Message Delta:  several
OpenAiController - =====>> Thread Message Delta:  medications
OpenAiController - =====>> Thread Message Delta:  are
OpenAiController - =====>> Thread Message Delta:  commonly
OpenAiController - =====>> Thread Message Delta:  used
OpenAiController - =====>> Thread Message Delta:  to
OpenAiController - =====>> Thread Message Delta:  manage
OpenAiController - =====>> Thread Message Delta:  the
OpenAiController - =====>> Thread Message Delta:  underlying
OpenAiController - =====>> Thread Message Delta:  constipation
OpenAiController - =====>> Thread Message Delta:  that
OpenAiController - =====>> Thread Message Delta:  can
OpenAiController - =====>> Thread Message Delta:  lead

However on the client it gets the entire response all at once after everything is finished.

When it works, I am running the server in VS Code using the following command:

& 'C:\Program Files\Java\jdk-21\bin\java.exe' '-agentlib:jdwp=transport=dt_socket,server=n,suspend=y,address=localhost:62203' '@C:\Users\xxx\AppData\Local\Temp\34\cp_396rcp08t5lealrto29q2ek8w.argfile' 'com.xxx.xxx.xxx'

Where it is not working I am running it as:

nohup java -server -Xms4096m -Xmx4096m -Dfile.encoding=UTF-8 -Dsun.jnu.encoding=UTF-8 -jar /mnt/xxx.jar >/dev/null 2>&1 &

Or just as

java -server -jar /mnt/xxx.jar

Solution

It turned out the reason it was not working was because the output buffer on the Java server running on the Linux machine was too big. I fixed the problem by adding random characters to the message so that it would fill up the buffer and flush immediately. Not sure how to change the buffer size though; couldn't find where the configuration is.