Flickering HttpClient sometimes throwing IOException

I'm using java.net.http.HttpClient.newHttpClient() under Java 19 (Temurin) and perform sendAsync(...) requests from different treads on the same instance. I assume this is ok, as the javadoc states:

Once built, an HttpClient is immutable...

However, some requests fail with:

java.io.IOException: HTTP/1.1 header parser received no bytes

The weird thing is, it depends on the speed of my requests:

Requests every 5 seconds: 30% failure
Requests every 3 seconds: 0% failure

I've written a test for it:

private final HttpRequest request = HttpRequest.newBuilder()
    .uri(URI.create("https://..."))
    .setHeader("Content-Type", "application/json")
    .POST(HttpRequest.BodyPublishers.ofByteArray("[]".getBytes()))
    .build();

@ParameterizedTest
@ValueSource(ints = {3, 5})
void httpClientTest(int intervalSeconds) throws Exception {
    HttpClient httpClient = HttpClient.newHttpClient();

    httpClient.sendAsync(request, HttpResponse.BodyHandlers.ofByteArray()).get();
    Thread.sleep(Duration.ofSeconds(intervalSeconds));
    httpClient.sendAsync(request, HttpResponse.BodyHandlers.ofByteArray()).get();
    Thread.sleep(Duration.ofSeconds(intervalSeconds));
    httpClient.sendAsync(request, HttpResponse.BodyHandlers.ofByteArray()).get();
    Thread.sleep(Duration.ofSeconds(intervalSeconds));
    httpClient.sendAsync(request, HttpResponse.BodyHandlers.ofByteArray()).get();
    Thread.sleep(Duration.ofSeconds(intervalSeconds));
    httpClient.sendAsync(request, HttpResponse.BodyHandlers.ofByteArray()).get();
}

I've already tried the following:

Doing the same with curl on the command line. No requests fail whatever interval I try. So it's probably not a problem with the server.
Running the tests multiple times in parallel. Still the 5-second-intervals fail (then multiple times in parallel). So it's probably not a problem with the server.
Creating an HttpClient.newHttpClient() for every request. No requests fail whatever interval. So it's probably not a problem with the server but with an internal state of the HttpClient (although it claims to be immutable?).

Do you have an idea what I could do, without needing to create a new HttpClient for every request?

Solution

Here is the answer for the record: the java.net.HttpClient has a long default HTTP/1.1 keepAlive time, which is longer than what usual servers are configured with. This often results in the server closing idle HTTP/1.1 connections before the client does. If the server closes the connection at about the same time than the client tries to reuse it, some IOException might get raised.

If such exceptions are observed too frequently applications should consider adapting the default keepAlive time in the client to some value shorter than what the servers it connects to are using. A default value for the HttpClient HTTP/1.1 keepAlive time can be specified on the command line with: -Djdk.httpclient.keepalive.timeout=duration-in-seconds

So for instance - if a server is configured with a keepAlive time of 5s, you could consider supplying -Djdk.httpclient.keepalive.timeout=3 or -Djdk.httpclient.keepalive.timeout=4 on the client's java command line.