javajava-http-client

Progress of download with java.net.http.HttpClient


I use java.net.http.HttpClient to download a large file.

How can I make HttpClient report the progress of the download? For example, can it call a callback every X byte that it has finished downloading?

The download code looks like this:

HttpClient client = HttpClient.newBuilder()
    .followRedirects(Redirect.NORMAL)
    .connectTimeout(Duration.ofSeconds(20))
    .build();

HttpRequest request = HttpRequest.newBuilder().uri(
    new URI("https://example.com/large-file")).build();

// The program blocks here until the download is finished
HttpResponse<Path> response = client.send(request, BodyHandlers.ofFileDownload(
    Path.of("."), StandardOpenOption.WRITE, StandardOpenOption.CREATE));

The file that is downloaded is a couple of hundred MBs. On the user system this takes maybe a minute, during which users are left wondering whether the download is progressing or whether the program is hung.

To help with this I'd like to, while the download is happening, indicate to the user how much of the file that has been downloaded and that the download is proceeding. For example by printing a dot in the terminal when 1 MB of data is completed, or regularly updating a progress bar in a GUI.

How can I do this with HttpClient?


Solution

  • This is rather clunky but the best solution I could find. It uses a custom BodyHandler and BodySubscriber that calls a callback:

    Example:

    BodyHandler<Path> responceHandler = callbackBodyHandler(
        1_000_000,
        nrBytesReceived -> System.out.print("."),
        BodyHandlers.ofFileDownload(Path.of("."),
            StandardOpenOption.WRITE, StandardOpenOption.CREATE));
    
    HttpResponse<Path> response = client.send(request, responceHandler);
    
    

    Implementation:

    private static <T> BodyHandler<T> callbackBodyHandler(int interval, LongConsumer callback, BodyHandler<T> h) {
        return info -> new BodySubscriber<T>() {
            private BodySubscriber<T> delegateSubscriber = h.apply(info);
            private long receivedBytes = 0;
            private long calledBytes = 0;
    
            @Override
            public void onSubscribe(Subscription subscription) {
                delegateSubscriber.onSubscribe(subscription);
            }
    
            @Override
            public void onNext(List<ByteBuffer> item) {
                receivedBytes += item.stream().mapToLong(ByteBuffer::capacity).sum();
    
                if (receivedBytes - calledBytes > interval) {
                    callback.accept(receivedBytes);
                    calledBytes = receivedBytes;
                }
    
                delegateSubscriber.onNext(item);
            }
    
            @Override
            public void onError(Throwable throwable) {
                delegateSubscriber.onError(throwable);
    
            }
    
            @Override
            public void onComplete() {
                delegateSubscriber.onComplete();
            }
    
            @Override
            public CompletionStage<T> getBody() {
                return delegateSubscriber.getBody();
            }
        };
    }