I am updating an old project's dependencies. One of the transitive changes pulled in is a version bump of apache-httpclient from 4.5 to 5.2. This project implements the recommended IdleConnectionMonitorThread from the 4.5.x Tutorial, Chapter 2 Connection Management, Section 5 Connection eviction policy, which says this:
One of the major shortcomings of the classic blocking I/O model is that the network socket can react to I/O events only when blocked in an I/O operation. When a connection is released back to the manager, it can be kept alive however it is unable to monitor the status of the socket and react to any I/O events. If the connection gets closed on the server side, the client side connection is unable to detect the change in the connection state (and react appropriately by closing the socket on its end).
HttpClient tries to mitigate the problem by testing whether the connection is 'stale', that is no longer valid because it was closed on the server side, prior to using the connection for executing an HTTP request. The stale connection check is not 100% reliable. The only feasible solution that does not involve a one thread per socket model for idle connections is a dedicated monitor thread used to evict connections that are considered expired due to a long period of inactivity. The monitor thread can periodically call ClientConnectionManager#closeExpiredConnections() method to close all expired connections and evict closed connections from the pool. It can also optionally call ClientConnectionManager#closeIdleConnections() method to close all connections that have been idle over a given period of time.
public static class IdleConnectionMonitorThread extends Thread {
private final HttpClientConnectionManager connMgr;
private volatile boolean shutdown;
public IdleConnectionMonitorThread(HttpClientConnectionManager connMgr) {
super();
this.connMgr = connMgr;
}
@Override
public void run() {
try {
while (!shutdown) {
synchronized (this) {
wait(5000);
// Close expired connections
connMgr.closeExpiredConnections();
// Optionally, close connections
// that have been idle longer than 30 sec
connMgr.closeIdleConnections(30, TimeUnit.SECONDS);
}
}
} catch (InterruptedException ex) {
// terminate
}
}
public void shutdown() {
shutdown = true;
synchronized (this) {
notifyAll();
}
}
}
There is a previous question Is Commons Components HttpClient IdleConnectionMonitorThread needed? exploring
why [this] is recommended when both implementations of HttpClientConnectionManager (PoolingHttpClientConnectionManager and BasicHttpClientConnectionManager) have logic for closing old connections?
However, it is limited to the context of httpclient-4.x. I noticed that Apache's documentation for v5.x does not include a tutorial, and does not reference the IdleConnectionMonitorThread anywhere in the quick start guide, nor migration from 4.x guide.
So is it still recommended to periodically clear idle and stale connections? Or is there something hidden in the guts of v5.x that manages this automatically in a way that 4.x didn't/couldn't? More broadly, is there an official Apache tutorial for v5.x on the scope of the 4.x tutorial that I am missing?
The limitation of the classic Java (blocking) i/o of only being able to react to i/o events when the underlying socket is blocked in a read / write operation applies to all versions of Apache HttpClient and all network code based on the classic I/O APIs.
Idle client connections kept in the connection pool have no way of reacting to the opposite endpoint closing the connection on its end. It may still be beneficial to monitor connections in the pool and pro-actively drop those that have been idle for too long.