Recently I migrated from Apache Commons Http Client 3.x to the new Apache 4.x http client. Everything was replaced like Multithreaded Http Client Connection Manager to PoolingHttpClientManager. Everything works except for uploading large files. It enters a put method that takes path and Input stream and it fails when I try to upload a large file. This is the behavior only when I put my program on cloud it works fine If I create it as exe file and install it & also in eclipse environment it works fine. I am using the webdav functionality of tomcat and in cloud webdav server is placed behind proxy. The following is the code of putMethod used for uploading files
HttpPut method = null;
try
{
method = new HttpPut(this.encodePath(path));
setProxy(method);
}
catch (MalformedURLException | URISyntaxException ex)
{
throw new HttpURIException();
}
String contentType = getGetContentType();
if ((contentType != null) && (contentType.length() > 0))
{
method.setHeader("Content-Type", contentType);
}
InputStreamEntity ise = new InputStreamEntity(inputStream);
ise.setChunked(true);
ise.setContentType(contentType);
BufferedHttpEntity bhe = new BufferedHttpEntity(ise);
method.setEntity(bhe);
try
{
HttpResponse response = this.client.execute(method);
return (response.getStatusLine().getStatusCode() >= 200) && (response.getStatusLine().getStatusCode() < 300);
}
finally
{
method.releaseConnection();
}
}
The following is the stack trace of the error
ERROR - FileUploadHandler - IO Error while file upload. java.net.SocketException: Connection reset by peer (Write failed) at java.net.SocketOutputStream.socketWrite0(Native Method) ~[?:?] at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:110) ~[?:?] at java.net.SocketOutputStream.write(SocketOutputStream.java:150) ~[?:?] at org.apache.http.impl.io.SessionOutputBufferImpl.streamWrite(SessionOutputBufferImpl.java:124) ~[httpcore-4.4.13.jar:4.4.13] at org.apache.http.impl.io.SessionOutputBufferImpl.write(SessionOutputBufferImpl.java:160) ~[httpcore-4.4.13.jar:4.4.13] at org.apache.http.impl.io.ContentLengthOutputStream.write(ContentLengthOutputStream.java:113) ~[httpcore-4.4.13.jar:4.4.13] at org.apache.http.impl.io.ContentLengthOutputStream.write(ContentLengthOutputStream.java:120) ~[httpcore-4.4.13.jar:4.4.13] at org.apache.http.entity.BufferedHttpEntity.writeTo(BufferedHttpEntity.java:105) ~[httpcore-4.4.13.jar:4.4.13] at org.apache.http.impl.DefaultBHttpClientConnection.sendRequestEntity(DefaultBHttpClientConnection.java:156) ~[httpcore-4.4.13.jar:4.4.13] at org.apache.http.protocol.HttpRequestExecutor.doSendRequest(HttpRequestExecutor.java:238) ~[httpcore-4.4.13.jar:4.4.13] at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:123) ~[httpcore-4.4.13.jar:4.4.13] at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272) ~[httpclient-4.5.13.jar:4.5.13] at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) ~[httpclient-4.5.13.jar:4.5.13] at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) ~[httpclient-4.5.13.jar:4.5.13] at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) ~[httpclient-4.5.13.jar:4.5.13] at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) ~[httpclient-4.5.13.jar:4.5.13] at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) ~[httpclient-4.5.13.jar:4.5.13] at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:108) ~[httpclient-4.5.13.jar:4.5.13] at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56) ~[httpclient-4.5.13.jar:4.5.13]
In the latest client I am using the following way to create a new client
PoolingHttpClientConnectionManager conManager = new PoolingHttpClientConnectionManager();
connectionManager.setMaxTotal(40);
connectionManager.setDefaultMaxPerRoute(20);
RequestConfig defaultRequestConfig = RequestConfig.custom().setConnectTimeout(CONNECTION_TIMEOUT).build();
SocketConfig socketConfig = SocketConfig.custom().setSoTimeout(SO_TIMEOUT).build();
HttpRequestRetryHandler retryHandler = new DefaultHttpRequestRetryHandler(HTTPMETHOD_RETRY_COUNT, false);
HttpClientBuilder clientBuilder = HttpClients.custom().useSystemProperties().setConnectionManager(conManager).setDefaultRequestConfig(defaultRequestConfig).setDefaultSocketConfig(socketConfig).setRetryHandler(retryHandler);
credentials = new UsernamePasswordCredentials(userName, password);
CredentialsProvider credentialsProvider = new BasicCredentialsProvider();
credentialsProvider.setCredentials(getAuthScope(aHttpUrl), credentials);
clientBuilder.setDefaultCredentialsProvider(credentialsProvider);
boolean shouldUseProxy = isProxy(aConfig);
String proxyHost = System.getProperty(PROPERTY_PROXY_HOST);
if (shouldUseProxy)
{
String proxyPort = System.getProperty(PROPERTY_PROXY_PORT, "80");
HttpHost proxyHosts = new HttpHost(proxyHost, Integer.parseInt(proxyPort));
RequestConfig config = RequestConfig.copy(defaultRequestConfig).setProxy(proxyHosts).build();
clientBuilder.setDefaultRequestConfig(config);
}
Is there something I should add or modify? or should I change some timeout settings for the proxy server? Via code I set the timeouts to 5000 and retry count to 3.
With the same configurations it worked with old http client.
I tried various ways of modifying the input stream and tried to implement custom input stream too.
It works by modifying the code while method execution. You can pass a context variable that can help you authenticate when the method executes.