I have went through all the question related to this and yet, I haven't found a solution that works for me.

Im using retrofit 2.8.1 and OkHttp 4.5.0.

My service interface looks like the following

public interface MlApiService
{
    @POST
    @Multipart
    Call<List<PreprocessedDocument>> postDocument( @Url String apiUrl, @Part MultipartBody.Part document,
    @Part ( "document_id") RequestBody documentId );
}

And I build the client like the following with requestTimeoutInSeconds set to 90 seconds.

public void init()
{
    GsonBuilder gson = new GsonBuilder();
    gson.registerTypeAdapter( new TypeToken<List<PreprocessedDocument>>() {}.getType(), new CustomResponseDeserializer() );

    HttpLoggingInterceptor logInterceptor = new HttpLoggingInterceptor();
    logInterceptor.setLevel( HttpLoggingInterceptor.Level.HEADERS );

    OkHttpClient client = new OkHttpClient.Builder().retryOnConnectionFailure( true ).addInterceptor( logInterceptor )
        .readTimeout( requestTimeoutInSeconds, TimeUnit.SECONDS ).build();
    //Dummy Base URL must be provided. otherwise client won't get initialized
    Retrofit retrofit = new Retrofit.Builder().baseUrl( "http://thisIsJustDummyUrlForTheSakeOfAddingBaseUrl.com/" )
        .client( client ).addConverterFactory( GsonConverterFactory.create( gson.setLenient().create() ) ).build();
    mlApiService = retrofit.create( MlApiService.class );
}

The request reaches the server and just when the server responds I get the following error:

Caused by: java.io.IOException: unexpected end of stream on Connection{34.XXX.XXX.9:8085, proxy=DIRECT hostAddress=/34.XXX.XXX.9:8085 cipherSuite=none protocol=http/1.1}
    at okhttp3.internal.http1.Http1Codec.readResponseHeaders(Http1Codec.java:203)
    at okhttp3.internal.http.CallServerInterceptor.intercept(CallServerInterceptor.java:88)
    at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)
    at okhttp3.internal.connection.ConnectInterceptor.intercept(ConnectInterceptor.java:45)
Caused by: java.io.EOFException: \n not found: limit=0 content=…
    at okio.RealBufferedSource.readUtf8LineStrict(RealBufferedSource.java:227)
    at okio.RealBufferedSource.readUtf8LineStrict(RealBufferedSource.java:211)
    at okhttp3.internal.http1.Http1Codec.readResponseHeaders(Http1Codec.java:187)

Few things which I have tried so far

1. retryOnConnectionFailure(true)
2. .addHeader("Connection","close")
3. .header("Accept-Encoding", "identity")

The API works fine from postman but it fails when I try from code. So I tried the same headers sent by postman. Still no luck.

Few observations:

1. It sometimes works. Doesn't fail always.(The same file works always with postman)
2. It always works for other files(never an issue).
3. The request reaches the server and process the request without any errors and responds too. I get the error immediately after server completes processing and responds back to client.

EDIT 1: The server I hit is backed by gunicorn/20.0.4 and uses Flask. I don't have access to the server code. And I doubt that the response sent received has some garbage characters causing the error. I don't know how to log the raw response before being read by retrofit/okhttp.

EDIT 2:

I ran the Curl command with verbose and this is what I got.

< HTTP/1.1 100 Continue

• Empty reply from server
• Connection #0 to host xx.xxx.xxx.9 left intact curl: (52) Empty reply from server

Short story

The problem was with the server I was hitting. It was not sending any response (literally nothing. No headers, no body, nothing).

Long story

So after going through all the related answers on stackoverflow, other good websites and trying out so many solution which I have mentioned in the question itself, it did not solve my issue.

After carefully reading the stack trace, I came across the following line.

okhttp3.internal.http1.Http1Codec.readResponseHeaders(Http1Codec.java:203)

The client (my code) is trying to read the Response header and that's when the error java.io.EOFException: \n not found: limit=0 content= is thrown.

This gave me a hint that the problem could be with the server and not with the client. So I thought I should try with a different client and see if I can see the raw response.

The first tool that came to my mind was Curl (Postman used to give the generic Could not get any response and this did not happen consistently). I hit the server using curl with verbose option and boom! I got the following response:

curl -v --location --request POST 'http://XX.XXX.XXX.9:8085/psc/document_upload' --form 'document=@/home/user376/Downloads/test-1.pdf' --form 'document_id=22004494_ae7f_4998_a1d8_73249bda9905'
Note: Unnecessary use of -X or --request, POST is already inferred.
*   Trying XX.XXX.XXX.9...
* Connected to XX.XXX.XXX.9 (XX.XXX.XXX.9) port 8085 (#0)
> POST /psc/document_upload HTTP/1.1
> Host: XX.XXX.XXX.9:8085
> User-Agent: curl/7.49.0
> Accept: */*
> Content-Length: 4684053
> Expect: 100-continue
> Content-Type: multipart/form-data; boundary=------------------------a8446c7eedb10689
> 
< HTTP/1.1 100 Continue
* Empty reply from server
* Connection #0 to host XX.XXX.XXX.9 left intact
curl: (52) Empty reply from server

And that confirmed the problem was with the server and not with the client(Retrofit / http).

Moral of the story: Sometimes you have to read the stacktrace word by word even if it seems overwhelming :)