HTC One bug? Parts of HTTP header appearing in URLConnection InputStream

I have some code to download an XML file that has been shipping in a paid app for a few years - never had any trouble until I recently saw this happen with an HTC One.

Wondering if there is something I am missing or if this should get reported somewhere.

Here is example code that forces the issue:

package com.lutron.davetest;

import java.io.BufferedInputStream;
import java.io.InputStream;
import java.net.URL;
import java.net.URLConnection;

import android.os.AsyncTask;
import android.os.Bundle;
import android.app.Activity;
import android.app.AlertDialog;
import android.content.DialogInterface;
import android.content.DialogInterface.OnClickListener;
import android.view.Menu;
import android.widget.EditText;
import android.widget.TextView;

public class MainActivity extends Activity {

    private TextView mainCenterText;

    @Override
    protected void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        setContentView(R.layout.activity_main);

        mainCenterText = (TextView) findViewById(R.id.mainCenterText);

        AlertDialog.Builder builder = new AlertDialog.Builder(this);
        builder.setTitle("Enter the IP address");
        final EditText edit = new EditText(this);
        builder.setView(edit);
        builder.setPositiveButton("OK", new OnClickListener() {
            @Override
            public void onClick(DialogInterface dialog, int which) {
                mainCenterText.setText("");
                String ip = edit.getText().toString();
                new DownloadTask().execute(ip);
            }
        });
        builder.create().show();
    }

    private class DownloadTask extends AsyncTask<String, Void, Void>{

        @Override
        protected Void doInBackground(String... params) {
            try{
                URL url = new URL("http", params[0], 80, "/file.xml");
                URLConnection connection = url.openConnection();
                InputStream in = new BufferedInputStream(connection.getInputStream());
                byte buf[] = new byte[4096];
                int len;
                while((len = in.read(buf)) != -1 ) {
                    final String data = new String(buf, 0, len);
                    runOnUiThread(new Runnable() {

                        @Override
                        public void run() {
                            mainCenterText.append(data);
                        }
                    });
                }

            }catch(Exception e){
                e.printStackTrace();
            }
            return null;
        }
    }

    @Override
    public boolean onCreateOptionsMenu(Menu menu) {
        getMenuInflater().inflate(R.menu.main, menu);
        return true;
    }

}

What happens is that on every other device when reading from the stream you only get the file contents.

On the HTC One you get THIS and then the file contents:

onnection: close

Last-Modified: ???

Content-Type: text/html

This is with Android 4.1.2.

Is there some way to configure the URLConnection to avoid this?

Wow.

Digging into the issue, we realized this only occurred on specific pages, and verified that when this phone navigates to any pages served by them in its browser, these HTTP headers still appear in the body causing errors.

I wiresharked a small HTTP session today and noticed that these headers are being considered part of the body!

All the line breaks in the HTTP layer were \n\r, or LF CR.

From HTTP 1.1 RFC 2616: http://www.ietf.org/rfc/rfc2616.txt

HTTP/1.1 defines the sequence CR LF as the end-of-line marker for all
protocol elements except the entity-body (see appendix 19.3 for
tolerant applications). The end-of-line marker within an entity-body
is defined by its associated media type, as described in section 3.7.

   CRLF           = CR LF

...

    […] a bare CR
or LF MUST NOT be substituted for CRLF within any of the HTTP control
structures (such as header fields and multipart boundaries).

...

6 Response

After receiving and interpreting a request message, a server responds
with an HTTP response message.

   Response      = Status-Line               ; Section 6.1
                   *(( general-header        ; Section 4.5
                    | response-header        ; Section 6.2
                    | entity-header ) CRLF)  ; Section 7.1
                   CRLF
                   [ message-body ]          ; Section 7.2

This seems like a big issue - however, the only way the app could have lasted until now appears the suggestion (19.3) that applications SHOULD tolerate a single LF.

So I guess HTC ignored this suggestion on the HTC One!! :-P

To work around this, I tried to use HttpClient (version included with Android SDK), but that also had the same problem. I wanted to try using the jar files from Apache HTTP Components, but of course those have a package name conflict with the version in the Android framework, so I repackaged them with Jar Jar Links. Once the imports were replaced with that it worked!