[SOLVED] Websocket SSL handshake failure

I have spring-boot Tomcat server for secure websocket connections. The server accepts Android 4.4, iOS, Firefox, and Chrome clients without failure with an authority-signed certificate. Android 5.0, however, fails the SSL handshake.

Caused by: javax.net.ssl.SSLHandshakeException: Handshake failed
        at com.android.org.conscrypt.OpenSSLEngineImpl.unwrap(OpenSSLEngineImpl.java:436)
        at javax.net.ssl.SSLEngine.unwrap(SSLEngine.java:1006)
        at org.glassfish.grizzly.ssl.SSLConnectionContext.unwrap(SSLConnectionContext.java:172)
        at org.glassfish.grizzly.ssl.SSLUtils.handshakeUnwrap(SSLUtils.java:263)
        at org.glassfish.grizzly.ssl.SSLBaseFilter.doHandshakeStep(SSLBaseFilter.java:603)
        at org.glassfish.grizzly.ssl.SSLFilter.doHandshakeStep(SSLFilter.java:312)
        at org.glassfish.grizzly.ssl.SSLBaseFilter.doHandshakeStep(SSLBaseFilter.java:552)
        at org.glassfish.grizzly.ssl.SSLBaseFilter.handleRead(SSLBaseFilter.java:273)
        at org.glassfish.grizzly.filterchain.ExecutorResolver$9.execute(ExecutorResolver.java:119)
        at org.glassfish.grizzly.filterchain.DefaultFilterChain.executeFilter(DefaultFilterChain.java:284)
        at org.glassfish.grizzly.filterchain.DefaultFilterChain.executeChainPart(DefaultFilterChain.java:201)
        at org.glassfish.grizzly.filterchain.DefaultFilterChain.execute(DefaultFilterChain.java:133)
        at org.glassfish.grizzly.filterchain.DefaultFilterChain.process(DefaultFilterChain.java:112)
        at org.glassfish.grizzly.ProcessorExecutor.execute(ProcessorExecutor.java:77)
        at org.glassfish.grizzly.nio.transport.TCPNIOTransport.fireIOEvent(TCPNIOTransport.java:561)
        at org.glassfish.grizzly.strategies.AbstractIOStrategy.fireIOEvent(AbstractIOStrategy.java:112)
        at org.glassfish.grizzly.strategies.WorkerThreadIOStrategy.run0(WorkerThreadIOStrategy.java:117)
        at org.glassfish.grizzly.strategies.WorkerThreadIOStrategy.access$100(WorkerThreadIOStrategy.java:56)
        at org.glassfish.grizzly.strategies.WorkerThreadIOStrategy$WorkerThreadRunnable.run(WorkerThreadIOStrategy.java:137)
        at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:565)
        at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.run(AbstractThreadPool.java:545)
at java.lang.Thread.run(Thread.java:818)
 Caused by: javax.net.ssl.SSLProtocolException: SSL handshake terminated: ssl=0xa1f34200: Failure in SSL library, usually a protocol error
error:1408E0F4:SSL routines:SSL3_GET_MESSAGE:unexpected message (external/openssl/ssl/s3_both.c:498 0xac526e61:0x00000000)
        at com.android.org.conscrypt.NativeCrypto.SSL_do_handshake_bio(Native Method)
        at com.android.org.conscrypt.OpenSSLEngineImpl.unwrap(OpenSSLEngineImpl.java:423)

I think the problem is with TLS or the cipher suites due to changes in Android 5.0 Lollipop, and not with the certificates because the other clients connect, but I cannot figure out how to tell what is happening on the client side of the connection because SSL debugging does not appear to be supported on Android. The problem is likely very similar to this one, which is also not resolved yet but suggests the problem is with cipher suites. The Android bugs 88313 81603 developer-preview-1989 seem to indicate the Android implementation is correct but server configuration or implementation of cipher suites may not be.

I have set the following server cipher suites

server.ssl.ciphers = TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA,TLS_RSA_WITH_AES_128_CBC_SHA,TLS_DHE_RSA_WITH_AES_128_CBC_SHA,TLS_DHE_DSS_WITH_AES_128_CBC_SHA

In particular, the TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA is on the list of supported protocols for Android for API 11+.

I verified the server supports this

openssl s_client -connect server:port

which returns

SSL-Session:
Protocol  : TLSv1.2
Cipher    : ECDHE-RSA-AES128-SHA

There is a slight mismatch in names between openssl and java, but the openssl documentation says these are the same cipher suite.

My server supports and negotiates first a cipher suite with the openssl client that is compatible with Android 5.0. I expect Android 5.0 to connect without issue, but it fails.

Has anyone successfully connected Android 5.0 secure websocket connections to Tomcat? Are there cipher suites that are known to work? Is there a way to debug the Android client side SSL implementation?

UPDATE

Network trace results:

SYN -->
<-- SYN, ACK
ACK -->
<-- Data
ACK -->
<-- certificates, SSL/TLS params? 1
<-- 2
<-- 3
<-- 4
ACK --> 
ACK --> 
ACK --> 
FIN(!), ACK -->

When the Android 5.0 device (a Nexus 5) receives the server certificate information sent in 4-5 packets, it responds with a variable number (2-4) ACKs then a FIN, ACK. In the successful trace, the client does not send a FIN. The Android 5 client does not like something it gets from the server.

For the failure, the server SSL debugging info says:

http-nio-8080-exec-10, called closeOutbound()
http-nio-8080-exec-10, closeOutboundInternal()
http-nio-8080-exec-10, SEND TLSv1.2 ALERT:  warning, description = close_notify
http-nio-8080-exec-10, WRITE: TLSv1.2 Alert, length = 2
[Raw write]: length = 7
0000: 15 03 03 00 02 01 00

UPDATE 2

Here is a bare-bones Tyrus Android application to use

package edu.umd.mindlab.androidssldebug;

import android.support.v7.app.ActionBarActivity;
import android.os.Bundle;
import android.util.Log;
import android.view.Menu;
import android.view.MenuItem;
import android.widget.TextView;

import org.glassfish.tyrus.client.ClientManager;

import java.io.ByteArrayOutputStream;
import java.io.PrintStream;
import java.net.URI;

import javax.websocket.ClientEndpoint;
import javax.websocket.CloseReason;
import javax.websocket.OnClose;
import javax.websocket.OnError;
import javax.websocket.OnMessage;
import javax.websocket.OnOpen;
import javax.websocket.Session;

@ClientEndpoint
public class MainActivity extends ActionBarActivity {
    public static final String TAG = "edu.umd.mindlab.androidssldebug";
    final Object annotatedClientEndpoint = this;

    @Override
    protected void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        setContentView(R.layout.activity_main);
    }

    @Override
    protected void onStart(){
        super.onStart();
        final Object annotatedClientEndpoint = this;
        new Thread(new Runnable(){
            @Override
            public void run() {
                try {
                    URI connectionURI = new URI("wss://mind7.cs.umd.edu:8080/test");
                    ClientManager client = ClientManager.createClient();
                    Object clientEndpoint = annotatedClientEndpoint;
                    client.connectToServer(clientEndpoint, connectionURI);
                }
                catch(Exception e){
                    ByteArrayOutputStream byteStream = new ByteArrayOutputStream();
                    PrintStream printStream = new PrintStream(byteStream);
                    e.printStackTrace(printStream);
                    final String message = byteStream.toString();
                    Log.e(TAG, message);
                    e.printStackTrace();
                    runOnUiThread(new Runnable() {
                        public void run() {
                            TextView outputTextView = (TextView) findViewById(R.id.outputTextView);
                            outputTextView.setText(message);
                        }
                    });
                }
            }
        }).start();

    }

    @Override
    public boolean onCreateOptionsMenu(Menu menu) {
        // Inflate the menu; this adds items to the action bar if it is present.
        getMenuInflater().inflate(R.menu.menu_main, menu);
        return true;
    }

    @Override
    public boolean onOptionsItemSelected(MenuItem item) {
        // Handle action bar item clicks here. The action bar will
        // automatically handle clicks on the Home/Up button, so long
        // as you specify a parent activity in AndroidManifest.xml.
        int id = item.getItemId();

        //noinspection SimplifiableIfStatement
        if (id == R.id.action_settings) {
            return true;
        }

        return super.onOptionsItemSelected(item);
    }

    @OnOpen
    public void onOpen(Session session) {
        Log.i(TAG, "opened");
        runOnUiThread(new Runnable() {
            public void run() {
                TextView outputTextView = (TextView) findViewById(R.id.outputTextView);
                outputTextView.setText("opened");
            }
        });

    }

    @OnMessage
    public void onMessage(String message, Session session) {
        Log.i(TAG, "message: " + message);
    }

    @OnClose
    public void onClose(Session session, CloseReason closeReason) {
        Log.i(TAG, "close: " + closeReason.toString() );
    }

    @OnError
    public void onError(Session session, Throwable t) {
        final String message = "error: " + t.toString();
        Log.e(TAG, message);
        runOnUiThread(new Runnable() {
            public void run() {
                TextView outputTextView = (TextView) findViewById(R.id.outputTextView);
                outputTextView.setText(message);
            }
        });
    }

}