javasocketswebrtcbouncycastledtls

DTLS handshaking failure due to SocketException though the socket was never closed menually


Apologies as the question might be a bit vague. Trying to establish webrtc connection to a webrtc-gateway. While performing the dtls handshaking with accept or connect function, it is throwing SocketException.

Here is the error:

   java.net.SocketException: Socket is closed
    at java.net.DatagramSocket.send(DatagramSocket.java:658)
    at org.bouncycastle.crypto.tls.UDPTransport.send(Unknown Source)
    at org.bouncycastle.crypto.tls.DTLSRecordLayer.sendRecord(Unknown Source)
    at org.bouncycastle.crypto.tls.DTLSRecordLayer.send(Unknown Source)
    at org.bouncycastle.crypto.tls.DTLSReliableHandshake$RecordLayerBuffer.sendToRecordLayer(Unknown Source)
    at org.bouncycastle.crypto.tls.DTLSReliableHandshake.writeHandshakeFragment(Unknown Source)
    at org.bouncycastle.crypto.tls.DTLSReliableHandshake.writeMessage(Unknown Source)
    at org.bouncycastle.crypto.tls.DTLSReliableHandshake.resendOutboundFlight(Unknown Source)
    at org.bouncycastle.crypto.tls.DTLSReliableHandshake.receiveMessage(Unknown Source)
    at org.bouncycastle.crypto.tls.DTLSServerProtocol.serverHandshake(Unknown Source)
    at org.bouncycastle.crypto.tls.DTLSServerProtocol.accept(Unknown Source)
    at callProcessor.DTLSManager.startDTLS(DTLSManager.java:421)
    at callProcessor.DTLSManager.processSTUNResponse(DTLSManager.java:554)

Checked multiple times if there is any other thread that is closing the socket but none and checked just before passing the socket if it is closed, getting false.(SoTimeOut is 60000)

Code Snippet:

tlsServer = new DefaultTlsServer2() {
                    public void notifyClientCertificate(org.bouncycastle.crypto.tls.Certificate clientCertificate) throws IOException {
                        org.bouncycastle.asn1.x509.Certificate[] chain = clientCertificate.getCertificateList();
                        
                        logger.debug("notifyClientCertificate: " + chain[0].getSignature());

                        
                        /*// JFLog.log("Received client certificate chain of length " + chain.length);
                        for (int i = 0; i != chain.length; i++) {
                            org.bouncycastle.asn1.x509.Certificate entry = chain[i];
                            // JFLog.log("fingerprint:SHA-256 " + KeyMgmt.fingerprintSHA256(entry.getEncoded()) + " (" + entry.getSubject() + ")");
                            // JFLog.log("cert length=" + entry.getEncoded().length);
                        }*/
                    }

                    protected ProtocolVersion getMaximumVersion() {
                        logger.debug("getMaximumVersion: " + ProtocolVersion.DTLSv10);
                        return ProtocolVersion.DTLSv10;
                    }

                    protected ProtocolVersion getMinimumVersion() {
                        logger.debug("getMinimumVersion: " + ProtocolVersion.DTLSv10);
                        return ProtocolVersion.DTLSv10;
                    }

                    protected TlsEncryptionCredentials getRSAEncryptionCredentials() throws IOException {
                        logger.debug("getRSAEncryptionCredentials");
                        return new DefaultTlsEncryptionCredentials(context, dtlsInfo.getCertChain(), dtlsInfo.getPrivateKey());
                    }

                    @SuppressWarnings("rawtypes")
                    protected TlsSignerCredentials getRSASignerCredentials() throws IOException {
                        SignatureAndHashAlgorithm signatureAndHashAlgorithm = null;
                        Vector sigAlgs = supportedSignatureAlgorithms;
                        if (sigAlgs != null) {
                            for (int i = 0; i < sigAlgs.size(); ++i) {
                                SignatureAndHashAlgorithm sigAlg = (SignatureAndHashAlgorithm) sigAlgs.elementAt(i);
                                if (sigAlg.getSignature() == SignatureAlgorithm.rsa) {
                                    signatureAndHashAlgorithm = sigAlg;
                                    break;
                                }
                            }

                            if (signatureAndHashAlgorithm == null) {
                                return null;
                            }
                        }
                        
                        logger.debug("getRSASignerCredentials");
                        return new DefaultTlsSignerCredentials(context, dtlsInfo.getCertChain(), dtlsInfo.getPrivateKey(), signatureAndHashAlgorithm);
                    }

                    @SuppressWarnings("rawtypes")
                    public Hashtable getServerExtensions() throws IOException {
                        //see : http://bouncy-castle.1462172.n4.nabble.com/DTLS-SRTP-with-bouncycastle-1-49-td4656286.html
                        Hashtable table = super.getServerExtensions();
                        if (table == null) table = new Hashtable();
                        int[] protectionProfiles = {
                            // TODO : need to pick ONE that client offers
                            SRTPProtectionProfile.SRTP_AES128_CM_HMAC_SHA1_80  //this is the only one supported for now
                            // SRTPProtectionProfile.SRTP_AES128_CM_HMAC_SHA1_32
                            // SRTPProtectionProfile.SRTP_NULL_HMAC_SHA1_32
                            // SRTPProtectionProfile.SRTP_NULL_HMAC_SHA1_80
                        };
                        byte mki[] = new byte[0];  //should match client or use nothing
                        UseSRTPData srtpData = new UseSRTPData(protectionProfiles, mki);
                        TlsSRTPUtils.addUseSRTPExtension(table, srtpData);
                        
                        logger.debug("getServerExtensions: " + table.size());

                        return table;
                    }
                
                    public void notifyHandshakeComplete() throws IOException {
                        logger.debug("SRTPChannel:DTLS:Server:Handshake complete");
                        super.notifyHandshakeComplete();
                        getKeys();
                        remoteKey = tlsServer.getRemoteKey();
                        remoteSalt = tlsServer.getRemoteSalt();
                        localKey = tlsServer.getLocalKey();
                        localSalt = tlsServer.getLocalSalt();
                                
                        logger.debug("keys got here server");
                        isHandshakeComplete = 1;
                        logger.debug("isHandshakeComplete: " + isHandshakeComplete);

                    }
                };
                
                
                
                
                


                try {
                    logger.debug("SRTPChannel: accept dtlsCLient by DTLS server");
                    logger.debug("DTLS before accept:socket state:Socket is closed ?"+socket.isClosed()+socket.isConnected());
                    dtlsServer.accept(tlsServer, new UDPTransport(socket, 1500 - 20 - 8));      
                    udp_started = true;
               } catch (Exception e) {
                  logger.fatal("Exception:  ",e);     
               }

Using bouncycastle.crypto.tls libraries from bcprov-ext-jdkon-159.jar and bcprov-jdk15on-160b04.jar

Note: This system was up and running and now getting this issue, couldn't identify what triggered this.


Solution

  • The issue was the fact that it was using DTLSv10, which has been dropped from the browsers.

    Upgrading the DTLSv10 to DTLS12 solved the socket close issue but introduced an interal_error in the same DTLSServerProtocol.accept function which is caused by an internal library bug of the bouncyCastle library, bcprov-ext-jdkon-159.jar.

    Upgrading the library jar to bcprov-ext-jdk15on-1.61.jar fixed the issue and now the server is successfully handshaking with browser for VoIP calling using webrtc.