[SOLVED] Java Grizzly server and Jersey client Received fatal alert: certificate

I have a Java application with an embedded SSL server and client.

My application uses client authentication to determine the identity of the client, so the server is configured with wantClientAuth=true and needClientAuth=true. The server is also configured with a server identity (cert/key pair). The server certificate SubjectDN does NOT contain the server's hostname in the CN portion of the distinguished name. The server certificate also does NOT contain the server's IP address in an x.509 alternate names extension.

My client is configured with a client identity. It's configured to NOT perform hostname verification. It's also configured with a trust-all trust manager (temporarily) defined in the usual manner. On the client side, the error received is:

javax.net.ssl.SSLHandshakeException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target

All of the attempted fixes I've made up to this point have only succeeded in making it fail more often.

I found this command line in another stackoverflow question and tried connecting:

openssl s_client -connect 10.200.84.48:9298 -cert cert.pem -key key.pem -state -debug

This works! I'm able to establish a connection using the openssl client and the client's private key and cert, but when I try to use my Java client to do it, it fails with the above error.

I'm certain I'm using the correct keys and certs on both ends.

For debugging purposes, I added print statements to the "trust-all" client-side trust store and I notice that none of the three methods are ever getting called to validate the server's cert (which it should do regardless of the content of the cert).

I did the same in the server-side trust store, which is dynamically managed, because client identities come and go. I understand that a new trust manager must be built whenever the trust store content is modified because the trust manager copies trust store content rather than holding a reference to the provided KeyStore object, so my code does this. When a client attempts to connect, the server does call checkClientTrusted and getAcceptedIssuers and the certificate contents displayed are correct.

Here's the really weird part - it works intermittently. Sometimes I get successful connections and data interchanges, and sometimes it fails with the title error (seen in the server's JSSE debug output) and the associated client-side errors about PKIX path building mentioned above.

One more fact: The server is using a grizzly embedded server created from SSLEngineConfigurator, and the client is a pure Jersey client, configured with an SSLContext.

I'm at a total loss. Has anyone seen anything like this before? Can I provide any more information which might help you understand the context better?

Update:

Here's a snippet from the server-side JSSE debug log:

javax.net.ssl|FINE|25|grizzly-nio-kernel(7) SelectorRunner|2022-05-24 03:06:01.221 UTC|Alert.java:238|Received alert message (
"Alert": {
  "level"      : "fatal",
  "description": "certificate_unknown"
}
)
javax.net.ssl|SEVERE|25|grizzly-nio-kernel(7) SelectorRunner|2022-05-24 03:06:01.221 UTC|TransportContext.java:316|Fatal (CERTIFICATE_UNKNOWN): Received fatal alert: certificate_unknown (
"throwable" : {
  javax.net.ssl.SSLHandshakeException: Received fatal alert: certificate_unknown
          at sun.security.ssl.Alert.createSSLException(Alert.java:131)
          at sun.security.ssl.Alert.createSSLException(Alert.java:117)
          at sun.security.ssl.TransportContext.fatal(TransportContext.java:311)

The fact that the server "Received fatal alert: certificate_unknown" tells me that the client is the one generating the alert and causing the problem. It seems the client does not like the server's certificate, event though I'm using a trust-all trust manager defined as follows:

    RestClientImpl(@Nonnull Endpoint endpoint, @Nonnull Credentials clientCreds,
            @Nullable KeyStore trustStore, @Nonnull Configuration cfg, @Nonnull ExecutorService es) {
        this.endpoint = endpoint;
        ClientBuilder builder = ClientBuilder.newBuilder();
        setupClientSecurity(builder, clientCreds, trustStore);
        this.client = builder
                .executorService(es)
                .register(JsonProcessingFeature.class)
                .register(LoggingFeature.class)
                .property(LoggingFeature.LOGGING_FEATURE_LOGGER_NAME_CLIENT, log.getName())
                .connectTimeout(cfg.getLong(CFG_REST_CLIENT_TMOUT_CONNECT_MILLIS), TimeUnit.MILLISECONDS)
                .readTimeout(cfg.getLong(CFG_REST_CLIENT_TMOUT_READ_MILLIS), TimeUnit.MILLISECONDS)
                .build();
        this.baseUri = "https://" + endpoint.getAddress() + ':' + endpoint.getPort() + '/' + BASE_PATH;
        log.debug("client created for endpoint={}, identity={}: client-side truststore {}active; "
                        + "hostname verification {}active", endpoint, osvIdentity,
                clientSideTrustStoreActive ? "" : "NOT ", hostnameVerifierActive ? "" : "NOT ");
    }

    private void setupClientSecurity(ClientBuilder builder, @Nonnull Credentials clientCreds,
            @Nullable KeyStore trustStore) {
        try {
            SSLContext sslContext = makeSslContext(clientCreds, trustStore);
            builder.sslContext(sslContext);
            if (trustStore != null) {
                hostnameVerifierActive = true;
            } else {
                builder.hostnameVerifier((hostname, session) -> true);
            }
        } catch (IOException | GeneralSecurityException e) {
            log.error("Failed to create SSL context with specified client credentials and "
                    + "server certificate for endpoint={}, osv identity={}", endpoint, osvIdentity);
            throw new IllegalArgumentException("Failed to create SSL context for connection to endpoint="
                    + endpoint + ", osv identity=" + osvIdentity, e);
        }
    }

    private SSLContext makeSslContext(@Nonnull Credentials clientCreds, @Nullable KeyStore trustStore)
            throws IOException, GeneralSecurityException {
        SSLContext context = SSLContext.getInstance(SSL_PROTOCOL);  // TLSv1.2
        X509Certificate clientCert = clientCreds.getCertificate();
        PrivateKey privateKey = clientCreds.getPrivateKey();

        // initialize key store with client private key and certificate
        KeyStore keyStore = KeyStore.getInstance(KeyStore.getDefaultType());
        keyStore.load(null);
        keyStore.setCertificateEntry(CLIENT_CERT_ALIAS, clientCert);
        keyStore.setKeyEntry(CLIENT_KEY_ALIAS, privateKey, KEYSTORE_PASSWORD, new Certificate[] {clientCert});
        KeyManagerFactory kmf = KeyManagerFactory.getInstance(KeyManagerFactory.getDefaultAlgorithm());
        kmf.init(keyStore, KEYSTORE_PASSWORD);
        KeyManager[] km = kmf.getKeyManagers();

        // initialize trust store with server cert or with no-verify trust manager if no server cert provided
        TrustManager[] tm;
        if (trustStore != null) {
            TrustManagerFactory tmf = TrustManagerFactory.getInstance(TrustManagerFactory.getDefaultAlgorithm());
            tmf.init(trustStore);
            tm = tmf.getTrustManagers();
            clientSideTrustStoreActive = true;
        } else {
            tm = new TrustManager[] {
                    new X509TrustManager() {

                        @Override
                        public X509Certificate[] getAcceptedIssuers() {
                            log.debug("client-side trust manager: getAcceptedIssuers (returning empty cert list)");
                            return new X509Certificate[0];
                        }
                        @Override
                        public void checkClientTrusted(X509Certificate[] certs, String authType) {
                            log.debug("client-side trust manager: checkClientTrusted authType={}, certs={}",
                                    authType, certs);
                        }
                        @Override
                        public void checkServerTrusted(X509Certificate[] certs, String authType) {
                            log.debug("client-side trust manager: checkServerTrusted authType={}, certs={}",
                                    authType, certs);
                        }
                    }
            };
        }
        context.init(km, tm, null);
        return context;
    }

Java Grizzly server and Jersey client Received fatal alert: certificate_unknown