I'm having some trouble with Application Layer Protocol Negotiation (ALPN) verification during a TLSv1.3 handshake. I want to check if the client sends an ALPN extension with a specific entry in its "Client Hello". If not, the handshake must be aborted.
I am using C++ in conjunction with Boost.Asio v1.83 and OpenSSL 3.1. The TLSv1.3 server uses the functions provided by Boost:
boost::asio::async_accept(...);
boost::asio::async_handshake(...);
boost::asio::async_read_some(...);
boost::asio::async_write(...);
boost::asio::async_shutdown(...);
So far everything works fine. If during the handshake an ALPN extension is registered in the "Client Hello", OpenSSL calls my callback function (alpn_select_cb). Here I can check the ALPN and depending on the return value abort the handshake.
// https://www.openssl.org/docs/manmaster/man3/SSL_select_next_proto.html
int ServerSession::alpn_select_cb(SSL *ssl,
const unsigned char **out,
unsigned char *outlen,
const unsigned char *in,
unsigned int inlen,
void *arg)
{
ServerSession* srvSession = static_cast<ServerSession*>(arg);
int status = SSL_select_next_proto(const_cast<unsigned char **>(out), // selected ALPN
outlen, // ALPN entry size
srvSession->m_serverAlpnList.data(), // list with supported ALPNs of the server
srvSession->m_serverAlpnList.size(), // list size (server)
in, // received list with supported ALPNs of the client
inlen); // list size (client)
switch (status)
{
case OPENSSL_NPN_NEGOTIATED: return SSL_TLSEXT_ERR_OK; // OK: ALPN negotiated
case OPENSSL_NPN_UNSUPPORTED: return SSL_TLSEXT_ERR_NOACK; // ERR: ALPN unsupported
case OPENSSL_NPN_NO_OVERLAP: return SSL_TLSEXT_ERR_ALERT_FATAL; // ERR: ALPN no overlap
}
}
If the handshake fails because of a wrong ALPN, it looks like this in Wireshark (client is 127.0.0.1 and server is 127.0.0.5):
No. Time Source Destination Protocol Length Info
---------------------------------------------------------------------------------------------------------------------------
22 38.136329 127.0.0.1 127.0.0.5 TLSv1.2 283 Client Hello --> client initiates TLSv1.3 connection (with a wrong ALPN)
24 38.137863 127.0.0.5 127.0.0.1 TLSv1.2 51 Alert (Level: Fatal, Description: No application Protocol)
The behavior is correct so far. However, if the client does not send an ALPN extension, OpenSSL will not call the ALPN callback, so the handshake will be completed. I can then close the TLS connection afterwards on the server side. In Wireshark it looks like this:
No. Time Source Destination Protocol Length Info
---------------------------------------------------------------------------------------------------------------------------
24 18.605087 127.0.0.1 127.0.0.5 TLSv1.3 273 Client Hello --> client initiates TLSv1.3 connection (without ALPN)
26 18.607975 127.0.0.5 127.0.0.1 TLSv1.3 1479 Server Hello, Change Cipher Spec, Application Data, Application Data, Application Data, Application Data
28 18.609183 127.0.0.1 127.0.0.5 TLSv1.3 124 Change Cipher Spec, Application Data
30 18.612081 127.0.0.5 127.0.0.1 TLSv1.3 554 Application Data, Application Data --> handshake finished
32 18.613404 127.0.0.5 127.0.0.1 TLSv1.3 68 Application Data --> server sends "close_notify" (connection shutdown)
34 22.126810 127.0.0.1 127.0.0.5 TLSv1.3 71 Application Data --> simultaneously the client is already trying to send data
36 22.127768 127.0.0.1 127.0.0.5 TLSv1.3 68 Application Data --> client sends "close_notify" (connection shutdown)
However, I want the handshake to be terminated as in the first case and a TLS alert to be sent. My idea was to use the message callback that OpenSSL calls in different TLS phases. This way I can check if an ALPN has been selected right after processing the "Client Hello" (see cb_ssl_msg()).
// https://www.openssl.org/docs/manmaster/man3/SSL_CTX_set_msg_callback.html
void ServerSession::cb_ssl_msg(int write_p, int version, int content_type, const void *buf, size_t len, SSL *ssl, void *arg)
{
// After processing the "Client Hello" the server can check if an ALPN exists and was selected
if (SSL_get_state(ssl) == TLS_ST_SW_SRVR_HELLO)
{
// get selected ALPN
unsigned int alpnLen;
const unsigned char* alpnData;
SSL_get0_alpn_selected(ssl, &alpnData, &alpnLen);
if (selectedAlpn.length() == 0) // no ALPN present
{
/*
... cancel the handshake process somehow ...
*/
// this does not work:
ServerSession* srvSession = static_cast<ServerSession*>(arg);
srvSession->m_socket.async_shutdown(boost::bind(&ServerSession::shutdownHandler, srvSession, boost::asio::placeholders::error));
return;
}
}
}
My problem: I don't know how to initiate the handshake termination in this callback (cb_ssl_msg()) if there is no ALPN. async_shutdown() doesn't seem to be the right approach since the handshake is performed anyway.
Can anyone help me or does anyone have any ideas?
SOLVED!
many thanks to Matt Caswell.
Yes, I just came across 'SSL_CTX_set_client_hello_cb()' as well. With this I can access the 'Client Hello' early.
first I set the callback function:
SSL_CTX_set_client_hello_cb(context.native_handle(), ServerSession::client_hello_cb, nullptr);
...and then the callback must be defined:
int ServerSession::client_hello_cb(SSL* ssl, int* al, void* arg)
{
const unsigned char* extData; // needed by SSL_client_hello_get0_ext(); on success: pointer to the 'length field' of the extension field
std::size_t extLen; // needed by SSL_client_hello_get0_ext(); on success: length of the extension field (length field + body)
if (SSL_client_hello_get0_ext(ssl, TLSEXT_TYPE_application_layer_protocol_negotiation, &extData, &extLen) == 1)
{
return SSL_CLIENT_HELLO_SUCCESS; // ALPN extension was found (success)
}
else
{
*al = TLS1_AD_NO_APPLICATION_PROTOCOL; // ALPN extension not found: set TLS Alert (Level: Fatal, Description: No application Protocol)
return SSL_CLIENT_HELLO_ERROR;
}
}
Thats it. To check the ALPN (if it exists), I still use SSL_CTX_set_alpn_select_cb(context.native_handle(), ServerSession::alpn_select_cb, this) like in my question.
thank you very much Matt :)