Problem

I am attempting to create an HTTPS server using TLS certificates using rustls_pki_types. I am attempting to read from files the server's certificate chain and the server's own private key with this function:

(copied from the Quinn HTTP/3 docs here)

fn read_certs_from_file() -> Result<(Vec<CertificateDer<'static>>,
    PrivateKeyDer<'static>), Box<dyn Error>> {
        let certs = CertificateDer::pem_file_iter("./src/serverChain.pem")
            .unwrap()
            .map(|cert| cert.unwrap())
            .collect();
        let key = PrivateKeyDer::from_pem_file("./src/serverPrivate.pem")
            .unwrap();
        Ok((certs, key))}

The chain is loaded correctly, but the let key line throws this runtime error (docs):

called 'Result::unwrap()' on an 'Err' value: NoItemsFound

The serverPrivate.pem is being created in OpenSSL 3.2.4 with the command:

openssl req -newkey rsa:4096 -subj "/C=XX/ST=XX/L=XX/O=XX/CN=XX" -addext "subjectAltName = IP.XX.XX.XX.XX"
-outform pem -keyout C:\filepath\serverPrivate.pem -out C:\filepath\serverRequest.pem

Question

This same OpenSSL command works for different server code I've written in Python, so the .pem itself must be valid, and the path is correct. If so, why would rustls_pki_types not load it?

I have tried using PrivatePkcs1Der and PrivatePkcs8Der instead of PrivateKeyDer, and these throw the same error.

I figured this out: rustls_pki_types does not support loading encrypted (i.e. password protected) private keys (source), which is what I was using. If you did not generate them, you can tell by the header --BEGIN ENCRYPTED PRIVATE KEY-- (RFC).

To create an unencrypted key, include -noenc (source) in your OpenSSL command. This is what I am going to do for now, but if any Googlers need encrypted private keys it looks like you can use the pkcs8 crate to unencrypt them with the password before passing them to rustls_pki_types.

This question on Server Fault discusses best practices.