Compass won't connect to MongoDB over TLS using OpenSSL certificates

Trying the tried and tested method of connecting Compass to MongoDB instance on VPS over TLS. If the certificates were from LetsEnctrypt CA (generated with CertBot), 2 files are needed:

CertAndKey.pem, containing host cert and it's priv key;
intermAndRoot.pem, containing Certbot's provided chain.pem and downloaded directly from LetsEncrypt website rootCA certificate.

This works fine. The connection gets established.

However the same isn't workingfor custom CA generated with OpenSSL. MongoDB log provides error "Unsuitable certificate purpose"

I followed this excellent playlist for creating the chain of OpenSSL certs: TechLAB

I believe I have done it correctly, since all the 3 certs are different and both interm and host certs do validate agains the root cert.

My Root CA is:

The intermediate cert is:

host cert is:

Any ideas about what is causing the problem are very appreciated!

Solution

Here the config files and commands to generate the certificates:

# root-ca.conf
[req]
distinguished_name = req_distinguished_name
prompt = no
[req_distinguished_name]
C = AU
O = giftbutton
OU = My Division
CN = Root CA
[v3_ca]
keyUsage = critical, keyCertSign, cRLSign
basicConstraints = critical, CA:true
subjectKeyIdentifier = hash


# intermediate-ca.conf
[req]
distinguished_name = req_distinguished_name
prompt = no
[req_distinguished_name]
C = AU
O = giftbutton
OU = My Division
CN = Intermediate CA
[v3_ca]
keyUsage = critical, keyCertSign, cRLSign
basicConstraints = critical, CA:true
subjectKeyIdentifier = hash


# server.conf
[req]
distinguished_name = req_distinguished_name
prompt = no
[req_distinguished_name]
C = AU
O = giftbutton
OU = My Division
CN = Mongo Server
[v3_ca]
keyUsage = digitalSignature, keyEncipherment
extendedKeyUsage = serverAuth
subjectAltName = @alt_names
[alt_names]
DNS.1 = giftbutton.com


# client.conf
[req]
distinguished_name = req_distinguished_name
prompt = no
[req_distinguished_name]
C = AU
O = giftbutton
OU = My Division
CN = Mongo Client
[v3_ca]
keyUsage = digitalSignature
extendedKeyUsage = clientAuth

Many tutorials generate the public/private key with openssl genrsa .... When you use the key only for one certificate, then it is easier to create the key automatically with the certificate (option -newkey 4096). It's one command less.

Typically when you need a certificate then you create a certificate request and send it to the person/department who owns the CA. They take your certificate request, sign it with their CA and return back the signed certificate to you. When you are the owner of the CA, then this step is not needed. You can create and sign the certificate request with a single command. It's one more command less.

As it looks like you like to use server and client certificates and intermediate CA. So, it ends up in 4 certificates created by 4 commands:

openssl req -config root-ca.conf -newkey 4096 -keyout root-ca.key -noenc -new -x509 -days 3650 -sha256 -copy_extensions copyall -extensions v3_ca -out root-ca.crt

openssl req -config intermediate-ca.conf -newkey 4096 -keyout intermediate-ca.key -noenc -new -x509 -days 3650 -CA root-ca.crt -CAkey root-ca.key -sha256 -copy_extensions copyall -extensions v3_ca -out intermediate-ca.crt

openssl req -config server.conf -newkey 4096 -keyout server.key -noenc -new -x509 -days 3650 -CA intermediate-ca.crt -CAkey intermediate-ca.key -sha256 -copy_extensions copyall -extensions v3_ca -out server.crt

openssl req -config client.conf -newkey 4096 -keyout client.key -noenc -new -x509 -days 3650 -CA intermediate-ca.crt -CAkey intermediate-ca.key -sha256 -copy_extensions copyall -extensions v3_ca -out client.crt

Instead of a config file, you can also put all parameters in command line. So, instead of -config client.conf it might be possible to use -subj "C=AU/O=giftbutton/OU=My\ Division/CN=Mongo\ Client" -addext "keyUsage=critical/keyCertSign/cRLSign" -addext "basicConstraints=critical,CA:true" -addext "subjectKeyIdentifier=hash" - but I did not test!

In order to use them, you have to combine them into files:

cat intermediate-ca.crt root-ca.crt > ca-chain.crt
cat client.crt client.key > client.pem
cat server.crt server.key > server.pem

Then they are ready to use. On the server side use

net:
  port: 27017
  bindIpAll: true
  tls:
    mode: requireTLS
    certificateKeyFile: server.pem
    CAFile: ca-chain.crt

And in Compass use connection string like this:

mongodb://user:secret@giftbutton.com:27017/?authSource=admin&tls=true&tlsCertificateKeyFile=client.pem&tlsCAFile=ca-chain.crt

Note:

Download latest version of openssl (version 3.4). In older version option -copy_extensions copyall was not supported and you would need to put the [v3_ca] section into an extension config-file and load this file with -extensions v3_ca -extfile ....

As already mentioned, I suggest to download and install XCA. It is very simple to use, you can import existing (working) certificate with simple copy/paste or drag/drop. Then you can check the properties, and create similar certificates or requests according to your need and you can export them in any format you may desire. It's really a helpful tool to learn the secrets of x.509 certificates.