Context
What I tryed to let my browser trust my cert
The cert is still not trusted, I still get the SSL_ERROR_BAD_CERT_DOMAIN in Firefox
Any idea what I still do wrong or how I can debug my issue further?
In my case, the problem was that my certificate had no SAN entries (Subject Alternative Name) but only a CN (Common Name), which is deprecated (see for example here).
I wrote that I added SAN entries in chapter ‘What I tried’ in the question but @steffen-ullrich pointed out to me (comment on my question) that people often forget to copy these SAN entries from the CSR when creating the certificate. And that was exactly the problem.
So if you have such a problem, make sure that you add SAN entries and that they find their way into the certificate.
In the course of this bug research I realised that the process of creating a certificate with root CA and SAN entries has become easier since OpenSSL v1.1.1 and again with OpenSSL v3. The sources I have found and used have unfortunately explained the old process which I have now gone through.
Here are sources that describe the modern approach
And here are sources that helped me create the Root CA and Cert (legacy process)