[SOLVED] Apache client authentication OCSP responder issue

Apache client authentication OCSP responder issue

I'm having an issue with setting up Apache 2.4.29 on Windows for client authentication with a working OCSP responder. Client authentication works fine when the OCSP responder is turned off. I am also able to verify my client certificate status is "good" when I manually use OpenSSL to make a request to the OCSP responder. This is only an issue when using it in Apache...

Certificate Authority (I am acting as my own CA):

Root CA > Intermediate CA
Intermediate CA > client certificate 1
Intermediate CA > OCSP signing certificate

Certificate Files

ca-chain.cert.pem (the Root CA and Intermediate CA certificates)
intermediate.cert.pem (the Intermediate CA certificate)
ocsp.mydomain.com.cert.pem (the OCSP signing certificate)
client1.cert.pem (the client certificate)

Windows Setup

Root CA and Intermediate CA certificates are imported into the "Trusted Root Certificate Authorities" and "Intermediate Certificate Authorities" stores respectively along with their private key (imported as .pfx)
Client certificate is imported into "Personal" certificate store along with its private key (imported as .pfx)

OCSP Responder server

openssl ocsp -port ocsp.mydomain.com:2560 -text -sha256 \
    -index intermediate/index.txt \
    -CA intermediate/certs/ca-chain.cert.pem \
    -rkey intermediate/private/ocsp.mydomain.com.key.pem \
    -rsigner intermediate/certs/ocsp.mydomain.com.cert.pem

Manual OCSP request (just to confirm all is setup right outside of Apache)

Request

openssl ocsp -CAfile intermediate/certs/ca-chain.cert.pem \
    -url http://ocsp.mydomain.com:2560 -resp_text \
    -issuer intermediate/certs/intermediate.cert.pem \
    -cert intermediate/certs/client1.cert.pem

Response (... represents some excluded verbose output and isn't actually in the response)

...
Certificate ID:
    ...
    Issuer Key Hash: 6FBE86C0DE4500EE4945D1ECC3E41F9DACF5CEEC
    ...
...
Response verify OK
intermediate/certs/client1.cert.pem: good

The "Issuer Key Hash" above matches the client certificate "Authority Key Identifier" in my "Personal" certificate store, all looks good

Apache setup

SSLVerifyClient require
SSLVerifyDepth 10
SSLOCSPEnable on
SSLOCSPDefaultResponder "http://ocsp.mydomain.com:2560"
SSLCACertificateFile "${SRVROOT}/conf/ssl/ca-chain.cert.pem"

Apache error

Library Error: OCSP_basic_verify:root ca not trusted (log info below)

    1973: connecting to OCSP responder 'ocsp.mydomain.com:2560'
    1975: sending request to OCSP responder
    AH02275: Certificate Verification, depth 2, CRL checking mode: none (0) [subject: CN=Generic Code Root CA,O=Generic Code,ST=New York,C=US / issuer: CN=Generic Code Root CA,O=Generic Code,ST=New York,C=US / serial: B0992B306BCDD3BD / notbefore: Mar 10 21:09:10 2018 GMT / notafter: Mar  5 21:09:10 2038 GMT]
    AH02275: Certificate Verification, depth 1, CRL checking mode: none (0) [subject: CN=Generic Code Intermediate CA,O=Generic Code,ST=New York,C=US / issuer: CN=Generic Code Root CA,O=Generic Code,ST=New York,C=US / serial: 1000 / notbefore: Mar 10 21:20:32 2018 GMT / notafter: Mar  7 21:20:32 2028 GMT]
    _util_ocsp.c(96):1973: connecting to OCSP responder 'ocsp.mydomain.com:2560'
    _util_ocsp.c(124):1975: sending request to OCSP responder
    _util_ocsp.c(234): 1981: OCSP response header: Content-type: application/ocsp-response
    _util_ocsp.c(234): 1981: OCSP response header: Content-Length: 2270
    _util_ocsp.c(282): 1987: OCSP response: got 2270 bytes, 2270 total
    1925: failed to verify the OCSP response
    Library Error: error:27069070:OCSP routines:OCSP_basic_verify:root ca not trusted
    AH02276: Certificate Verification: Error (50): application verification failure [subject: CN=Generic Code Intermediate CA,O=Generic Code,ST=New York,C=US / issuer: CN=Generic Code Root CA,O=Generic Code,ST=New York,C=US / serial: 1000 / notbefore: Mar 10 21:20:32 2018 GMT / notafter: Mar  7 21:20:32 2028 GMT]
    2008: library error 1 in handshake (server localhost:443)
    Library Error: error:14089086:SSL routines:ssl3_get_client_certificate:certificate verify failed
    1998: Connection closed to child 38 with abortive shutdown (server localhost:443)

OCSP Responder Server error response when Apache hits it

Response (... represents some excluded verbose output and isn't actually in the response)

...
Certificate ID:
    ...
    Issuer Key Hash: 79D4440D1471385397B194EF1038CEEEEFBBAC24
    ...
Cert Status: unknown
...

The "Issuer Key Hash" above matches the Root CA certificate "Authority Key Identifier" in my "Trusted Root Certificate Authorities" certificate store, WTF? Why?

Can anyone see anything wrong with what I have done or know why this isn't working?

Solution

I got this working.

Reissue the Intermediate CA with OCSP information
Setup a second OCSP responder for the OCSP information on Intermediate CA, the second OCSP responder signing certificate was signed by the Root CA
Re run the test and now everything is fine

Looks like mod_ssl has to verify the entire certificate chain instead of stopping at the client cert itself. I wish it was configurable but it isn't at this time...