Verify a signature file (PKCS7) with WinCrypt or CNG

I need to verify a signed JAR file using Windows crypto API methods. I have only a basic understanding of encryption and signing matters. I'm also new to those crypto APIs (WinCrypt, Bcrypt, Ncrypt). Verifying the file hashes wasn't a problem, but the signature part is blocking me.

Thanks to OpenSSL, the PKCS7 RFC (https://www.rfc-editor.org/rfc/rfc2315) and various other sources I was able to figure out the actual file contents of META-INF/LOCALSIG.DSA contained in the JAR. But after two weeks of digging, trial and error I'm still stuck and don't know what else to try.

OpenSSL has the nice command openssl smime -verify -inform der -in LOCALSIG.DSA -content LOCALSIG.SF -noverify, which does exactly what I want to do. Unfortunately, I couldn't find such a high-level command in the Windows APIs.

I've tried to use the VerifySignature family of functions from all three APIs, but I need to provide the public key for those, and I had no luck using any of the ImportKey functions. So I tried to manually dissect the ASN1 format using CryptDecodeObjectEx, in order to pass the individual parts as BLOBs to the API functions. While I had some success with this, I'm stuck again because I can't figure out how to parse sets. I don't want to write my own ASN1 parser from scratch...

So, how do I use a PKCS7 signature file with the Windows crypto APIs?

I guess it might be easier using OpenSSL, but then I would have to convince my employer to add OpenSSL to our codebase just for this one purpose...

UPDATE: The LOCALSIG.DSA file contains the signers certificate and the signed hash of the LOCALSIG.SF file. This can be verified using openssl pkcs7 -inform der -print_certs -text -in LOCALSIG.DSA or openssl cms -cmsout -inform DER -print -in LOCALSIG.DSA.

The certificate is self-signed by our company and in the certificate store. I might need to provide the whole chain of trust. That's why I added the -noverify option to openssl smime -verify.

In fact, there are two scenarios with different certificates (internal and external releases), one using DSA (sig file contains one cert), the other using RSA (sig file contains three certs). That means I can't hardcode which certificate or method to use. I need to extract that information from the provided file. How do I do that?

Solution

As i understand from your question you need to verify PKC7 with detached signature. For this purpose you can use function CryptVerifyDetachedMessageSignature.

Sample code:

CRYPT_VERIFY_MESSAGE_PARA vparam = { 0 };
    vparam.cbSize = sizeof(CRYPT_VERIFY_MESSAGE_PARA);
    vparam.dwMsgAndCertEncodingType = X509_ASN_ENCODING | PKCS_7_ASN_ENCODING;
    vparam.pfnGetSignerCertificate = nullptr;
    vparam.pvGetArg = nullptr;


    /* read your files somehow */
    std::vector<char> sig;
    if (!ReadFile("LOCALSIG.DSA", &sig))
    {
        std::cout << "Failed to read file" << std::endl;
        return;
    }

    std::vector<char> mes;
    if (!ReadFile("LOCALSIG.SF", &mes))
    {
        std::cout << "Failed to read file" << std::endl;
        return;
    }

    /* CryptVerifyDetachedMessageSignature requires array of messages to verify */
    const BYTE* marray[] = {
        reinterpret_cast<BYTE*>(mes.data())
    };
    DWORD marray_len[] = {
        static_cast<DWORD>(mes.size())
    };

    if (!CryptVerifyDetachedMessageSignature(vparam,
        0, 
        reinterpret_cast<BYTE*>(sig.data()),
        static_cast<DWORD>(sig.size()), 
        1, /* number of messages in marray */
        marray,
        marray_len,
        nullptr))
    {
        std::cout << "Failed to verify signature, error: " << GetLastError() << std::endl;
    }
    else
    {
        std::cout << "Verify success, signature valid" << std::endl;
    }

UPDATE
To verify certificate chain you need to use CertGetCertificateChain

Sample code:

PCCERT_CHAIN_CONTEXT pChain = nullptr;
CERT_CHAIN_PARA chainPara = {0};
HCERTSTORE hStore = nullptr;
/* you sig file */
DATA_BLOB db = {
    static_cast<DWORD>(sig.size()), reinterpret_cast<BYTE *>(sig.data())};

/* you can open your sig file as certificate store */
hStore = CertOpenStore(CERT_STORE_PROV_PKCS7, X509_ASN_ENCODING | PKCS_7_ASN_ENCODING, NULL, 0, reinterpret_cast<BYTE *>(&db));
if (!hStore)
{
    goto Exit;
}

chainPara.cbSize = sizeof(CERT_CHAIN_PARA);
chainPara.RequestedUsage.dwType = USAGE_MATCH_TYPE_AND;
chainPara.RequestedUsage.Usage.cUsageIdentifier = 0;

if (!CertGetCertificateChain(NULL,  /* use default chain engine */
                             pCert, /*  pCert - pointer to signer cert structure (the parameter that was obtained in the previous step) */
                             NULL,
                             hStore, /* point to additional store where need to search for certificates to build chain */
                             &chainPara,
                             CERT_CHAIN_REVOCATION_CHECK_CHAIN_EXCLUDE_ROOT,
                             NULL,
                             &pChain))
{
    std::cout << "failed to build chain: " << GetLastError();
    goto Exit;
}

if (pChain->TrustStatus.dwErrorStatus == CERT_TRUST_NO_ERROR)
{
    std::cout << "certificate valid";
    goto Exit;
}
if (pChain->TrustStatus.dwErrorStatus & CERT_TRUST_REVOCATION_STATUS_UNKNOWN)
{
    std::cout << "certificate revocation status unknown";
}
/* you need to place root certificate to the Trusted Root Store */
if (pChain->TrustStatus.dwErrorStatus & CERT_TRUST_IS_UNTRUSTED_ROOT)
{
    std::cout << "untrusted CA";
}
/* and so on */

Exit : 
if (pCert)
{
    CertFreeCertificateContext(pCert);
}
if (pChain)
{
    CertFreeCertificateChain(pChain);
}
if (hStore)
{
    CertCloseStore(hStore, 0);
}