c++cryptographyrsacryptoapimscapi

Verify OpenPGP based RSA signature with WinCrypt/CryptoAPI


I have code that parses OpenPGP packets and I have n, e of the public key packet as well as s of the signature packet as byte arrays.

In order to verify a signature I first initialize CryptAcquireContext (I also tried with PROV_RSA_FULL instead of PROV_RSA_AES)

HCRYPTPROV hCryptProv;
CryptAcquireContext(&hCryptProv, nullptr, nullptr, PROV_RSA_AES, CRYPT_VERIFYCONTEXT);

then create a hash

HCRYPTHASH hHash;
CryptCreateHash(hCryptProv, CALG_SHA1, 0, 0, &hHash); // as the digest algorithm of the signature was 2 => SHA1

and populate it using CryptHashData. This works so far as well as parsing and importing the public key using CryptImportKey.

typedef struct _RSAKEY
{
    BLOBHEADER blobheader;
    RSAPUBKEY rsapubkey;
    BYTE n[4096 / 8];
} RSAKEY;

static int verify_signature_rsa(HCRYPTPROV hCryptProv, HCRYPTHASH hHash, public_key_t &p_pkey, signature_packet_t &p_sig)
{
    int i_n_len = mpi_len(p_pkey.key.sig.rsa.n); // = 512; p_pkey.key.sig.rsa.n is of type uint8_t n[2 + 4096 / 8];
    int i_s_len = mpi_len(p_sig.algo_specific.rsa.s); // = 256; p_sig.algo_specific.rsa.s is of type uint8_t s[2 + 4096 / 8]

    HCRYPTKEY hPubKey;
    RSAKEY rsakey;
    rsakey.blobheader.bType = PUBLICKEYBLOB; // 0x06
    rsakey.blobheader.bVersion = CUR_BLOB_VERSION; // 0x02
    rsakey.blobheader.reserved = 0;
    rsakey.blobheader.aiKeyAlg = CALG_RSA_KEYX;
    rsakey.rsapubkey.magic = 0x31415352;// ASCII for RSA1
    rsakey.rsapubkey.bitlen = i_n_len * 8; // = 4096
    rsakey.rsapubkey.pubexp = 65537;

    memcpy(rsakey.n, p_pkey.key.sig.rsa.n + 2, i_n_len); // skip first two byte which are MPI length
    std::reverse(rsakey.n, rsakey.n + i_n_len); // need to convert to little endian for WinCrypt

    CryptImportKey(hCryptProv, (BYTE*)&rsakey, sizeof(BLOBHEADER) + sizeof(RSAPUBKEY) + i_n_len, 0, 0, &hPubKey); // no error

    std::unique_ptr<BYTE[]> pSig(new BYTE[i_s_len]);
    memcpy(pSig.get(), p_sig.algo_specific.rsa.s + 2, i_s_len); // skip first two byte which are MPI length
    std::reverse(p_sig.algo_specific.rsa.s, p_sig.algo_specific.rsa.s + i_s_len); // need to convert to little endian for WinCrypt

    if (!CryptVerifySignature(hHash, pSig.get(), i_s_len, hPubKey, nullptr, 0))
    {
        DWORD err = GetLastError(); // err=2148073478 -> INVALID_SIGNATURE
        CryptDestroyKey(hPubKey);
        return -1;
    }

    CryptDestroyKey(hPubKey);
    return 0;
}

CryptVerifySignature fails with GetLastError() decoding to INVALID_SIGNATURE.

On https://www.rfc-editor.org/rfc/rfc4880#section-5.2.2 I read

With RSA signatures, the hash value is encoded using PKCS#1 encoding
type EMSA-PKCS1-v1_5 as described in Section 9.2 of RFC 3447.  This
requires inserting the hash value as an octet string into an ASN.1
structure.

Is that needed or is that automatically done by CryptVerifySignature? If not, how to do that?


Solution

  • The error was in this line

    std::reverse(p_sig.algo_specific.rsa.s, p_sig.algo_specific.rsa.s + i_s_len); // need to convert to little endian for WinCrypt
    

    which should read

    std::reverse(pSig.get(), pSig.get() + i_s_len); // need to convert to little endian for WinCrypt
    

    because converting the source of the bytes from big to little endian does not convert another buffer after a copy.