CryptoAPI RSA Schannel Provider error

I am trying to generate digital signature using "Microsoft RSA SChannel Cryptographic Provider". After acquiring the handle to container, I am generating a signature using CryptGenKey(). But this function returns FALSE.

The dwError for CryptGenKey() returns 80090008.

The same works for any other provider type. Also, when I am trying to create key exchange pair for the same provider, it is working fine. What am I doing wrong?

#include <Windows.h>
#include <wincrypt.h>

int main()
{
    HCRYPTPROV phProv = 0;
    LPTSTR pszContainer = NULL;
    DWORD dwFlags = 0;
    bool flag;
    DWORD_PTR dwError;
    HCRYPTKEY phKey;

    flag = CryptAcquireContext(&phProv, pszContainer,
        MS_DEF_RSA_SCHANNEL_PROV, PROV_RSA_SCHANNEL, dwFlags);
    if (!flag)
    {
        flag = CryptAcquireContext(&phProv, pszContainer,
            MS_DEF_RSA_SCHANNEL_PROV, PROV_RSA_SCHANNEL, CRYPT_NEWKEYSET);
    }
    dwError = GetLastError();

    flag = CryptGenKey(phProv, AT_SIGNATURE, CRYPT_EXPORTABLE, &phKey);
    dwError = GetLastError();

    flag = CryptGetUserKey(phProv, AT_SIGNATURE, &phKey);
    dwError = GetLastError();

    return 0;
}

Thanks.

Solution

The SChannel provider doesn't support AT_SIGNATURE RSA keys, only AT_EXCHANGE. It's mainly a holdover from early TLS (back when it was still SSL) when keys were exchanged using RSA encryption instead of agreed upon using Diffie-Hellman Key Agreement signed with RSA signature... and then, presumably "well, everybody knows how the SChannel provider behaves, why change it?". (The closest I see to that being written down is https://msdn.microsoft.com/en-us/library/windows/desktop/aa387690(v=vs.85).aspx, which shows CALG_RSA_KEYX and doesn't talk about CALG_RSA_SIGN.)

In CAPI an AT_EXCHANGE key can do both encryption and signing, and an AT_SIGNATURE key can only do signing.

In general the Windows Cryptography team discourages new code written with CAPI (I don't have written evidence othen than what I just wrote; mainly this has come from being in meetings with them). CNG has a much more developer-friendly API, and is more powerful. CNG shipped back in in Windows Vista, and so it's available in every supported version of Windows. CAPI doesn't serve the purpose of "well, it has more coverage" anymore, it's just "the old, crufty, legacy API" (unless you're writing code for out-of-support OSes, like XP).

If you are using CAPI, I don't know why you'd want to use the SChannel provider. PROV_RSA_AES via MS_ENH_RSA_AES_PROV is the most functional RSA that CAPI has (SHA-2 based PKCS signatures). But it's out of date compared to the software provider in CNG (PSS signature, OAEP with SHA-2, and supports public exponent values larger than 2^32 (okay, that's not a common need, but it's something CNG fixed)).