c++pointersstack-smash

C++ stack smashing detected when returning from a function


I have the following function, which works fine and produces the expected output, but right upon returning, it produces

*** stack smashing detected ***: <unknown> terminated
Aborted (core dumped)

The code of the function:

bool derive_aes_key(const unsigned char* shared_secret, int shared_secret_len, unsigned char* aes_key) {
    
    memset(aes_key, 0, AES_KEY_SIZE);

    unsigned char prk[AES_KEY_SIZE];  // PRK must be at least AES_KEY_SIZE bytes
    unsigned int prk_len = AES_KEY_SIZE;

    // Use HMAC to derive a pseudorandom key (PRK) from the shared secret
    HMAC(EVP_sha256(), shared_secret, shared_secret_len, NULL, 0, prk, &prk_len)


    // Use PRK as the AES key
    memcpy(aes_key, prk, AES_KEY_SIZE);
    
    cout << "Code reaches this statement" << endl;
    return true;
}

How it's called from main:

unsigned char aes_key[AES_KEY_SIZE];
derive_aes_key(999c3293c8ed1e8173cc9d6e84ed9476884670422517291995183336a7f9b0f6, 32, aes_key);
cout << "Code doesn't reach this statement" << endl;

I saw many questions about stack smashing errors, including this one, which is similar, but was not able to figure out my specific mistake.


Solution

  • The error is here:

    unsigned char prk[AES_KEY_SIZE];  // PRK must be at least AES_KEY_SIZE bytes
    

    As for the HMAC man page

    It places the result in md (which must have space for the output of the hash function, which is no more than EVP_MAX_MD_SIZE bytes).

    You should have declared the array unsigned char prk[EVP_MAX_MD_SIZE].