cencryptionopensslcryptographyevp-cipher

Segmentation fault in EVP_DecryptFinal_ex when using AES-128/CBC


This is a continuation of my previous question: Intermittent decryption failures in EVP_DecryptFinal_ex when using AES-128/CBC.

I am trying to encrypt and decrypt using the C OpenSSL EVP library. After I received an answer to my question above, I updated my code accordingly:

This variable:

int len = outlen1 + outlen2;

Stores the number of bytes encrypted in the encrypt function. I then pass that variable to the decrypt function (the passing is not shown in the code below) which then uses what is supposedly the actual number of bytes.

However on some input strings, I get segmentation faults at the EVP_DecryptFinal_ex() function.

Something is obviously wrong with the number of bytes encrypted/decrypted or padding. I just don't know what that is.

 char* encrypt(char *key, char *s) {
            unsigned char iv[16] = {[0 ... 15 ] = 0};
            unsigned char outbuf[1024] = {[0 ... 1023] = 0};
            int outlen1, outlen2;

            EVP_CIPHER_CTX ctx;

            EVP_CIPHER_CTX_init(&ctx);
            EVP_EncryptInit_ex(&ctx, EVP_aes_128_cbc(), NULL, key, iv);
            if (EVP_EncryptUpdate(&ctx, outbuf, &outlen1, s, strlen(s)) == 1) {
                    if (EVP_EncryptFinal_ex(&ctx, outbuf + outlen1, &outlen2) == 1) {
                            EVP_CIPHER_CTX_cleanup(&ctx);
                            len = outlen1 + outlen2;
                            return strdup(outbuf);
                    }
            }
            EVP_CIPHER_CTX_cleanup(&ctx);
            return NULL;
    }

    char* decrypt(char *key, char *s, int len) {
            unsigned char iv[16] = {[0 ... 15 ] = 0};
            unsigned char outbuf[1024] = {[0 ... 1023] = 0};
            int outlen1, outlen2;

            printf("len: %d\n", len);
            printf("strlen(s): %d\n", strlen(s));

            EVP_CIPHER_CTX ctx;

            EVP_CIPHER_CTX_init(&ctx);
            EVP_DecryptInit_ex(&ctx, EVP_aes_128_cbc(), NULL, key, iv);
            if (EVP_DecryptUpdate(&ctx, outbuf, &outlen1, s, len) == 1) {
                    printf("After update\n");
                    if (EVP_DecryptFinal_ex(&ctx, outbuf + outlen1, &outlen2) == 1) {
                            printf("After final\n");
                            EVP_CIPHER_CTX_cleanup(&ctx);
                            return strdup(outbuf);
                    }
            }
            EVP_CIPHER_CTX_cleanup(&ctx);
            return NULL;
    }

NOTE:

I was able to fix the problems I previously had where decrypt final would fail to decrypt certain strings. Those strings can be decrypted fine now. However, some other strings are facing the same problem but this time I am getting seg faults.


Solution

  • You cannot use string functions on binary data. This is especially the case if that binary data is indistinguishable from random. Random binary data may contain null characters anywhere, or not at all. strdup uses strcpy internally, which relies on the null character to be present.