I have a problem with the following code (which decrypts a file using a known key):
#include <stdio.h>
#include <stdlib.h>
#include <windows.h>
#include <wincrypt.h>
void error(const char *what) {
fprintf(stderr, "%s failed with last error 0x%x\n", what, GetLastError());
exit(1);
}
#define AES_KEY_SIZE 32
typedef struct {
BLOBHEADER hdr;
DWORD dwKeySize;
BYTE rgbKeyData[AES_KEY_SIZE];
} AES256KEYBLOB;
BYTE *hex2byte(const char *hex) {
int len = strlen(hex) / 2;
BYTE *bytes = malloc(len);
if (bytes == NULL) {
error("malloc");
return NULL;
}
unsigned char val[2];
for (int i = 0; i < len; i++) {
sscanf(&hex[i * 2], "%2hhx", &val);
bytes[i] = val[0];
}
return bytes;
}
int main(int argc, char *argv[]) {
if (argc != 3) {
fprintf(stderr, "Usage: %s <input_file> <output_file>\n", argv[0]);
return 1;
}
BYTE *key = hex2byte("c2ecf7a4f3d1620f18cd38379269948fcd3aaf4fdce6d50d310464ea823a");
AES256KEYBLOB aes256KeyBlob;
aes256KeyBlob.hdr.bType = PLAINTEXTKEYBLOB;
aes256KeyBlob.hdr.bVersion = CUR_BLOB_VERSION;
aes256KeyBlob.hdr.reserved = 0;
aes256KeyBlob.hdr.aiKeyAlg = CALG_AES_256;
aes256KeyBlob.dwKeySize = AES_KEY_SIZE;
memcpy(aes256KeyBlob.rgbKeyData, key, AES_KEY_SIZE);
HCRYPTPROV hProv;
if (!CryptAcquireContextA(&hProv, NULL, MS_ENH_RSA_AES_PROV_A, PROV_RSA_AES, CRYPT_VERIFYCONTEXT)) {
error("CryptAcquireContext");
}
HCRYPTKEY hKey;
if (!CryptImportKey(hProv, (BYTE *) &aes256KeyBlob, sizeof(AES256KEYBLOB), 0, CRYPT_EXPORTABLE, &hKey)) {
CryptReleaseContext(hProv, 0);
error("CryptImportKey");
}
DWORD numBytes = 0;
if (!CryptEncrypt(hKey, 0, TRUE, 0, NULL, &numBytes, 0)) {
error("CryptEncrypt");
}
FILE *input_file = fopen(argv[1], "rb");
if (!input_file) {
perror("Error opening input file");
return 1;
}
FILE *output_file = fopen(argv[2], "wb");
if (!output_file) {
perror("Error opening output file");
fclose(input_file);
return 1;
}
BYTE blk[numBytes];
while (fread(blk, 1, numBytes, input_file) == numBytes) {
if (!CryptDecrypt(hKey, 0, TRUE, 0, blk, &numBytes)) {
//error("CryptDecrypt");
}
fwrite(blk, 1, numBytes, output_file);
}
free(key);
CryptDestroyKey(hKey);
CryptReleaseContext(hProv, 0);
fclose(input_file);
fclose(output_file);
return 0;
}
This code WORKS, but I had to comment the error checking line:
//error("CryptDecrypt");
I tried as suggested in other stackoverflow solution to increase the buffer size but it does not help. I don't care much because the code works, but I wish to understand the reason and how to avoid it. Note if I use:
CryptDecrypt(hKey, 0, FALSE, 0, blk, &numBytes)
I get no error but it works only for the first block.
Please don't mark this as a duplicate because none of the solutions work.
Since the encryption code was not posted, it is not clear how the ciphertext was generated. A test shows that a decryption with the posted code is only possible under the described conditions if the ECB mode was used (a decryption of a ciphertext generated with CBC fails).
The following changes are necessary in the posted code for a proper and error message-free decryption:
The current code uses the default mode, which is CBC with a Zero-IV (16 0x00 Bytes). Therefore, an explicit switch to the ECB mode is required:
// Set ECB mode
DWORD dwMode = CRYPT_MODE_ECB;
if (!CryptSetKeyParam(hKey, KP_MODE, (BYTE*)&dwMode, 0)) {
error("CryptSetKeyParam");
}
Also, the size of the ciphertext is determined incorrectly. This is due to the fact that CryptEncrypt() is used for the determination, which is actually intended for determining the ciphertext size during encryption (the ciphertext is generally larger than the plaintext due to the padding) and not for decryption.
The ciphertext size can be determined via the file size, e.g.
// Get ciphertext size
fseek(input_file, 0, SEEK_END);
DWORD ctSize = ftell(input_file);
fseek(input_file, 0, SEEK_SET);
Debugging the posted code shows that due to the incorrect use of CryptEncrypt() numBytes corresponds to the block size (i.e. 16 bytes), so that the data is read and decrypted block by block. Presumably the use of a chunk size of 16 bytes is not intended, but a larger chunk size or the reading of the file as a whole.
Although the file is read and decrypted block by block, the final flag is set to TRUE for all chunks. This must be changed so that the final flag is set to FALSE for all chunks and TRUE only for the last chunk, e.g. as in the following code:
// Read chunks
const DWORD chunkSize = 64; // multiple of 16
BYTE buffer[chunkSize];
DWORD dataLen = chunkSize;
DWORD read = 0;
while (read < ctSize) {
dataLen = fread(buffer, 1, chunkSize, input_file);
read += dataLen;
if (!CryptDecrypt(hKey, 0, !(read < ctSize) , 0, buffer, &dataLen)) { // final flag = FALSE except for the last chunk
error("CryptDecrypt");
}
fwrite(buffer, 1, dataLen, output_file);
}
With this, decryption is performed correctly and no more error message is displayed.
Why did the decryption still work with the code posted in the question (apart from the error message)? After all, the CBC mode with a Zero-IV is used by default, which is actually not compatible with the data generated in ECB mode.
The reason is: As already stated above, the ciphertext is read and decrypted block by block in the posted code. The final flag is set to TRUE for all blocks. A TRUE sets the IV back to the initial state (i.e. to a Zero-IV). In addition, encrypting a single block in CBC mode with a Zero-IV results in the same ciphertext as encrypting the same block in ECB mode. For this reason, every block is successfully decrypted.
On the other hand, a TRUE for the final flag also means an unpadding is performed. This generally fails (as only the last block is padded), which leads to an error with the error code 0x80090005 (Error: Bad Data).
The overall result is an error message with otherwise successful decryption.
However, if you now try to avoid unpadding by setting the final flag for all blocks to FALSE, the error message is no longer triggered, but the ciphertext is decrypted incorrectly because the IV is no longer reset. In order for the ciphertext to be decrypted correctly again, the mode must be set to ECB. And to ensure that unpadding takes place for the last block, the final flag for the last block must be set to TRUE. This in turn leads to the above solution (with chunk size 16).
Edit:
In the comments, it turned out that you are looking for an equivalent to
openssl enc -aes-256-ecb -d -in "encoded file" -out "decoded file" -nopad -K <hex key>
i.e. a decryption with ECB without padding. Since without padding the final flags of all blocks are to be set to FALSE, the loop can be simplified a little.
With regard to the performance issues mentioned in the comment: The above 64 bytes were intended for test purposes. For better performance choose a larger chunk size (to avoid unnecessarily many fread() calls), which is at the same time compatible with the available memory, e.g. 64 MB:
DWORD chunkSize = 64 * 1024 * 1024; // choose a proper chunk size (a multiple of 16 bytes)
BYTE* buffer = (BYTE*)malloc(chunkSize);
DWORD dataLen;
while ((dataLen = fread(buffer, 1, chunkSize, input_file)) > 0) {
if (!CryptDecrypt(hKey, 0, FALSE, 0, buffer, &dataLen)) {
error("CryptDecrypt");
}
fwrite(buffer, 1, dataLen, output_file);
}
free(buffer);