c++ecdsalibcryptoecdsasignature

EVP_DigestVerifyFinal fails - ECDSA P-256/SHA-256 with OpenSSL (Libcrypto)


While learning Libcrypto I reached a point where I need some external input.

I've the following working python example:

#!/usr/bin/env python3

import hashlib
from ecdsa import curves, VerifyingKey

public_key_bytes = bytearray.fromhex("0303b2ce64bc207bdd8bc4df859187fcb686320d63ffa091410fc158fbb77980ea")
public_key = VerifyingKey.from_string(public_key_bytes, curve=curves.NIST256p, hashfunc=hashlib.sha256)

ds = bytearray.fromhex("F8CD88299FA4605800207BFEBEAC55024053F30F7C69B35C15E60800AC3B6FE3ED0639952F7B028D86867445961FFE94FB226BFF7006E0C451EE3F8728C177FB")
m = bytearray.fromhex("8210492204E060610BDF26D77B5BF8C9CBFCF70422081475FD445DF0FF")

print(public_key.verify(ds, m))

When executing this code I get True. When I change any byte of m, ds or public_key I get verification errors as expected.

m is message and ds is digital signature.


Now the tricky part. I've created the following two functions:

First function to load the public key.

EVP_PKEY* load_public_key(const unsigned char* public_key_bytes, size_t key_len) {
    EVP_PKEY* pkey = nullptr;
    EVP_PKEY_CTX* ctx = EVP_PKEY_CTX_new_id(EVP_PKEY_EC, nullptr);
    if (!ctx) {
        std::cerr << "Failed to create EVP_PKEY_CTX" << std::endl;
        print_openssl_errors();
        return nullptr;
    }

    if (EVP_PKEY_fromdata_init(ctx) <= 0) {
        std::cerr << "EVP_PKEY_fromdata_init failed" << std::endl;
        print_openssl_errors();
        EVP_PKEY_CTX_free(ctx);
        return nullptr;
    }

    OSSL_PARAM params[3];
    params[0] = OSSL_PARAM_construct_utf8_string("group", (char*)"P-256", 0);
    params[1] = OSSL_PARAM_construct_octet_string("pub", (void*)public_key_bytes, key_len);
    params[2] = OSSL_PARAM_construct_end();

    if (EVP_PKEY_fromdata(ctx, &pkey, EVP_PKEY_PUBLIC_KEY, params) <= 0) {
        std::cerr << "EVP_PKEY_fromdata failed" << std::endl;
        print_openssl_errors();
        EVP_PKEY_CTX_free(ctx);
        return nullptr;
    }

    EVP_PKEY_CTX_free(ctx);
    return pkey;
}

Second function is supposed to verify the signature using the public key and message.

bool verify_signature(EVP_PKEY* pkey, const unsigned char* msg, size_t msg_len, const unsigned char* sig, size_t sig_len) {
    EVP_MD_CTX* mdctx = EVP_MD_CTX_new();
    if (!mdctx) {
        std::cerr << "Failed to create EVP_MD_CTX" << std::endl;
        print_openssl_errors();
        return false;
    }

    bool result = false;
    if (EVP_DigestVerifyInit(mdctx, nullptr, EVP_sha256(), nullptr, pkey) <= 0) {
        std::cerr << "EVP_DigestVerifyInit failed" << std::endl;
        print_openssl_errors();
        EVP_MD_CTX_free(mdctx);
    }

    if (EVP_DigestVerifyUpdate(mdctx, msg, msg_len) <= 0) {
        std::cerr << "EVP_DigestVerifyUpdate failed" << std::endl;
        print_openssl_errors();
        EVP_MD_CTX_free(mdctx);
    }

    int verify_result = EVP_DigestVerifyFinal(mdctx, sig, sig_len);
    if (verify_result == 1) {
        result = true; // Verification succeeded
    } else if (verify_result == 0) {
        std::cerr << "Signature is invalid" << std::endl;
    } else {
        // THIS CASE IS HAPPENING ALL THE TIME
        std::cerr << "EVP_DigestVerifyFinal failed" << std::endl;
        print_openssl_errors();
    }

    EVP_MD_CTX_free(mdctx);

    return result;
}

When calling these functions with the data used in the python example, it tells me that the EVP_DigestVerifyFinal failed, and the EVP_DigestVerifyFinal() returns -1. EVP_get_error() shows no error unfortunately and the documentation states that -1 means something is probably wrong with the signature.

EVP_DigestVerifyFinal() and EVP_DigestVerify() return 1 for success; any other value indicates failure. A return value of zero indicates that the signature did not verify successfully (that is, tbs did not match the original data or the signature had an invalid form), while other values indicate a more serious error (and sometimes also indicate an invalid signature form).

I think I've loaded the public key properly because when I change the public key slightly I'm getting an error.

My calling context looks like this:

std::array<std::uint8_t, 33> pk{0x03, 0x03, 0xB2, 0xCE, 0x64, 0xBC, 0x20, 0x7B, 0xDD, 0x8B, 0xC4, 0xDF, 0x85, 0x91, 0x87, 0xFC, 0xB6, 0x86, 0x32, 0x0D, 0x63, 0xFF, 0xA0, 0x91, 0x41, 0x0F, 0xC1, 0x58, 0xFB, 0xB7, 0x79, 0x80, 0xEA};
std::array<std::uint8_t, 64> ds{0xF8, 0xCD, 0x88, 0x29, 0x9F, 0xA4, 0x60, 0x58, 0x00, 0x20, 0x7B, 0xFE, 0xBE, 0xAC, 0x55, 0x02, 0x40, 0x53, 0xF3, 0x0F, 0x7C, 0x69, 0xB3, 0x5C, 0x15, 0xE6, 0x08, 0x00, 0xAC, 0x3B, 0x6F, 0xE3, 0xED, 0x06, 0x39, 0x95, 0x2F, 0x7B, 0x02, 0x8D, 0x86, 0x86, 0x74, 0x45, 0x96, 0x1F, 0xFE, 0x94, 0xFB, 0x22, 0x6B, 0xFF, 0x70, 0x06, 0xE0, 0xC4, 0x51, 0xEE, 0x3F, 0x87, 0x28, 0xC1, 0x77, 0xFB};
std::array<std::uint8_t, 29> m{0x82, 0x10, 0x49, 0x22, 0x04, 0xE0, 0x60, 0x61, 0x0B, 0xDF, 0x26, 0xD7, 0x7B, 0x5B, 0xF8, 0xC9, 0xCB, 0xFC, 0xF7, 0x04, 0x22, 0x08, 0x14, 0x75, 0xFD, 0x44, 0x5D, 0xF0, 0xFF};


// Load public key
EVP_PKEY* pkey = load_public_key(pk.data(), pk.size());
if (!pkey) {
    std::cerr << "Failed to load public key" << std::endl;
    return;
}

// Verify signature
bool is_valid = verify_signature(pkey, m.data(), m.size(), ds.data(), ds.size());
if (is_valid) {
    std::cout << "Signature is valid" << std::endl;
} else {
    // THIS ERROR IS SEEN
    std::cout << "Signature is invalid" << std::endl;
}

// Tidy up public key
EVP_PKEY_free(pkey);

When someone sees a chance to reduce the code in order to get a more minimal example, please let me know. Thanks!


Solution

  • As @Topaco pointed out, the problem was the signature format. The format of the signature in my calling context was in P1363 format. The implementation expects the signature to be ASN.1 DER Encoding for NIST P-256. Find more details here.

    I've implemented a function (in collaboration with ChatGPT) to have a corresponding conversion function.

    std::vector<uint8_t> convert_p1363_to_der(const std::array<uint8_t, 64>& signature) {
        std::unique_ptr<ECDSA_SIG, decltype(&ECDSA_SIG_free)> ecdsa_sig{ECDSA_SIG_new(), ECDSA_SIG_free};
    
        using BigNumUniquePtr = std::unique_ptr<BIGNUM, decltype(&BN_free)>;
        BigNumUniquePtr r{BN_bin2bn(signature.data(), 32, nullptr), BN_free};
        BigNumUniquePtr s{BN_bin2bn(signature.data() + 32, 32, nullptr), BN_free};
    
        if (!r || !s || !ecdsa_sig) {
            std::cerr << "Failed to convert ECDSA_SIG to DER" << std::endl;
            return {};
        }
    
        ECDSA_SIG_set0(ecdsa_sig.get(), r.release(), s.release());
    
        std::vector<std::uint8_t> der(72);
        auto* derPtr{der.data()};
        int der_size{i2d_ECDSA_SIG(ecdsa_sig.get(), &derPtr)};
        if (der_size < 0) {
            std::cerr << "Failed to convert ECDSA_SIG to DER" << std::endl;
            return {};
        }
    
        der.resize(der_size);
        return der;
    }
    

    My calling context looks like this now:

    std::array<std::uint8_t, 33> pk{0x03, 0x03, 0xB2, 0xCE, 0x64, 0xBC, 0x20, 0x7B, 0xDD, 0x8B, 0xC4, 0xDF, 0x85, 0x91, 0x87, 0xFC, 0xB6, 0x86, 0x32, 0x0D, 0x63, 0xFF, 0xA0, 0x91, 0x41, 0x0F, 0xC1, 0x58, 0xFB, 0xB7, 0x79, 0x80, 0xEA};
    std::array<std::uint8_t, 64> ds{0xF8, 0xCD, 0x88, 0x29, 0x9F, 0xA4, 0x60, 0x58, 0x00, 0x20, 0x7B, 0xFE, 0xBE, 0xAC, 0x55, 0x02, 0x40, 0x53, 0xF3, 0x0F, 0x7C, 0x69, 0xB3, 0x5C, 0x15, 0xE6, 0x08, 0x00, 0xAC, 0x3B, 0x6F, 0xE3, 0xED, 0x06, 0x39, 0x95, 0x2F, 0x7B, 0x02, 0x8D, 0x86, 0x86, 0x74, 0x45, 0x96, 0x1F, 0xFE, 0x94, 0xFB, 0x22, 0x6B, 0xFF, 0x70, 0x06, 0xE0, 0xC4, 0x51, 0xEE, 0x3F, 0x87, 0x28, 0xC1, 0x77, 0xFB};
    std::array<std::uint8_t, 29> m{0x82, 0x10, 0x49, 0x22, 0x04, 0xE0, 0x60, 0x61, 0x0B, 0xDF, 0x26, 0xD7, 0x7B, 0x5B, 0xF8, 0xC9, 0xCB, 0xFC, 0xF7, 0x04, 0x22, 0x08, 0x14, 0x75, 0xFD, 0x44, 0x5D, 0xF0, 0xFF};
    
    // THIS CALL IS NEW
    auto dsAsDer{convert_p1363_to_der(ds)};
    
    // Load public key
    EVP_PKEY* pkey = load_public_key(pk.data(), pk.size());
    if (!pkey) {
        std::cerr << "Failed to load public key" << std::endl;
        return;
    }
    
    // Verify signature - USING SIGNATURE IN DER FORMAT HERE
    bool is_valid = verify_signature(pkey, m.data(), m.size(), dsAsDer.data(), dsAsDer.size()); ds.size());
    if (is_valid) {
        std::cout << "Signature is valid" << std::endl;
    } else {
        std::cout << "Signature is invalid" << std::endl;
    }
    
    // Tidy up public key
    EVP_PKEY_free(pkey);
    

    Please feel free to hijack this answer, modify it and repost it. I'll delete this post and accept the best answer. I've only limited knowledge about libcrypto so far and relying on experienced input. When one might give more details about the different signature formats or a better implementation I'd appreciate that.