I am using MbedTLS in firmware. I successfully received a remote file and an encrypted hash of it that was encrypted with a private key (via OpenSSL). I used mbedtls_sha256() to locally make my own hash of the file and then mbedtls_pk_verify() to compare that with the encrypted hash I received.
The function mbedtls_pk_verify() fails, though. Looking at the memory with a debugger, I can see the two hashes themselves match while paused within mbedtls_pk_verify(), but I noticed there are many bytes that precede one hash that aren't present with the other (decrypted) hash. And I see that mbedtls_pk_verify() tries to compare those extra bytes...
I found out that these are likely "OID" bytes, used to declare the SHA-256 algorithm. Is there a way to automatically prepend these required bytes? Otherwise, what is the usual method of doing what I want?
I found out that mbedtls_pk_verify() failed in MbedTLS because I used the wrong command in OpenSSL to create my original encrypted hash/signature. I used openssl dgst... when I needed to use openssl sha256 -sign... This added the required (signature) bytes that MbedTLS' mbedtls_pk_verify() was looking for. Thanks to all respondents for helping me figure it out.