I am trying to write code to verify some RSA signatures. The signatures were made using the OpenSSL command-line tool, using the equivalent of this command line:
openssl dgst -sha1 -sign private_key_file.pem < binary_data_file > sig
I am trying to use libtomcrypt
to do the verify:
Here is the calling signature of the RSA verification function in libtomcrypt
:
int rsa_verify_hash_ex(
const unsigned char *sig, unsigned long siglen, // signature to verify
const unsigned char *hash, unsigned long hashlen, // hash value to check against sig
int padding, // defined constant value, see below
int hash_idx, // identifies which hash algorithm, see below
unsigned long saltlen, // specify salt length, see below
int *stat, // output parameter, returns whether verify succeeded or not
rsa_key *key); // RSA public key to use for verify
This function returns a 0 if it operates without error, otherwise returns an error code. If it operates without error, the stat
output parameter indicates whether the signature verified.
Most of the arguments seem straightforward: pass in the signature to check, the hash value to use to compare it, and the RSA key to use for the check. hash_idx
is clear from the example code included with libtomcrypt
; it is an index into a table of supported hash algorithms, and I can find the correct value to use with this code snippet: hash_idx = find_hash("sha1")
But I'm wondering about the padding
and saltlen
values. padding
doesn't worry me too much, as there are only two possible values, and I can just try them both. But what should I pass for saltlen
?
The OpenSSL documentation for the OpenSSL functions for RSA verify don't show a saltlen
parameter. The man page for openssl dgst
(i.e. the result of man dgst
) does not discuss salt.
So my questions:
dgst
command insert any extra stuff in the input, such as: (stdin)=
(I found that (stdin)=
thing by searching StackOverflow: Why are the RSA-SHA256 signatures I generate with OpenSSL and Java different?)
libtomcrypt
also has a function called pkcs_1_pss_decode()
which is documented to "decode a PSS encoded signature block". Is there any chance that this is the function I need to call?Thanks for any help you can give me.
EDIT: thanks to the help below, from @Jonathan Ben-Avraham, I was able to get this working today. The answers to my questions are, respectively:
(stdin)=
rsa_verify_hash_ex()
, and I needed to specify the padding
argument as LTC_LTC_PKCS_1_V1_5
.No salt:
First, generate a binary SHA1 hash of your data:
openssl dgst -sha1 -binary -out hash1 some_data_file
This is an SHA1 hash or digest. There is no salt prependended to the file some_data_file
. The openssl dgst -sha1
itself does not add salt. Note that the output file is just a 20 byte SHA1 hash with no salt. If there were salt, the hash would have to include it, probably prepended before the last 20 bytes that hold the SHA1 hash.
Next, sign the SHA1 hash file hash1
with your private key:
openssl pkeyutl -sign -in hash1 -inkey privkey.pem -pkeyopt digest:sha1 -out sig1
Now sign the some_data_file
with openssl dgst
:
openssl dgst -sha1 -sign privkey.pem < some_data_file > sig2
Finally, compare the two signatures:
diff sig1 sig2
and you should see that they are the same. This tells us that signing the raw SHA1 hash of a file with no salt is the same as using the openssl dgst -sha1 -sign
command to sign the file, so it must be that the openssl dgst -sha1 -sign
command also did not use any salt when generating its SHA1 hash for sig2
.
Note also that you cannot achieve the same result using the deprecated rsautl
:
openssl rsautl -sign -in hash1 -inkey privkey.pem -out sig1
instead of openssl pkeyutl
, because openssl rsautl -sign
does not do the ASN.1 encoding of DigestInfo as required by RSASSA-PKCS1-v1_5 defined in e.g. RFC3447 section 9.2 step 2. (This is why you need -pkeyopt digest:
even though pkeyutl -sign
itself doesn't do any hashing.) See this SE post for details.