node.jsnode-crypto

RSA-SHA256 signing algorithm not behaving as expected

I have recently adopted some third-party security software, and I am trying to understand how to process the signed messages we are receiving from them. In the process, I am using the Node Crypto library to solidify my understanding of the concepts.

As I understand the RSA-SHA256 signing algorithm, in the process of signing:

And then the client will send us the original message, the public key, and the signature. The process of verifying the signature is as follows:

If this is the case, then in the following code snippet:

const { 
  generateKeyPairSync,
  publicDecrypt, privateEncrypt,
  createSign, createVerify,
  createHash,
} = require('crypto');

const { privateKey, publicKey } = generateKeyPairSync('rsa', {
  modulusLength: 2048,
  publicKeyEncoding: {
    type: 'spki',
    format: 'pem',
  },
  privateKeyEncoding: {
    type: 'pkcs8',
    format: 'pem',
  },
});
const messageToSign = "Yma o hyd";

const signer = createSign('RSA-SHA256');
signer.update(messageToSign);
const signature = signer.sign(privateKey);
console.log('Signed message', signature.toString('hex'));

const verifier = createVerify('RSA-SHA256');
verifier.update(messageToSign);
const isVerified = verifier.verify(publicKey, signature);
console.log('isVerified:', isVerified);

const hash = createHash('SHA256').update(messageToSign).digest();
console.log('hash in hex', hash.toString('hex'));
const hashEncryptedWithPrivateKey = privateEncrypt(
  privateKey,
  hash,
);

console.log('encrypted hash', hashEncryptedWithPrivateKey.toString('hex'));
const signatureDecryptedWithPublicKey = publicDecrypt(
  publicKey,
  signature
);
console.log('Signature decrypted with public key', signatureDecryptedWithPublicKey.toString('hex'));

I would expect signatureDecryptedWithPublicKey.toString('hex') to match hash.toString('hex'), but it does not.

Does the signer do more than just a straightforward combination of RSA and SHA256?

Solution

  • You haven't taken padding into account in your considerations, s. RFC8017, RSASSA-PKCS1-v1_5.

    In order for hashEncryptedWithPrivateKey to be equal to signature, not hash must be passed to privateEncrypt(), but the DER encoding of the DigestInfo value T, which is for SHA256: 0x3031300d060960864801650304020105000420 || hash, thus:

    const hashEncryptedWithPrivateKey = privateEncrypt(
  privateKey,
  Buffer.concat([Buffer.from('3031300d060960864801650304020105000420', 'hex'), hash])
);

    Likewise,

    const signatureDecryptedWithPublicKey = publicDecrypt(
  publicKey,
  signature
);

    does not return hash but T.

    A bit more background: