'Encrypted message length is invalid.' using node-forge to decrypt the message encrypted from jsencrypt

currently using jsencrypt and node-forge for decrypt and encrypt the message using RSA, while jsencrypt is used in frontend and node-forge in backend.

import { JSEncrypt } from 'jsencrypt'
import * as forge from 'node-forge';

const message = 'data....'
const publicKey = '-----BEGIN PUBLIC KEY-----\nMIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQCqM+l9ZWy1Frt6felFFLmfZNls\nVbU1dKpF8Rx83FtKCsztO5k/iV5N9BbfHFUg9Y40b/EK2j/BPc1xlLYAHMXn6563\nXCwZ4IuCxvfOwz9qT9gkKBxkI5b0rnikkSWTGlJEk2PdZ7Plc73Fa+bx3PvuKvMd\ncKWvd80+vt9+b/7hrwIDAQAB\n-----END PUBLIC KEY-----'

const publicK = forge.pki.publicKeyFromPem(publicKey)
const encrypted = publicK.encrypt( message, 'RSA-OAEP')
console.log('encrpted：', encrypted) 

const privateKey = '-----BEGIN PRIVATE KEY-----\nMIICdgIBADANBgkqhkiG9w0BAQEFAASCAmAwggJcAgEAAoGBAKoz6X1lbLUWu3p9\n6UUUuZ9k2WxVtTV0qkXxHHzcW0oKzO07mT+JXk30Ft8cVSD1jjRv8QraP8E9zXGU\ntgAcxefrnrdcLBngi4LG987DP2pP2CQoHGQjlvSueKSRJZMaUkSTY91ns+VzvcVr\n5vHc++4q8x1wpa93zT6+335v/uGvAgMBAAECgYArxUnou6qnL39rUvIol9ncyfy4\nRZpicuxPLGCdI7Y+ZmSpJciVdGhSN9Gh8xFZdozpo1gj6Fi5A4HQEeR0RvIF9Rgh\nERblj1rRWqxPcsIddOO9VaknQPICWKqEW9+E1bEcyNUblCHA4LGyQwmuEFUb/Tkj\nxAghIHuEBCe0GFiVwQJBAN5i5QSoOIpdFHA0c981E4VhHc/muXwjx1HfE1pcuuFb\nTy3OwEoZdFp3LIjBnBkPRneLTNjo5WTIwrmfsy6VDF8CQQDD7c6d/nKiJwIESlr+\n/idqXAPNR/iS1YX3Nqtk9jgrgf5zULHr2nbk7MDas5S9Z9XPdUmxtnP44dhoGvDk\nzyyxAkB7XBxyQuZqSkvGGjKUhJq5iC/DXddSd35fegEARSQdUktPu7qK4Cfc7vKz\nQcLXW9PZCFqukDJ/f6YU1fPNSTy9AkADQ78hms/GK+g4shR6EzoM56OYlA5sQ+qL\nh/mrIP8mmm/m8/1C9MzuW5OLEVr1HPnPDyE/OM8N4pV8hpZk+Z7BAkEAzaFstazA\nxLzZOBWhvOzzo722glZ7HVezhMocLu7Y3EOXP/nbx09JpU3U7Egp5UVp0aiknh/Q\nez4Cc4ksMedxdA==\n-----END PRIVATE KEY-----\n'
const privateK = forge.pki.privateKeyFromPem(privateKey)

const decrypted = privateK.decrypt(encrypted, 'RSA-OAEP')
console.log('original：', decodeURIComponent(decrypted))

this worked.

then trying with jsencrypt with same pub/pri key and message.

  const encrypt = new JSEncrypt();
  encrypt.setPublicKey(publicKey);
  let encrypedQuery = encrypt.encrypt( message );
  console.log( encrypedQuery );

  try{
    const privateK = forge.pki.privateKeyFromPem(privateKey)
    const decrypted2 = privateK.decrypt(encrypedQuery)
    console.log('original：：', decodeURIComponent(decrypted2)) 
  }catch(err){
    console.log(err);
    
  }

this outputs an error saying 'Encrypted message length is invalid.'

Solution

The problem in the second code snippet is an encoding issue that arises because the JSEncrypt side returns a Base64 encoded ciphertext, while node-forge requires a binary/latin1 string. The issue can be fixed with forge.util.decode64() (or atob()):

const publicKey = '-----BEGIN PUBLIC KEY-----\nMIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQCqM+l9ZWy1Frt6felFFLmfZNls\nVbU1dKpF8Rx83FtKCsztO5k/iV5N9BbfHFUg9Y40b/EK2j/BPc1xlLYAHMXn6563\nXCwZ4IuCxvfOwz9qT9gkKBxkI5b0rnikkSWTGlJEk2PdZ7Plc73Fa+bx3PvuKvMd\ncKWvd80+vt9+b/7hrwIDAQAB\n-----END PUBLIC KEY-----'
const privateKey = '-----BEGIN PRIVATE KEY-----\nMIICdgIBADANBgkqhkiG9w0BAQEFAASCAmAwggJcAgEAAoGBAKoz6X1lbLUWu3p9\n6UUUuZ9k2WxVtTV0qkXxHHzcW0oKzO07mT+JXk30Ft8cVSD1jjRv8QraP8E9zXGU\ntgAcxefrnrdcLBngi4LG987DP2pP2CQoHGQjlvSueKSRJZMaUkSTY91ns+VzvcVr\n5vHc++4q8x1wpa93zT6+335v/uGvAgMBAAECgYArxUnou6qnL39rUvIol9ncyfy4\nRZpicuxPLGCdI7Y+ZmSpJciVdGhSN9Gh8xFZdozpo1gj6Fi5A4HQEeR0RvIF9Rgh\nERblj1rRWqxPcsIddOO9VaknQPICWKqEW9+E1bEcyNUblCHA4LGyQwmuEFUb/Tkj\nxAghIHuEBCe0GFiVwQJBAN5i5QSoOIpdFHA0c981E4VhHc/muXwjx1HfE1pcuuFb\nTy3OwEoZdFp3LIjBnBkPRneLTNjo5WTIwrmfsy6VDF8CQQDD7c6d/nKiJwIESlr+\n/idqXAPNR/iS1YX3Nqtk9jgrgf5zULHr2nbk7MDas5S9Z9XPdUmxtnP44dhoGvDk\nzyyxAkB7XBxyQuZqSkvGGjKUhJq5iC/DXddSd35fegEARSQdUktPu7qK4Cfc7vKz\nQcLXW9PZCFqukDJ/f6YU1fPNSTy9AkADQ78hms/GK+g4shR6EzoM56OYlA5sQ+qL\nh/mrIP8mmm/m8/1C9MzuW5OLEVr1HPnPDyE/OM8N4pV8hpZk+Z7BAkEAzaFstazA\nxLzZOBWhvOzzo722glZ7HVezhMocLu7Y3EOXP/nbx09JpU3U7Egp5UVp0aiknh/Q\nez4Cc4ksMedxdA==\n-----END PRIVATE KEY-----\n'
const message = 'data....'

const encrypt = new JSEncrypt();
encrypt.setPublicKey(publicKey);
let encryptedQuery = encrypt.encrypt(message);
console.log(encryptedQuery);

try {
    const privateK = forge.pki.privateKeyFromPem(privateKey);
    const decrypted2 = privateK.decrypt(forge.util.decode64(encryptedQuery)); // fix
    console.log('original：：', decodeURIComponent(decrypted2)); 
} catch(err){
    console.log(err);
}

<script src="https://cdnjs.cloudflare.com/ajax/libs/forge/1.3.1/forge.min.js"></script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/jsencrypt/3.3.2/jsencrypt.min.js"></script>

Note that the first code snippet uses OAEP as padding (explicitly specified), while the second code snippet applies PKCS#1 v1.5 padding (JSEncrypt exclusively uses PKCS#1 v1.5, node-forge applies it by default).
Since the padding is consistent within the same code snippet, decryption works. However, it would not be possible to decrypt a ciphertext generated with the first code snippet with the second or vice versa.

JSEncrypt with OAEP:

Although node-forge supports PKCS#1 v1.5 (by default) and OAEP as padding, the padding of the regular JSEncrypt library is PKCS#1 v1.5 and cannot be changed. Therefore, only PKCS#1 v1.5 can be used for interoperation between the two libraries.

As mentioned in the comment, the older PKCS#1 v1.5 padding is vulnerable to certain attacks and OAEP is the more secure alternative.
If you want to use JSEncrypt with OAEP: There is a JSEncrypt pull request on Github and a fork that extends JSEncrypt with OAEP, which might be an option for you. However, this uses SHA-256 for both the OAEP and MGF1 digests, while node-forge applies SHA-1 by default, but also supports other digests. Since the digest on the JSEncrypt side cannot be changed, the digest on the node-forge side must be changed to SHA-256 for interoperation, e.g.:

const publicKey = '-----BEGIN PUBLIC KEY-----\nMIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQCqM+l9ZWy1Frt6felFFLmfZNls\nVbU1dKpF8Rx83FtKCsztO5k/iV5N9BbfHFUg9Y40b/EK2j/BPc1xlLYAHMXn6563\nXCwZ4IuCxvfOwz9qT9gkKBxkI5b0rnikkSWTGlJEk2PdZ7Plc73Fa+bx3PvuKvMd\ncKWvd80+vt9+b/7hrwIDAQAB\n-----END PUBLIC KEY-----'
const privateKey = '-----BEGIN PRIVATE KEY-----\nMIICdgIBADANBgkqhkiG9w0BAQEFAASCAmAwggJcAgEAAoGBAKoz6X1lbLUWu3p9\n6UUUuZ9k2WxVtTV0qkXxHHzcW0oKzO07mT+JXk30Ft8cVSD1jjRv8QraP8E9zXGU\ntgAcxefrnrdcLBngi4LG987DP2pP2CQoHGQjlvSueKSRJZMaUkSTY91ns+VzvcVr\n5vHc++4q8x1wpa93zT6+335v/uGvAgMBAAECgYArxUnou6qnL39rUvIol9ncyfy4\nRZpicuxPLGCdI7Y+ZmSpJciVdGhSN9Gh8xFZdozpo1gj6Fi5A4HQEeR0RvIF9Rgh\nERblj1rRWqxPcsIddOO9VaknQPICWKqEW9+E1bEcyNUblCHA4LGyQwmuEFUb/Tkj\nxAghIHuEBCe0GFiVwQJBAN5i5QSoOIpdFHA0c981E4VhHc/muXwjx1HfE1pcuuFb\nTy3OwEoZdFp3LIjBnBkPRneLTNjo5WTIwrmfsy6VDF8CQQDD7c6d/nKiJwIESlr+\n/idqXAPNR/iS1YX3Nqtk9jgrgf5zULHr2nbk7MDas5S9Z9XPdUmxtnP44dhoGvDk\nzyyxAkB7XBxyQuZqSkvGGjKUhJq5iC/DXddSd35fegEARSQdUktPu7qK4Cfc7vKz\nQcLXW9PZCFqukDJ/f6YU1fPNSTy9AkADQ78hms/GK+g4shR6EzoM56OYlA5sQ+qL\nh/mrIP8mmm/m8/1C9MzuW5OLEVr1HPnPDyE/OM8N4pV8hpZk+Z7BAkEAzaFstazA\nxLzZOBWhvOzzo722glZ7HVezhMocLu7Y3EOXP/nbx09JpU3U7Egp5UVp0aiknh/Q\nez4Cc4ksMedxdA==\n-----END PRIVATE KEY-----\n'
const message = 'data....'

const encrypt = new JSEncrypt();
encrypt.setPublicKey(publicKey);
let encryptedQuery = encrypt.encryptOAEP(message); // OAEP with SHA256
console.log(encryptedQuery);

try {
    const privateK = forge.pki.privateKeyFromPem(privateKey);
    const decrypted2 = privateK.decrypt(forge.util.decode64(encryptedQuery), 'RSA-OAEP', {md: forge.md.sha256.create()}); // OAEP with SHA256
    console.log('original：：', decodeURIComponent(decrypted2)); 
} catch(err){
    console.log(err);
}

<script src="http://cdn.jsdelivr.net/gh/kingller/jsencrypt/bin/jsencrypt.min.js"></script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/forge/1.3.1/forge.min.js"></script>