RSA encrypt using JSEncrypt and decrypt using BouncyCastle (Java)

This might be a duplicate of this answered question, but I can't seem to get the same results. Hoping for some guidance here.

JSEncrypt (client)

let encrypt = new Encrypt.JSEncrypt();
encrypt.setPublicKey(this.publicKey);  // retrieved from server
encrypt.encrypt(password);

BouncyCastle (server) - RSA key generation

KeyPairGenerator generator = KeyPairGenerator.getInstance("RSA");
generator.initialize(1024);
KeyPair pair = generator.generateKeyPair();
PublicKey pubKey = pair.getPublic();
PrivateKey privKey = pair.getPrivate();

// returned to client
String publicKeyStr = new String(Base64.encodeBase64(pubKey.getEncoded()));
String privateKeyStr = new String(Base64.encodeBase64(privKey.getEncoded()));

BouncyCastle (server) - Decryption

Cipher cipher = Cipher.getInstance("RSA/None/PKCS1Padding");
cipher.init(Cipher.DECRYPT_MODE, privateKey);
// org.apache.commons.codec.binary.Hex

byte[] cipherText = cipher.doFinal(Hex.decodeHex(encrypted.toCharArray()));
decrypted = new String(cipherText, BaseConstant.ENC_UTF8);

Error

org.apache.commons.codec.DecoderException: Illegal hexadecimal character I at index 0 at org.apache.commons.codec.binary.Hex.toDigit(Hex.java:178) at org.apache.commons.codec.binary.Hex.decodeHex(Hex.java:89)

One thing I noticed is the length of encrypted text by JSEncrypt, which is 172, while encryption at server side produces 256.

The answered question mentioned to use RSA/None/PKCS1Padding, which I had already set. What else could I be missing?

Solution

The error occurs in Hex.decodeHex() method, which means that your data is not a Hex encoded string.

JSEncrypt.encrypt() method returns the encrypted data in Base64 (instead of Hex string). In order to decrypt it, you must decode it from base64 format.

So instead of:

byte[] cipherText = cipher.doFinal(Hex.decodeHex(encrypted.toCharArray()));

Do this:

byte[] cipherText = cipher.doFinal(Base64.decodeBase64(encrypted.toCharArray()));