encryptionopensslx509cfssl

Encrypt a file with a x509 certificate with cfssl and openssl

Noob question:

Given a x509 certificate created with cfssl:

server.pem
server-key.pem

issued by

ca.pem

Usages for server.pem are:

"server": {
  ...
  "usages": [ "signing", "key encipherment", "server auth", "data encipherment", "s/mime" ]
  ...
},

I'm able to verify the certificate with openssl:

openssl verify -CAfile ca.pem server.pem
server.pem: OK

I'm able to sign a plain text file:

openssl dgst -sha256 -sign server-key.pem -out signable.txt.sha256 signable.txt

And verify the signature

openssl x509 -pubkey -noout -in server.pem | tee server-pubkey.pem
openssl dgst -sha256 -verify server-pubkey.pem -signature signable.txt.sha256 signable.txt
Verified OK

But now I can't find out how to use the certificate for encryption/decryption:

Attempt 1

openssl smime -encrypt -aes-256-cbc -in secret.txt -out secret.txt.enc -outform DER server-key.pem

Could not read recipient certificate file from server-key.pem
4027E4F7E97F0000:error:1608010C:STORE routines:ossl_store_handle_load_result:unsupported:../crypto/store/store_result.c:151:
Unable to load recipient certificate file

Attempt 2

openssl smime -encrypt -aes-256-cbc -in secret.txt -out secret.txt.enc -outform DER server.pem

Error creating PKCS#7 structure
40F7A87F027F0000:error:10800096:PKCS7 routines:PKCS7_RECIP_INFO_set:encryption not supported for this key type:../crypto/pkcs7/pk7_lib.c:637:
40F7A87F027F0000:error:10800078:PKCS7 routines:PKCS7_encrypt_ex:error adding recipient:../crypto/pkcs7/pk7_smime.c:467:

Attempt 3

cat server.pem server-key.pem > server.pkcs12

openssl smime -encrypt -aes-256-cbc -in secret.txt -out secret.txt.enc -outform DER server.pkcs12

Error creating PKCS#7 structure
40C7B2B9947F0000:error:10800096:PKCS7 routines:PKCS7_RECIP_INFO_set:encryption not supported for this key type:../crypto/pkcs7/pk7_lib.c:637:
40C7B2B9947F0000:error:10800078:PKCS7 routines:PKCS7_encrypt_ex:error adding recipient:../crypto/pkcs7/pk7_smime.c:467:

Any clue?

Solution

    1. Meta: this is not about programming or development, and is out of scope for StackOverflow. However I can't fit the following in readable comments. I will delete if necessary to close or remove the question.

    2. do openssl x509 -in server.pem -text -noout and look at the line Public Key Algorithm. If it says dsaEncryption (horrible name BTW) or rsassaPss you can't encrypt with this cert (and key); these algorithms do not support encryption.* If it says id-ecPublicKey the smime command (which as you can see in the error message actually does PKCS7) cannot use it to encrypt but the cms command can -- this is one of the few differences between PKCS7 and CMS (the addition of the KeyAgreeRecipInfo choice aka KARI).

    * DSA was designed in an earlier century specifically to prevent encryption to allow its use without regard to then-current legal prohibition on exporting encryption 'technology' from the US. And also without regard to the Schnorr patent claims, but that's a more complicated story. Although RSA in general (and plain RSA keys) can be used for both signature and encryption, the PSS (Probabilistic Signature Scheme) variant is only defined for signature. There is a similar variant OAEP (Optimal Asymmetric Encryption Padding) for encryption but it does not use a different algorithm identifier in the certificate like PSS optionally does.

    1. To encrypt in any of PKCS7/CMS/SMIME you only need the certificate, not the privatekey. (Technically to encrypt in any PKC you need the publickey, but PKCS7/CMS/SMIME use other data in the certificate in addition to the publickey.) You will generally need the privatekey and certificate to decrypt.

    2. Concatenating two PEM files does not create a PKCS12 file, and naming such a file .pkcs12 is confusing, misleading, and deceptive to humans -- though the program ignores it.