How do I write embedded pgp signatures in Java using bouncyCastle?

I am trying to sign an encrypt a message in pgp, with an embedded signature.

I am able to create a detached signature with the code below, using bouncyCastle's pgp libraries to sign and encrypt :

public static byte[] pgpSign(PGPPrivateKey signingKey, byte[] data) throws PGPException, IOException {
        PGPSignatureGenerator sGen = new PGPSignatureGenerator(
                new BcPGPContentSignerBuilder(signingKey.getPublicKeyPacket().getAlgorithm(), SHA256));
        sGen.init(PGPSignature.BINARY_DOCUMENT, signingKey);
        sGen.update(data);
        PGPSignature signature = sGen.generate();
        ByteArrayOutputStream output = new ByteArrayOutputStream();
        signature.encode(output);
        output.close();
        return output.toByteArray();
}

public static byte[] pgpSignAndEncrypt_Detached(PGPPublicKey encryptionKey, PGPPrivateKey signingKey, byte[] data) throws IOException, PGPException, ClassCastException{
        PGPEncryptedDataGenerator encryptionGenerator = new PGPEncryptedDataGenerator(
                new BcPGPDataEncryptorBuilder(SymmetricKeyAlgorithmTags.AES_256)
                        .setSecureRandom(new SecureRandom())
                        .setWithIntegrityPacket(true)
        );
        encryptionGenerator.addMethod(new BcPublicKeyKeyEncryptionMethodGenerator(encryptionKey));
        ByteArrayOutputStream encryptedStream = new ByteArrayOutputStream();


        OutputStream literalDataStream = encryptionGenerator.open(encryptedStream, new byte[1048576]);
        OutputStream rawStream = new PGPLiteralDataGenerator().open(literalDataStream, PGPLiteralData.BINARY, "", new Date(), new byte[1048576]);
        rawStream.write(data);
        rawStream.close();
        literalDataStream.close();
        encryptedStream.write(pgpSign(signingKey, data));
        encryptedStream.close();
        return encryptedStream.toByteArray();
}

I was expecting to get an embedded signature, however, when trying to decrypt with gpg I get the following output :

gpg2 --decrypt testEmbedded.pgp

gpg: encrypted with 3072-bit RSA key, ID 9505FCA2AEF76A5B, created 2023-06-19

"userId (testKeypair) mailto:user@email.com"

Hello World

Detached signature.

Please enter name of data file:

What's the difference between detached and embedded signatures ? After all this "detached" signature is in the same .pgp file as the encrypted message. How do I create an embedded signature, such that I don't get prompted for the original data file to verify the signature when decrypting with gpg ?

Not a full answer as I don't have time to rewrite your code, but:

Unlike PKCS7/CMS/SMIME where an embedded (also called encapsulated) signature contains the data within the SignedData structure, in PGP a (non-detached) signed message is just a series of packets:

1. a "One-Pass Signature" packet, which announces the algorithms to be used

2. the data, normally as a "Literal Data" packet, but optionally can be a "Compressed Data" packet containing the "Literal Data"

3. a "Signature" packet, containing (to no one's great surprise) the signature of the data

See rfc4880 11.3. Technically there is also defined an older format which doesn't use the "One-Pass" packet and has the Signature packet before the data, but in practice that is obsolete.

In contrast a Signature packet by itself, which is what your code generates, is a detached signature, per 11.4 (one page down from the above link).

To generate the 'embedded' signed-message in BCPG, you must therefore

1. generate the One-Pass packet

2. write the data into a Literal packet (or Compressed/Literal packets) AND update the signature with that data; it's most convenient to do these together, although it would be sufficient to do them separately as long as both are in fact done and consistent

3. generate the Signature packet

See How to sign and verify the file in JAVA for a working example (after the correction in my answer) or https://github.com/bcgit/bc-java/blob/main/pg/src/main/java/org/bouncycastle/openpgp/examples/SignedFileProcessor.java .

When you (only) encrypt data in PGP you must first embed it in a Literal packet, or more commonly (and usually the default) Compressed/Literal packets, but when you sign-then-encrypt the signed-message is already a PGP message and is fed directly into Encrypted, or (again) more commonly into Compressed into Encrypted. I don't have to hand a good reference for that part.