We faced problem with REST calls from GCP Dataflow (Apache Beam/ Java SDK based) to our internal service within GCP with custom Self-Signed Certificate.
We tried to put certificate in Runtime to ../java/../cacerts file and it works well, but Java reads ../cacert during start and we are forced to override SSlContext and X509TrustManager, put it to HttpClient and reload cacert file in Runtime before http call and after custom cert loaded (followed this example).
From my perspective it looks like pretty hard implementation.
So the question: Does this problem has any easy workaround like run .sh script before GCP Dataflow Worker runs and update cacert file via cli before JVM started.
We also up Dataflow via gcloud cli so we cannot just put custom cacert to the project and point JVM to it via Java Options.
Thanks for any suggestions!
We did something similar for authenticating to services which use self-signed certificates.
You could always use custom containers for your workers and copy the certificate to cacerts in the dockerfile? This way the worker will always have the certificate in cacerts. (Custom containers are only supported for pipelines using Dataflow Runner v2)