Is it possible to deploy PaLM entirely on my own (GCP) cloud environment (hence ensuring no sensitive data leaves my environment) ? If not, is such deployment pattern on the roadmap ?
By design, the foundation models are deployed once by google and managed centrally by Google. So, you can't have access to those model and deploy them on your own project (on GCP or elsewhere).
However, as any services, Google Cloud guaranty the data privacy by default. Take the Cloud Storage or BigQuery, you never asked to have an instance in your project! You use a SaaS solution, managed by google. And it works seamlessly!!
If you want to fine tune the foundation model, again, you won't retrain directly the BIG model. But you will create an intermediary layer that will be used by the foundation model to tune the generation. It's named "parameter efficient tuning". You can find more details here: https://cloud.google.com/vertex-ai/docs/generative-ai/models/tune-models
(and at 2 minutes in the video)