In my project, the constraints/compute.requireOsLogin policy constraint is enabled. If I try running a dataproc batch job using this command:
gcloud dataproc batches submit --project <project-id> --region us-east1 pyspark --batch batch-123 gs://mybucket/test.py --version 2.1 --subnet test
I get the following error:
ERROR: (gcloud.dataproc.batches.submit.pyspark) Batch job is FAILED. Detail: Constraint constraints/compute.requireOsLogin violated for project 123456789.
The subnet has Private Google Access enabled. The test.py file can be anything, I took the example from here.
How can I fix that error?
You can setup OS login in your orginzation that will enable OS login to the newly created VMs, But OS Login in Dataproc Serverless is not supported. See list of not supported products below:
Cloud Data Fusion versions 6.1.4 and earlier
Cloud Composer Google Kubernetes Engine (GKE) public clusters that run versions earlier than 1.23.5
GKE private clusters that run node pool versions earlier than 1.20.5
Dataproc Serverless
Windows Server and SQL Server VMs
Fedora CoreOS VMs. To manage instance access to VMs created using these images, use the Fedora CoreOS ignition system
Reference:
https://cloud.google.com/compute/docs/oslogin/manage-oslogin-in-an-org#limitations