We are trying to configure a private GKE cluster, using PSC and autopilot, so that the control plane can be accessed from PUPI addresses (because we have to). The google documentation suggest it should be possible but we just can't see how to configure it.
If the cluster endpoint is public then we can set pupi addresses on the master authorised network configuration (because it allows public ip addresses) and kubectl can connect to the private endpoint (with a little hacking of the .kube/config) file.
However, if the cluster endpoint is private then it doesn't allow PUPI addresses to be used on the master authorised network configuration (which seems reasonable).
This document suggests it is possible to use PUPIs by setting the -enable-google-cloud flag (assuming that is the --enable-google-cloud-access flag in gcloud) but when trying to set this with a private endpoint, it says it is invalid.
The following works but only allows private IP addresses on the master authorized networks
gcloud container clusters create-auto <CLUSTER_NAME> \
--region=<REGION> \
--network=<NETWORK_NAME> \
--subnetwork=<SUBNET_NAME> \
--service-account=<SERVICE_ACCOUNT_EMAIL> \
--cluster-version=1.29.1 \
--enable-private-nodes \
--enable-private-endpoint \
--enable-master-authorized-networks \
--master-authorized-networks 10.10.10.0/24
The following creates the cluster but fails to add the master authorized networks as they are publicly addressable
finished with error: Invalid master authorized networks: network "11.10.10.0/24" is not a reserved network, which is required for private endpoints.
gcloud container clusters create-auto <CLUSTER_NAME> \
--region=<REGION> \
--network=<NETWORK_NAME> \
--subnetwork=<SUBNET_NAME> \
--service-account=<SERVICE_ACCOUNT_EMAIL> \
--cluster-version=1.29.1 \
--enable-private-nodes \
--enable-private-endpoint \
--enable-master-authorized-networks \
--master-authorized-networks 11.10.10.0/24
The following fails because it is not valid
invalid value for "cluster.master_authorized_networks_config": "cluster.master_authorized_networks_config.gcp_public_cidrs_access_enabled" cannot be enabled if private endpoint is enabled
gcloud container clusters create-auto <CLUSTER_NAME> \
--region=<REGION> \
--network=<NETWORK_NAME> \
--subnetwork=<SUBNET_NAME> \
--service-account=<SERVICE_ACCOUNT_EMAIL> \
--cluster-version=1.29.1 \
--enable-private-nodes \
--enable-private-endpoint \
--enable-master-authorized-networks \
--enable-google-cloud-access
butland -
Sorry for the super late response here. You are indeed correct that we only support using RFC 1918 in master authorized networks when using the private endpoint only. The reason for this is due to the fact that even when you only enable the private endpoint, the public endpoint still exists to support other Google services. If we allowed PUPI addresses in this case, the owner of the PUPI range could access the hidden public endpoint.
We have a new feature coming in early Q4 which adds a new (and better) option for exposing the control plane endpoint and would allow the use of PUPI addresses for access.