we are testing out the Ambassador Edge Stack and started with a brand new GKE private cluster in autopilot mode.
We installed from scratch following the quick start tour to get a feeling of it and ended up with the following error
Error from server: error when creating "mapping-test.yaml": conversion webhook for getambassador.io/v3alpha1, Kind=Mapping failed: Post "https://emissary-apiext.emissary-system.svc:443/webhooks/crd-convert?timeout=30s": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
We did a few rounds of DNS testing and deployed a few different test pods in different namespaces to validate that kube-dns is working properly, everything looks good at that end. Also the resolv.conf looks good.
Ambassador is using the hostname emissary-apiext.emissary-system.svc:443 (without the cluster.local) which should resolve fine. Doing a lookup with the FQN (with cluster.local) works fine btw.
Any clues?
Thanks a lot and take care.
That sounds like an issue related to the webhooks limitation in GKE Autopilot
Which version of GKE are you on ?
Also there is a limitation with which resources and namespaces we allow webhooks to intercept
Additionally, webhooks which specify one or more of following resources (and any of their sub-resources) in the rules, will be rejected:
- group: "" resource: nodes
- group: "" resource: persistentvolumes
- group: certificates.k8s.io resource: certificatesigningrequests
- group: authentication.k8s.io resource: tokenreviews
You probably have to check the manifests of Ambassador Edge Stack to figure this out.