I use Autopilot on GKE. I've created some log based metrics that I'd like to use to scale up pods.
To begin with - I'm not sure if it's great idea - the metric is just number of records in DB to process... I have a feeling using logs to scale app might bring in some weird infinite loop or something....
Anyhow - I've tried entering logging.googleapis.com|user|celery-person-count as an external metric and got HPA cannot read metric value. Installed Stackdriver adapter but not too sure how to use it either.
GKE Autopilot clusters have Workload Identity enabled for consuming other GCP services, including Cloud Monitoring.
You'll want to follow the steps here in order to deploy the Custom Metrics Adapter on Autopilot clusters.
kubectl create clusterrolebinding cluster-admin-binding \
--clusterrole cluster-admin --user "$(gcloud config get-value account)"
kubectl create namespace custom-metrics
kubectl create serviceaccount --namespace custom-metrics \
custom-metrics-stackdriver-adapter
gcloud iam service-accounts create GSA_NAME
gcloud projects add-iam-policy-binding PROJECT_ID \
--member "serviceAccount:GSA_NAME@PROJECT_ID.iam.gserviceaccount.com" \
--role "roles/monitoring.viewer"
gcloud iam service-accounts add-iam-policy-binding \
--role roles/iam.workloadIdentityUser \
--member "serviceAccount:PROJECT_ID.svc.id.goog[custom-metrics/custom-metrics-stackdriver-adapter]" \
GSA_NAME@PROJECT_ID.iam.gserviceaccount.com
kubectl annotate serviceaccount \
--namespace custom-metrics custom-metrics-stackdriver-adapter \
iam.gke.io/gcp-service-account=GSA_NAME@PROJECT_ID.iam.gserviceaccount.com
kubectl apply -f manifests/adapter_new_resource_model.yaml
Given that you've already deployed the adapter, you'll want to delete the deployment first, although you might just be able to run the steps starting at gcloud iam ...
You'll need to replace GSA_NAME with a name of your choosing and PROJECT_ID with your Google Cloud project ID.