We have a Prometheus Pushgateway running and listening to metrics push from our AWS Lambda function. However, the URL to the Pushgateway is accessible by the public, which might impose some security issues. We were wondering if there is any way we could add a layer of protection to the Pushgateway so that it is not publicly accessible?
I found this Github thread that may answered this question: https://github.com/prometheus/pushgateway/issues/281
It proposed to set up a reverse proxy in front of the pushgateway. However, I am still confused on how that may actually work? We are currently using Kubernetes to deploy the Prometheus.
You can include authentication in your ingress controller by using a TLS secret as an ingress rule. Here's an example that shows how to generate basic auth for your ingress:
https://kubernetes.github.io/ingress-nginx/examples/auth/basic/
Also, don't forget to include the Python handler function in your client to set the auth header as pointed out here:
https://github.com/prometheus/client_python#handlers-for-authentication