The FLink version is 1.12, I follow the step(https://ci.apache.org/projects/flink/flink-docs-release-1.12/deployment/metric_reporters.html#prometheuspushgateway-orgapacheflinkmetricsprometheusprometheuspushgatewayreporter), fill my config, run my job in Flink cluster. but after a few hours, I find cannot see metric data on grafana, so i loigin server and see pushgateway log, find like "Out of memory" error log.
i dont understand, actually i set deleteOnShutdown=true and some of my jobs is closed. why pushgateway will OOM?
This problem has always existed, However, it was not described in the previous v1.13 documents. you can see the pull request to get more info.
If you want to use push model in your Flink cluster, i recommend use influxdb.