I have a RoR application with installed Prometheus-client and Telegraf daemon with Prometheus input plugin working on the instance I want to monitor.
As far as I understand I need some kind of exporter middleware to collect metrics from Prometheus::Client.registry and expose them with /metrics HTTP endpoint.
What I don't really understand is how to pass all metrics from different envs (e.g from rake task and app's runtime code) into the same registry (it's an instance variable of Prometheus::Middleware::Exporter.new(registry)) of the same instance of Prometheus::Middleware::Exporter middleware?
Also, will urls = ["http://localhost:3000/metrics"] config of Prometheus input plugin for Telegraf work on EC2 instance for example?
Thank you for advices.
First of all, it's not recommended to use a catch-all exporter like Telegraf. You can read about some of the arguments in this blog post: https://www.robustperception.io/one-agent-to-rule-them-all/.
Then, if I understand your question correctly, it's not possible to use the same registry from multiple processes (like your Rails app and some rake task). Your Rails app will export its own metrics and you'll need to use a different approach for rake tasks.
As rake tasks are (usually) short-lived processes, they are not well suited to be pulled from. You have two options here, either you use the Pushgateway and the PGW-support in client_ruby to push all relevant metrics at the end of the rake task execution (like how long it took, how many items were processed, if there was any error, etc.). Alternatively, you can use the textfile collector in the node_exporter and write your metrics to disk at the end of your rake task execution. The node_exporter will then read that file and export the metrics when it gets scraped.
I don't actively monitor stackoverflow, you'll get more help with these questions on the prometheus-users mailing list, see https://prometheus.io/community/.