[SOLVED] Do we need to have metricbeat installed in the remote prometheus cluster to pull prometheus data to ELK Cluster using metricbeat prometheus module?

Do we need to have metricbeat installed in the remote prometheus cluster to pull prometheus data to ELK Cluster using metricbeat prometheus module?

Reference: Configuring Metricbeat Metricbeat Prometheus Module

From the second link I got the metricbeat Prometheus module configuration is as follows:-

- module: prometheus
  period: 10s
  hosts: ["localhost:9090"]
  metricsets: ["query"]
  queries:
  - name: 'up'
    path: '/api/v1/query'
    params:
      query: "up"

Regarding my use case I want to pull data from remote prometheus host which is outside my network to my ELK cluster using metricbeat prometheus queries.

In this regard I added my remote prometheus host name in the host section of the above config file for metricbeat prometheus module.

Now my question do we need to install metricbeat in the remote prometheus cluster also to pull the data (Ref: Configuring Metricbeat) or just adding the remote prometheus host name in the host section of metricbeat configuration is enough to do the trick?

Solution

You are not required to again configured MetricBeat on remote Prometheus host. You can use same configuration which you have given in question. But you can not give localhost:9090 as you are not running metricbeat on same host where Prometheus is running. Hence, you can update configuration like prometheus_ip:9090.

Also, You need to make sure that connectivity is allowed between host where you have installed metricbeat and host where you are running Prometheus.

You can use Elastic Agent & fleet as well instead of Metricbeat. because it provide centralized configuration management and it easy to configure. You can read more about Elastic agent and fleet here and it provide Prometheus integration.