I'm trying to install OpenSearch stack to ship data and visualize then on their Dashboard. Everything is locally for now.
I have installed Core, Dashboard and the service run.
Next I want to send metrics to follow CPU, RAM, Disk, Network... with MetricBeat OSS. I have setup Core to mimic Elasticsearch7 beacause it's seems to not reconize OpenSearch as it is.
curl -X GET "http://localhost:9200"
{
"name" : "node-1",
"cluster_name" : "opensearch-demo",
"cluster_uuid" : "MdYUP-7gTrqKf58qW8XY7g",
"version" : {
"number" : "7.10.2",
"build_type" : "rpm",
"build_hash" : "2e4741fb45d1b150aaeeadf66d41445b23ff5982",
"build_date" : "2025-02-27T01:16:53.890803852Z",
"build_snapshot" : false,
"lucene_version" : "9.12.1",
"minimum_wire_compatibility_version" : "7.10.0",
"minimum_index_compatibility_version" : "7.0.0"
},
"tagline" : "The OpenSearch Project: https://opensearch.org/"
}
When I run metricbeat I keep havig this error :
{
"log.level":"error",
"@timestamp":"2025-03-14T14:26:42.758+0100",
"log.logger":"publisher_pipeline_output",
"log.origin":{
"function":"github.com/elastic/beats/v7/libbeat/publisher/pipeline.(*netClientWorker).run",
"file.name":"pipeline/client_worker.go",
"file.line":149
},
"message":"Failed to connect to backoff(elasticsearch(http://10.29.165.222:9200)): Connection marked as failed because the onConnect callback failed: could not connect to a compatible version of Elasticsearch: 400 Bad Request: {\"error\":{\"root_cause\":[{\"type\":\"invalid_index_name_exception\",\"reason\":\"Invalid index name [_license], must not start with '_'.\",\"index\":\"_license\",\"index_uuid\":\"_na_\"}],\"type\":\"invalid_index_name_exception\",\"reason\":\"Invalid index name [_license], must not start with '_'.\",\"index\":\"_license\",\"index_uuid\":\"_na_\"},\"status\":400}",
"service.name":"metricbeat",
"ecs.version":"1.6.0"
}
Here my metricBeat conf :
sudo nano /etc/metricbeat/metricbeat.yml
metricbeat.config.modules:
# Glob pattern for configuration loading
path: ${path.config}/modules.d/*.yml
reload.enabled: false
module: system
period: 10s
metricsets:
- cpu
- memory
- process
- filesystem
- network
enabled: true
xpack.enabled: false
setup.dashboards.enabled: false
setup.ilm.enabled: false
setup.template.enabled: false
setup.template.settings:
index.number_of_shards: 1
output.elasticsearch:
# Array of hosts to connect to.
hosts: ["10.29.165.222:9200"]
protocol: "http"
preset: balanced
username: "admin"
password: "admin"
processors:
- add_host_metadata: ~
- add_cloud_metadata: ~
- add_docker_metadata: ~
- add_kubernetes_metadata: ~
monitoring.enabled: false
Check the compatibility matrix for metricbeat OSS and Opensearch. A similar discussion is here.
Make sure that
Both metricbeat and elasticsearch at the same version.
Both beat and ES is on OSS version.