apacheelasticsearchelkmetricbeat

Mapping ElasticSearch apache module field


I am new to ES and I am facing a little problem I am struggling with.

I integrated metricbeat apache module with ES and the it works fine.

The problem is that metricbeat apache module reports the KB of web traffic of apache (field apache.status.total_kbytes), instead I would like to create my own field, the name of which would be "apache.status.total_mbytes).

I am trying to create a new mapping via Dev Console using the followind api commands:

PUT /metricbeat-7.2.0/_mapping
{
  "settings":{

  },
      "mappings" : {
      "apache.status.total_mbytes" : {
        "full_name" : "apache.status.total_mbytes",
        "mapping" : {
          "total_mbytes" : {
            "type" : "long"
          }
        }
      }
    }
}

Still ES returns the following error:

{
  "error" : {
    "root_cause" : [
      {
        "type" : "mapper_parsing_exception",
        "reason" : "Root mapping definition has unsupported parameters:  [settings : {}] [mappings : {apache.status.total_mbytes={mapping={total_mbytes={type=long}}, full_name=apache.status.total_mbytes}}]"
      }
    ],
    "type" : "mapper_parsing_exception",
    "reason" : "Root mapping definition has unsupported parameters:  [settings : {}] [mappings : {apache.status.total_mbytes={mapping={total_mbytes={type=long}}, full_name=apache.status.total_mbytes}}]"
  },
  "status" : 400
}

FYI

The following may shed some light

GET /metricbeat-*/_mapping/field/apache.status.total_kbytes

Returns

{
  "metricbeat-7.9.2-2020.10.06-000001" : {
    "mappings" : {
      "apache.status.total_kbytes" : {
        "full_name" : "apache.status.total_kbytes",
        "mapping" : {
          "total_kbytes" : {
            "type" : "long"
          }
        }
      }
    }
  },
  "metricbeat-7.2.0-2020.10.05-000001" : {
    "mappings" : {
      "apache.status.total_kbytes" : {
        "full_name" : "apache.status.total_kbytes",
        "mapping" : {
          "total_kbytes" : {
            "type" : "long"
          }
        }
      }
    }
  }
}

What am I missing? Is the _mapping command wrong?

Thanks in advance,


Solution

  • A working example:

    Create new index

    PUT /metricbeat-7.2.0
    {
      "settings": {},
      "mappings": {
        "properties": {
          "apache.status.total_kbytes": {
              "type": "long"
            }
        }
      }
    }
    

    Then GET metricbeat-7.2.0/_mapping/field/apache.status.total_kbytes will result in (same as your example):

    {
      "metricbeat-7.2.0" : {
        "mappings" : {
          "apache.status.total_kbytes" : {
            "full_name" : "apache.status.total_kbytes",
            "mapping" : {
              "total_kbytes" : {
                "type" : "long"
              }
            }
          }
        }
      }
    }
    

    Now if you want to add a new field to an existing mapping use the API this way:

    Update an existing index

    PUT /metricbeat-7.2.0/_mapping
    {
      "properties": {
        "total_mbytes": {
          "type": "long"
        }
      }
    }
    

    Then GET metricbeat-7.2.0/_mapping will show you the updated mapping:

    {
     "metricbeat-7.2.0" : {
        "mappings" : {
          "properties" : {
            "apache" : {
              "properties" : {
                "status" : {
                  "properties" : {
                    "total_kbytes" : {
                      "type" : "long"
                    }
                  }
                }
              }
            },
            "total_mbytes" : {
              "type" : "long"
            }
          }
        }
      }
    }
    

    Also, take a look at Put Mapping Api