elasticsearchsplitlogstashelk

Logstash Split Issue


Below statement is simple incoming message and AccessList should be parsed by Logstash(v8.12.1):

<EventData>
    <Data Name="AccessList">%%4416 %%4417 %%4418 %%4419 %%4420 %%4423 %%4424 %%1538 </Data>
</EventData>

I scrape this text with following configuration but split in mutate filter plugin is not working correctly:

filter {
  xml {
    xpath => ["//Data[@Name='AccessList']/text()","access_text"]
  }
  mutate {
    gsub => ["access_text" ,"\s+", ""]
    gsub => ["access_text" ,"%%", ","]
    gsub => ["access_text","^,",""] #Delete first comma
  }
  mutate {
    convert => {
      "access_text" => "string"
    }
  }
  mutate {
    split => {"access_text" =>  ","}
  }
}

This is a output splitless text on Kibana:

4416,4417,4418,4419,4420,4423,4424,1538

My expected output is a array like this on Kibana:

[4416,4417,4418,4419,4420,4423,4424,1538]

Solution

  • Solution: I added force_array attribute in xml filter plugin like below:

    filter {
      xml {
        source => "message"
        store_xml => false
        force_array => false
        # other statements ...
      }
      mutate {
        gsub => ["access_text" ,"\s+", ""]
        gsub => ["access_text" ,"%%", ","]
        gsub => ["access_text","^,",""]
        split => {"access_text" => ","}
      }
    }