mongodbelasticsearchlogstashlogstash-filter

Filter MongoDB document's deep fields using Logstash

I am trying to rename some deep fields from my MongoDB document before uploading them to Elastic using Logstash.

For example: my MongoDB document looks like this:

name: "firstDoc",
infoArray: 
[
   {infoName: "i", elements:{e1: "e1", e2: "e2"}}, 
   {infoName: "j", elements:{e3: "e3", e4: "e4"}}
]

I was able to access the "name" field in logstash like this:

mutate { add_field => {"[otherDoc][name]" => "%{[document][name]}"} }

But I am not able to access the fields in infoArray, I have tried the rename:

mutate { rename => {"[document][infoArray][elements][e1]" => "[otherDoc][e1]"} }

I have also tried by adding it as a new field:

mutate { add_field => {"[otherDoc][e1]" => "%{[document][infoArray][elements][e1]}"} }

But still not working.

Is there any way to access deep fields?

Solution

  • I have found the solution, as follows:

    mutate { rename => {"[document][infoArray][0][elements][e1]" => "[otherDoc][e1]"} }

    or

    mutate { add_field => {"[otherDoc][e1]" => "%{[document][infoArray][0][elements][e1]}"} }

    The index of infoArray should be specified in Logstash.