I have log file that look for example like this:
2015-12-05 05:00:51 prefix1_sent 1
2015-12-05 05:00:52 prefix2_sent 2
2015-12-05 05:00:53 prefix1_received 1
2015-12-05 05:00:54 prefix2_received 2
I want to find out how much time particular event took. I am using elapsed plugin in logstash like this:
....
grok {
patterns_dir => "path"
match => {
"message" => "%{TIMESTAMP:logtime}, %{EVENT:event}, %{SPLIT:ID}"
}
}
filter{
if [event] == "(.)*sent" {
if [event] == "prefix1(.)*" {
kv {
add_tag => [ "prefix1_sent" ]
}
} else if [event] == "prefix2(.)*" {
kv {
add_tag => [ "prefix2_sent" ]
}
} else if [event] == "(.)*received" {
if [event] == "prefix1(.)*" {
kv {
add_tag => [ "prefix1_received" ]
}
} else if [event] == "prefix2(.)*" {
kv {
add_tag => [ "prefix2_received" ]
}
}
}
}
filter{
elapsed {
start_tag => "prefix1_sent"
end_tag => "prefix1_received"
unique_id_field => "ID"
new_event_on_match => false
}
}
filter{
elapsed {
start_tag => "prefix2_sent"
end_tag => "prefix2_received"
unique_id_field => "ID"
new_event_on_match => false
}
}
....
I am not sure if my syntax of nested if statement is correct of if I understand the use of "kv". I am new to logstash. However I can't find any tag in kibana with this configuration. Where am I making mistake?
Thanks
Several comments:
Those changes would probably get you on track.
For some unsolicited advice, take a look at your workflow. First, you apply a regexp with grok to make three fields. Then you regexp one of them again (2-3 times) before you do the add_field. Try this:
Not only is it more "grok-ish", but it would be faster when applied to a large data set.