I am trying to trend the below data over the archived timestamp. I am not sure why my dates and times aren't parsed. According to my grok debugger it works just fine.
https://i.sstatic.net/GDFAy.png
Sample Input:
[15/06/02@11:05:31.233-0700] P-007158 T-4131301152 2 WS 4GLTRACE Run htmAssociate "vsess vsess 1349" [htmOffsets - dpa/setup/vsysadv.w @ 9563]
Config file
input {
file {
path => "/Users/philipp/Documents/Performance/ProductionMetrics/4gltrace_logs/4gltrace_log_bstash.txt"
start_position => beginning
}
}
filter {
grok {
match => ["message", "\[%{DATE}@%{TIME}-%{INT:TIMEZONE}] %{NOTSPACE:PID} % {NOTSPACE:T} %{INT:NUM} %{WORD:WS} %{WORD:4GLTRACE} %{GREEDYDATA} \[%{DATA:PROGRAM}]"]
}
}
output {
elasticsearch { host => localhost protocol => "http" port => "9200" }
stdout { codec => rubydebug }
}
I am sure its a silly oversight but not sure where it is. Any help is appreciated.
You need to use the date{} filter in logstash to take a field from your event and replace @timestamp with that value.
If you had a field called my_timestamp
with the following format, this would do it:
date {
match => [ 'my_timestamp', "dd/MMM/yyyy:HH:mm:ss Z" ]
remove_field => [ 'my_timestamp' ]
}