I have a similar to following logs
4294967295,"adult,low-risk",6564ec78-4995-45b7-b73d-44ee50851dcb,"everything,lost,bla",0
The value inside double quotes should be on the same field so i would get something like
field1 => 4294967295
field2 => "adult,low-risk"
field3 => 36564ec78-4995-45b7-b73d-44ee50851dcb
field4 => "everything,lost,bla"
field5 => 0
But, if the value empty or has a single value the double quotes would not present like:
4294967295,,6564ec78-4995-45b7-b73d-44ee50851dcb,everything,0
Then if i place my dissect/grok like:
%{field1},%{field2},%{field3},%{field4},%{field5}
it would return:
field1 => 4294967295
field2 => "adult
field3 => low-risk"
field4 => 36564ec78-4995-45b7-b73d-44ee50851dcb
field5 => "everything,lost,bla",0
and if i place my dissect/grok like:
%{field1},"%{field2}",%{field3},"%{field4}",%{field5}
it would work but once the value empty or has a single value like i mentioned above it would return _grokparsefailure or __dissectfailure
How do i solve this? Any help would be appreciated, thanks
Using dissect in preference to grok is often a good idea because it has limited functionality, which means it is cheaper. However, dissect does not know about the quoting conventions for commas in csv files. A csv filter does, so if you use
csv { columns => [ "field1", "field2", "field3", "field4", "field5" ] }
you will get the result you want.