[SOLVED] Logstash kv filter

Logstash kv filter

I have a file with the following format:

10302\t<document>.....</document>   
12303\t<document>.....</document>   
10054\t<document>.....</document>   
10034\t<document>.....</document>

as you can see there are two values separated by a tab char. I need to

index the first token (e.g. 10302, 12303...) as ID
extract (and then index) some information from the second token (the XML document). In other words, the second token would be used with the xml filter for extracting some information

Is it possibile to do that separating the two values using the kv filter? Ideally I should end, for each line, with a document like this:

id:10302       
msg:<document>....</document>

I could use a grok filter but I'd like to avoid any regex as the field detection is very easy and can be accomplished with a simple key-value logic. However, using a plain kv detection I'm ending with the following:

"10302": <document>.....</document>   
"12303": <document>.....</document>   
"10054": <document>.....</document>   
"10034": <document>.....</document>

and this is not want I need.

Solution

It is not possible to use kv for the job you want to do, as far as I know, since there are no possible key for the id (10302, 10303, 10304...). There are no possible key since there is nothing before the id.

This grok configuration would work, assuming each id + document is on the same line :

grok {
  match => { "message" => "^%{INT:ID}\t%{GREEDYDATA:msg}"}
}