splunksplunk-query

Splunk: Show all the ids that are present in different events


My data is as follows:

event_A: { [-]
     processingTimeMillis: 2178
      event_A_recipients: [ [-]
       id1,
       id2,
       id3,
       id4,
     ]
   }
   eventType: event_A

event_B: { [-]
     processingTimeMillis: 2178
      event_B_recipients: [ [-]
       id1,
       id3,
     ]
   }
   eventType: event_B

event_B: { [-]
     processingTimeMillis: 2178
      event_B_recipients: [ [-]
       id2
     ]
   }
   eventType: event_B

Above is an example of what my source data looks like. There are two types of events, event_A and then event_B (json data). I am trying to list out all the ids that have an event_A associated with them, but not event_B. I tried using spath and sub-searches but nothing worked so far.

The above data should out id4.


Solution

  • Ok, this seems quite tricky (of course the last time I said that someone else came along and answered it way more easily). My way of doing this is:

    Here is a run anywhere example:

    | makeresults
    | eval JSON="{\"event_A\":{\"processingTimeMillis\":\"2178\",\"event_A_recipients\":[\"id1\",\"id2\",\"id4\"]},\"eventType\":\"event_A\"}|{\"event_B\":{\"processingTimeMillis\":\"2178\",\"event_B_recipients\":[\"id1\",\"id3\"]},\"eventType\":\"event_B\"}|{\"event_B\":{\"processingTimeMillis\":\"2178\",\"event_B_recipients\":[\"id2\"]},\"eventType\":\"event_B\"}"
    | eval JSON=split(JSON,"|")
    | mvexpand JSON
    
    | spath input=JSON
    | rename event_A.event_A_recipients{} as A
    | rename event_B.event_B_recipients{} as B
    
    | eval id=""
    | transaction id
    
    | eval A_copy=A
    | mvexpand A_copy
    
    | eval C=mvdedup(mvappend(B,A_copy))
    | eval matches=if(mvcount(C)>mvcount(B),null(),A_copy)
    
    | table _time A B matches
    | stats values(*) as * by _time