aggregatesumologic

Aggregating Wildcards in Sumologic


I'm trying to aggregate the API logs based on the different endpoints I have. There are a total of 4 endpoints:

1: /v1/vehicle_locations

2: /v1/vehicle_locations/id

3: /v1/driver_locations

4: /v1/driver_locations/id

The way I'm currently doing this is:

_sourceCategory=production | keyvalue auto | where (path matches "/v1/driver_locations" OR path matches "/v1/driver_locations/*" or path matches "/v1/vehicle_locations" or path matches "/v1/vehicle_locations/*") | count by path

The problem with this is that while I get the correct aggregate for /v1/vehicle_locations and /v1/driver_locations, I get individual results for /v1/driver_locations/id and /v1/vehicle_locations/id since the id is a wildcard. Is there a way I can aggregate these wildcards as well?


Solution

  • There are several ways to achieve what you ask. I think the most straightforward one and suggested is to use | parse operator so that you can treat the top-most element of your path as a field, e.g.

    _sourceCategory=production
    | keyvalue auto 
    | parse field=path "*/*" as topmost, rest
    | where (topmost = "vehicle_locations" or topmost = "driver_locations")
    | count by topmost
    

    Note that by default | parse operator works on the raw message (e.g. the original log line), but you can make it parse a field - using the field= syntax and this is what it's used above.

    You might want to tweak the parse expression or use a regex depending on the actual paths you encounter.

    (Disclaimer: I am currently employed by Sumo Logic)