azurekqlazure-data-explorerkusto-explorerazure-analytics

Azure KQL list filtering


I'm trying to filter a list by another list.

The objective is to get a list of all apps on a server, that doesn't have any logging against it. So I've created 2 variables, appsInServer which lists all the apps in the server, and appsBeingLogged that lists all the apps being logged.

Then I'm just trying to filter out the apps from appsInServer that appear in appsBeingLogged.

Individually the first 2 queries to generate the list all works as expected, but the last query to filter the list errors

let appsInServer = arg("").resources
| where type == 'microsoft.web/sites' and properties.serverFarmId contains ('ASP-CPRD-CardiffApp-Hybrid01')
| extend serverName = case(properties.serverFarmId contains ('ASP-CPRD-CardiffApp-Hybrid01'), 'Hybrid01', 'Unkown')
| project  Name = tolower(tostring(name)); 
let appsBeingLogged = AppPerformanceCounters 
| extend Name=replace_string(tostring(split(_ResourceId, "/")[-1]), "appi-cprd-", "")
| where ((Category == "Process" and Counter == "% Processor Time Normalized") or Name == "processCpuPercentage")
| summarize by Name
| sort by Name
| project Name = strcat("app-cprd-", tolower(Name));
appsInServer
| where Name !in~ (appsBeingLogged)
| summarize by Name
| sort by Name
| project Name;

The error I get isn't very useful - enter image description here


I know filtering by a list should work because this query works as expected -

let appsInServer = arg("").resources
| where type == 'microsoft.web/sites' and properties.serverFarmId contains ('ASP-CPRD-CardiffApp-Hybrid01')
| extend serverName = case(properties.serverFarmId contains ('ASP-CPRD-CardiffApp-Hybrid01'), 'Hybrid01', 'Unkown')
| project  Name = tolower(tostring(name)); 
AppPerformanceCounters 
| extend Name=replace_string(tostring(split(_ResourceId, "/")[-1]), "appi-cprd-", "")
| where ((Category == "Process" and Counter == "% Processor Time Normalized") or Name == "processCpuPercentage")
| where strcat("app-cprd-", tolower(Name)) in~ (appsInServer)
| summarize AvgCPUPercentage = sum(todouble(Value)) / count() by _ResourceId, Name
| sort by  AvgCPUPercentage
| project Name, round(AvgCPUPercentage, 2)


Solution

  • I think there is a bug in the !in~, converted the above query to use a join instead and it all works -

    Here is the final query -

    let appsInServer = arg("").resources
        | where type == 'microsoft.web/sites' and properties.serverFarmId contains ('ASP-CPRD-CardiffApp-Hybrid01')
        | extend serverName = case(properties.serverFarmId contains ('ASP-CPRD-CardiffApp-Hybrid01'), 'Hybrid01', 'Unkown')
        | project Name = tolower(tostring(name)); 
    let appsBeingLogged = AppPerformanceCounters 
        | extend Name=replace_string(tostring(split(_ResourceId, "/")[-1]), "appi-cprd-", "")
        | summarize by Name
        | sort by Name
        | project Name = strcat("app-cprd-", tolower(Name));
    appsInServer
        | join kind=anti (appsBeingLogged) on $left.Name == $right.Name
        | extend Name=replace_string(Name, "app-cprd-", "")
        | summarize by Name
        | sort by Name asc
        | project Name