azurekqlazure-log-analytics

KQL: how to exclude first and last incomplete bin?


When using the bin function of the Kusto Query Language (KQL) on a time range, the first and last bin are most of the time incomplete, giving "strange" results. E.g. consider the following LogAnalytics query that attempt to show the number of requests per week:

AppRequests
| where AppRoleName contains '-apim' // Exclude FunctionApps, only include Azure API Management
| where TimeGenerated  > ago(90d)
| summarize count() by bin(TimeGenerated, 7d)
//| where TimeGenerated > ago(90d) and TimeGenerated  < ago(2d) // Get rid of the first and last incomplete datapoints
| render timechart

The first and last bin are incomplete resulting in an anomaly. How to exclude those incomplete bins? My solution is commented out in the above script, but is a very dirty solution and there should be something better ;-)

And as a bonus question (that might also solve the above problem): how can I turn this query in using a moving average for the count() instead of using bins?


Solution

  • If bins are incomplete only way to remove them is the way you are using or you can simply use below query:

    let rith_ed = now();  
    let rith_strt = datetime(2024-09-01T00:00:00Z); 
    AppRequests
    | where TimeGenerated >= rith_strt and TimeGenerated < rith_ed
    | summarize TotalRequests = sum(RequestCount) by bin_at(TimeGenerated, 7d, rith_ed)
    | where TimeGenerated >= rith_strt + 7d and TimeGenerated < rith_ed - 7d
    

    Output:

    enter image description here

    Fiddle

    Or Alternatively, if you do not want to give start date, you can use below query:

    let cho = 7d;     
    let test_rith = 90d; 
    let rith_strt = bin(ago(test_rith), cho); 
    let rith_ed = bin(now(), cho);      
    AppRequests
    | where TimeGenerated >= rith_strt and TimeGenerated < rith_ed
    | summarize TotalRequests = sum(RequestCount) by bin_at(TimeGenerated, cho, rith_strt)
    | where TimeGenerated >= rith_strt + cho and TimeGenerated < rith_ed - cho
    

    Output:

    enter image description here

    Fiddle.