azurepowershellworkspaceazure-log-analyticsazure-log-analytics-workspace

is it possible to get the ingestion volume of a log analytics workspace using powershell or any programming language?


I am getting a summary of our log analytics workspaces in the company, this includes the tables that are being used in each workspace, as well as other data such as the ingestion volume.

The closest thing to "get" this is to use this command in PowerShell

Get-AzOperationalInsightsWorkspaceUsage -ResourceGroupName "RG_name" -Name "WS_name"

And it shows me this info:

Id            : DataAnalyzed
CurrentValue  : 0
Unit          : Bytes
Limit         : -1
NextResetTime : 7/24/2023 8:00:00 AM
QuotaPeriod   : 1.00:00:00

Which is not enough, I am looking for this:

image with the data I am looking to get through powershell or any other language

I searched for anything similar but didn't find anything else. Hope there is a solution that I am missing.


Solution

  • Assuming you will be using your user account to query the Log Analytics Rest API and you have access to the Az Module plus Reader roles over the target Log Analytics Workspace, this is how you can get the ingestion volume by querying the Usage table.

    # connect impersonating user
    Connect-AzAccount
    # the GUID of the LAW goes here
    $workspaceId = 'xxxxx-xxxxx-xxxxx...'
    $resource = 'https://api.loganalytics.io'
    # get a token with permissions to query the LAW API
    $token = Get-AzAccessToken -ResourceUrl $resource
    
    $invokeRestMethodSplat = @{
        Headers     = @{
            Authorization = '{0} {1}' -f $token.Type, $token.Token
        }
        Uri         = '{0}/v1/workspaces/{1}/query' -f $resource, $workspaceId
        ContentType = 'application/json'
        Method      = 'Post'
        Body        = @{
            query = '
            Usage
            | where TimeGenerated > ago(24h)
            | summarize ["TotalIngestionVolume(GB)"] = sum(Quantity) / 1024.0 by DataType
            | order by ["TotalIngestionVolume(GB)"]
            '
        } | ConvertTo-Json
    }
    $response = Invoke-RestMethod @invokeRestMethodSplat
    

    Up to this point in $response you would have the ingestion volume per table in your Log Analytics Workspace, problem is the response from this API is really bad so you have to enumerate the columns and rows to get objects out of it like so:

    $columns = @($response.tables.columns.name)
    $result = [ordered]@{}
    
    foreach ($row in $response.tables.rows) {
        for ($i = 0; $i -lt $columns.Count; $i++) {
            $result[$columns[$i]] = $row[$i]
        }
     
        [pscustomobject] $result
        $result.Clear()
    }
    

    If using a Service Principal instead of impersonating our user account, the logic is pretty much the same, the only change is the way we acquire the token:

    $clientId = 'xxxxx-xxxx-xxxx....'
    $tenantId = 'xxxxx-xxxx-xxxx....'
    $secret = 'fo0B4rB4z'
    
    $cred = [pscredential]::new(
        $clientId,
        (ConvertTo-SecureString $secret -AsPlainText -Force))
    
    Connect-AzAccount -ServicePrincipal -Tenant $tenantId -Credential $cred
    
    $resource = 'https://api.loganalytics.io'
    # get a token with permissions to query the LAW API
    $token = Get-AzAccessToken -ResourceUrl $resource
    
    # rest stays the same