powershellazure-powershellazure-data-explorerazure-rest-apidata-ingestion

Kusto Data Ingestion using POST Rest Api call in Powershell


As part of a POC, I need to ingest data into Kusto table using RestAPI using PowerShell.

I came up with below lines of script after referencing Kusto Streaming Ingestion Doc

However, Script just runs infinitely doing nothing and is never completing. I have to end up closing the PowerShell window and reopen it. How can I correct the below POST REST API call to make it work? Or if I change anything in $authHeader, the script is failing with BadRequest(400) error.

$Collection = Get-AzResource | select ResourceName,ResourceGroupName,ResourceType,Location,ResourceId,Tags | ConvertTo-Json

$accessToken = (Get-AzAccessToken -ResourceUrl "https://MyKustoClustername.westus2.kusto.windows.net").Token

$authHeader = @{
        'Authorization'='Bearer ' + $accessToken
        'Accept-Encoding'='deflate'
        'Content-Length'='8000'
        'Host'='MyKustoClustername.westus2.kusto.windows.net'
        }

$Uri = -join("https://MyKustoClustername.westus2.kusto.windows.net/v1/rest/ingest/MyDatabasename/VinnyRunBookTest?streamFormat=Json&mappingName=VinnyRunBookTestJsonMapping")
Invoke-RestMethod -Uri $Uri -Method POST -Headers $authHeader -Body $TestCollection

Solution

  • Figured it out. Kusto does not accepts perfect JSON format. It only accepts the payload in Unchained Serialized json format. I am not too sure what is the actual name of this format :)

    Below code works.

    $accessToken = (Get-AzAccessToken -ResourceUrl https://MyKustoClusterName.westus2.kusto.windows.net).Token
    
    $authHeader = @{
            'Content-Type'='application/json'
            'Authorization'='Bearer ' + $accessToken
            'Host'='MyKustoClusterName.westus2.kusto.windows.net'
            }
    
    $Collection = @'
    {"ResourceName":"Test1","ResourceGroupName":"Test1","ResourceType":"Test1","Location":"Test1","ResourceId":"Test1","Tags":"Desktop"}
    {"ResourceName":"Test2","ResourceGroupName":"Test2","ResourceType":"Test2","Location":"Test2","ResourceId":"Test2","Tags":"Desktop"}
    '@
    $Collection
    
    $Uri = https://MyKustoClusterName.westus2.kusto.windows.net/v1/rest/ingest/MyDBName/VinnyRunBookTest?streamFormat=json
    $Response = Invoke-RestMethod -Uri $Uri -Method POST -Headers $authHeader -Body $Collection -Verbose