azureazure-data-explorerkqladx

Azure Data Explorer - Continuous data export using MI

We are testing usage of user assigned managed-identity in ADX for external table - [Microsoft documentation][1]

However, continuous exports are failing due to error, not sure what configuration we are missing.

Query execution has resulted in error (0x80004003): Partial query failure: Invalid pointer (message: 'Argument '[CreateCredentialsProviderForManagedIdentity] CredentialsStore shouldn't be null when acquiring Managed Identity Credentials Provider.' is null: at CreateCredentialsProviderForManagedIdentity in C:\source\Src\Common\Kusto.Common.Svc\AccessControl\ResourceConnectionStringTokenCredentialsBuilder.cs: line 103 Parameter name: [CreateCredentialsProviderForManagedIdentity] CredentialsStore shouldn't be null when acquiring Managed Identity Credentials Provider. ==> ExecutePluginOperator failure: ', details: 'Source: Kusto.Cloud.Platform System.ArgumentNullException: Argument '[CreateCredentialsProviderForManagedIdentity] CredentialsStore shouldn't be null when acquiring Managed Identity Credentials Provider.' is null: at CreateCredentialsProviderForManagedIdentity in C:\source\Src\Common\Kusto.Common.Svc\AccessControl\ResourceConnectionStringTokenCredentialsBuilder.cs: line 103 Parameter name: [CreateCredentialsProviderForManagedIdentity] CredentialsStore shouldn't be null when acquiring Managed Identity Credentials Provider.

Followed the steps and have created external table and continuous export using code below -

create-or-alter external table test_ext_raw_logs (ingestiondt: datetime, full_record: string)
    kind=storage
    partition by (Day: datetime = startofday(ingestiondt))
    pathformat = (datetime_pattern("'year='yyyy'/month='MM'/day='dd", Day))
    dataformat=parquet 
    (
    h@'abfss://Container@StorageAccountName.dfs.core.windows.net/log;managed_identity=****'
    )
    with
    (
    docstring = "External Table",
    folder = "Testing",
    namePrefix = "export",
    validateNotEmpty = true
    )

.create-or-alter continuous-export test_raw_logs over (raw_logs)
to table test_ext_raw_logs with (intervalBetweenRuns = 10min) <| (raw_logs | project   ingestiondt = ingestion_time(), full_record = tostring(full_record)|limit 10);
Solution

  • Assuming you set up the managed identity policy as described in the docs, there's nothing wrong with your configuration.

    However, we did identified a bug in our code that produces this error. We are deploying a fix to all affected clusters.