azureazure-monitor

Workspace transformation data flow applied to multiple streams?


I created a Log Analytics workspace and want to extend the incoming telemetry data from Application Insights service with custom columns/properties (e.g. system_CF). Initially I used the Azure Portal to setup the transformations via Log Analytics workspace | Tables view and Edit Transformation form from the context menu of specific Application Insights tables.

It results in several dataFlows entries in ARM template/JSON view, which only differ in the streams property.

{
  "properties": {
    "dataFlows": [{
        "destinations": ["dc...d1"],
        "streams": ["Microsoft-Table-AppDependencies"],
        "transformKql": "source \n| extend system_CF=case(...))\n"
      },
      {
        "destinations": ["dc...d1"],
        "streams": ["Microsoft-Table-AppRequests"],
        "transformKql": "source \n| extend system_CF=case(...))\n"
      },
      ...
    ]
  }
}

Since the streams property is an array, I tried to update the Data Collection Rule via REST API via az rest (based on procedure from Tutorial: Edit a data collection rule (DCR)). While the response returns status code 200, the PATCH request had no effect (also with the existing dataFlows property value but small changes in the transformKql value).

  1. How can I modify the dataFlows property from commandline?
  2. How can I set multiple streams?

Solution

  • I found a solution using Azure CLI (version 2.65.0) with the extension monitor-control-service (version 1.2.0) to add additional streams entries for an existing dataFlows entry. See az monitor data-collection rule update.

    az monitor data-collection rule update \
      --ids $test_rule_id 
      --add dataFlows[0].streams 'Microsoft-Table-AppTraces'