azureazure-data-factoryazure-resource-graph

Azure Resource Graph API - Get Resources table via Data Factory

I'm creating a Data Factory pipeline with Web Activity, and that activity is using Azure Resource Graph API to query the resources table. And after I'll use copy data activity to copy the data to blob container. I already created it but the output file returns the same Web Activity results.

For example, and using the Web Activity,it is working well, but only returns 201 rows and if I execute the kusto query in Azure Graph Explorer returns 286 rows.

And if I filter by type = "microsoft.desktopvirtualization/hostpools" it will return 0 rows via API, but in Azure Graph Explorer returns 2 rows I think it will only queries the first 200 lines and not the whole set, because I tried with order by asc or desc and with "| take 500" (for example) and it didn't work either.

I checked the microsoft documentation and it says that the limit is 1000 rows, so I don't understand why it doesn't return more rows:

enter image description here

Results with type filter:

Azure Graph Explorer:

enter image description here

ADF Output:

enter image description here

ADF Pipeline:

URL: https://management.azure.com/providers/Microsoft.ResourceGraph/resources?api-version=2021-03-01

Method: POST

Body:

{
  "query": "resources"
}

Resource: https://management.azure.com

Headers: Content-Type; application/json enter image description here

Does anyone have similar issue/experience?

Thanks a lot!

Solution

  • I would check the access your Azure Data Factory has. It needs to have Reader Role on all of the Azure resources you are expecting to return in the API Call:

    Add Role Assignment

    You can assign a role at a Resource, Resource Group, Subscription, or Management Group Level.

    Cheers!