pythonazurepaginationazure-resource-graph

Azure python sdk resourcegraph no skip_token


no matter what i try skip_token is None. my code:

from azure.identity import DefaultAzureCredential
from azure.mgmt.resourcegraph import ResourceGraphClient
from azure.mgmt.resourcegraph.models import QueryRequest, QueryRequestOptions

credential = DefaultAzureCredential()
client = ResourceGraphClient(credential)

query = """
Resources
| where type == "microsoft.network/networkinterfaces"
| where isnotnull(managedBy)
| mv-expand ipConfig = properties.ipConfigurations
| where isnotnull(ipConfig.properties.privateLinkConnectionProperties.fqdns)
| project fqdns = ipConfig.properties.privateLinkConnectionProperties.fqdns
"""

fqdns = []
skip_token = None

while True:
    request = QueryRequest(
        query=query,
        options=QueryRequestOptions(
            skip_token=skip_token,
            top=1000,
        ),
    )
    response = client.resources(request)
    # do smth here

    skip_token = response.skip_token # always None

sample response:

{'additional_properties': {}, 'total_records': 10070, 'count': 1000, 'result_truncated': 'true', 'skip_token': None, 'data': [ xyz ], 'facets': [] }

WORKING query\code:

query = """
Resources
| where type == "microsoft.network/networkinterfaces"
| where isnotnull(managedBy) and not(managedBy == "")
| where location == "eastus2"
| mv-expand ipConfig = properties.ipConfigurations
| where isnotnull(ipConfig.properties.privateLinkConnectionProperties.fqdns) and array_length(ipConfig.properties.privateLinkConnectionProperties.fqdns) > 0
| project name, fqdns = ipConfig.properties.privateLinkConnectionProperties.fqdns
| order by name asc
"""
fqdns = []
skip_token = None
while True:
    request = QueryRequest(
        query=query,
        options=QueryRequestOptions(
            skip_token=skip_token
        ),
    )
    response = client.resources(request)
    for item in response.data:
        # process results here
    if response.skip_token == None:
        break
    else:
        skip_token = response.skip_token

Solution

  • Apparently skip_token is only populated when you sort results:

    To enable proper pagination and receive a valid skip_token, modify your query to include a stable order by clause.

    also keep in mind that result_truncated is true only if you do not receive a skip_token (this is even specified in the docs!). which makes no sense, because you cant paginate without one. skipping first X results only makes sense if you sort, because else results are random. extremely weird implementation.

    None of the docs I was able to find cover the quote above, they do cover this:

    The response won't include the $skipToken if:

    The query contains a limit or sample/take operator
    All output columns are either dynamic or null type.

    https://learn.microsoft.com/en-us/azure/governance/resource-graph/concepts/work-with-data#paging-results