pythonamazon-web-servicesboto3aws-organizations

in aws orgazination, use paginate to list all accounts


This should be a generic question about the usage paginate in boto3.

In this case, when I get a lot of accounts (100+) under AWS Orgazinations, use list_account() directly without with NextToken, you can't list all accounts.

response = client.list_accounts()

I knew the correct way is adding NextToken and MaxResults, but that needs more coding.

response = client.list_accounts(
    NextToken='string',
    MaxResults=123
)

So I switch to use another method, called paginate, ref class Organizations.Paginator.ListAccounts. It reports more accounts than list_accounts(), but still can't list all of them.

the Request Syntax has similar MaxItems and PageSize as in list_accounts()

response_iterator = paginator.paginate(
    PaginationConfig={
        'MaxItems': 123,
        'PageSize': 123,
        'StartingToken': 'string'
    }
)

So two questions from me:

Will be appreciated.


Solution

  • So i am wrong, maybe at beginning.

    the paginator does handle the loop automatically. I can't run test on Organization list_accounts(), because I don't have so many accounts to run the test, but I did a test on s3 bucket objects.

    aws cli to get s3 objects

    $ aws s3api list-objects --bucket bucket-demo |jq '.Contents|length'
    
    8696
    

    So I can confirm there are 8000+ objects in this bucket.

    Get s3 object by python sdk with paginator

    >>> import boto3
    >>> client = boto3.client('s3')
    >>> paginator = client.get_paginator('list_objects')
    >>> response_iterator = paginator.paginate(Bucket="bucket-demo")
    >>> for i in response_iterator:
    ...   print(len(i['Contents']))
    ...
    1000
    1000
    1000
    1000
    1000
    1000
    1000
    1000
    696
    

    it proves the paginator does the loop automatically.

    show the problem why we need paginator

    >>> import boto3
    >>> client = boto3.client('s3')
    >>> response = client.list_objects(Bucket="bucket-demo")
    >>> len(response['Contents'])
    1000
    

    So paginator can similify your codes a lot and avoid to develop own loop with normal way.