amazon-web-servicesaws-lambdaamazon-cloudwatchlogs

how to get all log groups names


I have a lambda that exports all of our loggroups to s3, and am currently using cloudwatchlogs.describeLogGroups to list all of our logGroups.

const logGroupsResponse = await cloudwatchlogs.describeLogGroups({ limit: 50 })

The issue is that we have 69 logGroups is there any way to list (ids, and names) of absolutely all logGroups in an aws account. I see it's possible to have 1000 log groups. This is a screenshot of our console: enter image description here

How come cloudwatchlogs.describeLogGroups just allows a limit of 50 which is very small?


Solution

  • Assuming that you are using AWS JS SDK v2, describeLogGroups API provides a nextToken in its response and also accepts a nexToken. This token is used for retrieving multiple log groups (more than 50) by sending multiple requests. We can use the following pattern to accomplish this:

    const cloudwatchlogs = new AWS.CloudWatchLogs({region: 'us-east-1'});
    let nextToken = null;
    do {
        const logGroupsResponse = await cloudwatchlogs.describeLogGroups({
          limit: 50,
          nextToken: nextToken
        }).promise();
        
        // Do something with the retrieved log groups
        console.log(logGroupsResponse.logGroups.map(group => group.arn));
    
        // Get the next token. If there are no more log groups, the token will be undefined
        nextToken = logGroupsResponse.nextToken;
    } while (nextToken);
    

    We are querying the AWS API in loop until there are no more log groups left.