amazon-web-servicesaws-cliaws-secrets-manager

`aws secretsmanager list-secrets` command to return secrets and filter them by tag


How do I call the aws secretsmanager list-secrets command and filter secrets by their tags? I don't see examples of this here: https://docs.aws.amazon.com/cli/latest/reference/secretsmanager/list-secrets.html

Also, Amazon's docs seem to be wrong. It says --max-items on that page but really should be --max-results. Also there is no mention of how to filter on that wiki page as well.


Solution

  • [Original: December 2019]

    You can use jq, for example:

    aws secretsmanager list-secrets \
        | jq '.SecretList[] | select((.Tags[]|select(.Key=="Name")|.Value) | test("^Production$|^Staging$"))'
    

    You can also use the awscli's in-built query option, for example:

    aws secretsmanager list-secrets \
        --query "SecretList[?Tags[?Key=='Name' && Value=='Production']]"
    

    You can use boolean tests with the awscli's in-built query option, for example:

    aws secretsmanager list-secrets \
        --query "SecretList[?Tags[?Key=='Name' && (Value=='Production' || Value='Staging')]]"
    

    Here's an outline of a solution using Python and boto3:

    from functools import partial
    import boto3
    
    def filter_tags(key, values, secret):
        for tag in secret['Tags']:
            if tag['Key'] == key and tag['Value'] in values:
                return True
        return False
    
    sm = boto3.client('secretsmanager')
    
    paginator = sm.get_paginator('list_secrets')
    
    secrets_list_iterator = paginator.paginate()
    
    filter_production = partial(filter_tags, 'Name', ['Production', 'Staging'])
    
    for secrets in secrets_list_iterator:
        for s in filter(filter_production, secrets['SecretList']):
            print(s['Name'], s['Tags'])
    

    [Updated: January 2021]

    The aws secretsmanager list-secrets command now supports filtering via the --filters option. But .. I recommend that you do NOT use it unless you understand how it actually works (see below) and you would benefit from its particular implementation.

    Here's an example of how to use it to filter on secrets with a name that begins with Production:

    aws secretsmanager list-secrets \
        --filters Key=name,Values=Production
    

    Note that you cannot do an exact match with the --filters option, just a 'begins with' match, so be cautious when using it. If you have secrets with names of Production and Production-Old, both will be returned. That may not be what you want so in that case use the original client-side queries described above.

    Here's an example of how to use it to filter on secrets with a name that begins with Production or Staging:

    aws secretsmanager list-secrets \
        --filters Key=name,Values=Production,Staging
    

    Here's an example of how to use it to filter on secrets with a tag key that begins with stage or a tag value that begins with dev:

    aws secretsmanager list-secrets \
        --filters Key=tag-key,Values=stage Key=tag-value,Values=dev
    

    Note: the --filters option implements logical OR, not logical AND.

    Here's a boto3 example, filtering on tag keys that begin with Name or tag values that begin with Production or Staging:

    import boto3
    sm = boto3.client('secretsmanager')
    
    res = sm.list_secrets(Filters=[
        { 'Key': 'tag-key', 'Values': ['Name'] },
        { 'Key': 'tag-value', 'Values': ['Production', 'Staging'] },
    ])
    
    for secret in res['SecretList']:
        print(secret['Name'], secret['Tags'])