amazon-web-servicesaws-lambdaaws-cloudformationaws-secrets-manageramazon-kinesis-firehose

How to correctly pass a key stored in secret manager to another resource in cloud formation template


I have a case where I am taking some Licensekey as input to a cloud formation template, then I have to use this to create a firehose delivery stream. This is working fine,because I can just pass the license key directly to the firehose delivery stream creation logic, something like this: Here is the parameter:

Parameters:
  LicenseKey:
    Type: String
    Decription: Some description

here is the code snippet from firehose delivery stream :

 StreamNameXyzabc:
    Type: AWS::KinesisFirehose::DeliveryStream
     Properties:
      HttpEndpointDestinationConfiguration:
        RequestConfiguration:
          ContentEncoding: GZIP
          EndpointConfiguration:
            Name: xyz
            Url : 'some url'
            Accesskey: !Ref Licensekey

The above code is working fine. I am able to send logs. Now if I want to store the Licensekey in secret manager while using stackId in the name of secret manager, I am doing something like this:

LicenseKeySecret:
    Type: 'AWS::SecretsManager::Secret'
    Properties:
      Description: 
      Name : !Join ['-', ['himanshu-', !Select [4, !Split ['-', !Select [2, !Split ['/', !Ref AWS::StackId]]]]]]
      SecretString: !Sub '{ "LicenseKey": "${LicenseKey}"}'

I want to use this secret manager in my FireHoseDelivery Stream code mentioned above:

EndpointConfiguration:
            Name: xyz
            Url : 'some url'
            Accesskey: !Ref Licensekey

So far I have tried accessing it in many ways but not able to, how to access it. Some of the methods I have tried:

 1. AccessKey: Fn::Subresolve:secretsmanager:himanshu-${AWS::StackId}-LicenseKey}}
 2. AccessKey: !Sub '{{resolve:secretsmanager:himanshu-key-${AWS::StackId}:SecretString:LicenseKey}}'

and some more permutation combination, in some cases stack creation itself is failing, but in some case stack creation is not failing, but the LicenseKey is going incorrectly, and hence firehose is not able to send the logs.

I need answer to below two questions:

  1. can some one help me with writing the correct code for AcessKey ?
  2. Is there a way to debug the accesskey that is being used for creation , I can get some idea what exactly is getting passed and may be correct it. I tried writing a custom lambda using python and getting accessKey, but to no use because seems like we can't see accessKey. Code used to debug:
 FirehoseAccessKeyLoggerLambda:
    Type: 'AWS::Serverless::Function'
    DependsOn: someResource
    Properties: 
      InlineCode: |
          import json
          import boto3
          import logging

          logger = logging.getLogger()
          logger.setLevel(logging.INFO)

          def handler(event, context):
              firehose_client = boto3.client('firehose')
              stream_name = event['ResourceProperties']['StreamName']
              
              try:
                  response = firehose_client.describe_delivery_stream(DeliveryStreamName=stream_name)
                  logger.info(f'Response: {response}')

                  #access_key = response['DeliveryStreamDescription']['Destinations'][0]['HttpEndpointDestinationDescription']['AccessKey']
                  #logger.info(f'Access Key: {access_key}')
    #accessKey is not present in the response it seems.
              except Exception as e:
                  logger.error(f'Error retrieving access key: {str(e)}')
                  raise e
      Handler: index.handler
      Runtime: python3.12
      Role: !GetAtt FirehoseAccessKeyLoggerLambdaExecutionRole.Arn
      Timeout: 120
      MemorySize: 128

Solution

  • If your LicenseKeySecret resource looks like this:

    LicenseKeySecret:
      Type: 'AWS::SecretsManager::Secret'
      Properties:
        Description: License Key Secret
        Name: !Join ['-', ['himanshu-', !Select [4, !Split ['-', !Select [2, !Split ['/', !Ref AWS::StackId]]]]]]
        SecretString: !Sub '{"LicenseKey": "${LicenseKey}"}'
    

    You can reference the secret’s value in your Firehose AccessKey property like this:

    StreamNameXyzabc:
      Type: AWS::KinesisFirehose::DeliveryStream
      Properties:
        HttpEndpointDestinationConfiguration:
          RequestConfiguration:
            ContentEncoding: GZIP
            EndpointConfiguration:
              Name: xyz
              Url: 'some url'
              AccessKey: !Sub '{{resolve:secretsmanager:${LicenseKeySecret}:SecretString:LicenseKey}}'
    

    With ${LicenseKeySecret} we will get direct the secret ARN.

    PS: corrected code version