amazon-web-servicesaws-lambdaamazon-cloudwatchamazon-cloudwatchlogscloudwatch-alarms

Can AWS email errors logged to CloudWatch by multiple Lambda functions?


I have many functions in AWS Lambda that are invoked on various periodic schedules by CloudWatch Rules. If their Node JavaScript encounters an error, it’s logged to CloudWatch Log Groups.

Can either or both of these two complaints be solved by these tools, or should other tools be used?


Solution

  • Partially informed by this answer to a similar question, and by trial/error and who knows what other googling, I ended up pivoting away from AWS Metrics and AWS Alarms, and instead utilized a new Lambda function that is triggered by any number of my select Log groups. I kept the SNS functionality, which is very easy to set up if you own your emailing domain.

    Upon creation of a new Lambda function, the option to search for blueprints is presented, and searching log yields (among other options) the cloudwatch-logs-process-data blueprint. It can also be created from scratch, as its code is merely this:

    const zlib = require('zlib');
    
    exports.handler = async (event, context) => {
        const payload = Buffer.from(event.awslogs.data, 'base64');
        const parsed = JSON.parse(zlib.gunzipSync(payload).toString('utf8'));
        console.log('Decoded payload:', JSON.stringify(parsed));
        return `Successfully processed ${parsed.logEvents.length} log events.`;
    };
    

    I then chose for it as multiple Triggers the Log groups I wish to monitor, setting their Filter pattern to the word ERROR, and setting their Filter name to my desired email subject.

    Using the console log from the blueprint, I created a test event of this format:

    {
      "awslogs": {
        "data": "LONG_STRING_OF_ENCODED_CHARACTERS_FROM_ABOVE_CONSOLE_LOG_HERE"
      }
    }
    

    And I modified the blueprint code more to my liking as this:

    const zlib = require('zlib');
    const aws = require('aws-sdk');
    const ses = new aws.SES({ region: '<some-region-1>' });
    
    exports.handler = async function (event) {
        const payload = Buffer.from(event.awslogs.data, 'base64');
        const parsed = JSON.parse(zlib.gunzipSync(payload).toString('utf8'));
    
        const bodyText = JSON.stringify(parsed, null, 2).replace(/\\n/g, ' \\\n         ').replace(/\\t/g, '\\\n         ');
        console.log('bodyText:', bodyText);
    
        var params = {
            Source: '<some_email@example.com>',
            Destination: {
                ToAddresses: ['<some_email@example.com>'],
            },
            Message: {
                Subject: { Data: parsed.subscriptionFilters[0] },
                Body: {
                    Text: { Data: bodyText },
                },
            },
        };
        return ses.sendEmail(params).promise();
    };
    

    And it works, I get an immediate email if a Lambda function logs an error (with no noticeable increase to my monthly AWS bill as well).