amazon-sqsamazon-cloudwatchaws-event-bridge

Input Transformer in EventBridge Rule for Events sending to Cloudwatch Logs does not work


I have a very basic setup to write some of the AWS events in my account to a CloudWatch log group.

A Rule with pattern

{
  "detail-type": ["Batch Job State Change"],
  "source": ["aws.batch"]
}

Targets Type CloudWatch log group and using an existing log group

If I chose "Matched events" as target input, it works perfectly, all expected events are visible in the logs.

If I switch to "Input Transformer"

with Input Path:

{"eventId":"$.id","jobId":"$.detail.jobId","jobName":"$.detail.jobName","jobQueue":"$.detail.jobQueue","jobStatus":"$.detail.status","jobTime":"$.time"}

with Template:

{
  "id": "<eventId>",
  "time": "<jobTime>",
  "detail": {
    "jobName": "<jobName>",
    "jobId": "<jobId>",
    "jobQueue": "<jobQueue>",
    "status": "<jobStatus>"
  }
}

the events do not appear in the logs. I can see in the rules monitoring that there are now failed invocations.

So I configured a SQS as DLQ for the rule.

It shows now the expected transformed events, body looks as expected like the message changed by the input transformer.

Attributes of the SQS message say:

ERROR_CODE String SDK_CLIENT_ERROR
ERROR_MESSAGE String Unknown exception.
RULE_ARN String arn:aws:events:<my region>:<my account>:rule/<my rule name>
TARGET_ARN String arn:aws:logs:<my region>:<my account>:log-group:<my log group>

I have the resource based policy, so events can be written to CloudWatch as it is working fine if using "Matched events". I do not know what is missing. Another IAM policy?

I consulted: https://docs.aws.amazon.com/eventbridge/latest/userguide/eb-input-transformer-tutorial.html https://docs.aws.amazon.com/eventbridge/latest/userguide/eb-transform-input-rule.html

I tried successfully with the basic setup using "Matched events" and found the messages sent to CloudWatch logs. I expect to find the transformed event messages in CloudWatch logs. For debugging purposes I configured a DLS in SQS but the error details are not helpful


Solution

  • For CW Logs as the target, the input transformer needs to follow a specific format. Please see the table section "Amazon CloudWatch log group" on page https://docs.aws.amazon.com/eventbridge/latest/userguide/eb-transform-target-input.html