I have a Java application that runs in AWS Elastic Container Service. Application polls a queue periodically. Sometimes there is no response from the queue and the application hanging forever. I have enclosed the methods with try-catch blocks with logging exceptions. Even though there are no logs in the Cloudwatch after that. No exceptions or errors. Is there a way that I can identify this situation. ? (No logs in the Cloudwatch). Like filtering an error log pattern. So I can restart the service. Any trick or solution would be appreciated.
public void handleProcess() {
try {
while(true) {
Response response = QueueUitils.pollQueue(); // poll the queue
QueueUitils.processMessage(response);
TimeUnit.SECONDS.sleep(WAIT_TIME); // WAIT_TIME = 20
}
} catch (Exception e) {
LOGGER.error("Data Queue operation failed" + e.getMessage());
throw e;
}
}
You can do this with CloudWatch Alarms. I've set up a test Lambda function for this which runs every minute and logs to CloudWatch.
IncomingLogEvents
metric for the relevant log group (the log group to which your application is logging). In my case it's /aws/lambda/test-log-silence