amazon-web-servicesaws-lambdaamazon-sqslong-polling

AWS SQS Long Polling doesn't reduce empty receives

Currently, I have an AWS SQS as a trigger to my AWS Lambda function.

I would like to implement long polling to reduce costs since I've used up 70% of my monthly free tier, mostly from empty receives.

I tried setting up long polling by changing the queue attribute ReceiveMessageWaitTimeSeconds to 20 seconds:

SQS Details showing ReceiveMessageWaitTimeSeconds set to 20s

However, this didn't seem to reduce the number of empty receives, where the settings were changed on 11/19, between 2:00 - 3:00. Empty Receives graph shows the same trend even after setting long polling

According to the AWS Documentation, WaitTimeSeconds has priority over the queue attribute ReceiveMessageWaitTimeSeconds

Short polling occurs when the WaitTimeSeconds parameter of a ReceiveMessage request is set to 0 in one of two ways:

  • The ReceiveMessage call sets WaitTimeSeconds to 0.
  • The ReceiveMessage call doesn’t set WaitTimeSeconds, but the queue attribute ReceiveMessageWaitTimeSeconds is set to 0.

Note

For the WaitTimeSeconds parameter of the ReceiveMessage action, a value set between 1 and 20 has priority over any value set for the queue attribute ReceiveMessageWaitTimeSeconds.

Since AWS Lambda is receiving the SQS requests, I don't think WaitTimeSeconds can be configured.

Why doesn't my long polling configuration work in this situation? Am I misunderstanding something, or did I configure it wrong?

Thank you!

Solution

  • Actually Long Polling is working in your situation.

    5 lambdas * polling / 20 seconds * 3600 seconds in an hour = 900 receives/hour

    What I think you've missed is the "5 minimum concurrent lambdas". This is implied in the Lambda Scaling Behaviour documentation, but is more helpfully and explicitly laid out in the "Additional Information" section of the announcement/deep-dive blog.

    When an SQS event source mapping is initially created and enabled, or when messages first appear after a period with no traffic, then the Lambda service will begin polling the SQS queue using five parallel long-polling connections. The Lambda service monitors the number of inflight messages, and when it detects that this number is trending up, it will increase the polling frequency by 20 ReceiveMessage requests per minute and the function concurrency by 60 calls per minute.