socketslambdaaws-lambda

Lambda + API Gateway: Long executing function return before task finished


I have a function in Lambda that executes for up to 30s depending on the inputs. That function is linked to an API gateway so I am able to call it through POST, unfortunatly the API Gateway is limited to 30s exaclty, so if my function runs for longer I get an "internal server error" back from my POST.

I see 2 solutions to this:

The function is coded in Python3.6 I therefore have access to the event and context object.


Solution

  • Return a notification to api gateway before the lambda function is actually finished. This is acceptable for my use case, but I am not sure again how would this work.

    Unfortunately, you will not be able to return a result until the Lambda is finished. Otherwise, AWS will interrupt execution of Lambda if you try to do it, for example, via multithreading.

    I suggest you create the following system:

    1. API Gateway puts a request to Kinesis stream and returns a response of successfully putting. The response will contain SequenceNumber of the Kinesis Record. (How to do it)
    2. Lambda is invoked by Kinesis Stream Event, processes the request and saves that your job is done (or completed result) to DynamoDB with SequenceNumber as id.
    3. You invoke another API Gateway method and SequenceNumber to check the status of your job in DynamoDB (if necessary).

    If you need just to run the Lambda without knowing of job result you can avoid using DynamoDB.