google-app-enginegoogle-cloud-platformgoogle-appengine-node

AppEngine - What's the timeout for Node cloud tasks handlers?


I have an application that does some work in background, using default Cloud Tasks for scheduling/executing the process.

I would like the job to be able to run for a few minutes, or at least understand what the actual limitations are and what I can do about them.

According to docs on Push Queues (which seem to be equivalent to the modern Cloud Tasks?), the deadline is 10 minutes for auto-scaling, and 24 hours for basic scaling.

However, my job seems to crash after 2 minutes. 115 seconds is fine, 121 seconds is a crash. The workload and resource consumption is the same in all cases. The message is always the unhelpful "The process handling this request unexpectedly died. This is likely to cause a new process to be used for the next request to your application. (Error code 203)".

It does not matter if I use an auto-scaling F2 instance, or basic-scaling B2. It gets terminated after 2 minutes.

According to docs on Node request handling, there is a 60-second timeout for "request handlers"

What is the timeout in the end? Is it 1 minute, 2 minutes, or 10 minutes? Is there anything I can do to change it, if I want my job to run for 5 or 30 minutes.


Solution

  • In short summary, I think the best deduction that can help your scenario is Node's Request Timeout which has exactly 2 minutes timeout by default


    In Long, after reading your question. I decided to create PoC out of it

    1. created the Dummy Node 8 Service which only uses a built-in HTTP server
    2. created a URL path that can have an artificially long response (using setTimeout) and can specify the duration from the request (e.g. /lr/300 means it gonna response approximately in 5 minutes)
    3. deployed it to GAE service another than default (Node8, Automatic Scaling)
    4. created Cloud Tasks "task" that request /lr/540 to the aforementioned service

    Before: Before

    As you can see, the Cloud Tasks and App Engine have problems waiting longer than 2 minutes, and have the same unhelpful message that you got (The process handling this request unexpectedly died...)

    And then: Code

    I wrote this line in order to increase Global Request Timeout

    And the result: Result

    In my case, I can safely say that it's Node Request Timeout that causes the problem. I hope this can be of use to you too.