I have an application that does some work in background, using default Cloud Tasks for scheduling/executing the process.
I would like the job to be able to run for a few minutes, or at least understand what the actual limitations are and what I can do about them.
According to docs on Push Queues (which seem to be equivalent to the modern Cloud Tasks?), the deadline is 10 minutes for auto-scaling, and 24 hours for basic scaling.
However, my job seems to crash after 2 minutes. 115 seconds is fine, 121 seconds is a crash. The workload and resource consumption is the same in all cases. The message is always the unhelpful "The process handling this request unexpectedly died. This is likely to cause a new process to be used for the next request to your application. (Error code 203)".
It does not matter if I use an auto-scaling F2 instance, or basic-scaling B2. It gets terminated after 2 minutes.
According to docs on Node request handling, there is a 60-second timeout for "request handlers"
What is the timeout in the end? Is it 1 minute, 2 minutes, or 10 minutes? Is there anything I can do to change it, if I want my job to run for 5 or 30 minutes.
In short summary, I think the best deduction that can help your scenario is Node's Request Timeout which has exactly 2 minutes timeout by default
In Long, after reading your question. I decided to create PoC out of it
setTimeout
) and can specify the duration from the request (e.g. /lr/300
means it gonna response approximately in 5 minutes)default
(Node8, Automatic Scaling)/lr/540
to the aforementioned serviceBefore: Before
As you can see, the Cloud Tasks and App Engine have problems waiting longer than 2 minutes, and have the same unhelpful message that you got (The process handling this request unexpectedly died...)
And then: Code
I wrote this line in order to increase Global Request Timeout
And the result: Result
In my case, I can safely say that it's Node Request Timeout that causes the problem. I hope this can be of use to you too.