google-app-enginecelerygoogle-app-engine-python

How do you start workers in Google App Engine?


I want to use Celery in a Django app I have on Google App Engine.

My question is: how do I start/run the workers?


Solution

  • I'm not familiar with Celery but the docs tell me it is a task queuing library. In that case, you are probably better off using Google Tasks Service instead. The task queuing service has long been one of the big strengths of App Engine out of the box.

    On App Engine classic all executions are started via an HTTP request. So If you absolutely must stick with Celery on App Engine classic, you would need to figure out a way to start the workers via an HTTP request possibly on a backend service that can run indefinitely, or via a cron task that triggers the worker at fixed intervals. It's unclear if either of these are workable, but they might be, it will take some trial and error.

    Alternatively, you might want to look at using a Google Compute Engine instance to run your workers, or possibly App Engine Flexible, which is sort of middle ground between the GCE and App Engine classic.