cronceleryscheduled-tasksperiodic-task

Celery - Start and stop periodic task from tornado interface


I am trying to start and stop celery periodic tasks from a tornado application interface.

As an example, let say there are two tasks: A and B. I would like the user to be able to select the periodicity from an HTML form (every minute, every month, every 5 minutes, etc) and click start on task A. The user can do the same with task B. And then come back on a page where there is a button to stop task A and/or task B whenever he wants.

I have been browsing a lot stackoverflow on questions around the topic and none of them are answering this simple question.

As of now, my tornado app handle simple celery workers without any problem, the problem I face is for periodic tasks.(https://docs.celeryproject.org/en/latest/reference/celery.html#celery.Celery.on_after_configure)

continuous_monitoring_worker.py:

continuous_tracking_worker_app.conf.timezone = 'Europe/London'

@continuous_tracking_worker_app.on_after_configure.connect
def setup_periodic_tasks(sender, **kwargs):
    print('OK inside setup periodic tasks ...')
    sender.add_periodic_task(10.0, test.s('Hello World'), name='add every 10')

    # In case I want to stop the task later
    #time.sleep(35)
    #print('beat_schedule = {}'.format(continuous_tracking_worker_app.conf.beat_schedule))
    #del continuous_tracking_worker_app.conf.beat_schedule['add every 10']

@continuous_tracking_worker_app.task
def test(name):
    print('Periodic task called, name = {}'.format(name))

With this code, I run through the following problem: When I start the app, it launches the task every 10 seconds (I guess because of the @continuous_tracking_worker_app.on_after_configure.connect decorator). But I want the task to be launch on demand from the interface (on the backend in my tornado view file by calling setup_periodic_tasks(continuous_tracking_worker_app), and not on start of the tornado app !


Solution

  • Celery Beat can't do that.

    You may consider using the Celery Redbeat that makes it possible.