After a couple days, my celery service will repeat a task over and over indefinitely. This is somewhat difficult to reproduce, but happens regularly once a week or more frequently depending on the tasks volume being processed.
I will appreciate any tips on how to get more data about this issue, since I don't know how to trace it. When it occurs, restarting celery will solve it temporarily.
I have one celery node running with 4 workers (version 3.1.23). Broker and result backends are on Redis. I'm posting to one queue only and I don't use celery beat.
The config in Django's setting.py is:
BROKER_URL = 'redis://localhost:6380'
CELERY_RESULT_BACKEND = 'redis://localhost:6380'
Relevant part of the log:
[2016-05-28 10:37:21,957: INFO/MainProcess] Received task: painel.tasks.indicar_cliente[defc87bc-5dd5-4857-9e45-d2a43aeb2647]
[2016-05-28 11:37:58,005: INFO/MainProcess] Received task: painel.tasks.indicar_cliente[defc87bc-5dd5-4857-9e45-d2a43aeb2647]
[2016-05-28 13:37:59,147: INFO/MainProcess] Received task: painel.tasks.indicar_cliente[defc87bc-5dd5-4857-9e45-d2a43aeb2647]
...
[2016-05-30 09:27:47,136: INFO/MainProcess] Task painel.tasks.indicar_cliente[defc87bc-5dd5-4857-9e45-d2a43aeb2647] succeeded in 53.33468166703824s: None
[2016-05-30 09:43:08,317: INFO/MainProcess] Task painel.tasks.indicar_cliente[defc87bc-5dd5-4857-9e45-d2a43aeb2647] succeeded in 466.0324719119817s: None
[2016-05-30 09:57:25,550: INFO/MainProcess] Task painel.tasks.indicar_cliente[defc87bc-5dd5-4857-9e45-d2a43aeb2647] succeeded in 642.7634702899959s: None
Tasks are sent by user request with:
tasks.indicar_cliente.delay(indicacao_db.id)
Here's the source code of the task and the celery service configuration.
Why are the tasks being received multiple times after some time the service is running? How can I get a consistent behavior?
Solved by using rabbitmq broker instead of redis.