[SOLVED] After scheduling a new task, previously scheduled tasks are not getting executed by Celery Beat (Django)

After scheduling a new task, previously scheduled tasks are not getting executed by Celery Beat (Django)

I have a Django (4.2.2) app running with Python (3.10.12), Celery (5.4.0), Celery Beat (2.6.0), Django Celery Results (2.5.1), Redis and Postgres.

Here is my celery configuration:

CELERY_BROKER_URL = "redis://localhost:6379/3"

from __future__ import absolute_import, unicode_literals
from celery import Celery
from django.conf import settings
import os
import django

# Set the DJANGO_SETTINGS_MODULE before calling django.setup()
os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'auctopus.settings')
django.setup()

# Initialize Celery
app = Celery('proj')
app.conf.enable_utc = False
app.conf.broker_connection_retry_on_startup = True
app.conf.task_serializer = 'json'
app.conf.accept_content = ['application/json']
app.conf.result_serializer = 'json'
app.conf.timezone = 'Asia/Kolkata'
app.conf.cache_backend = 'default'
app.conf.database_engine_options = {'echo': True}
app.conf.result_backend = 'django-db'
app.conf.result_expires = 3600
app.conf.task_time_limit = 1000
app.conf.task_default_queue ='default'
app.conf.worker_concurrency = 6

app.config_from_object(settings, namespace='CELERY')

# Autodiscover tasks from installed apps
app.autodiscover_tasks(lambda: settings.INSTALLED_APPS)

app.conf.beat_scheduler = 'django_celery_beat.schedulers:DatabaseScheduler'

After Django application starts, it creates several schedulers automatically such as internet_status_check, system_resource_check, etc. using IntervalSchedule.

At first, it runs smoothly, but as soon as I create any new scheduler be it an IntervalSchedule or CrontabSchedule, previously running schedulers stop execution. I get a message in celery beat terminal also saying DatabaseScheduler: Schedule changed.

After this message it does not send any task for execution even if tasks are available for execution which can be seen from Django admin panel.

I tried changing broker to RabbitMQ and result_backend to Redis/django-db. But same issue persists. Even if I restart my celery beat, it is not picking any tasks.

Only thing that worked is changing the last_run_at for all the tasks to None and then calling changed() method using django shell at runtime. But this can't be the solution.

from django_celery_beat.models import PeriodicTask, PeriodicTasks

PeriodicTask.objects.all().update(last_run_at=None)
for task in PeriodicTask.objects.all():
    PeriodicTasks.changed(task)

How do I resolve this issue? Is there any configuration missing from my end or it is a bug in the celery beat scheduler itself?

Solution

After looking into the issue for 3 days I found that issue was with time zone settings.

I had set USE_TZ = False in my settings.py of Django app and if we do not set DJANGO_CELERY_BEAT_TZ_AWARE=False, Celery Beat sets it to True by default.

So, setting DJANGO_CELERY_BEAT_TZ_AWARE=False in settings.py resolved the issue.