DJCelery
is not storing task results in my Django
SQLite
DB.
I have an existing Django project that I have started setting up Celery w/ RabbitMQ on. I started my RabbitMQ server. I can run Celery python manage.py celeryd --verbosity=2 --loglevel=DEBUG
and Celerybeat python manage.py celerybeat --verbosity=2 --loglevel=DEBUG
. Everything starts up w/ out error and my periodic example tasks also runs without error.
I used pip install django-celery
to install. I have djcelery in my installed apps and ran python manage.py migrate djcelery
. I added CELERY_RESULT_BACKEND='djcelery.backends.database:DatabaseBackend'
to the end of my settings.py file.
When I run python manage.py celeryd --verbosity=2 --loglevel=DEBUG
, the startup text shows:
...
- ** ---------- .> transport: amqp://guest:**@localhost:5672//
- ** ---------- .> results:
- *** --- * --- .> concurrency: 1 (prefork)
...
The results section being blank indicates to me that the configuration isn't right somehow but I can't figure out how. I tried using app.conf.update in my celery.py file to set the CELERY_RESULT_BACKEND but got the same results. I left out CELERY_RESULT_BACKEND, but that defaulted to no results. I also tried putting 'database'
instead of 'djcelery.backends.database:DatabaseBackend'
but that indicated it was attempting to use sqlalchemy
instead of djcelery
.
When I run python manage.py runserver
I can see a DJCELERY
section with tables Crontabs
, Intervals
, Periodic tasks
, Tasks
, and Workers
. There isn't any data on my Tasks though.
Can anyone point out what could be wrong or missing? Thank you for your time.
tutuDajuju led me in the right direction - there's more to it so I'll write it all up. I abandoned using djcelery
in favor of sqlalchemy
with a separate back-end database outside of Django
.
Inside my venv
I ran pip install sqlalchemy
. I then put CELERY_RESULT_BACKEND = 'db+sqlite:///celery_results.sqlite3'
in settings.py
. This connected Celery
to the new SQLite
database to use for state/results.
Running celery -A <projectapp>.celery:app worker
then showed the database in the startup message:
...
- ** ---------- .> transport: amqp://guest:**@localhost:5672//
- ** ---------- .> results: sqlite:///celery_results.sqlite3
- *** --- * --- .> concurrency: 1 (prefork)
...
At first I was worried because the database file wasn't created in my Django project dir. This is because I hadn't ran a task yet. Once I ran my first task, the database & tables were created correctly.
I verified task results were stored in the database by running a script:
from sqlalchemy import create_engine
engine = create_engine("sqlite:///celery_results.sqlite3")
connection = engine.connect()
result = connection.execute("select * from celery_taskmeta")
for row in result:
print(row)
connection.close()
I found the table names by:
print(engine.table_name())
Hope this helps someone out.