DJCelery is not storing task results in my Django SQLite DB.
I have an existing Django project that I have started setting up Celery w/ RabbitMQ on. I started my RabbitMQ server. I can run Celery python manage.py celeryd --verbosity=2 --loglevel=DEBUG and Celerybeat python manage.py celerybeat --verbosity=2 --loglevel=DEBUG. Everything starts up w/ out error and my periodic example tasks also runs without error.
I used pip install django-celery to install. I have djcelery in my installed apps and ran python manage.py migrate djcelery. I added CELERY_RESULT_BACKEND='djcelery.backends.database:DatabaseBackend' to the end of my settings.py file.
When I run python manage.py celeryd --verbosity=2 --loglevel=DEBUG, the startup text shows:
...
- ** ---------- .> transport: amqp://guest:**@localhost:5672//
- ** ---------- .> results:
- *** --- * --- .> concurrency: 1 (prefork)
...
The results section being blank indicates to me that the configuration isn't right somehow but I can't figure out how. I tried using app.conf.update in my celery.py file to set the CELERY_RESULT_BACKEND but got the same results. I left out CELERY_RESULT_BACKEND, but that defaulted to no results. I also tried putting 'database' instead of 'djcelery.backends.database:DatabaseBackend' but that indicated it was attempting to use sqlalchemy instead of djcelery.
When I run python manage.py runserver I can see a DJCELERY section with tables Crontabs, Intervals, Periodic tasks, Tasks, and Workers. There isn't any data on my Tasks though.
Can anyone point out what could be wrong or missing? Thank you for your time.
tutuDajuju led me in the right direction - there's more to it so I'll write it all up. I abandoned using djcelery in favor of sqlalchemy with a separate back-end database outside of Django.
Inside my venv I ran pip install sqlalchemy. I then put CELERY_RESULT_BACKEND = 'db+sqlite:///celery_results.sqlite3' in settings.py. This connected Celery to the new SQLite database to use for state/results.
Running celery -A <projectapp>.celery:app worker then showed the database in the startup message:
...
- ** ---------- .> transport: amqp://guest:**@localhost:5672//
- ** ---------- .> results: sqlite:///celery_results.sqlite3
- *** --- * --- .> concurrency: 1 (prefork)
...
At first I was worried because the database file wasn't created in my Django project dir. This is because I hadn't ran a task yet. Once I ran my first task, the database & tables were created correctly.
I verified task results were stored in the database by running a script:
from sqlalchemy import create_engine
engine = create_engine("sqlite:///celery_results.sqlite3")
connection = engine.connect()
result = connection.execute("select * from celery_taskmeta")
for row in result:
print(row)
connection.close()
I found the table names by:
print(engine.table_name())
Hope this helps someone out.