pythondjangorabbitmqcelerysystem-design

Django celery error while adding tasks to RabbitMQ message queue : AttributeError: 'ChannelPromise' object has no attribute '__value__'


I have setup celery, rabbitmq and django web server on digitalocean. RabbitMQ runs on another server where my Django app is not running. When I am trying to add the tasks to the queue using delay I am getting an error

AttributeError: 'ChannelPromise' object has no attribute 'value'

From django shell I am adding the task to my message queue.

python3 manage.py shell

Python 3.8.10 (default, Mar 15 2022, 12:22:08)
[GCC 9.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
(InteractiveConsole)
>>> from app1.tasks import add
>>> add.delay(5, 6)

But getting error

Traceback (most recent call last):
  File "/etc/myprojectenv/lib/python3.8/site-packages/kombu/utils/functional.py", line 30, in __call__
    return self.__value__
AttributeError: 'ChannelPromise' object has no attribute '__value__'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/etc/myprojectenv/lib/python3.8/site-packages/kombu/connection.py", line 446, in _reraise_as_library_errors
    yield
  File "/etc/myprojectenv/lib/python3.8/site-packages/kombu/connection.py", line 433, in _ensure_connection
    return retry_over_time(
  File "/etc/myprojectenv/lib/python3.8/site-packages/kombu/utils/functional.py", line 312, in retry_over_time
    return fun(*args, **kwargs)
  File "/etc/myprojectenv/lib/python3.8/site-packages/kombu/connection.py", line 877, in _connection_factory
    self._connection = self._establish_connection()
  File "/etc/myprojectenv/lib/python3.8/site-packages/kombu/connection.py", line 812, in _establish_connection
    conn = self.transport.establish_connection()
  File "/etc/myprojectenv/lib/python3.8/site-packages/kombu/transport/pyamqp.py", line 201, in establish_connection
    conn.connect()
  File "/etc/myprojectenv/lib/python3.8/site-packages/amqp/connection.py", line 323, in connect
    self.transport.connect()
  File "/etc/myprojectenv/lib/python3.8/site-packages/amqp/transport.py", line 129, in connect
    self._connect(self.host, self.port, self.connect_timeout)
  File "/etc/myprojectenv/lib/python3.8/site-packages/amqp/transport.py", line 184, in _connect
    self.sock.connect(sa)
ConnectionRefusedError: [Errno 111] Connection refused

Started celery as :

celery -A myproject worker -l info

which gives me

User information: uid=0 euid=0 gid=0 egid=0

  warnings.warn(SecurityWarning(ROOT_DISCOURAGED.format(

 -------------- celery@ubuntu-s-1vcpu-1gb-blr1-01 v5.2.7 (dawn-chorus)
--- ***** -----
-- ******* ---- Linux-5.4.0-107-generic-x86_64-with-glibc2.29 2022-06-09 17:24:14
- *** --- * ---
- ** ---------- [config]
- ** ---------- .> app:         myproject:0x7fd64fa5d970
- ** ---------- .> transport:   amqp://himanshu:**@IPADDRESS2:5672/vhostcheck
- ** ---------- .> results:
- *** --- * --- .> concurrency: 1 (prefork)
-- ******* ---- .> task events: OFF (enable -E to monitor tasks in this worker)
--- ***** -----
 -------------- [queues]
                .> celery           exchange=celery(direct) key=celery


[tasks]
  . app1.tasks.add

[2022-06-09 17:24:14,309: INFO/MainProcess] Connected to amqp://himanshu:**@IPADDRESS:5672/vhostcheck
[2022-06-09 17:24:14,313: INFO/MainProcess] mingle: searching for neighbors
[2022-06-09 17:24:15,333: INFO/MainProcess] mingle: all alone
[2022-06-09 17:24:15,349: WARNING/MainProcess] /etc/myprojectenv/lib/python3.8/site-packages/kombu/pidbox.py:70: UserWarning: A node named celery@ubuntu-s-1vcpu-1gb-blr1-01 is already using this process mailbox!


[2022-06-09 17:24:15,352: WARNING/MainProcess] /etc/myprojectenv/lib/python3.8/site-packages/celery/fixups/django.py:203: UserWarning: Using settings.DEBUG leads to a memory
            leak, never use this setting in production environments!
  warnings.warn('''Using settings.DEBUG leads to a memory

[2022-06-09 17:24:15,352: INFO/MainProcess] celery@ubuntu-s-1vcpu-1gb-blr1-01 ready.

Inside app1 project :

tasks.py

from __future__ import absolute_import, unicode_literals
from celery import shared_task

@shared_task
def add(x, y):
    return x + y

settings.py

INSTALLED_APPS = [
    ...
    'app1',
    'django_celery_results',
]

CELERY_RESULT_BACKEND = 'django-db'
CELERY_CACHE_BACKEND = 'django-cache'
CELERY_BROKER_URL = 'amqp://himanshu:password@IPADDRESS:5672/vhostcheck'

CELERY_ACCEPT_CONTENT = ['application/json']
CELERY_TASK_SERIALIZER = 'json'
CELERY_RESULT_SERIALIZER = 'json'
CELERY_TIMEZONE = 'Europe/Amsterdam'

Solution

  • To ensure the app is loaded when Django starts, we need to import the Celery app we defined in myproject/__init__.py:

    sudo nano myproject/__init__.py
    
    # This will make sure the app is always imported when
    # Django starts so that shared_task will use this app.
    from .celery import app as celery_app
    __all__ = ['celery_app']