linuxcelerydaemonsystemd

Problem setting up celery as a daemon service

I've been trying to set up celery as a daemon for three days now and I am still struggling with it. Every time I start the service and check the status, it shows this,

The documentation had Type=forking, and when using it, the service fails to start. Status shows,

● celery.service - Celery Service
     Loaded: loaded (/etc/systemd/system/celery.service; disabled; vendor preset: enabled)
     Active: failed (Result: exit-code) since Thu 2020-07-23 05:36:18 UTC; 7s ago
    Process: 4256 ExecStart=/bin/sh -c ${CELERY_BIN} multi start ${CELERYD_NODES}    -A ${CELERY_APP} --pidfile=${CELER>
   Main PID: 4272 (code=exited, status=1/FAILURE)

... systemd[1]: Starting Celery Service...
... sh[4257]: celery multi v4.4.6 (cliffs)
... sh[4257]: > Starting nodes...
... sh[4257]:         > w1@dexter-server: OK
... systemd[1]: Started Celery Service.
... systemd[1]: celery.service: Main process exited, code=exited, status=1/FAILURE
... systemd[1]: celery.service: Failed with result 'exit-code'.

Nothing in the logs. And when I remove Type, I get this,

● celery.service - Celery Service
     Loaded: loaded (/etc/systemd/system/celery.service; disabled; vendor preset: enabled)
     Active: inactive (dead)

... systemd[1]: Started Celery Service.
... sh[2683]: celery multi v4.4.6 (cliffs)
... sh[2683]: > Starting nodes...
... sh[2683]:         > w1@dexter-server: OK
... sh[2690]: celery multi v4.4.6 (cliffs)
... sh[2690]: > w1@dexter-server: DOWN
... systemd[1]: celery.service: Succeeded.

Nothing in the log as well. Here are my current files.

/etc/systemd/system/celery.service

[Unit]
Description=Celery Service
After=network.target

[Service]
User=dexter
Group=dexter
RuntimeDirectory=celery
RuntimeDirectoryMode=0755
EnvironmentFile=/etc/conf.d/celery
WorkingDirectory=/var/www/example.com/myapp
ExecStart=/bin/sh -c '${CELERY_BIN} multi start ${CELERYD_NODES} \
  -A ${CELERY_APP} --pidfile=${CELERYD_PID_FILE} \
  --logfile=${CELERYD_LOG_FILE} --loglevel=${CELERYD_LOG_LEVEL} ${CELERYD_OPTS}'
ExecStop=/bin/sh -c '${CELERY_BIN} multi stopwait ${CELERYD_NODES} \
  --pidfile=${CELERYD_PID_FILE}'
ExecReload=/bin/sh -c '${CELERY_BIN} multi restart ${CELERYD_NODES} \
  -A ${CELERY_APP} --pidfile=${CELERYD_PID_FILE} \
  --logfile=${CELERYD_LOG_FILE} --loglevel=${CELERYD_LOG_LEVEL} ${CELERYD_OPTS}'

[Install]
WantedBy=multi-user.target

/etc/tmpfiles.d/celery.conf

d /var/run/celery 0755 dexter dexter -
d /var/log/celery 0755 dexter dexter -

/etc/conf.d/celery

# Name of nodes to start
# here we have a single node
CELERYD_NODES="w1"
# or we could have three nodes:
#CELERYD_NODES="w1 w2 w3"

# Absolute or relative path to the 'celery' command:
CELERY_BIN="/var/www/example.com/venv/bin/celery"
#CELERY_BIN="/virtualenvs/def/bin/celery"

# App instance to use
# comment out this line if you don't use an app
CELERY_APP="myapp.celery:app"
# or fully qualified:
#CELERY_APP="proj.tasks:app"

# How to call manage.py
CELERYD_MULTI="multi"

# Extra command-line arguments to the worker
CELERYD_OPTS="--time-limit=300 --concurrency=8"

# - %n will be replaced with the first part of the nodename.
# - %I will be replaced with the current child process index
#   and is important when using the prefork pool to avoid race conditions.
CELERYD_PID_FILE="/var/run/celery/%n.pid"
CELERYD_LOG_FILE="/var/log/celery/%n%I.log"
CELERYD_LOG_LEVEL="INFO"

# you may wish to add these options for Celery Beat
CELERYBEAT_PID_FILE="/var/run/celery/beat.pid"
CELERYBEAT_LOG_FILE="/var/log/celery/beat.log"

CELERY_CREATE_DIRS=1

I have no idea how to configure beyond this or where to look for errors at this point. Could someone help me identify the issue and solve this? I have a very basic understanding of concepts like forking, so please go easy on me.

Solution

  • I solved this by replacing the block in celery.service with the following.

    ExecStart='${CELERY_BIN} ${CELERy_NODES} \
  -A ${CELERY_APP} worker --pidfile=${CELERYD_PID_FILE} \
  --logfile=${CELERYD_LOG_FILE} --loglevel=${CELERYD_LOG_LEVEL} ${CELERYD_OPTS}'
ExecStop='${CELERY_BIN} stopwait ${CELERYD_NODES} \
  --pidfile=${CELERYD_PID_FILE}'
ExecReload='${CELERY_BIN} restart ${CELERYD_NODES} \
  -A ${CELERY_APP} worker --pidfile=${CELERYD_PID_FILE} \
  --logfile=${CELERYD_LOG_FILE} --loglevel=${CELERYD_LOG_LEVEL} ${CELERYD_OPTS}'

    I got help from #celery IRC.