amazon-web-servicesamazon-elastic-beanstalkcelerydaemon

AWS Elasticbeanstalk post deploy script problem


I'm trying to run the celery worker after my application has been deployed to AWS Elasticbeanstalk.

99_celery_start.sh

#!/usr/bin/env bash

#make dirs
sudo mkdir -p /usr/etc/
sudo chmod 755 /usr/etc/
sudo touch /usr/etc/celery.conf
sudo touch /usr/etc/supervisord.conf

# Get django environment variables
celeryenv=`cat /var/app/rootfolder/myprject/.env | tr '\n' ',' | sed 's/export //g' | sed 's/$PATH/%(ENV_PATH)s/g' | sed 's/$PYTHONPATH//g' | sed 's/$LD_LIBRARY_PATH//g' | sed 's/%/%%/g'`
celeryenv=${celeryenv%?}

# Create celery configuration script
celeryconf="[program:celery-worker]
user=root
directory=/var/app/rootfolder
; Set full path to celery program if using virtualenv
command=/var/app/venv/*/bin/celery -A myprject worker -P solo --loglevel=INFO        
.
.
.
.

celery.config

container_commands:
  01_celery_configure:
    command: "mkdir -p /.platform/hooks/postdeploy/ && cp .ebextensions/99_celery_start.sh /.platform/hooks/postdeploy/ && chmod 774 /.platform/hooks/postdeploy/99_celery_start.sh"
  02_run_celery:
    command: "sudo /.platform/hooks/postdeploy/99_celery_start.sh"

So what I'm trying is to copy the celery-worker script in the .ebextension folder and paste it into the post-deploy hook folder so that the script runs after the application is deployed on the instance.But the command 02_run_celery is executing before the application is extracted and deployed on the instance. Since the script requires the application folder /var/app/rootfolder/myprjct/.env, the deployment process gives an error cat: /var/app/rootfolder/myprjct/.env: No such file or directory.


Solution

  • If anyone still looking for the answer then this is how I did it:

    Folder structure:

    |-- .ebextensions/
    |   |-- celery.config        # Option settings
    |   `-- cloudwatch.config     # Other .ebextensions sections, for example files and container commands
    `-- .platform/
        |-- nginx/                # Proxy configuration
        |   |-- nginx.conf
        |   `-- conf.d/
        |       `-- custom.conf
        |-- hooks/                # Application deployment hooks
        |   `-- postdeploy/
        |       `-- 99_celery_start.sh
    

    Now add permissions for 99_celery_start.script in celery.config:

    01_celery_perm:
        command: "sudo chmod +x .platform/hooks/postdeploy/99_celery_start.sh"
    02_dos2unix:
        command: "perl -i -pe's/\r$//;' .platform/hooks/postdeploy/99_celery_start.sh"
    

    IMPORTANT: Make sure the script should be saved in LF line endings instead of CRLF.