python-3.xdjangoubuntuamazon-ec2gunicorn

Unable to bind Django application with Gunicorn


It has been very difficult for me trying to deploy to aws EC2 ubuntu server SINCE I'm coming from windows background. I encounter an error while trying to bind django application to gunicorn. The command I'm running is sudo gunicorn --bind 0.0.0.0:8000 logistics.wsgi:application And the error log is show below:

(venv) ubuntu@ip-172-31-18-196:/var/www/html$ sudo gunicorn --bind 0.0.0.0:8000 logistics.wsgi:application
[2021-09-08 11:21:00 +0000] [29379] [INFO] Starting gunicorn 20.1.0
[2021-09-08 11:21:00 +0000] [29379] [INFO] Listening at: http://0.0.0.0:8000 (29379)
[2021-09-08 11:21:00 +0000] [29379] [INFO] Using worker: sync
[2021-09-08 11:21:00 +0000] [29382] [INFO] Booting worker with pid: 29382
[2021-09-08 11:21:00 +0000] [29382] [ERROR] Exception in worker process
Traceback (most recent call last):
  File "/usr/local/lib/python3.5/dist-packages/gunicorn/arbiter.py", line 589, in spawn_worker
    worker.init_process()
  File "/usr/local/lib/python3.5/dist-packages/gunicorn/workers/base.py", line 134, in init_process
    self.load_wsgi()
  File "/usr/local/lib/python3.5/dist-packages/gunicorn/workers/base.py", line 146, in load_wsgi
    self.wsgi = self.app.wsgi()
  File "/usr/local/lib/python3.5/dist-packages/gunicorn/app/base.py", line 67, in wsgi
    self.callable = self.load()
  File "/usr/local/lib/python3.5/dist-packages/gunicorn/app/wsgiapp.py", line 58, in load
    return self.load_wsgiapp()
  File "/usr/local/lib/python3.5/dist-packages/gunicorn/app/wsgiapp.py", line 48, in load_wsgiapp
    return util.import_app(self.app_uri)
  File "/usr/local/lib/python3.5/dist-packages/gunicorn/util.py", line 359, in import_app
    mod = importlib.import_module(module)
  File "/usr/lib/python3.5/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 986, in _gcd_import
  File "<frozen importlib._bootstrap>", line 969, in _find_and_load
  File "<frozen importlib._bootstrap>", line 958, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 673, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 665, in exec_module
  File "<frozen importlib._bootstrap>", line 222, in _call_with_frames_removed
  File "/var/www/html/logistics/wsgi.py", line 12, in <module>
    from django.core.wsgi import get_wsgi_application
  File "/usr/local/lib/python3.5/dist-packages/django/core/wsgi.py", line 2, in <module>
    from django.core.handlers.wsgi import WSGIHandler
  File "/usr/local/lib/python3.5/dist-packages/django/core/handlers/wsgi.py", line 3, in <module>
    from django.conf import settings
  File "/usr/local/lib/python3.5/dist-packages/django/conf/__init__.py", line 19, in <module>
    from django.utils.deprecation import RemovedInDjango40Warning
  File "/usr/local/lib/python3.5/dist-packages/django/utils/deprecation.py", line 5, in <module>
    from asgiref.sync import sync_to_async
  File "/usr/local/lib/python3.5/dist-packages/asgiref/sync.py", line 115
    launch_map: "Dict[asyncio.Task[object], threading.Thread]" = {}
              ^
SyntaxError: invalid syntax
[2021-09-08 11:21:00 +0000] [29382] [INFO] Worker exiting (pid: 29382)
[2021-09-08 11:21:00 +0000] [29379] [INFO] Shutting down: Master
[2021-09-08 11:21:00 +0000] [29379] [INFO] Reason: Worker failed to boot.

When I run gunicorn --bind 0.0.0.0:8000 logistics.wsgi:application, (that is, without sudo) I get another error:

(venv) ubuntu@ip-172-31-18-196:/var/www/html$ gunicorn --bind 0.0.0.0:8000 logistics.wsgi:application
Traceback (most recent call last):
  File "/home/ubuntu/.local/bin/gunicorn", line 7, in <module>
    from gunicorn.app.wsgiapp import run
ModuleNotFoundError: No module named 'gunicorn'

But I have already install gunicorn with the command pip3 install gunicorn --user. The reason why I added --user at the end is that running pip3 install gunicorn within the activated virtual enviroment is throwing back permission error as shown below:

(venv) ubuntu@ip-172-31-18-196:/var/www/html$ pip3 install gunicorn
Collecting gunicorn
  Using cached https://files.pythonhosted.org/packages/e4/dd/5b190393e6066286773a67dfcc2f9492058e9b57c4867a95f1ba5caf0a83/gunicorn-20.1.0-py3-none-any.whl
Requirement already satisfied: setuptools>=3.0 in ./venv/lib/python3.6/site-packages (from gunicorn) (40.6.2)
Installing collected packages: gunicorn
Could not install packages due to an EnvironmentError: [Errno 13] Permission denied: '/var/www/html/venv/lib/python3.6/site-packages/gunicorn-20.1.0.dist-info'
Consider using the `--user` option or check the permissions.

You are using pip version 18.1, however version 21.2.4 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.

Again, I have already upgraded from python3.5 to python3.6 such that when I run python3 on the terminal I get the following output

(venv) ubuntu@ip-172-31-18-196:/var/www/html$ python3 --version
Python 3.6.13

Yet, I don't know why the error log is making reference to python3.5 instead of python3.6 as shown here: File "/usr/local/lib/python3.5/dist-packages/gunicorn/app/wsgiapp.py" whenever I run sudo gunicorn --bind 0.0.0.0:8000 logistics.wsgi:application Please I want to know why I'm getting that error


Solution

  • How to upload your Django app in AWS Ubuntu using Gunicorn and NGINX

    This answer is courtesy of WOLFx Digital Agency

    You have to login as Ubuntu user and NOT sudo su/root

    Stage 1: Binding your Gunicorn to your Django app and checking if the upstream gunicorn is working fine. Please note the deployment is incomplete without other stages

    1. sudo apt-get update

    2. sudo apt-get upgrade

    3. Optional - If it shows a popup/options then just select the pkg maintainer version.

    4. python3 -m venv env

    5. sudo apt-get install python3-venv

    6. source env/bin/activate

    7. pip3 install django

    8. git clone <your-repo-url>

    9. pip3 install gunicorn

    10. sudo apt-get install -y nginx

    11. cd to your project directory where settings.py, db.sqlite3 and all those files of your project is stored.

    12. pip3 install -r requirements.txt

    13. gunicorn --bind 0.0.0.0:8000 <project_name>.wsgi:application
      Note: your project name is the main app name which you created in the beginning with django-admin startproject <project_name> command

    14. You will see

      [2021-09-08 15:20:17 +0000] [12789] [INFO] Starting gunicorn 20.1.0
      [2021-09-08 15:20:17 +0000] [12789] [INFO] Listening at: http://0.0.0.0:8000 (12789)
      [2021-09-08 15:20:17 +0000] [12789] [INFO] Using worker: sync
      [2021-09-08 15:20:17 +0000] [12791] [INFO] Booting worker with pid: 12791
      

    which means you have successfully bonded your gunicorn to run your Django app and your Django app is now ready to get linked with a webserver (NGINX in our case). This marks the completion of Stage 1. To test out Stage 1 success you can type in your IP with port :8000 and see your application run (make sure your aws security is allowing port 8000 else you will see a 404) but the above Booting worker with pid confirms that it's working.


    Stage 2: Setting up supervisor so that your Gunicors autostarts your Django app on reboot and after first boot.

    1. sudo apt-get install -y supervisor

    2. cd /etc/supervisor/conf.d/

    3. sudo touch gunicorn.conf

    4. sudo nano gunicorn.conf This will open up an editor where you have to type in the script for gunicorn (The bind which we did in Stage 1 but now we are telling supervisor to bind gunicorn every time the server/instance start)

       [program:gunicorn]
       directory = /home/ubuntu/<path to manage.py>
       command = /home/ubuntu/env/bin/gunicorn --workers 3 --bind unix:/home/ubuntu/<App Directory>/app.sock <App Name>.wsgi:application
       autostart = true
       autorestart = true
       stderr_logfile = /var/log/gunicorn/gunicorn.err.log
       stdout_logfile = /var/log/gunicorn/gunicorn.out.log
      
       [group:guni]
       programs:gunicorn
      
    5. sudo mkdir /var/log/gunicorn, here we are creating the log folder for our gunicorn out and error logs.

    6. sudo supervisorctl reread

    7. If you see guni:available here it means your supervisor is all set and you have properly done everything till now.

    8. sudo supervisorctl update

    9. if you see guni: added process group this is another marker that you have properly done the steps.

    10. sudo supervisorctl status

    11. If you see guni:gunicorn RUNNING this is the third and final indicator that your gunicorn is now properly set up. After all these steps it is confirmed that your gunicorn is now bi-directionally communicating to an app.sock file which is automatically created inside your project directory.


    Stage 3: Final step to link your Gunicorn upstream server to your NGINX

    1. cd /etc/nginx/sites-available

    2. sudo touch django.conf

    3. sudo nano django.conf This will open up your nano editor where you have to type in these exact server settings.

      server {
          listen 80;
          server_name <ipaddress or domain name> ;
          #server_name 192.168.0.1 yourdomain.com your_alternate_domain.com; this is how you can add multiple hosts. Do not add any comma just separate it with spaces
      
          location / {
              include proxy_params;
              proxy_pass http://unix:/home/ubuntu/Appdir/app.sock;
          }
      }
      
    4. sudo nginx -t (This will test the syntax of your config file)

    5. sudo ln django.conf /etc/nginx/sites-enabled/ <---- this is a very crucial step make sure there is not typing mistake here, it creates a symlink

    6. sudo nginx -t

    7. sudo service nginx restart

    With this you now have linked your Nginx to your gunicorn upstream on app.sock so head on to your browser and type in the IP address of your instance and you will see your app live.

    If you can see your website without any CSS then you have followed everything properly, I will re-edit my answer on how to serve static files of your Django app in Nginx once you confirm that everything works.