Hi just deployed my Flask application on my remote ubuntu server on Digital Ocean following this tutorial: https://www.digitalocean.com/community/tutorials/how-to-serve-flask-applications-with-uswgi-and-nginx-on-ubuntu-18-04
It all works fine except that when I import pyPandoc it cannot find pandoc. My code to import pypandoc is this:
import pypandoc
try:
from pypandoc.pandoc_download import download_pandoc
except:
pass
and this was working fine on my local but I get an error on the server:
[2020-04-09 16:16:37,027] ERROR in app: Exception on /get_all_articles [POST]
Traceback (most recent call last):
File "/home/fmagarelli/Infant_Competition_Platform/venv3/lib/python3.6/site-packages/flask/app.py", line 2447, in wsgi_app
response = self.full_dispatch_request()
File "/home/fmagarelli/Infant_Competition_Platform/venv3/lib/python3.6/site-packages/flask/app.py", line 1952, in full_dispatch_request
rv = self.handle_user_exception(e)
File "/home/fmagarelli/Infant_Competition_Platform/venv3/lib/python3.6/site-packages/flask/app.py", line 1821, in handle_user_exception
reraise(exc_type, exc_value, tb)
File "/home/fmagarelli/Infant_Competition_Platform/venv3/lib/python3.6/site-packages/flask/_compat.py", line 39, in reraise
raise value
File "/home/fmagarelli/Infant_Competition_Platform/venv3/lib/python3.6/site-packages/flask/app.py", line 1950, in full_dispatch_request
rv = self.dispatch_request()
File "/home/fmagarelli/Infant_Competition_Platform/venv3/lib/python3.6/site-packages/flask/app.py", line 1936, in dispatch_request
return self.view_functions[rule.endpoint](**req.view_args)
File "./main.py", line 285, in get_all_articles
return jsonify(myFirebase.get_all_articles(pending))
File "services/myFirebase.py", line 343, in get_all_articles
text = pypandoc.convert_text(text,'html',format='md')
File "/home/fmagarelli/Infant_Competition_Platform/venv3/lib/python3.6/site-packages/pypandoc/__init__.py", line 103, in convert_text
outputfile=outputfile, filters=filters)
File "/home/fmagarelli/Infant_Competition_Platform/venv3/lib/python3.6/site-packages/pypandoc/__init__.py", line 260, in _convert_input
_ensure_pandoc_path()
File "/home/fmagarelli/Infant_Competition_Platform/venv3/lib/python3.6/site-packages/pypandoc/__init__.py", line 544, in _ensure_pandoc_path
raise OSError("No pandoc was found: either install pandoc and add it\n"
OSError: No pandoc was found: either install pandoc and add it
to your PATH or or call pypandoc.download_pandoc(...) or
install pypandoc wheels with included pandoc.
I tried to install Pandoc via pip, apt, and even downloading the Debian file as suggested by the official Pandoc doc. Nothing seems to work. on pyPandoc website: https://pypi.org/project/pypandoc/
it says something about using wheel to install Pandoc but I'm not sure I understood what it means and I'm not familiar with the procedure. Do you have any idea on how to make pyPandoc working on my deployed Flask application?
ADDITIONAL INFO:
my flask app is running in a virtual environment as by the tutorial linked above. I tried to access Pandoc from within the virtual environment and this is my output:
fmagarelli@InfantTestServer:~/Infant_Competition_Platform$ source venv3/bin/activate
(venv3) fmagarelli@InfantTestServer:~/Infant_Competition_Platform$ pandoc --version
pandoc 2.9.2.1
Compiled with pandoc-types 1.20, texmath 0.12.0.1, skylighting 0.8.3.2
Default user data directory: /home/fmagarelli/.local/share/pandoc or /home/fmagarelli/.pandoc
Copyright (C) 2006-2020 John MacFarlane
Web: https://pandoc.org
This is free software; see the source for copying conditions.
There is no warranty, not even for merchantability or fitness
for a particular purpose.
EDIT2 adding PDB:
So change my imports to:
from pypandoc.pandoc_download import download_pandoc
import pypandoc
import pdb; pdb.set_trace()
and now this is my output:
[2020-04-09 17:31:46,358] ERROR in app: Exception on /get_all_articles [POST]
Traceback (most recent call last):
File "/home/fmagarelli/Infant_Competition_Platform/venv3/lib/python3.6/site-package
s/flask/app.py", line 2447, in wsgi_app
response = self.full_dispatch_request()
File "/home/fmagarelli/Infant_Competition_Platform/venv3/lib/python3.6/site-package
s/flask/app.py", line 1952, in full_dispatch_request
rv = self.handle_user_exception(e)
File "/home/fmagarelli/Infant_Competition_Platform/venv3/lib/python3.6/site-package
s/flask/app.py", line 1821, in handle_user_exception
reraise(exc_type, exc_value, tb)
File "/home/fmagarelli/Infant_Competition_Platform/venv3/lib/python3.6/site-package
s/flask/_compat.py", line 39, in reraise
raise value
File "/home/fmagarelli/Infant_Competition_Platform/venv3/lib/python3.6/site-package
s/flask/app.py", line 1950, in full_dispatch_request
rv = self.dispatch_request()
File "/home/fmagarelli/Infant_Competition_Platform/venv3/lib/python3.6/site-package
s/flask/app.py", line 1936, in dispatch_request
return self.view_functions[rule.endpoint](**req.view_args)
File "./main.py", line 285, in get_all_articles
return jsonify(myFirebase.get_all_articles(pending))
File "services/myFirebase.py", line 344, in get_all_articles
art['title'] = line.replace('#','')
File "/home/fmagarelli/Infant_Competition_Platform/venv3/lib/python3.6/site-package
s/pypandoc/__init__.py", line 103, in convert_text
outputfile=outputfile, filters=filters)
File "/home/fmagarelli/Infant_Competition_Platform/venv3/lib/python3.6/site-package
s/pypandoc/__init__.py", line 260, in _convert_input
_ensure_pandoc_path()
File "/home/fmagarelli/Infant_Competition_Platform/venv3/lib/python3.6/site-packages/pypandoc/__init__.py", line 544, in _ensure_pandoc_path
raise OSError("No pandoc was found: either install pandoc and add it\n"
OSError: No pandoc was found: either install pandoc and add it
to your PATH or or call pypandoc.download_pandoc(...) or
install pypandoc wheels with included pandoc.
[pid: 1159|app: 0|req: 2/8] 46.7.82.176 () {48 vars in 893 bytes} [Thu Apr 9 17:31:46 2020] POST /get_all_articles => generated 290 bytes in 325 msecs (HTTP/1.1 500) 2 headers in 99 bytes (1 switches on core 0)
SIGINT/SIGQUIT received...killing workers...
worker 1 buried after 1 seconds
worker 2 buried after 1 seconds
worker 3 buried after 1 seconds
worker 4 buried after 1 seconds
worker 5 buried after 1 seconds
goodbye to uWSGI.
VACUUM: unix socket main.sock removed.
*** Starting uWSGI 2.0.18 (64bit) on [Thu Apr 9 17:41:27 2020] ***
compiled with version: 7.5.0 on 09 April 2020 15:04:16
os: Linux-4.15.0-66-generic #75-Ubuntu SMP Tue Oct 1 05:24:09 UTC 2019
nodename: InfantTestServer
machine: x86_64
clock source: unix
detected number of CPU cores: 1
current working directory: /home/fmagarelli/Infant_Competition_Platform
detected binary path: /home/fmagarelli/Infant_Competition_Platform/venv3/bin/uwsgi
!!! no internal routing support, rebuild with pcre support !!!
your processes number limit is 3842
your memory page size is 4096 bytes
detected max file descriptor number: 1024
lock engine: pthread robust mutexes
thunder lock: disabled (you can enable it with --thunder-lock)
uwsgi socket 0 bound to UNIX address main.sock fd 3
Python version: 3.6.9 (default, Nov 7 2019, 10:44:02) [GCC 8.3.0]
*** Python threads support is disabled. You can enable it with --enable-threads ***
Python main interpreter initialized at 0x55c9e3a74470
your server socket listen backlog is limited to 100 connections
your mercy for graceful operations on workers is 60 seconds
mapped 437520 bytes (427 KB) for 5 cores
*** Operational MODE: preforking ***
> /home/fmagarelli/Infant_Competition_Platform/services/myFirebase.py(13)<module>()
-> import random
(Pdb)
Traceback (most recent call last):
File "./wsgi.py", line 1, in <module>
from main import app
File "./main.py", line 21, in <module>
import myFirebase
File "services/myFirebase.py", line 13, in <module>
import random
File "services/myFirebase.py", line 13, in <module>
import random
File "/usr/lib/python3.6/bdb.py", line 51, in trace_dispatch
return self.dispatch_line(frame)
File "/usr/lib/python3.6/bdb.py", line 70, in dispatch_line
if self.quitting: raise BdbQuit
bdb.BdbQuit
unable to load app 0 (mountpoint='') (callable not found or import error)
*** no app loaded. going in full dynamic mode ***
*** uWSGI is running in multiple interpreter mode ***
spawned uWSGI master process (pid: 893)
spawned uWSGI worker 1 (pid: 1040, cores: 1)
spawned uWSGI worker 2 (pid: 1041, cores: 1)
spawned uWSGI worker 3 (pid: 1042, cores: 1)
spawned uWSGI worker 4 (pid: 1043, cores: 1)
spawned uWSGI worker 5 (pid: 1044, cores: 1)
--- no python application found, check your startup logs for errors ---
[pid: 1040|app: -1|req: -1/1] 46.7.82.176 () {42 vars in 793 bytes} [Thu Apr 9 17:41:34 2020] GET /main.html => generated 21 bytes in 0 msecs (HTTP/1.1 500) 2 headers in 83 bytes (1 switches on core 0)
I solved, apparently it was such a common issue that they made a pull request in the pypandoc repo:
https://github.com/bebraw/pypandoc/issues/72#issuecomment-219283647
https://github.com/bebraw/pypandoc/pull/99/commits/5be3973b2a2c7eeb1abd6521c13374d82aa5ba99
so after activating the virtual environment, run which pandoc to get the path to Pandoc:
#activate env:
source myenv/bin/activate
which pandoc
OUT: '/usr/bin/pandoc'
Then add this to the flask app:
os.environ.setdefault('PYPANDOC_PANDOC','/usr/bin/pandoc')