I've developed a simple flask website that is using a remote PostgreSQL database. I've used sqlalchemy for defining the database and pyscopg for connecting to the database. I store the database URL, username and password in a .env file which I then load in my app.py file. When I run this locally on my laptop everything works fine and I can connect to the database.
Local app.py file:
from web import create_app
from web.forms import csrf
from database.config import DevConfig, ProdConfig
from database import init_db
config = DevConfig
application = create_app(config)
csrf.init_app(application)
init_db(application, config)
if __name__ == "__main__":
application.run(debug=True, use_debugger=False, use_reloader=False)
init_db is the function that creates the connection to the database.
from sqlalchemy import create_engine
from sqlalchemy.orm import sessionmaker, scoped_session, Session
def init_db(app, config):
engine = create_engine(
config.DATABASE_URI,
echo=False,
)
app.db_session = scoped_session(sessionmaker(engine))
I'm now trying to set up the flask website on a webhosting server that is using cPanel, following the instructions here: https://docs.cloudlinux.com/shared/lve_manager/#python-selector-client-plugin
For simple flask apps that don't connect to a database that works fine. However I'm unable to get my set-up to work.
I've removed the if __name__ == "__main__": block from app.py and I'm using the following passenger_wsgi.py that is created by default when starting the app:
import imp
import os
import sys
sys.path.insert(0, os.path.dirname(__file__))
wsgi = imp.load_source('wsgi', 'app.py')
application = wsgi.application
This initially seems to work and the site appears at the URL and the pages render and everything looks great. However, when I go to one of the site pages that query the database to return a table of data the app crashes and dies with the following messages displayed "Incomplete response received from application".
Checking the passenger.log file in cPanel I get the stack trace of the error which is pretty long, but essentially says the connection was closed.
App 131002 output: /opt/cpanel/ea-ruby27/root/usr/share/passenger/helper-scripts/wsgi-loader.py:26: DeprecationWarning: the imp module is deprecated in favour of importlib and slated for removal in Python 3.12; see the module's documentation for alternative uses
App 131002 output: import sys, os, re, imp, threading, signal, traceback, socket, select, struct, logging, errno
App 131002 output: [ pid=131002, time=2023-11-02 17:07:56,254 ]: WSGI application raised an exception!
App 131002 output: Traceback (most recent call last):
App 131002 output: File "/home/db/virtualenv/dbsite/3.11/lib64/python3.11/site-packages/sqlalchemy/engine/base.py", line 145, in __init__
App 131002 output: self._dbapi_connection = engine.raw_connection()
App 131002 output: ^^^^^^^^^^^^^^^^^^^^^^^
App 131002 output: File "/home/db/virtualenv/dbsite/3.11/lib64/python3.11/site-packages/sqlalchemy/engine/base.py", line 3293, in raw_connection
App 131002 output: return self.pool.connect()
App 131002 output: ^^^^^^^^^^^^^^^^^^^
App 131002 output: File "/home/db/virtualenv/dbsite/3.11/lib64/python3.11/site-packages/sqlalchemy/pool/base.py", line 452, in connect
App 131002 output: return _ConnectionFairy._checkout(self)
App 131002 output: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
App 131002 output: File "/home/db/virtualenv/dbsite/3.11/lib64/python3.11/site-packages/sqlalchemy/pool/base.py", line 1268, in _checkout
App 131002 output: fairy = _ConnectionRecord.checkout(pool)
App 131002 output: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
App 131002 output: File "/home/db/virtualenv/dbsite/3.11/lib64/python3.11/site-packages/sqlalchemy/pool/base.py", line 716, in checkout
App 131002 output: rec = pool._do_get()
App 131002 output: ^^^^^^^^^^^^^^
App 131002 output: File "/home/db/virtualenv/dbsite/3.11/lib64/python3.11/site-packages/sqlalchemy/pool/impl.py", line 168, in _do_get
App 131002 output: with util.safe_reraise():
App 131002 output: File "/home/db/virtualenv/dbsite/3.11/lib64/python3.11/site-packages/sqlalchemy/util/langhelpers.py", line 147, in __exit__
App 131002 output: raise exc_value.with_traceback(exc_tb)
App 131002 output: File "/home/db/virtualenv/dbsite/3.11/lib64/python3.11/site-packages/sqlalchemy/pool/impl.py", line 166, in _do_get
App 131002 output: return self._create_connection()
App 131002 output: ^^^^^^^^^^^^^^^^^^^^^^^^^
App 131002 output: File "/home/db/virtualenv/dbsite/3.11/lib64/python3.11/site-packages/sqlalchemy/pool/base.py", line 393, in _create_connection
App 131002 output: return _ConnectionRecord(self)
App 131002 output: ^^^^^^^^^^^^^^^^^^^^^^^
App 131002 output: File "/home/db/virtualenv/dbsite/3.11/lib64/python3.11/site-packages/sqlalchemy/pool/base.py", line 678, in __init__
App 131002 output: self.__connect()
App 131002 output: File "/home/db/virtualenv/dbsite/3.11/lib64/python3.11/site-packages/sqlalchemy/pool/base.py", line 902, in __connect
App 131002 output: with util.safe_reraise():
App 131002 output: File "/home/db/virtualenv/dbsite/3.11/lib64/python3.11/site-packages/sqlalchemy/util/langhelpers.py", line 147, in __exit__
App 131002 output: raise exc_value.with_traceback(exc_tb)
App 131002 output: File "/home/db/virtualenv/dbsite/3.11/lib64/python3.11/site-packages/sqlalchemy/pool/base.py", line 898, in __connect
App 131002 output: self.dbapi_connection = connection = pool._invoke_creator(self)
App 131002 output: ^^^^^^^^^^^^^^^^^^^^^^^^^^
App 131002 output: File "/home/db/virtualenv/dbsite/3.11/lib64/python3.11/site-packages/sqlalchemy/engine/create.py", line 637, in connect
App 131002 output: return dialect.connect(*cargs, **cparams)
App 131002 output: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
App 131002 output: File "/home/db/virtualenv/dbsite/3.11/lib64/python3.11/site-packages/sqlalchemy/engine/default.py", line 616, in connect
App 131002 output: return self.loaded_dbapi.connect(*cargs, **cparams)
App 131002 output: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
App 131002 output: File "/home/db/virtualenv/dbsite/3.11/lib/python3.11/site-packages/psycopg/connection.py", line 728, in connect
App 131002 output: raise ex.with_traceback(None)
App 131002 output: psycopg.OperationalError: connection failed: server closed the connection unexpectedly
App 131002 output: This probably means the server terminated abnormally
App 131002 output: before or while processing the request.
App 131002 output: The above exception was the direct cause of the following exception:
App 131002 output: Traceback (most recent call last):
App 131002 output: File "/opt/cpanel/ea-ruby27/root/usr/share/passenger/helper-scripts/wsgi-loader.py", line 199, in main_loop
App 131002 output: socket_hijacked = self.process_request(env, input_stream, client)
App 131002 output: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
App 131002 output: File "/opt/cpanel/ea-ruby27/root/usr/share/passenger/helper-scripts/wsgi-loader.py", line 333, in process_request
App 131002 output: result = self.app(env, start_response)
App 131002 output: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
App 131002 output: File "/home/db/virtualenv/dbsite/3.11/lib/python3.11/site-packages/flask/app.py", line 2548, in __call__
App 131002 output: return self.wsgi_app(environ, start_response)
App 131002 output: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
App 131002 output: File "/home/db/virtualenv/dbsite/3.11/lib/python3.11/site-packages/flask/app.py", line 2528, in wsgi_app
App 131002 output: response = self.handle_exception(e)
App 131002 output: ^^^^^^^^^^^^^^^^^^^^^^^^
App 131002 output: File "/home/db/virtualenv/dbsite/3.11/lib/python3.11/site-packages/flask/app.py", line 2525, in wsgi_app
App 131002 output: response = self.full_dispatch_request()
App 131002 output: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
App 131002 output: File "/home/db/virtualenv/dbsite/3.11/lib/python3.11/site-packages/flask/app.py", line 1822, in full_dispatch_request
App 131002 output: rv = self.handle_user_exception(e)
App 131002 output: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
App 131002 output: File "/home/db/virtualenv/dbsite/3.11/lib/python3.11/site-packages/flask/app.py", line 1820, in full_dispatch_request
App 131002 output: rv = self.dispatch_request()
App 131002 output: ^^^^^^^^^^^^^^^^^^^^^^^
App 131002 output: File "/home/db/virtualenv/dbsite/3.11/lib/python3.11/site-packages/flask/app.py", line 1796, in dispatch_request
App 131002 output: return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)
App 131002 output: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
App 131002 output: File "/home/db/dbsite/web/templates/materials/bl_materials.py", line 23, in materials
App 131002 output: materials = get_materials()
App 131002 output: ^^^^^^^^^^^^^^^
App 131002 output: File "/home/db/dbsite/database/em_db.py", line 36, in get_materials
App 131002 output: materials_view_query = conn.scalars(select(Material)).all()
App 131002 output: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
App 131002 output: File "/home/db/virtualenv/dbsite/3.11/lib64/python3.11/site-packages/sqlalchemy/orm/session.py", line 2344, in scalars
App 131002 output: return self._execute_internal(
App 131002 output: ^^^^^^^^^^^^^^^^^^^^^^^
App 131002 output: File "/home/db/virtualenv/dbsite/3.11/lib64/python3.11/site-packages/sqlalchemy/orm/session.py", line 2117, in _execute_internal
App 131002 output: conn = self._connection_for_bind(bind)
App 131002 output: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
App 131002 output: File "/home/db/virtualenv/dbsite/3.11/lib64/python3.11/site-packages/sqlalchemy/orm/session.py", line 1984, in _connection_for_bind
App 131002 output: return trans._connection_for_bind(engine, execution_options)
App 131002 output: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
App 131002 output: File "<string>", line 2, in _connection_for_bind
App 131002 output: File "/home/db/virtualenv/dbsite/3.11/lib64/python3.11/site-packages/sqlalchemy/orm/state_changes.py", line 137, in _go
App 131002 output: ret_value = fn(self, *arg, **kw)
App 131002 output: ^^^^^^^^^^^^^^^^^^^^
App 131002 output: File "/home/db/virtualenv/dbsite/3.11/lib64/python3.11/site-packages/sqlalchemy/orm/session.py", line 1111, in _connection_for_bind
App 131002 output: conn = bind.connect()
App 131002 output: ^^^^^^^^^^^^^^
App 131002 output: File "/home/db/virtualenv/dbsite/3.11/lib64/python3.11/site-packages/sqlalchemy/engine/base.py", line 3269, in connect
App 131002 output: return self._connection_cls(self)
App 131002 output: ^^^^^^^^^^^^^^^^^^^^^^^^^^
App 131002 output: File "/home/db/virtualenv/dbsite/3.11/lib64/python3.11/site-packages/sqlalchemy/engine/base.py", line 147, in __init__
App 131002 output: Connection._handle_dbapi_exception_noconnection(
App 131002 output: File "/home/db/virtualenv/dbsite/3.11/lib64/python3.11/site-packages/sqlalchemy/engine/base.py", line 2431, in _handle_dbapi_exception_noconnection
App 131002 output: raise sqlalchemy_exception.with_traceback(exc_info[2]) from e
App 131002 output: File "/home/db/virtualenv/dbsite/3.11/lib64/python3.11/site-packages/sqlalchemy/engine/base.py", line 145, in __init__
App 131002 output: self._dbapi_connection = engine.raw_connection()
App 131002 output: ^^^^^^^^^^^^^^^^^^^^^^^
App 131002 output: File "/home/db/virtualenv/dbsite/3.11/lib64/python3.11/site-packages/sqlalchemy/engine/base.py", line 3293, in raw_connection
App 131002 output: return self.pool.connect()
App 131002 output: ^^^^^^^^^^^^^^^^^^^
App 131002 output: File "/home/db/virtualenv/dbsite/3.11/lib64/python3.11/site-packages/sqlalchemy/pool/base.py", line 452, in connect
App 131002 output: return _ConnectionFairy._checkout(self)
App 131002 output: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
App 131002 output: File "/home/db/virtualenv/dbsite/3.11/lib64/pyt
App 131002 output: hon3.11/site-packages/sqlalchemy/pool/base.py", line 1268, in _checkout
App 131002 output: fairy = _ConnectionRecord.checkout(pool)
App 131002 output: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
App 131002 output: File "/home/db/virtualenv/dbsite/3.11/lib64/python3.11/site-packages/sqlalchemy/pool/base.py", line 716, in checkout
App 131002 output: rec = pool._do_get()
App 131002 output: ^^^^^^^^^^^^^^
App 131002 output: File "/home/db/virtualenv/dbsite/3.11/lib64/python3.11/site-packages/sqlalchemy/pool/impl.py", line 168, in _do_get
App 131002 output: with util.safe_reraise():
App 131002 output: File "/home/db/virtualenv/dbsite/3.11/lib64/python3.11/site-packages/sqlalchemy/util/langhelpers.py", line 147, in __exit__
App 131002 output: raise exc_value.with_traceback(exc_tb)
App 131002 output: File "/home/db/virtualenv/dbsite/3.11/lib64/python3.11/site-packages/sqlalchemy/pool/impl.py", line 166, in _do_get
App 131002 output: return self._create_connection()
App 131002 output: ^^^^^^^^^^^^^^^^^^^^^^^^^
App 131002 output: File "/home/db/virtualenv/dbsite/3.11/lib64/python3.11/site-packages/sqlalchemy/pool/base.py", line 393, in _create_connection
App 131002 output: return _ConnectionRecord(self)
App 131002 output: ^^^^^^^^^^^^^^^^^^^^^^^
App 131002 output: File "/home/db/virtualenv/dbsite/3.11/lib64/python3.11/site-packages/sqlalchemy/pool/base.py", line 678, in __init__
App 131002 output: self.__connect()
App 131002 output: File "/home/db/virtualenv/dbsite/3.11/lib64/python3.11/site-packages/sqlalchemy/pool/base.py", line 902, in __connect
App 131002 output: with util.safe_reraise():
App 131002 output: File "/home/db/virtualenv/dbsite/3.11/lib64/python3.11/site-packages/sqlalchemy/util/langhelpers.py", line 147, in __exit__
App 131002 output: raise exc_value.with_traceback(exc_tb)
App 131002 output: File "/home/db/virtualenv/dbsite/3.11/lib64/python3.11/site-packages/sqlalchemy/pool/base.py", line 898, in __connect
App 131002 output: self.dbapi_connection = connection = pool._invoke_creator(self)
App 131002 output: ^^^^^^^^^^^^^^^^^^^^^^^^^^
App 131002 output: File "/home/db/virtualenv/dbsite/3.11/lib64/python3.11/site-packages/sqlalchemy/engine/create.py", line 637, in connect
App 131002 output: return dialect.connect(*cargs, **cparams)
App 131002 output: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
App 131002 output: File "/home/db/virtualenv/dbsite/3.11/lib64/python3.11/site-packages/sqlalchemy/engine/default.py", line 616, in connect
App 131002 output: return self.loaded_dbapi.connect(*cargs, **cparams)
App 131002 output: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
App 131002 output: File "/home/db/virtualenv/dbsite/3.11/lib/python3.11/site-packages/psycopg/connection.py", line 728, in connect
App 131002 output: raise ex.with_traceback(None)
App 131002 output: sqlalchemy.exc.OperationalError: (psycopg.OperationalError) connection failed: server closed the connection unexpectedly
App 131002 output: This probably means the server terminated abnormally
App 131002 output: before or while processing the request.
App 131002 output: (Background on this error at: https://sqlalche.me/e/20/e3q8)
I'm not a developer or even remotely familiar with cPanel or hosting but thought that if the flask app works on my desktop machine and I recreate the python environment, and upload the same code, then everything would work!
I assume the issue is related to the init_db function not being loaded by the webhost but don't really understand the problem fully.
Would be grateful if anyone could point me in the right direction to resolve this database connection issue.
I have figured this out. You need to use a virtual private server because access to the 5432 port essential for postgreSQL is probably being blocked.