Using multiple gunicorn workers cause the error with status PGRES_TUPLES_OK and no message from the libpq

I have a Flask website that uses the peewee ORM. The connection is managed by FlaskDB.

When I only use 1 gunicorn worker, it works well. But as soon as I use 2 or more gunicorn workers, I start getting this error:

Traceback (most recent call last):
  File "/home/user/.local/lib/python3.10/site-packages/flask/app.py", line 2073, in wsgi_app
    response = self.full_dispatch_request()
  File "/home/user/.local/lib/python3.10/site-packages/flask/app.py", line 1519, in full_dispatch_request
    return self.finalize_request(rv)
  File "/home/user/.local/lib/python3.10/site-packages/flask/app.py", line 1540, in finalize_request
    response = self.process_response(response)
  File "/home/user/.local/lib/python3.10/site-packages/flask/app.py", line 1888, in process_response
    self.session_interface.save_session(self, ctx.session, response)
  File "/home/user/project/session.py", line 113, in save_session
    saved_session.save()
  File "/home/user/.local/lib/python3.10/site-packages/peewee.py", line 6497, in save
    rows = self.update(**field_dict).where(self._pk_expr()).execute()
  File "/home/user/.local/lib/python3.10/site-packages/peewee.py", line 1886, in inner
    return method(self, database, *args, **kwargs)
  File "/home/user/.local/lib/python3.10/site-packages/peewee.py", line 1957, in execute
    return self._execute(database)
  File "/home/user/.local/lib/python3.10/site-packages/peewee.py", line 2442, in _execute
    cursor = database.execute(self)
  File "/home/user/.local/lib/python3.10/site-packages/peewee.py", line 3112, in execute
    return self.execute_sql(sql, params, commit=commit)
  File "/home/user/.local/lib/python3.10/site-packages/peewee.py", line 3096, in execute_sql
    with __exception_wrapper__:
  File "/home/user/.local/lib/python3.10/site-packages/peewee.py", line 2873, in __exit__
    reraise(new_type, new_type(exc_value, *exc_args), traceback)
  File "/home/user/.local/lib/python3.10/site-packages/peewee.py", line 183, in reraise
    raise value.with_traceback(tb)
  File "/home/user/.local/lib/python3.10/site-packages/peewee.py", line 3099, in execute_sql
    cursor.execute(sql, params or ())
peewee.DatabaseError: error with status PGRES_TUPLES_OK and no message from the libpq

The peewee documentation states that the connection should be thread-safe, but it seems there are thread-safety issues here.

I use playhouse.pool.PooledPostgresqlDatabase if that matters.

What is the solution to this problem?

Solution

I believe this is possibly due to the connection being opened before the workers are forked. I'm not too familiar w/gunicorns worker model, but googling your error reveals similar problems w/multiprocessing. Specifically,

When a program uses multiprocessing or fork(), and an Engine object is copied to the child process, Engine.dispose() should be called so that the engine creates brand new database connections local to that fork. Database connections generally do not travel across process boundaries.

That is for SQLAlchemy, but the same should apply to Peewee. Just ensure that all your connections are closed (pool.close_all()) when your workers first start running. Or similarly, if you're opening any database connections at module scope, ensure you call close_all() after using them. This way when your workers start up they will each have an empty pool of connections.

db = FlaskDB(app)
# ...
db.database.close_all()