pythonflasksqlalchemyflask-sqlalchemy

Even if SQLALCHEMY_POOL_RECYCLE is set, Flask-SQLAlchemy will not automatically recycle MySQL connections


Recently, my flask program encountered the error of Lost connection to MySQL server during query. I searched some documents and discussions. It seems that MySQL timed out and disconnected the connection, but flask did not recycle these expired connections.
I am currently using Flask_SQLAlchemy==2.4.4. From the documentation of flask 2.x, I learned that setting SQLALCHEMY_POOL_RECYCLE can adjust the time for flask to recycle expired connections. I tried setting it but it did not solve my problem.
This is my demo:

import time

import sqlalchemy as sql
from flask import Flask
from flask_sqlalchemy import SQLAlchemy

db = SQLAlchemy()


class TestData(db.Model):
    id = sql.Column(sql.BigInteger, primary_key=True)

def create_app() -> Flask:
    app: Flask = Flask(__name__)
    app.config["SQLALCHEMY_DATABASE_URI"] = "mysql+pymysql://mysql_host"
    app.config["SQLALCHEMY_POOL_SIZE"] = 1
    app.config["SQLALCHEMY_POOL_RECYCLE"] = 3
    db.init_app(app)
    return app


def test():
    app = create_app()
    with app.app_context():
        q1 = TestData.query.get(0)
        print(q1)
        time.sleep(10)
        q2 = TestData.query.get(1)
        print(q2)

test()

I ran SET GLOBAL wait_timeout = 8; to set the MySQL disconnection time to 8s, and also configured app.config["SQLALCHEMY_POOL_RECYCLE"] = 3. After time.sleep(10), the query of q2 still failed.

package version

SQLAlchemy==1.3.22
Flask_SQLAlchemy==2.4.4
Flask==1.1.2

Solution

  • From the docs on Setting Pool Recycle:

    Note that the invalidation only occurs during checkout - not on any connections that are held in a checked out state.

    Your first .get() checks out a connection, begins a transaction, and issues a SELECT query. Since you don't complete the transaction the connection remains "open" (held in a checked out state), and after 10 seconds of inactivity you try using that same connection again.

    The fix is to complete the transaction after the first .get() so the connection gets returned to the pool. After 10 seconds your next .get() will check out the connection and it will be recycled.

            q1 = TestData.query.get(0)
            print(q1)
            db.session.rollback()  # complete the transaction
            time.sleep(10)
            q2 = TestData.query.get(1)
            print(q2)