djangopymongo

How to Reuse MongoClient Connections Efficiently in Django with PyMongo?


I'm working on a project using Django version 3.0.8 and PyMongo version 3.11.3. Currently, for each incoming request, a new MongoClient is created, and the connection is destroyed once the request is processed. For example:

def get_all_logs(request):
    req = json.loads(request.body.decode('utf-8'))
    record_id = str(req.get('record_id')).strip()
    with NewMongoClient(db='space').db() as db:
        log_info_with_record_id = list(db.database_alternative.find({'recordID': record_id}))

Here, NewMongoClient is a wrapper class around pymongo.MongoClient I think this approach incurs too much overhead due to the constant creation and destruction of connections. Is there a more efficient way to reuse MongoClient connections across requests?


Solution

  • Yea, creating new MongoClient for each request can be quite resource-intensive.

    You could use Singleton solution for creating MongoClient:

    # get_mongo_client.py
    
    from pymongo import NewMongoClient
    
    mongo_client = None
    
    def get_mongo_client():
        global mongo_client
        if mongo_client is None:
            mongo_client = NewMongoClient(db='space')
        return mongo_client
    

    After this, you could use this in your view.py:

    # views.py
    
    from .mongo_client import get_mongo_client
    
    def my_view(request):
        client = get_mongo_client()
        with client.db() as db:
            
        ...
    

    Thread-safe solution:

    from pymongo import MongoClient
    import threading
    
    mongo_client = None
    mongo_client_lock = threading.Lock()
    
    def get_mongo_client():
        global mongo_client
        if mongo_client is None:
            with mongo_client_lock:
                if mongo_client is None:
                    mongo_client = MongoClient('mongodb://localhost:123/')
        return mongo_client