pythondjangodatabaselogging

How do I override django.db.backends logging to work when DEBUG=False?


In django's LOGGING configuration for the builtin django.db.backends it states that: "For performance reasons, SQL logging is only enabled when settings.DEBUG is set to True, regardless of the logging level or handlers that are installed."

As a result the following LOGGING configuration, which is correctly set up to issue debug level logs showing DB queries, will NOT output the messages I need:


DEBUG = False
LOGGING = {
    "version": 1,
    "disable_existing_loggers": True,
    "root": {"handlers": [ "gcp_structured_logging"]},
    "handlers": {
        "gcp_structured_logging": {
            "level": "DEBUG",
            "class": "django_gcp.logging.GoogleStructuredLogsHandler",
        }
    },
    "loggers": {
        'django.db.backends': {
            'handlers': ["gcp_structured_logging"],
            'level': 'DEBUG',
            'propagate': True,
        },
    },
}

This is preventing me from activating this logging in production, where of course I'm not going to durn on DEBUG=True in my settings but where I need to log exactly this information.

Ironically, I need this in order to debug a performance issue (I plan to run this for a short time in production and cat my logs so I can set up a realistic scenario for a load test and some benchmarking on the database).

How can I override django's override so that sql queries get logged as I intend?

NOTE willeM_ Van Onsem's answer as accepted is correct (because it totally is) but it's worth noting that in the end, I came across a library called django-silk. Whilst not an answer to this question directly, silk actually covers the capability I was trying to build for myself when I found this peculiarity of how the db logging works. Perhaps someone else trying to achieve the same thing will make good use of it.


Solution

  • Fortunately we can override this. Indeed, by setting the .force_debug_cursor of the connection to True, for example in one of the AppConfigs (any app config) exists:

    # my_app/apps.py
    
    from django.apps import AppConfig
    from django.db import connection
    
    
    class MyAppConfig(AppConfig):
        name = 'my_app'
    
        def ready(self):
            connection.force_debug_cursor = True

    This works because Django decides whether to log with [GitHub]:

    @property
    def queries_logged(self):
        return self.force_debug_cursor or settings.DEBUG
    

    Another option is to work with the CaptureQueriesContext [GitHub], this is a context processor that sets .force_debug_cursor to true in the scope of the context. So if you know where to look for the slow query, you can use:

    from django.db import connection
    from django.test.utils import 
    
    with CaptureQueriesContext(connection):
        list(MyModel.objects.all())  # sample query