pythonloggingpython-loggingstructured-logging

Can structured logging be done with Pythons standard library?


I recently read about structured logging (here). The idea seems to be to log not by appending simple strings as a line to a logfile, but instead JSON objects. This makes it possible to analyze the logfile by automatic tools.

Can Pythons logging library do structured logging? If not, is there a "mainstream" solution for it (e.g. like numpy/scipy is the mainstream solution for scientific calculations)? I found structlog, but I'm not sure how widespread it is.


Solution

  • If you install python-json-logger and have a logging configuration (YAML) like the following, you will get a structured logging file.

    version: 1
    formatters:
        detailed:
            class: logging.Formatter
            format: '[%(asctime)s]:[%(levelname)s]: %(message)s'
        json:
            class: pythonjsonlogger.jsonlogger.JsonFormatter
            format: '%(asctime)s %(levelname)s %(message)s'
    handlers:
        console:
            class: logging.StreamHandler
            level: INFO
            formatter: detailed
        file:
            class: logging.FileHandler
            filename: logfile.log
            level: DEBUG
            formatter: json
    root:
        level: DEBUG
        handlers:
            - console
            - file
    

    Exceptions

    You might also want to make exceptions / tracebacks use the structured format.

    See Can I make Python output exceptions in one line / via logging?