I recently read about structured logging (here). The idea seems to be to log not by appending simple strings as a line to a logfile, but instead JSON objects. This makes it possible to analyze the logfile by automatic tools.
Can Pythons logging
library do structured logging? If not, is there a "mainstream" solution for it (e.g. like numpy/scipy is the mainstream solution for scientific calculations)? I found structlog
, but I'm not sure how widespread it is.
If you install python-json-logger
and have a logging configuration (YAML) like the following, you will get a structured logging file.
version: 1
formatters:
detailed:
class: logging.Formatter
format: '[%(asctime)s]:[%(levelname)s]: %(message)s'
json:
class: pythonjsonlogger.jsonlogger.JsonFormatter
format: '%(asctime)s %(levelname)s %(message)s'
handlers:
console:
class: logging.StreamHandler
level: INFO
formatter: detailed
file:
class: logging.FileHandler
filename: logfile.log
level: DEBUG
formatter: json
root:
level: DEBUG
handlers:
- console
- file
You might also want to make exceptions / tracebacks use the structured format.
See Can I make Python output exceptions in one line / via logging?