I have added log messages in my Django application and it was successfully logging log messages to the log file.
Now, I tried to add log messages to AWS CloudWatch. When I run the application it creates log group
in AWS CloudWatch but log stream
is not created and log messages also not uploaded.
I have also manually created log stream
in AWS CloudWatch but still log messages were not uploaded.
I have the following logging configuration in my Django application.
logger_boto3_client = boto3.client(
"logs",
aws_access_key_id=CLOUDWATCH_AWS_ID,
aws_secret_access_key=CLOUDWATCH_AWS_KEY,
region_name=AWS_DEFAULT_REGION,
)
LOGGING = {
'version': 1,
'disable_existing_loggers': False,
'formatters': {
'simple': {
'format': "[cid: %(cid)s] [%(asctime)s.%(msecs)03d] %(levelname)s [%(name)s:%(lineno)s] [%(funcName)s] %(message)s",
'datefmt': '%Y-%m-%d %H:%M:%S',
},
},
'handlers': {
'logger': {
'level': 'DEBUG',
'class': 'logging.handlers.RotatingFileHandler',
'filename': str(BASE_DIR) + '/logs/test.log',
'formatter': 'simple',
'filters': ['correlation'],
},
'watchtower': {
"level": "DEBUG",
"class": "watchtower.CloudWatchLogHandler",
"boto3_client": logger_boto3_client,
"log_group": "DemoLogs2",
# Different stream for each environment
"stream_name": "logs",
"formatter": "simple",
'filters': ['correlation'],
}
},
'filters': {
'correlation': {
'()': 'cid.log.CidContextFilter'
},
},
'loggers': {
'root': {
'handlers': ['logger', 'watchtower'],
'level': 'DEBUG',
'filters': ['correlation'],
'propagate': True,
}
}
}
In my application I'm using logger like this,
import logging
logger = logging.getLogger(__name__)
logger.info("log message.")
My aws cloudwatch
console.
The issue is with calling the logger
in the API.
LOGGING = {
'version': 1,
'disable_existing_loggers': False,
'formatters': {
'default': {
'format': "[cid: %(cid)s] [%(asctime)s.%(msecs)03d] %(levelname)s [%(name)s:%(lineno)s] [%(funcName)s] %(message)s",
'datefmt': '%Y-%m-%d %H:%M:%S',
}
},
'handlers': {
'file': {
'level': 'DEBUG',
'class': 'logging.handlers.RotatingFileHandler',
'filename': str(BASE_DIR) + '/logs/test.log',
'formatter': 'default',
'filters': ['correlation'],
}
'cloudwatch': {
"level": "DEBUG",
"class": "watchtower.CloudWatchLogHandler",
"boto3_client": logger_boto3_client,
"log_group": CLOUDWATCH_LOG_GROUP,
"stream_name": f"{ENV}-{CLOUDWATCH_DEFAULT_LOG_STREAM_NAME}",
"formatter": "default",
'filters': ['correlation'],
}
},
'filters': {
'correlation': {
'()': 'cid.log.CidContextFilter'
},
},
'loggers': {
# previously named as 'root'
'default': {
'level': 'DEBUG',
'handlers': ['file', 'cloudwatch'],
'filters': ['correlation'],
'propagate': False,
},
}
}
In the API,
import logging
# logger = logging.getLogger(__name__)
logger = logging.getLogger("default")
logger.info("log message.")
Previously I was passing __name__
to the getLogger
method. Now, I passed the logger name default
to the getLogger
method and now I could able to see the log messages in both the log
file and cloudwatch
.