I want to set up a logger in Databricks that does two things everytime its called:
print()
command (eg "This is a log")notebook_name_{date}.log
with more granular info (eg "2024-09-12 12:51:49,694 - main - INFO - This is a log")The goal would be to just call this once, similar to logger.info('log')
, so if it needs to be wrapped in a function that is fine.
This is my code so far but it prints the timestamp, name, etc in the cell output which I don't want:
import logging
from datetime import datetime
def get_current_date():
# Get the current date and time
now = datetime.now()
# Format the date as DD Uppercase Three Letter Month YEAR
formatted_date = now.strftime("%d%b%Y").upper()
return formatted_date
log_file = f'logfile_{formatted_date}.log'
# Create a logger
logger = logging.getLogger(__name__)
logger.setLevel(logging.DEBUG)
# Create a file handler
file_handler = logging.FileHandler(log_file)
# Create a stream handler
stream_handler = logging.StreamHandler()
# Create a formatter and add it to the handlers
formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')
file_handler.setFormatter(formatter)
stream_handler.setFormatter(formatter)
# Add the handlers to the logger
logger.addHandler(file_handler)
logger.addHandler(stream_handler)
# Test the logger
logger.info('This is a test!!!')
logger.info('this is a test again')
//etc in the cell output which I don't want://
Create another formatter variable for Streamingoutput.
sformatter = logging.Formatter('%(message)s')
stream_handler.setFormatter(sformatter)
This way it will display only the message in Databricks and Log file will have more details.