Python: Logging from multiple modules

Hi, In one of my tasks at work I was designing a system where there would be one main module of my application and the application had several modules all of which did their part to make the application record success or failure.

However, at the time I did not know how to have python log the generated messages from multiple modules to the same logger object that was initiated in the main module. Hence this post.

Let us say your application looks like the below

The main application file should look as below:

import logging
#Function to initialize the logger, notice that it takes 2 arguments
#logger_name and logfile
def setup_logger(logger_name,logfile):
                logger = logging.getLogger(logger_name)
                logger.setLevel(logging.INFO)
                # create file handler which logs even debug messages
                fh = logging.FileHandler(logfile)
                fh.setLevel(logging.INFO)
                # create console handler with a higher log level
                ch = logging.StreamHandler()
                ch.setLevel(logging.INFO)
                # create formatter and add it to the handlers
                formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')
                fh.setFormatter(formatter)
                ch.setFormatter(formatter)
                # add the handlers to the logger
                logger.addHandler(fh)
                logger.addHandler(ch)
                return logger
#setup the logger as below with mylogger being the name of the #logger and myloggerfile.log being the name of the logfile
mylogger=setup_logger('mylogger','myloggerfile.log')
mylogger.info("My Logger has been initialized")
##########Very important to import the module after the logger has #been created, if the module is imported before setting up the #logger the module wont log to the desired file
import mymodule
#Then you can continue the rest of the code.

The module file should look like below:

import logging
hostlogger = logging.getLogger('mylogger.module')
hostlogger.info("Logger has been initialised in the host module")

This should help you get logs from the modules of your applications into a single common file as below:

# cat myloggerfile.log
2022-05-05 17:06:10,024 - mylogger - INFO - My Logger has been initialized
2022-05-05 17:06:10,028 - mylogger.module - INFO - Logger has been initialised in the host module

Notice the name of the logger object, the message with the tag mylogger is coming from you main application and the message mylogger.module is coming from the module

This can also be tweaked to log to multiple files and have multiple logger objects created in the main application file and the modules can refer to them accordingly.