Python Logging Patterns for Production Scripts

#python #logging #production #scripts

Introduction

Logging is an essential part of any production software. It helps in debugging, monitoring, and maintaining the software. Python’s built-in logging module provides a flexible and powerful logging framework. This article explains common Python logging patterns and best practices for production scripts.

Log Levels

Python's logging module defines log levels which determine the severity of the message. Common log levels include DEBUG, INFO, WARNING, ERROR, and CRITICAL. Each log level corresponds to a numeric value, with DEBUG being the lowest and CRITICAL being the highest.

import logging
logging.basicConfig(level=logging.INFO)

In this example, we set the log level to INFO, meaning only messages at or above INFO level will be captured.

Handlers and Formatters

Handlers control where and how logs are written. Common handlers include StreamHandler for console output, FileHandler for file output, and SMTPHandler for sending emails. Each handler can have its own formatter, which specifies the format of the log message. Here’s an example of a custom handler with a formatter:

from logging import Handler, Formatter

class CustomHandler(Handler):
    def __init__(self, level=logging.INFO):
        super().__init__(level)

    def emit(self, record):
        # Implement custom emit logic
        pass

    def format(self, record):
        return f'{record.levelname}: {record.getMessage()}'

Common Logging Patterns

Structured Logging: Use structured data like dictionaries to provide context and meaning to log messages. This pattern is especially useful for traceability and querying.

import logging

logging.basicConfig(level=logging.INFO)
logger = logging.getLogger(__name__)

data = {'user_id': 123, 'action': 'view', 'timestamp': '2023-01-01 12:00:00'}
logger.info('User viewed page', extra=data)

Rotating File Handlers: Use RotatingFileHandler to limit log file size or number of files. This ensures logs don’t consume excessive disk space.

import logging
from logging.handlers import RotatingFileHandler

handler = RotatingFileHandler('app.log', maxBytes=1000000, backupCount=5)
handler.setLevel(logging.INFO)
formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')
handler.setFormatter(formatter)
logging.getLogger(__name__).addHandler(handler)

Logging Exceptions: Use the logging.exception() method to log exceptions without causing the program to crash. This helps in tracking down issues.

import logging

logging.basicConfig(level=logging.INFO)
logger = logging.getLogger(__name__)

try:
    # Code that may raise an exception
    1 / 0
except Exception as e:
    logger.exception('An error occurred')

Common Pitfalls

Ignoring Logs: Don’t ignore logs by not configuring handlers. Logs are a crucial part of the debugging process and must be written somewhere.
Over-Logging: Avoid over-logging by using appropriate log levels and avoiding unnecessary logging. Log only what is needed for debugging and monitoring.
Incorrect Handlers: Ensure handlers are correctly set up and configured. Mixing up handlers or not setting up a handler at all can lead to logs being ignored or incorrectly formatted.

Conclusion

Python’s logging module offers a robust framework for logging. By understanding and utilizing log levels, handlers, and formatters, developers can create a reliable logging system that aids in debugging, monitoring, and maintenance. Structured logging and rotating handlers are particularly useful for traceability and resource management. Avoiding common pitfalls such as ignoring logs and over-logging will lead to more efficient and effective logging practices.

DEV Community

Python Logging Patterns for Production Scripts

Top comments (0)