Simple Python and Django logging

Introduction

In this article, discuss logging in Python, specifically in the context of Django projects. Logging is essential for understanding the behavior of your application, diagnose problems, and monitor performance. More verbose logging makes it easier to pinpoint problems, but on the other hand, logging everything can lead to so manny log information that it is harder to find actual problems. The right amount of logging is different for every app and developer, but it is almost always more than "none".

In Python, the logging module is a standard way to implement logging functionality. The common pattern for using the logging module is by importing the getLogger function and creating a logger object with the module's name:

from logging import getLogger

log = getLogger(__name__)

log.info("Something interesting happened")

The logging module provides a flexible and powerful framework for capturing log messages in your applications. You can log messages with different levels of severity, such as DEBUG, INFO, WARNING, ERROR, and CRITICAL, depending on the nature of the message. Using the getLogger function, you can create a logger object for each Python module, allowing you to then specify logging level and different handlers for each module.

You can also use the logger functions directly on the logging module. This uses the root logger instead of creating a new one for each module:

import logging
logging.info("Something interesting happened")

This is useful for small scripts, but creating per-module logger is recommended for non-trivial Python applications because it makes it easier to show or handle logs differently based on where they originated.

Basic configuration for Python logging

Basic configuration that just outputs all logs above certain severity level can be setup using basicConfig:

from logging import getLogger, basicConfig, INFO

basicConfig(level=INFO)
log = getLogger(__name__)

# won't show up because DEBUG is less severe than INFO
log.debug("Not terribly interesting")  
# this will be shown in the log
log.info("Something interesting happened")

basicConfig has more options such as customizing the log format or configuring where the logs will be output (if not to console).

Logging exception details

The logger methods support including exception information in the log message, making it easy to provide stack traces helpful for diagnosing issues in your application. When logging a message, you can set the exc_info parameter to True to automatically include the exception information in the log message, like in this example:

import logging
log = getLogger(__name__)
try:
    result = 1 / 0
except ZeroDivisionError:
    logging.error("An error occurred while dividing by zero", exc_info=True)

Simple logging in Django

Django provides a default logging configuration out of the box. It outputs INFO or more severe logs to the console only if DEBUG is set to True . Otherwise, it only sends email to admins on server errors.

In many modern developments, it's preferrable to log everything interesting to console and leave it up to the platform (such as Docker, Systemd or Kubernetes) to take care of gathering and storing the logs. The default configuration can easily be customized by modifying the LOGGING dictionary in the Django settings file.

First, let's create a LOG_LEVEL variable that pulls the desired log level from an environment variable:

import os
import logging

LOG_LEVEL = os.environ.get("LOG_LEVEL", logging.INFO)

Next, update the LOGGING dictionary to output log messages to the console (only showing messages with severity equal to or higher than our configured LOG_LEVEL):

LOGGING = {
    "version": 1,
    # This will leave the default Django logging behavior in place
    "disable_existing_loggers": False,
    # Custom handler config that gets log messages and outputs them to console
    "handlers": {
        "console": {
            "class": "logging.StreamHandler",
            "level": LOG_LEVEL,
        },
    },
    "loggers": {
        # Send everything to console
        "": {
            "handlers": ["console"],
            "level": LOG_LEVEL,
        },
    },
}

Disabling loggers

Sometimes it's useful to disable some loggers. An example in Django is silencing the log error message when the site visitor uses an incorrect host name (ie. not among the ones whitelisted in ALLOWED_HOSTS Django setting). While in debugging this might be useful, it is often an annoyance once you deploy to production. Since the public web is full with automated crawlers checking for vulnerabilities, you'll get a ton of these as soon as you set up a public-facing web app.

Here's a modified logging configuration that works the same as before but ignores these messages:

LOGGING = {
    "version": 1,
    "disable_existing_loggers": False,
    "handlers": {
        "console": {"class": "logging.StreamHandler"},
        # A null handler ignores the mssage
        "null": {"level": "DEBUG", "class": "logging.NullHandler"},
    },
    "loggers": {
        "": {
            "handlers": ["console"],
            "level": ENV_STR("LOG_LEVEL", "INFO"),
        },
        "django.security.DisallowedHost": {
            # Redirect these messages to null handler
            "handlers": ["null"],
            # Don't let them reach the root-level handler
            "propagate": False,
        },
    },
}

Further reading

This simple configuration can be useful for many Django projects, but it is really just scratching the surface in terms of capability of Python logging module and the Django integration.

If you'd like to explore more, here are a few links with more in-depth information:

• Real Python's tutorial on Python logging
• Python logging documentation
• Django logging documentation