How to Use Python's logging Module Like a Pro — From Beginner to Production Setup

How to Use Python's logging Module Like a Pro — From Beginner to Production Setup

Python's built-in logging module is one of those tools every developer knows about, but few use well. When I started, I used print() everywhere. When I moved to production apps, that approach didn't scale.

Here's what I learned about logging in Python, from basic setup to production-ready patterns.

The Minimum Viable Logger

Instead of print(), start with this:

import logging

logging.basicConfig(level=logging.INFO)
logging.info("Application started")
logging.error("Something went wrong: %s", err)

That's it. You now have timestamps, log levels, and proper output formatting. The %s formatting is intentional — it delays string interpolation until the message is actually logged.

Why Levels Matter

Most beginners use logging.info() for everything. Here's a better mental model:

• DEBUG (10): Detailed information for diagnosing problems. Ship with this off.
• INFO (20): Confirmation that things are working as expected.
• WARNING (30): Something unexpected happened, but the app still works.
• ERROR (40): The software couldn't perform a function.
• CRITICAL (50): A serious error that may prevent the program from continuing.

Set your default level to INFO in production, DEBUG during development.

Rotating File Handlers

If you log to a file, you'll eventually run out of disk space. Use RotatingFileHandler:

from logging.handlers import RotatingFileHandler

handler = RotatingFileHandler(
    'app.log', maxBytes=10_000_000, backupCount=5
)
logging.basicConfig(
    level=logging.INFO,
    handlers=[handler],
    format='%(asctime)s - %(name)s - %(levelname)s - %(message)s'
)

This keeps 5 backup files of 10MB each. When the current log fills up, it rotates automatically.

Structured Logging with JSON

For production systems, especially if you use log aggregation tools, JSON logs are superior:

import json

class JSONFormatter(logging.Formatter):
    def format(self, record):
        log_entry = {
            "timestamp": self.formatTime(record),
            "level": record.levelname,
            "logger": record.name,
            "message": record.getMessage(),
        }
        if record.exc_info and record.exc_info[0]:
            log_entry["exception"] = self.formatException(record.exc_info)
        return json.dumps(log_entry)

handler = logging.StreamHandler()
handler.setFormatter(JSONFormatter())
logging.basicConfig(level=logging.INFO, handlers=[handler])

Now your logs parse cleanly in tools like Logstash, Datadog, or Grafana.

Logging Decorators for Function Tracing

Here's a pattern I use for debugging complex flows:

import logging
from functools import wraps

logger = logging.getLogger(__name__)

def log_call(func):
    @wraps(func)
    def wrapper(*args, **kwargs):
        logger.debug("Entering %s", func.__name__)
        try:
            result = func(*args, **kwargs)
            logger.debug("Exiting %s", func.__name__)
            return result
        except Exception as e:
            logger.exception("Error in %s: %s", func.__name__, e)
            raise
    return wrapper

@log_call
def process_order(order_id: str) -> bool:
    # Your business logic here
    return True

The Logger Object Pattern

For larger applications, create loggers per module:

# In each module
logger = logging.getLogger(__name__)

# This automatically gives you module-qualified logger names
# like "myapp.services.payment" instead of flat "myapp"

Context Filters for Request Tracing

When debugging, you often need to trace a single request across multiple modules:

import threading

class RequestContextFilter(logging.Filter):
    """Add request_id to every log record."""

    _context = threading.local()

    @classmethod
    def set_request_id(cls, request_id):
        cls._context.request_id = request_id

    def filter(self, record):
        record.request_id = getattr(
            self._context, 'request_id', 'N/A'
        )
        return True

# Usage
handler = logging.StreamHandler()
handler.addFilter(RequestContextFilter())
logging.basicConfig(level=logging.INFO, handlers=[handler])
logging.info("Request started")  # Includes request_id field

What I Actually Use in Production

Here's my production logging setup, simplified:

import logging
import sys
from logging.handlers import RotatingFileHandler

def setup_logging(env: str = "development"):
    log_format = (
        "%(asctime)s | %(levelname)-8s | %(name)s | "
        "%(message)s"
    )

    root_logger = logging.getLogger()
    root_logger.setLevel(logging.DEBUG if env == "development" else logging.INFO)

    # Console output
    console = logging.StreamHandler(sys.stdout)
    console.setFormatter(logging.Formatter(log_format))
    root_logger.addHandler(console)

    # File output with rotation
    if env != "development":
        file_handler = RotatingFileHandler(
            "logs/app.log", maxBytes=10_000_000, backupCount=5
        )
        file_handler.setFormatter(logging.Formatter(
            '%(asctime)s - %(name)s - %(levelname)s - %(message)s'
        ))
        root_logger.addHandler(file_handler)

    # Suppress noisy third-party logs
    logging.getLogger("urllib3").setLevel(logging.WARNING)
    logging.getLogger("httpx").setLevel(logging.WARNING)

Quick Tips

1. Never use logging.getLogger() without __name__ in modules — you lose context.
2. Use logging.exception() in except blocks — it includes the full traceback.
3. Don't format strings manually — let the logger do it: logging.info("User %s logged in", user.id) (faster, and avoids work if the log level is suppressed).
4. Add a startup log — log your app version, config path, and environment on startup. Saves hours of debugging.

---

I use these patterns in my own automation scripts. If you're building Python automation tools and want ready-to-run scripts (file organizers, log analyzers, data pipelines), check out my Python Automation Scripts Pack — 50 copy-paste ready scripts with proper logging already built in.

What's your go-to logging pattern? Drop a comment below.