Python Logging Best Practices: From print() to Production-Ready Logging

Every Python developer starts with print() for debugging. It works fine... until your application grows, runs in production, and you find yourself scrolling through terminal output wondering what went wrong three hours ago.

The built-in logging module is Python's answer to this problem, and it's far more powerful than most developers realize. Let's walk through practical logging patterns that scale from a single script to a distributed application.

Why Not Just Use print()?

print() has three fundamental limitations:

1. No levels — you can't distinguish between debug info, warnings, and errors
2. No configuration — you can't redirect output without modifying code
3. No context — no timestamps, module names, or line numbers unless you manually add them

The logging module solves all of these, and it's already in the standard library — zero dependencies required.

Basic Setup That Actually Works

Here's a production-ready logger configuration that you can drop into any project:

import logging
import sys
from pathlib import Path

def setup_logger(
    name: str = __name__,
    log_file: str | None = None,
    level: int = logging.INFO,
) -> logging.Logger:
    """Configure and return a logger with consistent formatting."""
    logger = logging.getLogger(name)
    logger.setLevel(level)

    # Avoid duplicate handlers if called multiple times
    if logger.handlers:
        return logger

    # Console handler with simple format
    console_handler = logging.StreamHandler(sys.stdout)
    console_handler.setLevel(level)
    console_format = logging.Formatter(
        "[%(levelname)s] %(message)s"
    )
    console_handler.setFormatter(console_format)
    logger.addHandler(console_handler)

    # File handler with detailed format (if log_file is provided)
    if log_file:
        Path(log_file).parent.mkdir(parents=True, exist_ok=True)
        file_handler = logging.FileHandler(log_file)
        file_handler.setLevel(logging.DEBUG)
        file_format = logging.Formatter(
            "%(asctime)s | %(name)s | %(levelname)-8s | %(filename)s:%(lineno)d | %(message)s",
            datefmt="%Y-%m-%d %H:%M:%S"
        )
        file_handler.setFormatter(file_format)
        logger.addHandler(file_handler)

    return logger

# Usage
log = setup_logger("my_app", log_file="logs/app.log")
log.info("Application started")
log.debug("This only goes to the file")

The Five Log Levels and When to Use Them

Level	Numeric Value	When to Use
DEBUG	10	Detailed diagnostic info during development
INFO	20	Confirmation that things are working as expected
WARNING	30	Something unexpected happened, but the app can continue
ERROR	40	A serious problem; a specific function failed
CRITICAL	50	The application cannot continue running

A common pattern is to log at DEBUG during development and INFO in production, so you don't overwhelm your production logs:

import os

level = logging.DEBUG if os.getenv("DEBUG") else logging.INFO
log = setup_logger("app", level=level)

Structured Logging with Extra Context

When debugging production issues, raw log messages are rarely enough. You need context — request IDs, user IDs, transaction numbers. Python's logging supports this through the extra parameter and custom formatters.

import logging
import json

class JSONFormatter(logging.Formatter):
    """Format log records as JSON for machine parsing."""
    def format(self, record: logging.LogRecord) -> str:
        log_entry = {
            "timestamp": self.formatTime(record),
            "level": record.levelname,
            "logger": record.name,
            "message": record.getMessage(),
        }
        # Include any extra context passed to the logger
        if hasattr(record, "extra_fields"):
            log_entry.update(record.extra_fields)
        return json.dumps(log_entry)

logger = logging.getLogger("api")
handler = logging.StreamHandler()
handler.setFormatter(JSONFormatter())
logger.addHandler(handler)
logger.setLevel(logging.INFO)

# Using extra context
logger.info("Request completed", extra={
    "extra_fields": {
        "method": "POST",
        "endpoint": "/api/orders",
        "status_code": 201,
        "duration_ms": 145,
        "user_id": "u_8a3f2c"
    }
})

Output:

{"timestamp": "2025-06-25 10:30:15", "level": "INFO", "logger": "api", "message": "Request completed", "method": "POST", "endpoint": "/api/orders", "status_code": 201, "duration_ms": 145, "user_id": "u_8a3f2c"}

This is the foundation of centralized logging systems like the ELK stack or Datadog.

Rotating File Handlers — Don't Let Logs Eat Your Disk

Production logs grow fast. Without rotation, a single verbose application can fill a disk in days. Python provides RotatingFileHandler and TimedRotatingFileHandler out of the box:

from logging.handlers import RotatingFileHandler, TimedRotatingFileHandler

# Rotate by size: 10 MB per file, keep 5 backups
size_handler = RotatingFileHandler(
    "app.log", maxBytes=10_485_760, backupCount=5
)

# Rotate by time: new file at midnight, keep 30 days
time_handler = TimedRotatingFileHandler(
    "app.log", when="midnight", interval=1, backupCount=30
)

A practical pattern: use size-based rotation for development, time-based for production where log retention policies matter.

The Logger Hierarchy — Don't Create Loggers Ad Hoc

Python loggers follow a hierarchical namespace based on dots. A logger named myapp.api.orders is a child of myapp.api, which is a child of myapp. This means:

• You can set a level on myapp and all children inherit it
• You can silence myapp.api while keeping myapp.api.orders active
• Log messages propagate up the hierarchy by default

The best practice is to create one logger per module using __name__:

# In myapp/api/orders.py
import logging

logger = logging.getLogger(__name__)  # Gets "myapp.api.orders"

def create_order(data):
    logger.info("Creating order: %s", data["order_id"])
    # ...

Then in your main entry point, configure once:

# main.py
import logging

logging.basicConfig(
    level=logging.INFO,
    format="%(asctime)s | %(name)s | %(levelname)s | %(message)s"
)

# Silence third-party libraries if they're too verbose
logging.getLogger("urllib3").setLevel(logging.WARNING)
logging.getLogger("boto3").setLevel(logging.WARNING)

Exception Logging Without Losing the Traceback

One of the most common mistakes is logging exceptions without the full traceback:

# BAD — loses the traceback
try:
    result = risky_operation()
except Exception as e:
    logger.error(f"Operation failed: {e}")

# GOOD — preserves the full traceback
try:
    result = risky_operation()
except Exception:
    logger.exception("Operation failed")

The logger.exception() method automatically includes the stack trace at ERROR level. For other levels, use exc_info=True:

logger.critical("Database unreachable", exc_info=True)

Practical Pattern: Request Logging Middleware

Here's a complete, production-ready example for a web application (works with FastAPI, Flask, or similar):

import logging
import time
import uuid
from functools import wraps

request_logger = logging.getLogger("request")

def log_request(func):
    """Decorator to log request details with timing."""
    @wraps(func)
    def wrapper(*args, **kwargs):
        request_id = str(uuid.uuid4())[:8]
        start = time.perf_counter()

        try:
            result = func(*args, **kwargs)
            duration = time.perf_counter() - start

            request_logger.info(
                "Request completed", extra={
                    "extra_fields": {
                        "request_id": request_id,
                        "duration_ms": round(duration * 1000, 2),
                        "status": "success"
                    }
                }
            )
            return result
        except Exception:
            duration = time.perf_counter() - start
            request_logger.exception(
                "Request failed", extra={
                    "extra_fields": {
                        "request_id": request_id,
                        "duration_ms": round(duration * 1000, 2),
                        "status": "error"
                    }
                }
            )
            raise
    return wrapper

Quick Checklist: Is Your Logging Production-Ready?

• [ ] You're using log levels correctly (DEBUG doesn't show in production)
• [ ] You have rotating log files or log shipping
• [ ] All exception handlers use logger.exception() or exc_info=True
• [ ] Log messages include identifiable context (request IDs, user IDs)
• [ ] You're not logging sensitive data (passwords, tokens, PII)
• [ ] Third-party library loggers are not overwhelming your output
• [ ] Log files are accessible only to authorized processes

Configuring Logging via Environment Variables

Hardcoding log levels and file paths works for small projects, but production deployments usually need runtime configuration. A clean approach is to use environment variables with sensible defaults:

import os
import logging

LOG_LEVEL = os.getenv("LOG_LEVEL", "INFO").upper()
LOG_FILE = os.getenv("LOG_FILE", "")  # Empty means console-only
LOG_FORMAT = os.getenv(
    "LOG_FORMAT",
    "%(asctime)s | %(levelname)-8s | %(message)s"
)

numeric_level = getattr(logging, LOG_LEVEL, logging.INFO)
logger = logging.getLogger("app")
logger.setLevel(numeric_level)

handler = logging.StreamHandler()
handler.setFormatter(logging.Formatter(LOG_FORMAT))
logger.addHandler(handler)

if LOG_FILE:
    file_handler = logging.FileHandler(LOG_FILE)
    file_handler.setFormatter(logging.Formatter(LOG_FORMAT))
    logger.addHandler(file_handler)

This pattern allows DevOps teams to adjust logging without touching code — simply set LOG_LEVEL=DEBUG in a .env file or Kubernetes ConfigMap when troubleshooting.

Avoid Common Pitfalls

Pitfall 1: F-strings in Logging Calls

# BAD — string formatting happens even if log level is WARNING
logger.debug(f"Processing {len(large_list)} items: {expensive_computation()}")

# GOOD — lazy formatting, no performance hit when disabled
logger.debug("Processing %d items: %s", len(large_list), expensive_computation())

With f-strings, Python evaluates all arguments before calling logger.debug(). If your log level is INFO or WARNING, that expensive computation still runs — wasted CPU cycles. The old %s formatting is evaluated lazily inside the logging module.

Pitfall 2: Logging Sensitive Information

Never log passwords, API keys, credit card numbers, or personal identifiable information (PII). If you need to log request data for debugging, implement a sanitization layer:

import re

def sanitize_for_logs(data: dict) -> dict:
    """Remove sensitive fields from log entries."""
    sensitive_keys = {"password", "secret", "token", "api_key", "credit_card", "ssn"}
    return {
        k: "***" if k.lower() in sensitive_keys else v
        for k, v in data.items()
    }

logger.info("User data: %s", sanitize_for_logs(request_data))

Integration with Modern Python Frameworks

Most web frameworks already use Python's logging module under the hood. FastAPI and Flask both configure a default logger for their request/response cycle. You just need to configure the root logger to capture everything in one place:

import logging

# This captures both your app logs AND framework logs
logging.basicConfig(
    level=logging.INFO,
    format="%(asctime)s | %(name)s | %(levelname)s | %(message)s"
)

# Your app-specific logger
logger = logging.getLogger("myapp")

# FastAPI/uvicorn logs will also use the configured format

This is especially useful when deploying behind Gunicorn or uvicorn — all logs go to the same stream with consistent formatting.

Summary

The logging module is one of Python's most underutilized standard library gems. Moving from print() to structured, leveled, rotatable logging is one of the highest-ROI improvements you can make to a Python application. It adds zero external dependencies, takes minutes to set up, and saves hours of debugging time.

Start with the setup_logger() function from this article, add file rotation when your app runs longer than a day, and reach for logger.exception() instead of print(e). Your future self — debugging at 2 AM in production — will thank you.

---

What logging patterns do you use in your projects? Share your setup in the comments below.