DEV Community

Brad
Brad

Posted on

Production Python Automation: 5 Rules That Prevent Your Scripts From Breaking

Most Python automation tutorials show you code that works in a blog post but breaks in production.

Here's what actually works in real business environments, based on running these scripts daily:

Rule 1: Always Use Proper Logging

Scripts that run silently make debugging a nightmare. Use structured logging from the start:

import logging
from datetime import datetime

def setup_logger(name, log_file=None):
    logger = logging.getLogger(name)
    logger.setLevel(logging.INFO)

    # Console handler
    console = logging.StreamHandler()
    console.setFormatter(logging.Formatter(
        '%(asctime)s [%(levelname)s] %(message)s',
        datefmt='%Y-%m-%d %H:%M:%S'
    ))
    logger.addHandler(console)

    # File handler (optional but recommended)
    if log_file:
        file_handler = logging.FileHandler(log_file)
        file_handler.setFormatter(logging.Formatter(
            '%(asctime)s [%(levelname)s] %(name)s: %(message)s'
        ))
        logger.addHandler(file_handler)

    return logger

# Usage
log = setup_logger("invoice_bot", "logs/invoice_bot.log")
log.info("Starting invoice processing run")
log.error("Failed to connect to database: %s", str(error))
Enter fullscreen mode Exit fullscreen mode

Rule 2: Handle Errors Gracefully (Don't Crash Silently)

import smtplib
from email.mime.text import MIMEText

def send_with_retry(to_email, subject, body, retries=3):
    last_error = None

    for attempt in range(retries):
        try:
            msg = MIMEText(body)
            msg['Subject'] = subject
            msg['To'] = to_email
            msg['From'] = "your@email.com"

            with smtplib.SMTP_SSL('smtp.gmail.com', 465) as server:
                server.login("your@email.com", "your-app-password")
                server.send_message(msg)

            print("Email sent successfully on attempt " + str(attempt + 1))
            return True

        except smtplib.SMTPException as e:
            last_error = e
            print("Attempt " + str(attempt + 1) + " failed: " + str(e))

            import time
            time.sleep(5 * (attempt + 1))  # Exponential backoff

    print("All retries exhausted. Last error: " + str(last_error))
    return False
Enter fullscreen mode Exit fullscreen mode

Rule 3: Use Environment Variables for Credentials

Never hardcode passwords or API keys:

import os
from pathlib import Path

def load_config():
    # Try .env file first (for local dev)
    env_file = Path('.env')
    if env_file.exists():
        with open(env_file) as f:
            for line in f:
                line = line.strip()
                if line and not line.startswith('#') and '=' in line:
                    key, value = line.split('=', 1)
                    os.environ[key.strip()] = value.strip().strip('"')

    # Now read from environment
    config = {
        'STRIPE_KEY': os.environ.get('STRIPE_KEY'),
        'GMAIL_PASSWORD': os.environ.get('GMAIL_PASSWORD'),
        'DATABASE_URL': os.environ.get('DATABASE_URL', 'sqlite:///app.db'),
    }

    # Validate required fields
    missing = [k for k, v in config.items() if v is None and k != 'DATABASE_URL']
    if missing:
        raise EnvironmentError("Missing required env vars: " + str(missing))

    return config

config = load_config()
# Use config['STRIPE_KEY'] throughout your script
Enter fullscreen mode Exit fullscreen mode

Rule 4: Make Scripts Idempotent

Your script should be safe to run multiple times without duplicating work:

import json
from pathlib import Path
from datetime import datetime

class ProcessedTracker:
    def __init__(self, state_file="processed.json"):
        self.file = state_file
        self.processed = self._load()

    def _load(self):
        if Path(self.file).exists():
            with open(self.file) as f:
                return json.load(f)
        return {}

    def _save(self):
        with open(self.file, 'w') as f:
            json.dump(self.processed, f, indent=2)

    def is_processed(self, item_id):
        return str(item_id) in self.processed

    def mark_done(self, item_id, metadata=None):
        self.processed[str(item_id)] = {
            "timestamp": datetime.now().isoformat(),
            "metadata": metadata or {}
        }
        self._save()

# Usage: process each invoice only once
tracker = ProcessedTracker("invoices_processed.json")

for invoice in get_all_invoices():
    if tracker.is_processed(invoice['id']):
        print("Skipping already-processed invoice: " + invoice['id'])
        continue

    send_invoice(invoice)
    tracker.mark_done(invoice['id'], {"amount": invoice['amount']})
Enter fullscreen mode Exit fullscreen mode

Rule 5: Schedule Reliably with Error Notifications

import subprocess, logging
from datetime import datetime

def run_with_notification(script_path, admin_email=None):
    log = setup_logger("scheduler")

    log.info("Running script: " + script_path)
    start = datetime.now()

    try:
        result = subprocess.run(
            ['python', script_path],
            capture_output=True,
            text=True,
            timeout=300  # 5-minute timeout
        )

        elapsed = (datetime.now() - start).seconds

        if result.returncode == 0:
            log.info("Script succeeded in " + str(elapsed) + "s")
            return True
        else:
            log.error("Script failed with code " + str(result.returncode))
            log.error("STDERR: " + result.stderr[:500])

            if admin_email:
                send_with_retry(
                    admin_email,
                    "Script Failed: " + script_path,
                    "Error:\n" + result.stderr
                )
            return False

    except subprocess.TimeoutExpired:
        log.error("Script timed out after 300s")
        return False
    except Exception as e:
        log.error("Unexpected error: " + str(e))
        return False

# Cron entry: 0 */4 * * * python scheduler.py run invoice_bot.py admin@yourdomain.com
Enter fullscreen mode Exit fullscreen mode

The Toolkit That Follows These Rules

I built these patterns from scratch after running automation scripts that kept breaking in production. Every script in my Python Business Automation Toolkit ($29) follows all 5 rules above.

The 12 scripts in the toolkit include:

  • Invoice generator (PDF, with retry + logging)
  • Email automation (with error handling)
  • File organizer (idempotent, won't duplicate)
  • Lead tracker (state-aware)
  • Business dashboard (production-ready)

All production-tested, all following best practices.


What's the worst production bug you've hit with a Python script? Drop it in the comments.

Top comments (0)