Practical Secrets Management in Python: From .env Files to a Secrets Manager

Many leaked credentials are not stolen by clever attackers. They get committed to git, printed in a stack trace (the error report Python shows when something crashes), or pasted into a chat thread. Good secrets management in Python is mostly about boring discipline: one place to load secrets, strict validation at startup, no accidental logging, and a path to a real secrets manager. This post builds that up step by step.

What counts as a secret?

A secret is any value that gives access to something. Database passwords, API tokens, cloud access keys, signing keys and webhook secrets are all secrets. If one leaks, someone else can act as your app. Secrets management is the set of habits and tools that keep these values safe: where you store them, how your Python code loads them, how you change (rotate) them, and how you stop them from showing up in logs or in git.

The five rules below are the base. Each step after that turns one rule into working Python code.

The rules

1. Secrets never live in source code or the repository.
2. The app reads secrets from the environment or a secrets manager, in one module.
3. If a secret is missing or in the wrong format, the app stops at startup with a clear error, not on the first request.
4. Secret values never appear in logs, printed objects, or error messages.
5. Rotation does not require a redeploy.

Step 1: A single config module with validation

Use the standard library only, so this works anywhere:

# config.py
import os
from dataclasses import dataclass, field


class ConfigError(RuntimeError):
    pass


class Secret:
    """Wraps a sensitive string so it is never printed by accident."""

    __slots__ = ("_value",)

    def __init__(self, value: str):
        self._value = value

    def reveal(self) -> str:
        return self._value

    def __repr__(self) -> str:
        return "Secret('**********')"

    __str__ = __repr__


def _require(name: str) -> str:
    value = os.environ.get(name)
    if value is None or not value.strip():
        raise ConfigError(f"Missing required environment variable: {name}")
    return value.strip()


@dataclass(frozen=True)
class Settings:
    database_url: Secret
    api_token: Secret
    log_level: str = field(default="INFO")

    @classmethod
    def from_env(cls) -> "Settings":
        db = _require("DATABASE_URL")
        if not db.startswith(("postgresql://", "postgres://")):
            raise ConfigError("DATABASE_URL must be a PostgreSQL URL")
        token = _require("API_TOKEN")
        if len(token) < 32:
            raise ConfigError("API_TOKEN looks too short (expected >= 32 chars)")
        return cls(
            database_url=Secret(db),
            api_token=Secret(token),
            log_level=os.environ.get("LOG_LEVEL", "INFO").upper(),
        )

Notice the error messages name the variable but never echo its value. Because Secret overrides __repr__, printing Settings gives you:

Settings(database_url=Secret('**********'), api_token=Secret('**********'), log_level='INFO')

Call Settings.from_env() once in main() and pass the result around. Code that needs the raw value must call .reveal(), which is easy to grep for in review.

Step 2: .env files for local development only

.env files are convenient locally. Load them only if they exist, and never in production:

# dev_env.py
import os
from pathlib import Path


def load_dotenv(path: str = ".env") -> None:
    """Minimal .env loader: KEY=VALUE lines, # comments, no overrides."""
    p = Path(path)
    if not p.is_file():
        return
    for raw in p.read_text(encoding="utf-8").splitlines():
        line = raw.strip()
        if not line or line.startswith("#") or "=" not in line:
            continue
        key, _, value = line.partition("=")
        key = key.strip()
        value = value.strip().strip('"').strip("'")
        os.environ.setdefault(key, value)

(python-dotenv does the same with more edge cases handled.) Add .env to .gitignore and commit a .env.example with placeholder values so new developers know what to set.

Step 3: Fetch from a secrets manager in production

Environment variables are visible to anything that can inspect the process, and they can't change without a restart. In production, fetch from a managed store such as AWS Secrets Manager, with a short cache time (a TTL, or "time to live": how many seconds a saved value is trusted before the code fetches it again), so a rotated secret is picked up without a redeploy:

# secrets_store.py
import json
import threading
import time

import boto3
from botocore.exceptions import ClientError


class CachedSecret:
    def __init__(self, secret_id: str, ttl_seconds: int = 300, client=None):
        self._secret_id = secret_id
        self._ttl = ttl_seconds
        self._client = client or boto3.client("secretsmanager")
        self._lock = threading.Lock()
        self._value = None
        self._fetched_at = 0.0

    def _fetch(self) -> dict:
        try:
            resp = self._client.get_secret_value(SecretId=self._secret_id)
        except ClientError as exc:
            code = exc.response["Error"]["Code"]
            raise RuntimeError(
                f"Could not read secret {self._secret_id!r}: {code}"
            ) from None
        return json.loads(resp["SecretString"])

    def get(self, force_refresh: bool = False) -> dict:
        with self._lock:
            expired = time.monotonic() - self._fetched_at > self._ttl
            if force_refresh or self._value is None or expired:
                self._value = self._fetch()
                self._fetched_at = time.monotonic()
            return self._value

raise ... from None drops the original exception chain, which keeps request details out of logs. Give the app's IAM role (its AWS identity) the secretsmanager:GetSecretValue permission only for the secrets it needs, listed by their ARN (each secret's full AWS ID).

Handling rotation gracefully

When a database password rotates, open connections keep working but new ones with the cached password fail. Retry once with a fresh secret:

import psycopg

db_secret = CachedSecret("prod/app/db")


def connect():
    for attempt in (1, 2):
        creds = db_secret.get(force_refresh=(attempt == 2))
        try:
            return psycopg.connect(
                host=creds["host"],
                dbname=creds["dbname"],
                user=creds["username"],
                password=creds["password"],
                connect_timeout=5,
            )
        except psycopg.OperationalError:
            if attempt == 2:
                raise

Step 4: Keep secrets out of logs

Logs are copied to many places: log tools, error trackers, support tickets. A secret in a log line can be read by far more people than the secret itself. So we add a safety net.

Even with Secret, a raw value can slip into a log line through an f-string or an exception from a library. Add a logging filter that redacts known values and common token patterns:

# redact.py
import logging
import re

TOKEN_PATTERNS = [
    re.compile(r"AKIA[0-9A-Z]{16}"),                 # AWS access key id
    re.compile(r"ghp_[A-Za-z0-9]{36}"),              # GitHub token
    re.compile(r"(?i)(password|passwd|pwd)=([^\s&]+)"),
]


class RedactFilter(logging.Filter):
    def __init__(self, known_secrets=()):
        super().__init__()
        self._known = [s for s in known_secrets if s]

    def filter(self, record: logging.LogRecord) -> bool:
        msg = record.getMessage()
        for s in self._known:
            msg = msg.replace(s, "[REDACTED]")
        for pat in TOKEN_PATTERNS:
            msg = pat.sub("[REDACTED]", msg)
        record.msg, record.args = msg, None
        return True


def install(known_secrets=()):
    handler = logging.StreamHandler()
    handler.addFilter(RedactFilter(known_secrets))
    logging.basicConfig(level=logging.INFO, handlers=[handler], force=True)

Usage: install([settings.api_token.reveal()]). The filter sits on the root logger's handler, so it covers log lines from every logger that passes its records up to the root logger (the default).

Step 5: Stop secrets before they reach git

Prevention beats cleanup. Run a scanner as a pre-commit hook (a check that runs every time you commit) and in CI (your automated build and test pipeline). With pre-commit and gitleaks:

# .pre-commit-config.yaml
repos:
  - repo: https://github.com/gitleaks/gitleaks
    rev: v8.21.2
    hooks:
      - id: gitleaks

pip install pre-commit
pre-commit install
pre-commit run --all-files

If a secret is committed anyway, rotate it first. Rewriting history with git filter-repo is useful cleanup, but forks, clones, and CI caches may already have the value.

Common pitfalls

Even teams with good tools make the same few mistakes. Watch for these in code review.

• Logging the whole config or environment. logger.debug(os.environ) is a classic leak. Log key names only.
• Secrets in Docker build args or layers. ARG/ENV values persist in image history. Use a BuildKit secret mount (RUN --mount=type=secret) instead. It gives the build step the secret without saving it in the image.
• Caching forever. No TTL means rotation needs a restart. A few minutes is a good default.
• One secret shared across environments. A dev leak becomes a prod incident. Use separate secrets per environment.
• Broad IAM permissions. secretsmanager:* on * lets one compromised service read everything.
• Validation errors that echo values. "Invalid token: abc123..." puts the secret in your error tracker.

The short checklist

Centralize loading, validate at startup, wrap values so they don't print, fetch from a manager with a TTL cache, and scan every commit. None of these steps is complicated, and together they close the most common ways credentials leak from Python services.

Notes from the ByteBridge Talent LLC software engineering team.