Originally published on kuryzhev.cloud
Context
A Python AWS script built on boto3 argparse logging seems trivial to write correctly — until it runs against the wrong account, exits with a traceback CI can't parse, or writes signed credentials to a log file. This trio has become the de-facto skeleton for internal automation: snapshot cleanup jobs, tag auditors, cost reports, one-off migration scripts. Each library "just works" out of the box, which is exactly why the defaults are dangerous.
All three fail silently in different ways. botocore resolves region and credentials through a chain that doesn't always match what the operator expects. argparse exits the process via SystemExit instead of raising a normal exception, which breaks naive error handling. logging.basicConfig() is a no-op if a handler already exists on the root logger, so a "fix" to increase verbosity can be silently ignored.
None of this is exotic. It's documented behavior in the argparse docs and the boto3 credentials guide. The failure patterns below come from reading how these libraries are built, not from a specific outage — but they recur often enough in internal tooling to be worth documenting as a checklist rather than tribal knowledge.
Common failure: wrong account, wrong region, no warning
boto3 resolves credentials through a chain: environment variables, then shared config/profile files, then container or instance metadata. If a script assumes AWS_PROFILE is always set and doesn't validate it, an empty environment falls through to whatever identity is available next — which might be a different account than the one intended.
Region resolution is stricter than credential resolution, but it can still surprise. boto3.client() raises NoRegionError immediately at construction time if no region can be resolved from an explicit argument, an environment variable, or the shared config file — for services that require a region. The catch is that this failure surfaces wherever the client happens to get built, which in a larger script can be buried inside a helper function called conditionally, well after argument parsing and logging setup have already run and looked fine.
A common mistake pattern: a script accepts --profile as optional with a "sensible default," meant to save typing during testing. Months later, that same default silently points at a production account when someone forgets to pass the flag from a cron job or CI runner. Deregistering AMIs or deleting snapshots against the wrong account isn't a boto3 bug — it's the documented credential chain doing exactly what it's designed to do, just not what the operator expected.
Watch out for: instance and container metadata credentials silently taking priority in environments where AWS_PROFILE isn't explicitly exported, especially inside containers that inherit an IAM task role.
Common failure: argparse errors that vanish into broad except blocks
When parser.parse_args() encounters invalid input, it doesn't raise a normal exception — it calls sys.exit(2), which raises SystemExit. This is a subclass of BaseException, not Exception. A try/except Exception: block wrapped around the whole entrypoint won't catch it, which is correct behavior. But scripts that go one step further and wrap everything in try/except BaseException: "for clean output" will swallow it too — masking real usage errors and turning a clear --help message or validation exit into a silently continuing script.
A second, quieter issue: argparse's type=int or type=str only coerce type, not business rules. A negative --retention-days value or an empty --resource-id string passes parsing cleanly and fails much later, deep inside a boto3 call, with a stack trace that has nothing to do with the actual root cause.
Verify with python script.py --help (exit code 0) and then with a deliberately invalid flag (exit code 2) — in both cases, argparse's SystemExit should propagate untouched, not be caught anywhere in the code path. Custom validation logic for value ranges belongs in a post-parse check or a custom type= callable, not left implicit.
Common failure: logging that hides the problem or leaks it
logging.basicConfig() only takes effect if the root logger has no handlers attached yet — a widely-cited gotcha in the Python logging docs. If any other imported module has already called basicConfig() or attached its own handler to the root logger, a later call in the entrypoint script is silently ignored. This means a developer trying to raise the log level to DEBUG for troubleshooting sees no change at all, and assumes the bug they're chasing doesn't produce log output.
The opposite failure is more serious. Setting the root logger to DEBUG to "see what's happening" also enables botocore's internal debug logging, which in some SDK versions includes full request and response bodies — including signed headers and, in specific cases, temporary session credentials. Verbose CI logs are a documented and common accidental credential-leak vector; this is worth checking against the boto3 logging documentation for the SDK version in use.
A third pattern shows up as multiple modules each calling logging.getLogger(__name__), each attaching its own handler, with propagate=True left at its default. The result is duplicated log lines for every event, which makes troubleshooting slower rather than faster — exactly the opposite of the intended effect.
Safer operating pattern
The fix for all three failure classes is the same shape: make defaults explicit, and validate identity before mutation. Construct the session directly with required --profile and --region flags, and call sts.get_caller_identity() immediately afterward — it's a cheap, read-only call that confirms account and region before anything destructive runs.
# main.py — pattern for combining boto3 + argparse + logging safely
import argparse
import logging
import sys
import boto3
from botocore.exceptions import ClientError, NoRegionError
def build_parser() -> argparse.ArgumentParser:
parser = argparse.ArgumentParser(description="Snapshot cleanup utility")
parser.add_argument("--profile", required=True, help="AWS profile to use")
parser.add_argument("--region", required=True, help="AWS region, e.g. us-east-1")
parser.add_argument("--retention-days", type=int, default=30)
parser.add_argument("--dry-run", action="store_true")
parser.add_argument("--verbose", action="store_true", help="Enable debug logging for this script's logger only")
return parser
def configure_logging(verbose: bool) -> logging.Logger:
logger = logging.getLogger("cleanup_tool") # named logger, not root
handler = logging.StreamHandler()
handler.setFormatter(logging.Formatter("%(asctime)s %(levelname)s %(message)s"))
logger.addHandler(handler)
logger.setLevel(logging.DEBUG if verbose else logging.INFO)
logger.propagate = False # avoid duplicate lines via root logger
# Keep the SDK quiet even if our app logger is DEBUG
logging.getLogger("botocore").setLevel(logging.WARNING)
logging.getLogger("boto3").setLevel(logging.WARNING)
return logger
def main() -> int:
args = build_parser().parse_args() # SystemExit here is intentional, don't swallow it
log = configure_logging(verbose=args.verbose)
session = boto3.Session(profile_name=args.profile, region_name=args.region)
try:
identity = session.client("sts").get_caller_identity()
log.info("Running as %s in %s", identity["Arn"], args.region)
except NoRegionError:
log.error("Region not resolved — check --region or profile config")
return 1
except ClientError as e:
log.error("Failed to verify identity: %s", e)
return 1
if args.retention_days < 1:
log.error("--retention-days must be >= 1")
return 1
# ... cleanup logic using session.client("ec2") once, reused, not per-loop ...
return 0
if __name__ == "__main__":
sys.exit(main())
For tools with more than one action, argparse's add_subparsers() scales better than accumulating mutually exclusive flags. Explicit sys.exit(1) on every handled failure path — rather than relying on an unhandled traceback for a nonzero exit code — is what lets a CI pipeline or cron wrapper reliably detect failure. Testing boto3 calls with botocore.stub.Stubber or the moto library catches parameter and logic errors before a script ever touches real infrastructure.
A checklist form of this pattern is easier to apply consistently across a team than a single reference script:
Pre-flight checklist before writing another boto3 CLI script:
[ ] --profile / --region are explicit CLI args, not silently defaulted
[ ] sts.get_caller_identity() called once, logged, before any mutating action
[ ] argparse validation errors are never caught by a broad except block
[ ] custom type= or post-parse checks cover value ranges, not just types
[ ] logging configured once, in main(), on a named logger (not root)
[ ] botocore/boto3 loggers explicitly capped at WARNING even in verbose mode
[ ] one boto3 client per resource type, created once, reused across calls
[ ] AWS_RETRY_MODE / max_attempts set explicitly, not left on legacy defaults
[ ] sys.exit(1) on every handled failure path for CI/cron detection
[ ] no credentials passed as CLI args or printed in logs
Two smaller details are worth adding to that list. First, creating a new boto3 client inside a loop instead of reusing one instance adds per-call connection setup overhead that's easy to miss in a short script but adds up at scale. Second, switching retry mode from the legacy default to standard or adaptive via AWS_RETRY_MODE improves resilience against throttling, but without an explicit max_attempts cap it can multiply API calls — and cost — during sustained throttling events. Details vary by botocore version, so verify current behavior against the botocore retries documentation before relying on it in production tooling.
None of these boto3 argparse logging fixes require a framework or a rewrite — they're a handful of explicit checks added to a skeleton that most internal AWS scripts already share. For broader patterns on structuring AWS automation and CI pipelines, see the DevOps_DayS archive.
Top comments (0)