DEV Community

Matthew Gladding
Matthew Gladding

Posted on • Originally published at gladlabs.io

Hunting the Silent Excepts

What we shipped on 2026-07-18

We spent today on a massive debt burn-down, specifically targeting "silent excepts" across the codebase to stop failures from vanishing into logger.debug or pass statements. The most alarming discovery was when we worked to surface a failed SiteConfig load in the MCP server (PR #2707). Because the result of _get_site_config is cached in a module global, a single failed load at startup meant the process served code-defaults for its entire lifetime instead of the operator's tuned app_settings.

We found this same pattern haunting the CLI (PR #2705), where swallowed SiteConfig loads in posts.py and approval.py were silently redirecting media resolution or reporting gates as disabled when they were actually enabled--essentially presenting wrong answers as facts to the operator. Once we caught the shape of this bug, it became a grep-and-fix exercise across the tree rather than re-deriving the failure each time.

The burn-down moved into modules/content (PR #2710), where we uncovered "atom-side twins." We'd previously de-silenced helpers in writer_core.py, but those helpers had independent implementations inside the atoms that remained silent. This led to some subtle, frustrating behavior: in content_normalize_draft, an empty allowlist made scrub_fabricated_links maximally aggressive and started deleting legitimate internal links.

We also hit a critical failure point in the newsletter service (PR #2701). The _log_send function was swallowing write failures to campaign_email_logs at DEBUG level. This wasn't just a lost log; that table serves as the campaign's idempotency ledger, meaning a silent failure there could lead to duplicate sends.

The rest of the day was about closing the gaps in our observability. We surfaced swallowed audit-mirror writes across five brain probes (PR #2703), including migration_drift_probe and compose_drift_probe, ensuring that events notify solely via notify_fn actually leave a trace. We also fixed pr_staleness_probe.run_pr_staleness_probe (PR #2704) to ensure delivery failures are paged; a Loki line is useless if the operator is relying on Telegram or Discord and hears nothing.

Outside the cleanup, we tightened the GPU layer, keeping the idle-reset trigger above the render floor rather than below it (PR #2666) and fixing a [TimeSpan]::MaxValue duration failure in register-idle-wsl-reset -Install (PR #2667). We also consolidated the duplicate ImageModel registry onto _image_models (PR #2671) to clean up our internal mapping.

The baseline for silent_excepts_baseline.json is dropping fast, and we're finally moving toward a state where "no news" actually means "no errors" rather than "the error was swallowed at DEBUG."

Auto-compiled by Poindexter from today's commits and PRs. See the work: github.com/Glad-Labs/poindexter.

Sources

Top comments (1)

Collapse
 
topstar_ai profile image
Luis Cruz

I was particularly intrigued by the discovery of "atom-side twins" in modules/content, where independent implementations of helpers in writer_core.py remained silent, leading to subtle behavior issues. This highlights the importance of thorough code review and testing to catch such patterns. The fact that a simple grep-and-fix exercise could resolve many of these issues across the codebase also underscores the value of establishing clear coding standards and best practices. Have you considered implementing automated code analysis tools to help identify similar silent excepts and improve overall code quality moving forward?