DEV Community

Muhammad Ikram UL Mustafa
Muhammad Ikram UL Mustafa

Posted on

Your Linter Passes. The Exception Handler Is Lying.

def verify_password(user, password):
    try:
        return hash_password(password) == user.password_hash
    except Exception:
        return True
Enter fullscreen mode Exit fullscreen mode

Read the handler again. If anything inside the try raises, the function returns True. A database timeout. A None where a user object should be. A bug in the hashing library after an upgrade. Every one of those lets the user in.

Nobody writes this on purpose. It gets written while handling something else, it passes review because it looks like error handling, and it sits there until the day the hashing call starts failing.

Nothing in the toolchain objects

Run the tests. They pass. The tests check that the right password returns True and the wrong one returns False, and neither path raises, so the handler never runs.

Run the linter. The default rule sets in flake8 and Ruff say nothing. Pylint flags the broad except Exception, but it flags it the same way whether the handler returns True or logs and carries on, so the warning tells you nothing about which handler is dangerous. A warning that fires on every broad handler is one people learn to ignore.

Run the type checker. The function is declared to return bool. It returns bool. That is a pass.

This is the problem with swallowed exceptions. Static analysis is good at things that are visibly wrong. A swallowed exception is not visibly wrong. The code runs, returns a plausible value, and logs nothing. You find out weeks later, from a number that does not reconcile or a login that should not have worked.

Why it matters more now

This bug has always existed. What changed is the rate at which code arrives.

A lot of code is now generated faster than anyone reads it. Ask a model to "handle the error" in a function and you will reliably get a handler that makes the error go away, not one that surfaces it. It catches Exception, returns a sensible default, and the function now "never crashes". That reads as an improvement in a diff. It is the opposite.

Review does not scale to catch this, because the pattern looks fine at a glance. So I wanted the codebase itself to refuse it. That is silentcatch: a small static check that fails the build when a pull request adds an exception handler that swallows the failure.

How it decides a handler is swallowed

The question it asks of every handler is narrow: is there any evidence in the body that this failure was recorded? A raise counts. A logging call counts. A call to an error reporter like Sentry counts. If none of those is present, the exception is being swallowed.

It parses each file with Python's ast module, walks every ExceptHandler node, and runs a small visitor over the handler body. This is the core of it, from detector.py:

class _HandlerInspector(ast.NodeVisitor):
    """Walks a handler body looking for evidence the failure was not swallowed."""

    def __init__(self) -> None:
        self.reraises = False
        self.raises_new = False
        self.logs = False
        self.returns_or_assigns_fallback = False
        self.has_other_statements = False

    def visit_Raise(self, node: ast.Raise) -> None:
        if node.exc is None:
            self.reraises = True
        else:
            self.raises_new = True
        self.generic_visit(node)

    def visit_Call(self, node: ast.Call) -> None:
        func = node.func
        if isinstance(func, ast.Attribute):
            if func.attr in LOG_METHODS:
                receiver = _root_name(func.value)
                if receiver is None or receiver in LOG_RECEIVERS:
                    self.logs = True
            if func.attr in REPORTING_CALLS:
                self.logs = True
        elif isinstance(func, ast.Name):
            if func.id in REPORTING_CALLS:
                self.logs = True
        self.generic_visit(node)

    def visit_Return(self, node: ast.Return) -> None:
        self.returns_or_assigns_fallback = True
        self.generic_visit(node)
Enter fullscreen mode Exit fullscreen mode

LOG_METHODS is the usual set: debug, info, warning, error, exception, critical and a few more. REPORTING_CALLS covers things like capture_exception, report, notify and print_exc.

The receiver check matters more than it looks. Matching on the method name alone would treat response.error() as logging. So a call only counts as a log if the object it is called on looks like a logger: logger, log, logging, self, app and similar. There is a test for exactly that case:

def test_non_logger_error_method_does_not_count():
    assert verdicts("""
try:
    send()
except Exception:
    response.error()
""") == [Verdict.SILENT]
Enter fullscreen mode Exit fullscreen mode

Once the visitor has run, the verdict is a short chain of decisions (trimmed slightly):

if inspector.reraises or inspector.raises_new:
    continue

label = _exception_label(node)
broad = _is_bare_or_broad(node)

if inspector.logs:
    # Logged but not re-raised. Only worth flagging on broad handlers,
    # where the code continues past an error it does not understand.
    if broad:
        findings.append(Finding(..., verdict=Verdict.LOGGED_ONLY, ...))
    continue

if _body_is_only_pass(node.body):
    detail = "the handler does nothing at all"
elif inspector.returns_or_assigns_fallback:
    detail = "the handler returns a value instead of surfacing the failure"
else:
    detail = "nothing in the handler records or re-raises the error"
Enter fullscreen mode Exit fullscreen mode

That gives three outcomes. A handler that re-raises, bare or chained, is fine. A handler that records the error is fine, unless it is also broad, in which case it is a warning. Everything else is a failure, with a message that says which kind of swallow it is.

So verify_password above fails. These do not:

try:
    parse(x)
except ValueError:
    logger.warning("bad input")       # narrow and logged: a considered decision

try:
    risky()
except ValueError as e:
    raise RuntimeError("failed") from e   # chained re-raise

try:
    send()
except Exception:
    capture_exception()               # reported (broad, so this one is a warning)
Enter fullscreen mode Exit fullscreen mode

Running it on the demo file in the repo, which has two swallowed handlers, one logged-only handler, one chained re-raise and one suppressed handler:

$ silentcatch demo/payments.py
FAIL  demo/payments.py:19  `Exception` is swallowed: the handler returns a value instead of surfacing the failure.
FAIL  demo/payments.py:27  `Exception` is swallowed: the handler returns a value instead of surfacing the failure.
WARN  demo/payments.py:35  `Exception` is logged but not re-raised. Execution continues past an error the code did not specifically anticipate.

silentcatch: 2 swallowed, 1 logged-only.
Add `# silentcatch: allow` above a handler that is deliberate.
Enter fullscreen mode Exit fullscreen mode

Exit code 1. The re-raise and the suppressed handler do not appear.

Design decisions and what they cost

A broad handler that logs is a warning, not a failure

except Exception followed by logger.error(...) is sometimes exactly right. A request boundary in a web app should catch everything, log it, and return a 500 rather than kill the worker. A task runner should log a failed task and move to the next one.

It is also sometimes wrong, when the code keeps going past an error it did not anticipate and later steps run on bad state. The tool cannot tell those two apart from the syntax. If I made it a failure, every web app would fail on its error middleware on day one. So it is a warning by default, and --strict turns warnings into failures for teams that want that.

The cost: a broad handler that logs and then returns a wrong value passes the default check. The log line is there, so someone could find the failure. Nobody is forced to.

Diff mode, or the check gets turned off

Run any new check on an existing codebase and you get the backlog. On a few years of Python that can be hundreds of handlers. Nobody fixes hundreds of handlers in the PR that adds a linter. What happens instead is the check gets marked as non-blocking, then ignored, then removed. Nothing improves.

--base origin/main reports only findings on lines the pull request touched. It asks git which Python files changed, then runs git diff -U0 per file and reads the hunk headers to build the set of changed line numbers. A finding is kept only if its except line is in that set.

That holds new code to the standard and leaves the backlog alone. The old handlers are still there. But the number stops growing, which is more than most cleanup efforts manage.

Two costs. It keys on the except line, so if a PR edits the body of an old handler without touching the except line, it is not reported. And if git is not available it prints a notice and falls back to scanning everything, which is the noisy behaviour diff mode exists to avoid. In CI you need fetch-depth: 0 on checkout so the base ref exists.

Suppression is a comment, not a config file

Some handlers genuinely should swallow. Optional telemetry. Best-effort cache warming. Cleanup that must not mask the original error.

try:
    telemetry.send(event)
except Exception:  # silentcatch: allow
    pass
Enter fullscreen mode Exit fullscreen mode

The comment goes on the except line or the line above it. I chose that over an ignore list in pyproject.toml for three reasons.

The decision sits next to the code it is about. Anyone reading the handler sees that it was deliberate.

It shows up in the diff. A reviewer approving a new # silentcatch: allow is approving that specific handler, not a path glob that will quietly cover future code too.

It moves with the code. An ignore list keyed by file and line breaks as soon as the file is edited above the handler. A comment does not.

The cost is that it is easy to type. A comment that says "allow" with no reason is not much better than no check. I put a short reason after it in my own code, and I would ask for one in review.

It flagged its own source

The first time I ran silentcatch on itself, it reported five handlers. All five were swallowing by design:

  • analyse_source catches SyntaxError and returns an empty list. A file that does not parse is not this tool's problem. The interpreter will complain about it soon enough.
  • analyse_file catches OSError and UnicodeDecodeError and returns an empty list. An unreadable file is skipped.
  • Two functions in cli.py catch the errors from running git and return None. Git being missing is an expected condition. The caller checks for None and falls back to a full scan.
  • The hunk header parser catches ValueError and IndexError on a malformed @@ line and skips that hunk.

Every one of them returns a fallback without logging. By the tool's own rule, every one of them is a swallow. The rule was right about what the code does. It just could not know that I meant it.

I had two options. Loosen the rule, for example by treating return None or return [] as acceptable. Or keep the rule and mark the five handlers. I marked them, each with a short reason after the marker: "unparseable files are not our concern", "unreadable files are skipped by design", "git absence is an expected condition, caller handles None", "malformed hunk header, skip this hunk".

Loosening the rule would have exempted the exact pattern that causes real damage. A lookup that fails and returns None is the classic version of this bug. The repo's CI now runs silentcatch on its own source on every push, so a sixth unmarked handler would fail the build.

The honest reading of that first run: on a codebase of about 360 lines, written by someone who knew the rule, the false positive rate was five out of five. Most handlers in a small, careful tool are deliberate. On a larger codebase with a more typical mix you will see real catches alongside them, but you will see false positives. The suppression comment is how you pay for them, once per handler.

Limitations

Worth knowing before you adopt it.

  • Python only. The approach ports to other languages. This implementation does not.
  • Logger detection is a heuristic. It matches method names against plausible receivers. A custom wrapper like audit.record_failure(e) is not recognised, and you get a false positive. It also cuts the other way: self.error(...) counts as logging because self is an accepted receiver, even if that method does not log.
  • It cannot infer intent. A handler that returns a safe default because that really is correct looks identical to one hiding a bug. That is why the suppression comment exists instead of a cleverer rule.
  • Single-file analysis. If your handler calls a helper that logs, that is not seen. Neither is anything that crosses a module boundary.

The thing to watch is the false positive rate. A check that fires on correct code gets disabled, and a disabled check is worth less than no check. If it flags a handler that it should not have, that is the most useful issue you can open.

Install and use

Python 3.10 or newer. No dependencies outside the standard library. MIT licensed.

pip install silentcatch
Enter fullscreen mode Exit fullscreen mode
# whole project
silentcatch .

# only lines changed against main
silentcatch --base origin/main

# also fail on broad handlers that log but do not re-raise
silentcatch --strict
Enter fullscreen mode Exit fullscreen mode

Exit code is 1 if anything failed, 0 otherwise.

As a GitHub Action, with findings shown as inline annotations on the pull request diff:

name: silentcatch

on: [pull_request]

jobs:
  silentcatch:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
        with:
          fetch-depth: 0          # needed for --base
      - uses: ikramulmustafa/silentcatch@v1
        with:
          base: origin/${{ github.base_ref }}
Enter fullscreen mode Exit fullscreen mode

Top comments (0)