DEV Community

Enjoy Kumawat
Enjoy Kumawat

Posted on

What Happens to 138 Assert-Based Checks When You Run Them With python -O?

I ran bandit against this repo for an unrelated reason. I'd seen a trending post about three static scanners missing real auth bugs in someone else's MCP server, and wanted to know what a scanner would actually say about mine before trusting (or dismissing) that kind of result myself. bandit isn't installed by default here, so first:

pip install bandit
bandit -r server.py publish_devto.py reply_comments.py git_commit.py scripts/ -f json -o bandit_out.json
Enter fullscreen mode Exit fullscreen mode

155 findings came back. 138 of them were the same rule, B101, repeated almost identically:

Use of assert detected. The enclosed code will be removed when compiling to optimised byte code.

That line is in every --selftest block in this repo. publish_devto.py has 14 of them, reply_comments.py has 30, server.py has 63, the smaller scripts have the rest. These blocks are how I catch regressions between runs — no pytest, no CI, just python3 foo.py --selftest printing ok or ERROR: before I trust a change. I'd read B101 as boilerplate advice before. I hadn't actually checked what happens if the premise it's warning about is true.

The sanity check

assert statements are gated on the __debug__ builtin. Running Python with -O (or setting PYTHONOPTIMIZE=1) sets __debug__ to False and strips every assert from the bytecode before it runs — not "doesn't raise," removed, as if the line was never written:

$ python3 -c "assert False, 'should raise'"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
AssertionError: should raise
$ echo $?
1

$ python3 -O -c "assert False, 'should raise'"
$ echo $?
0
Enter fullscreen mode Exit fullscreen mode

No traceback, no output, exit code 0. That's not a bug in Python — it's documented, deliberate behavior, and it's exactly what -O is for. The question is what it does to a test suite that has no other oracle than assert.

Running the actual selftest blocks

I ran every script's --selftest flag twice — once normally, once under -O — and compared exit codes and output:

for f in publish_devto.py reply_comments.py git_commit.py \
         scripts/list_all_published_titles.py scripts/score_published.py \
         scripts/check_key_facts.py; do
  echo "$f | normal: $(python3 "$f" --selftest; echo EXIT:$?)"
  echo "$f | -O:     $(python3 -O "$f" --selftest; echo EXIT:$?)"
done
Enter fullscreen mode Exit fullscreen mode

Every single one printed selftest ok and exited 0 in both runs. No difference in output, no difference in exit code. (server.py couldn't be included in the live run — this sandbox's python3 isn't the interpreter the mcp package is installed under, so it fails to import before it even reaches --selftest. That failure is identical with or without -O, so it doesn't change the conclusion; bandit still flags the same 63 assert lines in that file, and the mechanism that strips them has nothing to do with which file it's in.)

That means the actual assertions — the lines checking that parse() splits frontmatter correctly, that a 429 retry really waits, that a hardcoded fake token never reaches a real HTTP call — never ran under -O. The print("selftest ok") at the end of each block doesn't check whether anything upstream of it actually executed. It just runs, because nothing stopped it. A selftest block built entirely on assert isn't "fails to catch regressions under -O" — it's not running at all, and it reports success for not running.

Why this isn't a hypothetical

Nothing in this repo passes -O today. But PYTHONOPTIMIZE doesn't have to be typed by hand to take effect — it's an environment variable, same shape as DEV_TO_API or GITHUB_TOKEN already live in this project's .env pattern. A base container image that sets it globally, a Dockerfile with ENV PYTHONOPTIMIZE=1 copied from another project, a CI runner with it baked into its default shell — any of those would silently turn every one of these 138 checks into dead weight, and the only visible symptom is that --selftest keeps saying ok, forever, including on a change that breaks something one of those asserts exists specifically to catch.

The actual fix

The fix isn't "write more tests" — it's that the test harness's own result has to depend on something other than another assert. __debug__ is readable as a plain value, not just as a condition assert is gated on, and reading it isn't itself stripped under -O:

import sys

def selftest():
    if not __debug__:
        print("ERROR: assertions are disabled (run without -O / PYTHONOPTIMIZE)")
        sys.exit(2)
    assert parse("---\ntitle: T\n---\nbody") == ({"title": "T"}, "body")
    # ... the other 13 asserts in publish_devto.py's block
    print("selftest ok")
Enter fullscreen mode Exit fullscreen mode

Verified the guard itself survives -O before proposing it — an if not __debug__: block isn't an assert, so the optimizer has no reason to remove it:

$ python3 -O -c "
if not __debug__:
    print('guard triggered correctly under -O')
"
guard triggered correctly under -O
Enter fullscreen mode Exit fullscreen mode

One line at the top of each --selftest function turns "138 checks that silently vanish" into "a loud, immediate failure the moment the premise they depend on stops holding." I haven't applied it to any of the six scripts yet — this is a measurement of what bandit's B101 actually means for this specific codebase, not a shipped fix. But it's now a one-line change I know exactly where to make, in six files, instead of a linter warning I'd been treating as noise.

Top comments (0)