mypy says your code is clean. Your CI pipeline agrees. Then someone passes None where you annotated str, and the AttributeError arrives at runtime with no warning from the type checker.
This is the structural problem with AI-generated type annotations. The annotation is syntactically correct — mypy parses it, validates it against call sites, and reports no issues. The semantic problem lives in the gap between what the annotation claims and what the code actually does. AI coding assistants close that gap less often than the type-check green light implies.
Why AI Generates Structurally Valid but Semantically Wrong Type Hints
AI coding assistants generate type hints that satisfy mypy's structural validator — the annotation is syntactically correct and type-checks — but carry semantic meaning that doesn't match the code's actual behavior, because mypy validates annotation consistency, not the correctness of the annotator's intent. BrassCoders runs Pyre/Pysa as one of its 12 scanners, adding taint analysis that traces data flows across type-annotated boundaries rather than just checking annotation consistency.
The root cause is how AI assistants learn annotation patterns. An assistant trained on millions of Python files has seen Optional[str] used in thousands of contexts. It pattern-matches the annotation to the function signature shape — not to the behavior of every caller. When the rest of your codebase dereferences the return value without a None check, the assistant doesn't trace that call graph. mypy, per its documented design, validates that the annotation is structurally consistent, not that it's behaviorally correct.
That distinction matters for typed codebases at scale. When an annotation lies, mypy's green light trains reviewers to trust type-check output as behavioral proof. It isn't.
The Four Type Hint Patterns That Fool mypy
BrassCoders's Pyre/Pysa scanner performs taint analysis across type-annotated code and can flag type-unsafe data flows, but the four patterns AI generates that mypy accepts without complaint are: Optional[str] on a field the downstream code never handles as None, Dict[str, Any] collapsing a structure mypy could enforce, incorrect return type annotations on async functions, and cast() calls that lie about the type without converting it.
Optional[str] is the most common. An AI assistant sees a field that could theoretically be absent and annotates it Optional[str] — technically correct in isolation. The problem is that every downstream caller dereferences it directly: user.name.upper(). mypy checks the callers against their own annotations, not against whether the None branch is handled. The AttributeError waits.
Dict[str, Any] is the annotation that collapses type safety entirely. Python's typing module provides TypedDict for exactly this case — a dict with a known key shape that mypy can enforce. AI assistants default to Dict[str, Any] because it never fails type checking. Neither does it provide any enforcement.
Return type mismatches on async functions are subtler. An async function annotated -> str that returns a coroutine object compiles and type-checks. At runtime, calling code that awaits a str gets an exception, not a string. mypy catches this inconsistency only when the function is explicitly called in type-checked context. Unannotated callers let it through.
cast() is the most direct lie. It tells mypy "trust me, this value is type T" without modifying the value at runtime. cast() is a pure no-op at runtime. AI assistants reach for it to silence a type error the checker can't resolve. The underlying inconsistency remains.
When the Annotation Lies: Runtime vs Static Analysis
mypy's type checker operates on annotations at analysis time: it never executes the code. An annotation that says a function returns str is trusted by mypy even if the function returns None in a code path mypy didn't follow. BrassCoders's Pysa scanner traces these flows interprocedurally — following the None across function calls until it reaches a sink that would crash or produce incorrect behavior.
The difference in scope is significant. mypy performs local type inference: it checks that each function's body is consistent with the function's declared signature, and that call sites match the signature. It does not execute a global reachability analysis of what values actually flow through the call graph at runtime. When a function is annotated -> str but has a code path that returns None, mypy flags the inconsistency only if the None return is in a branch mypy can analyze. A None that arrives via a third-party API call with an Any return type passes through undetected.
Pysa's taint analysis starts from sources — user input, external API responses, database reads — and follows those values through the call graph interprocedurally. It doesn't just check annotations. It tracks what's reachable. When a tainted None crosses a type-annotated boundary labeled str, that's a finding, not a green light.
What BrassCoders Flags in Typed Python Code
BrassCoders runs Pyre/Pysa as one of its 12 scanners — Pysa performs interprocedural taint analysis on typed Python code and flags data flows where taint crosses type boundaries. It also flags cast() usage patterns and unannotated code paths adjacent to security-sensitive operations.
Running BrassCoders alongside mypy gives you two different analyses on the same codebase. mypy finds annotation inconsistencies at call sites where the types don't match. Pysa finds reachability problems: code paths where dangerous values reach sensitive sinks regardless of what the annotations say. The two tools are complementary, not redundant.
Add BrassCoders to your CI pipeline alongside mypy with --strict and Pyright in your editor for real-time annotation validation. On macOS, Linux, and Windows (WSL2), install with:
pip install brasscoders
brasscoders scan .
The OSS core runs locally with no outbound network calls. Apache 2.0. No account needed. Pyre/Pysa are included as one of the 12 scanners, so taint analysis runs automatically on every scan.
BrassCoders Paid adds embedding-based noise reduction on top of the OSS core for $12/dev/month — useful once your scan output exceeds the volume you want to triage manually. The OSS core with Pysa is sufficient for catching the type-hint patterns described here.
The benchmark: BrassCoders catches 11 out of 12 AI-generated bugs in the published corpus. Bandit catches 6 of those 12. Type-hint failures are a category Bandit doesn't see at all.
Top comments (0)