DEV Community

Evan Mercer
Evan Mercer

Posted on

Measuring PR Cognitive Load with Deterministic Python AST Heuristics

AI-assisted code generation is not inherently a software defect—most engineering teams rely on coding assistants daily to accelerate boilerplate, test scaffolding, and migrations.

The real architectural bottleneck is review throughput versus generation velocity. When pull requests grow from 150 lines to 1,200+ lines, human reviewers hit cognitive fatigue. Two failure modes tend to slip past fatigued code reviews:

  1. Loss of Cognitive Ownership: Deeply nested control flow, redundant defensive wrappers, or silent exception swallowing (except Exception: pass) that compile cleanly and pass happy-path tests while obscuring failure states.
  2. Unverified Dependency Surface: Unfamiliar or hallucinated package imports introduced during rapid iteration without supply-chain verification.

Rather than relying on probabilistic "AI text detectors" (which are noisy and largely irrelevant to production reliability), teams get far higher signal from deterministic AST heuristics wired directly into PR triage.

Minimal AST Heuristics in Pure Python (stdlib)

Using Python's built-in ast module, we can flag review-friction hotspots in milliseconds before a human reviewer even opens the diff:

import ast

class ReviewFrictionVisitor(ast.NodeVisitor):
    def __init__(self):
        self.max_depth = 0
        self._depth = 0
        self.silent_broad_excepts = 0

    def _enter_branch(self, node):
        self._depth += 1
        self.max_depth = max(self.max_depth, self._depth)
        self.generic_visit(node)
        self._depth -= 1

    visit_If = visit_For = visit_While = _enter_branch
    visit_Try = _enter_branch

    def visit_ExceptHandler(self, node):
        is_broad = node.type is None or (
            isinstance(node.type, ast.Name) and node.type.id == "Exception"
        )
        if is_broad and len(node.body) == 1 and isinstance(node.body[0], ast.Pass):
            self.silent_broad_excepts += 1
        self.generic_visit(node)
Enter fullscreen mode Exit fullscreen mode

Why Deterministic Structural Signals Beat Authorship Guessing

For Engineering Leads, CISOs, and Staff Reviewers, who (or what) typed a function matters much less than how expensive it will be to debug at 3 AM:

  • Control-flow nesting depth > 5 correlates strongly with missed edge cases during code review.
  • Silent broad except blocks mask upstream timeouts and schema drift.
  • Dependency delta gates force explicit justification whenever a PR adds a package outside the organization's verified lockfile baseline.

I put together a small zero-dependency reference snippet here if you want to experiment with custom AST checks in CI: evan-mercer-dev/ast-review-signals.

How is your team adapting PR review gates as commit volume scales up? Curious what deterministic checks have given you the highest signal-to-noise ratio.

Top comments (0)