"Learn Deterministic and AI-Assisted Findings With One Buggy Function"

This function accepts 0 even though the requirement says quantities must be positive. A deterministic test can prove the mistake every time. An AI-assisted reviewer might explain the intent or propose a patch, but its finding and fix still need review. That difference is the lesson, not a contest between two kinds of tools.

GitHub announced Code Quality general availability on July 20 for Enterprise Cloud and Team. The latest verified official signal describes deterministic CodeQL plus AI-assisted detection and reviewable Autofix. GitHub also reports an internal resolution metric; treat that number as GitHub-reported, not a universal benchmark. Unverified secondary July 27 claims are excluded; no newer official replacement was found.

Prerequisites

You need Python 3.11 or later and no third-party packages. This tiny exercise imitates the categories, not GitHub Code Quality or CodeQL internals.

Create quality_lesson.py:

def subtotal(quantity: int, unit_cents: int) -> int:
    """Return cents for a strictly positive quantity and price."""
    if quantity < 0:                 # bug: zero slips through
        raise ValueError("quantity must be positive")
    if unit_cents <= 0:
        raise ValueError("price must be positive")
    return quantity * unit_cents


def deterministic_rule(source: str) -> list[str]:
    finding = "if quantity < 0:"
    return ["QTY_ZERO: use <= for a positive-only contract"] if finding in source else []

Then create test_quality_lesson.py:

import unittest
from pathlib import Path
from quality_lesson import subtotal, deterministic_rule

class QualityLessonTest(unittest.TestCase):
    def test_normal_purchase(self):
        self.assertEqual(subtotal(2, 350), 700)

    def test_zero_is_rejected(self):
        with self.assertRaises(ValueError):
            subtotal(0, 350)

    def test_rule_finds_fixture(self):
        source = Path("quality_lesson.py").read_text()
        self.assertEqual(len(deterministic_rule(source)), 1)

if __name__ == "__main__":
    unittest.main()

Run:

python -m unittest -v

Expected output includes a failure for test_zero_is_rejected and passes for the other two tests. That visible contradiction teaches more than a green demo: the rule recognizes a known syntax fixture, while the behavior test checks the requirement.

Three evidence roles

Role	Example in this lesson	What it can establish	What it cannot establish
deterministic rule	exact < 0 pattern	repeatable match on this source	full program meaning
AI-assisted finding	proposed explanation	a review hypothesis	guaranteed correctness
reviewable Autofix	suggested <= 0 patch	candidate change to inspect	permission to merge

Real CodeQL is far richer than this string rule. Deterministic means repeatable for defined input, not complete or error-free.

Apply the candidate fix:

-    if quantity < 0:
+    if quantity <= 0:

Rerun the tests. Expected normal path: all three tests pass, the rule no longer reports its fixture, and a human confirms that rejecting zero matches the written contract.

Now try the failure path by changing the requirement to “zero means remove the line item.” The same patch would violate the revised contract even though the original test passes. Update the behavior tests first; this shows why a plausible Autofix must remain reviewable.

Common mistakes are treating assisted suggestions as facts, deterministic alerts as proof of impact, green tests as completeness, or vendor metrics as reproduced results.

Extension: add a discount_percent range, write boundary and invalid-input tests first, then demonstrate a semantically equivalent implementation that a naive source rule misses.

Use this checklist to grade yourself:

• Can you state the requirement without referring to the implementation?
• Does the normal input have an exact expected result?
• Does at least one invalid input fail before the correction?
• Can the deterministic check be rerun without judgment calls?
• Is every assisted finding labeled as a hypothesis until verified?
• Is the proposed change reviewed as code rather than accepted by origin?

The exercise does not provide CodeQL coverage, enterprise setup guidance, or evidence about GitHub's reported metric. Check current plan terms before adoption.

Optional platform evaluation

Once the lesson works locally, MonkeyCode is a separate candidate learning environment: an open-source AGPL-3.0 AI development platform with an overseas online option, managed server-side cloud development environments, model/task/requirement management, and build/test/preview. It is free to start, but I have not executed this exercise there. Keep the test as the judge if you open the official campaign option.

Disclosure: This article promotes MonkeyCode using an official campaign link. I’m a MonkeyCode user, not affiliated with the project, and I receive no commission from this link.

AI assistance disclosure: This article was drafted with AI assistance and reviewed against the cited primary sources.