DEV Community

Cover image for AI Generated Code Detection: Practical Guide for Builders
Ayush Kumar
Ayush Kumar

Posted on • Originally published at logiclooptech.dev

AI Generated Code Detection: Practical Guide for Builders

#ai #codedetection #security #python

If you need to know whether a piece of code in your repo was written by an LLM, you can run a detection step in your CI pipeline and act on the result before it reaches production. In short, AI generated code detection is a set of lightweight checks that flag suspicious snippets, let you review them, and give you a safety net against hidden bugs or licensing issues. Below I’ll walk through why it matters, the tools that actually work, how to stitch them into a FastAPI-centric workflow, and what to watch out for on the security and legal fronts.

Overview of AI-generated code and why detection matters

AI-generated code looks like any other Python file: proper indentation, sensible names, even docstrings that pass a linter. The difference is that the author is a model, not a person, so the usual assumptions about intent and testing discipline don’t always hold. In production I’ve seen three concrete failures that stem directly from unchecked AI output:

  1. Silent performance regressions – a model-generated loop that looks correct but runs O(N²) on large payloads.
  2. Missing edge-case handling – a FastAPI endpoint that forgets to validate a query param, leading to 500 errors under real traffic.
  3. License contamination – a snippet copied verbatim from an MIT-licensed repo, pulling the whole project under an unexpected license.

Detecting AI-written code early catches these problems before they cost you downtime, expensive refactors, or legal headaches. It also gives you a chance to apply the extra review rigor that AI code often needs.

What techniques actually detect AI-written code?

Not every “AI detection” tool lives up to the hype. Below are the approaches that have proven useful in a production setting.

1. Token-level entropy analysis

Models tend to produce text with lower lexical diversity than a human author. By calculating the Shannon entropy of token n-grams across a file you can flag sections that are unusually uniform. The entropy-detector Python package does this in a single function call:

from entropy_detector import file_entropy

def is_suspicious(path: str, threshold: float = 3.5) -> bool:
    ent = file_entropy(path)
    return ent < threshold

# Example usage
if is_suspicious("app/router.py"):
    print("⚠️ Possible AI‑generated code")
Enter fullscreen mode Exit fullscreen mode

It’s cheap, runs in milliseconds, and works on any language as long as you have a tokenizer.

2. Classifier models trained on human vs. AI corpora

OpenAI released detectgpt, a lightweight classifier that returns a probability that a snippet was generated by GPT-3.5 or GPT-4. The model can be called locally:

import torch
from transformers import AutoTokenizer, AutoModelForSequenceClassification

tokenizer = AutoTokenizer.from_pretrained("openai/detectgpt")
model = AutoModelForSequenceClassification.from_pretrained("openai/detectgpt")

def detect_ai_code(code: str) -> float:
    inputs = tokenizer(code, return_tensors="pt", truncation=True, max_length=512)
    logits = model(**inputs).logits
    prob = torch.softmax(logits, dim=-1)[0, 1].item()  # probability of AI origin
    return prob

# Flag if > 0.8
if detect_ai_code(open("app/services.py").read()) > 0.8:
    print("🚩 High AI‑code confidence")
Enter fullscreen mode Exit fullscreen mode

The downside is a modest GPU requirement for speed, but on a CI runner you can cache the model and run a few seconds per job.

3. Metadata and provenance checks

Many AI assistants embed comments like # Generated by Claude or # @cursor. A simple regex scan can surface those clues:

import re, pathlib

GEN_COMMENT = re.compile(r"#\s*Generated by (Claude|Cursor|ChatGPT|Bolt)", re.I)

def has_generator_comment(file_path: pathlib.Path) -> bool:
    return any(GEN_COMMENT.search(line) for line in file_path.read_text().splitlines())

# Quick scan across repo
for p in pathlib.Path("app").rglob("*.py"):
    if has_generator_comment(p):
        print(f"{p} contains generator comment")
Enter fullscreen mode Exit fullscreen mode

This won’t catch “clean” output, but it’s a zero-cost first filter.

Trade-offs

Entropy analysis is fast but noisy; you’ll get false positives on heavily templated code (e.g., Pydantic models). Classifiers are more accurate but add latency and a dependency on a GPU-enabled runner. Metadata checks are cheap but easy to evade. In practice I combine all three: cheap filters first, then the classifier on anything that passes.

How can I integrate AI code detection into CI/CD pipelines?

The easiest place to enforce detection is the pull-request stage. Here’s a minimal GitHub Actions workflow that runs the three checks and fails the job if any file crosses the danger threshold.

name: AI Code Detection

on:
  pull_request:
    paths:
      - '**/*.py'

jobs:
  detect:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3

      - name: Set up Python
        uses: actions/setup-python@v4
        with:
          python-version: '3.11'

      - name: Install deps
        run: |
          pip install entropy-detector torch transformers

      - name: Run detection
        id: detect
        run: |
          python - <<'PY'
          import pathlib, sys, json
          from entropy_detector import file_entropy
          from detect_ai import detect_ai_code, has_generator_comment  # assume we placed earlier code in a module

          def flag(file_path):
              # entropy
              if file_entropy(file_path) < 3.5:
                  return True
              # comment
              if has_generator_comment(pathlib.Path(file_path)):
                  return True
              # classifier (sample 200 chars)
              with open(file_path) as f:
                  snippet = f.read(200)
              if detect_ai_code(snippet) > 0.8:
                  return True
              return False

          flagged = [str(p) for p in pathlib.Path('.').rglob('*.py') if flag(str(p))]
          if flagged:
              print(json.dumps({"flagged": flagged}))
              sys.exit(1)
          PY

      - name: Report
        if: failure()
        run: |
          echo "AI‑generated code detected. Review the flagged files before merging."
Enter fullscreen mode Exit fullscreen mode

A few practical notes:

  • Cache the model – add actions/cache@v3 to store the detectgpt weights between runs.
  • Fail fast – the job exits with status 1 as soon as a file is flagged, preventing the merge.
  • Human review – configure the PR to require a reviewer comment on the detection step. That way the team decides whether to keep, rewrite, or replace the snippet.

If you’re using FastAPI, you can also hook this into your own internal CI (e.g., Jenkins) by importing the same detection module. I once added a pre-deployment hook that scans the generated OpenAPI schema for stray # Generated by comments, which saved us from shipping an undocumented endpoint.

Which security vulnerabilities show up in AI-generated code and how to mitigate them?

AI models are great at stitching together known patterns, but they lack a mental model of threat modeling. The most common security gaps I’ve observed are:

Vulnerability Typical AI slip Mitigation
SQL injection Direct string interpolation in execute() Enforce ORM usage (SQLAlchemy) and run static analysis (Bandit).
Open redirects return RedirectResponse(url) with user-supplied url Whitelist domains, add urlparse checks.
Missing auth Forgetting Depends(get_current_user) on a new route Use a linter rule that flags any APIRouter without the dependency.
Over-permissive CORS allow_origins=["*"] generated for quick testing Fail CI if * appears in production config files.

A practical way to catch these is to augment the detection step with security linters. For example, integrating bandit into the same workflow:

- name: Run Bandit
  run: |
    pip install bandit
    bandit -r app/ -ll
Enter fullscreen mode Exit fullscreen mode

Bandit will raise issues on the patterns above, and you can treat those as hard failures. The combination of AI detection + security linting creates a safety net: you catch both “who wrote it” and “what it does”.

Are there legal or copyright issues with AI-generated code?

The legal landscape is still evolving, but there are two practical concerns:

  1. License contamination – If the model reproduces a snippet from an Apache-2.0 repo, your whole codebase might inherit that license. A simple diff against known open-source corpora (e.g., using git clone of popular repos and difflib) can surface exact matches.
  2. Attribution requirements – Some tools (Claude, Cursor) embed a comment that is itself a license notice. Stripping it might violate the tool’s terms of service.

My rule of thumb: treat any AI-generated file as “needs review for licensing”. Add a step in the CI that runs a git grep for known license headers and fails if it finds a match that isn’t already accounted for in your project’s LICENSE file.

If you’re unsure, consult counsel early. The cost of a retroactive license audit can dwarf the few seconds you spend on a detection check.

FAQ

How accurate are AI detection models?

Current classifiers hit around 80 % precision at a 0.8 confidence threshold. They’re good enough to flag suspicious code, but never assume they’re 100 % correct - always pair with human review.

Can I run detection on compiled bytecode?

Most tools work on source text. Decompiling .pyc files is possible but adds noise, so it’s not recommended for CI pipelines.

Do I need a GPU for the classifier?

A modern CPU can run the model in ~5 seconds per file; a small GPU brings it under a second. Cache the model and batch files to keep runtimes low.

What if I generate code on the fly in production?

Treat runtime generation like any other dynamic code: sandbox it, run unit tests before loading, and never expose it directly to the internet.

Key Takeaways

  • Detect early – add entropy, comment, and classifier checks to your PR workflow.
  • Combine with security linters – Bandit, sqlfluff, or custom rules catch the most common AI-generated bugs.
  • Treat AI output as “needs review” – both for security and licensing.
  • Cache models – keep CI fast and inexpensive.
  • Don’t rely on a single signal – false positives are inevitable; layered detection reduces noise.

If you’ve hit a wall trying to wrangle AI-generated code into a stable FastAPI service, I can help you set up a robust detection pipeline and audit the existing codebase. Feel free to reach out through the hire page for a hands-on session.

Related reads:

Top comments (0)