DEV Community

squarelight
squarelight

Posted on

I reverse-engineered how AI detectors work (and built a scanner to beat them)

Hi guys,

Do you guys know how AI watermark detection actually works?

Most AI-generated text has invisible signatures—token patterns, distribution anomalies, and watermark signals that detectors can identify.

Here's how we built a tool to detect them.

The detection methods

Most commercial AI detectors use 3 main approaches:

1. Perplexity analysis

Measures how "surprised" a model is by each token. AI text tends to have low perplexity—too predictable.

2. Burstiness analysis

Human writing varies in sentence structure and length. AI text is more consistent.

3. Token-pattern fingerprinting

Each LLM leaves subtle patterns in token selection—this is the hardest to defeat.

How we built GPT Scanner

Our team (former Big Tech NLP engineers + NeurIPS/ACL researchers) built a multi-LLM detection engine:

Input text

Token pattern analysis (GPT + Claude + Gemini)

Compare against known detector signatures

Generate risk report with specific weak points

Tech stack

  • Detection: Ensemble of GPT-4, Claude 3, Gemini Pro
  • Analysis: Token-level pattern matching
  • Calibration: Tested against Turnitin, GPTZero, Originality, Copyleaks
  • Backend: Python + FastAPI
  • Frontend: [사용 기술]

Results

  • 200,000+ users
  • 99% accuracy in our test benchmark (1,000 samples)
  • #6 on GPTs Store

Ethics question

Is it wrong to help people evade AI detection?

I don't think so. Here's why:

  1. False positives are common – Human-written text gets flagged
  2. AI assistance is now normal – Using AI for grammar/ideas shouldn't disqualify you
  3. Screening is opaque – Applicants can't see what's being filtered

We believe in transparency: Let people see what scanners see, and make their own decisions.

Please try it and give me some feedbacks for better tool.

gpt-scanner.com

Top comments (0)