If you write documentation, API references, or runbooks for a living, and you've ever run a draft through an AI detector out of curiosity or because a client asked you to, you may have noticed something uncomfortable: your score tends to run higher than you'd expect. Not because you used AI to write any of it, but because of how disciplined technical writing is structured at a sentence level, which happens to overlap statistically with the patterns detectors are built to catch.
The style guide is working against you
Most technical writing style guides mandate the exact features an AI detector treats as suspicious signals. Active voice, consistent terminology used the same way every time it appears, short declarative sentences, and parallel structure across similar sections of a document. A style guide that says "use the same term for the same concept every single time" is directly asking you to lower your own vocabulary diversity score, because vocabulary diversity is literally a measure of how much you vary your word choices across a passage. A guide that says "keep sentences short and direct, avoid subordinate clauses where possible" is asking you to lower your sentence-length variance, which is exactly what detectors read as low burstiness, one of their core suspicious signals.
None of this is a coincidence in the sense that both language models and disciplined technical writers are optimizing toward the same statistical target: predictable, low-noise prose that's easy to parse quickly under time pressure. The model gets there because it was trained explicitly to minimize surprise at each next word. The technical writer gets there because a confused reader in the middle of a production incident at 3am is a reader who cannot afford ambiguity in a runbook step. Two completely different reasons for writing, arriving at the same statistical destination, and a detector cannot tell the difference between the two just by looking at the finished text.
Where this actually causes real, costly problems
Some organizations now run detector checks on contributed documentation, whether from external contractors or from community contributors to open source docs, as a spam filter or a plagiarism check before accepting a contribution. A contractor who has been writing clean, consistent API reference documentation for a decade can trip a detector's threshold simply by doing their job well and following the house style guide, and then has to explain a statistical artifact to a project maintainer who has never heard the word "perplexity" used this way and doesn't know what a false positive rate even means in this context.
This is a real, measurable cost, not a hypothetical inconvenience. A flagged contribution that gets held up for extra review, or a contractor invoice that gets questioned because of a detector score attached to the deliverable, wastes time on a signal that was never validated for this use case in the first place. The fix isn't to deliberately write worse, more chaotic documentation just to dodge a score, that would be actively harmful to the people relying on the docs. The fix is for whoever is reviewing flagged content to understand what the score is actually measuring, and what it demonstrably is not measuring, before treating it as evidence of anything at all.
What actually distinguishes AI text from disciplined human text
If detector scores alone aren't reliable for this specific population of writers, what does actually help separate the two in practice? A few things a purely statistical model has no access to at all: whether the writing references specific internal system names, unusual edge cases, or incident history that the writer would only plausibly know from direct, hands-on experience with the system being documented; whether the phrasing matches the same author's prior documentation in tone, structure, and characteristic word choices over time; and whether there are small, characteristic imperfections, a slightly awkward but functionally correct explanation of a genuinely tricky edge case, that tend to show up specifically in writing from someone who actually debugged the thing they're describing rather than someone summarizing it secondhand.
None of these checks are automatable the way a single detector score is, which is exactly why they get skipped in practice in favor of the faster, cheaper, and considerably less reliable signal. The Google Developer Documentation Style Guide and the Microsoft Writing Style Guide both codify, in explicit detail, the consistency-first approach that produces this exact statistical overlap with machine-generated text, for anyone who wants to see the actual published guidance technical writers across the industry are following and understand why the overlap exists structurally rather than coincidentally.
A concrete example of the mismatch in practice
Take a real sentence from a hypothetical API changelog: "The rate limit was increased from 100 to 500 requests per minute for authenticated endpoints." That's a textbook agentless passive, following house style exactly as written, because the changelog reader cares about the rate limit change, not about which engineer on the platform team flipped the config value. Run that same sentence through a detector tuned on general web prose and it will likely register as unusually predictable: common vocabulary, a passive construction, no rhetorical flourish anywhere in it. Now compare it to a sentence from that same writer's personal blog written the same week: "I spent way too long last night trying to figure out why our rate limiter kept doubling requests, and it turned out to be a stale cache entry nobody had touched in months." Same person, wildly different statistical profile, because the genre and the audience are different. A detector comparing only the changelog sentence against a generic human-writing baseline has no way to know it's looking at disciplined technical prose rather than machine output, and that's the exact gap that produces a false positive on a writer who did nothing wrong.
A reasonable process fix for teams that rely on these scores
If your team relies on an AI detector as part of a contribution review process for documentation, it's worth running a few samples of your own team's most experienced, most trusted writers' historical documentation through the same detector first, before using any threshold from that tool against outside contributors. Seeing your own baseline false-positive rate on writing you already know for a fact is human and original is the fastest way to calibrate how much weight the score deserves in your actual review process, and it often produces a genuinely uncomfortable but useful result. The Write the Docs community guidelines are a good general reference for documentation review practices that don't lean on a single automated signal as a gate.
There's a fuller breakdown of the underlying statistical reasoning, including the specific research on which populations of writers get flagged most often and why, over on EvvyTools' recent piece on why detectors misfire on genuine human writing. If you want to check where your own writing currently sits before anyone else runs it through a detector and hands you a number with no context, the site's own AI content detection tool breaks the score down sentence by sentence instead of handing you one opaque figure to argue about after the fact.
Top comments (0)