DEV Community

Cover image for "Write Like a Human" Wasn't Enough, So I Built a Skill That Measures AI Prose
ilien
ilien

Posted on

"Write Like a Human" Wasn't Enough, So I Built a Skill That Measures AI Prose

Most of the documentation I read at work these days was written with AI. READMEs, internal documentation and notes shared within the team. I don't have a problem with that. Nobody is going to write the documentation for every project by hand, because that would throw away most of the time AI saves you. The problem is that I have to read those texts again and again, and they're tiring. They read bland and impersonal, they pile up filler before getting to the point, and sometimes the structure is so unnatural that they end up confusing. You read them to stay informed and to not leave everything in the hands of the AI, and it gets heavy.

So I built Quirón, a skill for Claude Code (and other agents) that edits that kind of text and then measures the result. It wasn't my first try. It's the result of several experimental skills that never quite did what I wanted, plus trying other skills very close to it. This post is about what I learned on the way, including one result that didn't go the way I hoped.

Telling the model to "write like a human" makes it overshoot

The first thing everyone tries is adding "write like a human" to the prompt. It doesn't work, and the reason is interesting.

A 2026 working paper (TextPulse Research, not peer reviewed) gave four flagship models a detailed brief on what separates human prose from assistant prose: uneven sentence length, plain words, no lists of three, no connective openers. The models followed it. Then they kept going. Sentence-length variation went to 0.58 where the humans were at 0.40. Claude Opus 5 overshot the most of the four. The one property the brief didn't name, vocabulary range, didn't move at all.

So you trade one tell for another: prose choppier and plainer than any person writes. You can't feel where the ceiling is while you're writing. You have to measure it.

Measuring against real dev.to posts

That's the core of the skill. It has a list of 33 patterns to remove (the "it's not X, it's Y" contrasts, lists of three, a summary section at the end, headings everywhere, the one-line closers that restate the paragraph), and a script that measures 23 features of the text.

The human baseline is a bit of a love letter to this site: 167 dev.to posts by 21 authors, all published before 2022, so before ChatGPT. For each feature the script takes the 10th to 90th percentile range of those posts. Then it tells you whether your text sits inside it, on the AI side, or past the human side. Every fourth human post was held out and never used to pick anything.

This is what it says about an assistant-written Stripe tutorial from the repository's evaluation set:

feature                   this   human band      verdict
long words (7+) /1k     353.83   159.68 - 250.46  above band, AI side
nominalizations /1k      55.21     7.29 - 32.65   above band, AI side
em dashes /1k             3.76     0.00 - 2.74    above band, AI side
lists of three /1k       11.29     0.00 - 5.51    above band, AI side
plain words /1k          33.88    43.19 - 95.24   below band, AI side
...
13/23 features inside the human band (p10-p90 of 167 human texts)
Enter fullscreen mode Exit fullscreen mode

The rewrite of that same post, built from the author's own notes, scores 23 of 23.

A few things surprised me when the numbers came in. Delve barely matters anymore: it showed up in 1% of 2026 assistant posts or fewer, about as often as in human ones. The stronger signal is the plain words models leave out, like very or able, which people use all the time. Lists of three turned out to be the strongest single feature. And assistant posts carry about twice the headings of human ones (a median of 12.2 against 6.6 per 1,000 words).

A checklist that doesn't grade itself

Next to the meter there's a checklist with four states. PASS and FAIL are what you'd expect, but FAIL is kept narrow on purpose: the wrapper lines a chat assistant puts around its answers, or several AI signals at once. My first version failed any em dash and any Title Case heading, and it failed every single one of the held-out human posts. The current rules pass 98% of them and still fail most of the Sonnet and GPT posts written on the same titles.

The other two states are TELL, a pattern humans use too but assistants use more, and READ, something no regular expression can settle, like whether the text is generic or whether a fact got lost. A READ item isn't a pass. The model has to write a ruling on each one, and the loop only stops after two clean runs in a row.

One more check exists because every rewriter I tested slipped in small invented details. A version number here, "in the same week" there. An invented detail reads as more human, so no style check can see it. The skill diffs the rewrite against your draft and notes and fails on any number or link that isn't in them.

The result that humbled me

I had fresh Claude Opus instances judge posts one at a time and give the probability that each one was AI. Human originals got 5% to 8%. Untouched assistant drafts got 90%.

Then the rewrites. With every metric inside the human range they were still judged AI 12 times out of 12, at 82%. Adding samples of the author's writing didn't move it. What did move it was the author's own notes: what actually happened, real names, links, opinions. With those it came down to 37% and 53% in two rounds, and 10 of 24 posts were judged AI. On 8 fresh titles it went back up to 68%, all 8 judged AI.

Twelve posts per row is small. The judge and the writers are also the same model family, so read that with care. But the direction was clear. Style alone doesn't do it. Your own material does. That's why the skill now asks for your notes before rewriting anything, and says plainly when it doesn't have them.

What surprised me most, though, wasn't a number. Once it started working, the texts became fluid and familiar. I could read an AI-assisted document from start to finish without the filler, the known patterns or the heavy structure. That was what I was really after.

What it's for, and what it isn't

I know it can produce text that reads like a person wrote it, even when the AI wrote all of it. Even if you get great results that way, please don't. Passing off text the AI generated as your own, when you didn't write any of it, is wrong, and I'm not encouraging anyone to do it. The skill itself has a warning about this. It also doesn't beat detectors like GPTZero, which read token probabilities and never look at style.

Using AI to end up with a better-written text is a good thing, though. It's very good at catching the mistakes and habits we all have. It works best on text you wrote yourself and want to polish, or text written with AI help that still has real human input in it. And for that project documentation nobody will write by hand, it should at least be easy to read when you need to get familiar with a concept or an internal workflow at the company.

When to use something else

humanizer and stop-slop both do their job well, and I don't want to take credit away from them. If you want something lightweight, stop-slop is a short set of rules you can even paste into your custom instructions. humanizer has a very complete pattern list and a draft, audit and rewrite flow. Quirón makes more sense when you want a script to verify the result instead of the model grading itself, when you want to catch overshoot, or when you write in Spanish. I think it covers what they do and goes further, but try them and judge for yourself.

On languages: it's tuned for English and Spanish, the two I speak (English barely, I'm still working on it). It also works in other languages and identifies the structural patterns, but the best results are in those two. If you speak something else, try it and tell me how it went. Contributions for new languages are welcome.

Don't take my word for any of this. Run it on your own documentation and look at the measurements. The README was written with its help, so it's a quick way to see how it reads, and the install steps are there too.

GitHub logo ilien-dev / quiron

AI humanizer skill for Claude Code, Codex and Cursor. Removes AI slop and the signs of AI writing, then measures the rewrite against human baselines.

Quirón: a centaur holding up an eight-point star

English · Español

Quirón: an AI humanizer skill that measures its own work

An agent skill for Claude Code, Codex and Cursor. It strips the AI slop that marks prose as machine-written,
then checks the result against measured human writing instead of a hunch.

Python 3 standard library only Registers: blog, fiction, Spanish Claude Code skill skills.sh installs Website

Hand it a draft and ask it to humanize the text, or to make a README sound less like ChatGPT. It rewrites the prose, then measures the result. You get the rewrite and a measurement that says whether it now reads like a person wrote it.

Install

With the skills CLI, for Claude Code and every other agent it supports, Gemini CLI and GitHub Copilot among them:

npx skills add ilien-dev/quiron
Enter fullscreen mode Exit fullscreen mode

It needs no API key. It runs on the model you already use, and the scripts need nothing beyond the Python 3 standard library.

As a Claude Code plugin

Run these two commands inside Claude…

How are you dealing with AI-generated documentation in your team? And if you try it, I'd like to know what it gets wrong.

Top comments (0)