I built a 36-pattern checklist to catch AI writing tells in my own drafts, calibrated against everything I've published. So when a theory about why AI can't write well went semi-viral last week, I read it the way I read a bug report: I wanted to know if the explanation actually matched the failure I was seeing in my own tool's flagged patterns, or if it just sounded right.
One of the theories didn't match. It's wrong, and it's the kind of wrong that spreads because it sounds technical enough to not get checked.
The claim: inference-time optimizations — the tricks labs use to make models respond faster — are why creative writing sounds worse than it used to. Specifically, speculative decoding.
Quick version of what that is, because the claim only sounds plausible if you don't know this part. Instead of one big model generating a response token by token, you pair it with a small, fast "draft" model. The small model guesses several words ahead; the big model checks the guesses in parallel and keeps the ones it agrees with. It's a shortcut for speed. The claim going around is that this shortcut quietly degrades writing quality.
It doesn't. Speculative decoding is built to be lossless by design — the output distribution is mathematically identical to what the big model would've produced generating alone, word by word. There's no quality trade happening anywhere in the math. What's real is narrower: creative writing benefits less from the speedup, because high-temperature prose has more genuine surprise in it, so the small model's guesses get rejected more often. One inference-benchmarking writeup put creative-fiction guess-acceptance around 50-65%, versus 75-85% for code — not a peer-reviewed number, but it lines up with how the technique works. That's a story about how much faster your writing gets served, not about how good it is.
Quantization — shrinking a model's precision to save memory — can genuinely hurt output quality. Real, separate lever. But it's not the same thing as speculative decoding, and I watched people conflate the two for a full afternoon before anyone pushed back.
The part that actually holds up isn't the part people were arguing about.
Every major model goes through a training step called RLHF — reinforcement learning from human feedback — where it gets nudged toward responses that human raters rated highly. Sounds fine until you notice what raters actually reward: responses that are pleasant to skim, hard to disagree with. Over enough training, the model doesn't just get better at avoiding bad answers. It narrows toward one "safe" register and stops producing the wider range of responses it was originally capable of. That's called mode collapse, and Kirk et al. measured it directly: RLHF-trained models show meaningfully lower output diversity than the same models before that training step, across every metric in their study.
This is why the "ten drafts, then a panel of models grades and picks the best parts" trick doesn't work, and I know because I've run some version of it myself. If every draft came from a model that already narrowed toward the same safe average, grading and merging those drafts just averages the average. You don't escape mode collapse by voting inside it. You get the same failure, run twice, dressed up as rigor.
Writing has no equivalent to "the code compiles." There's no automatic checkable signal for "this is insightful," so training has nothing to push against except rater preference — and rater preference, in bulk, rewards smooth and safe over sharp and specific. That's the actual bottleneck: the training incentive itself, not the prompt, not the inference trick.
Once you name mode collapse, it's tempting to treat "AI can't write well" as one flat, unsolved problem. It isn't. It splits into two, and I write in the easier half without having fully clocked that until this week.
Nonfiction — technical writing, essays, arguments — has a nameable failure mode: conflicting optimization objectives. The traits that make a model pleasant in consumer chat (warm, hedged) are directly at odds with what good technical writing needs (precise, willing to be blunt). One reward signal, trying to serve two audiences that want different things, so the model gets pulled toward the middle of both and satisfies neither. That's a solvable engineering problem, the same way you'd split an API endpoint that's trying to serve two incompatible callers. Not fully solved, but there's already public evidence of labs shipping partial fixes aimed at exactly this (OpenAI's November 2024 GPT-4o writing update is one example).
Fiction is a different kind of hard, and I don't think it's mine to weigh in on with any authority. A good novel gets built over years of iteration on structure and character, closer to how Pixar's story team reworks a film's plot for years before a single frame gets animated. Published novels only show the finished result. The failed drafts and the years of restructuring never got kept as training data. Even if that data existed, there's limited economic incentive to solve it yet, because "does this novel move a reader emotionally over 300 pages" isn't something you can currently measure the way you can measure "does this code pass its tests."
Update (Jul 7): a sharper version of this argument surfaced after this went up, worth naming honestly. The "no training data for the process" framing has a real hole — finished novels already exist in these training corpora by the millions, so a lack of raw text isn't the bottleneck. Models like GPT-4.5 and Llama 3.1 405B suggest pretraining alone can already produce genuinely strong long-form prose. The likelier story: post-training optimization (RLHF and shrinking parameter counts chasing efficiency) is actively degrading a capability that already existed, not filling a gap that was never there. Doesn't change the nonfiction half of this piece. Does mean the paragraph above needs a rewrite I haven't done yet.
Two different problems, wearing the same complaint. One has a name and a rough fix already shipping somewhere. The other doesn't have a dataset yet, or a business case to build one.
I only write the first kind. Tutorials, arguments built from things I've actually built and broken. That puts me on the solvable side of this line, and for most of the week I was following this argument, I didn't separate that from the harder, maybe-unsolvable half everyone was actually upset about.
"Solvable" doesn't mean "solved," though, and the fix that exists today isn't a model retrain I have access to. It's an editing pass. Which is the actual reason I built that 36-pattern checklist I mentioned at the start — calibrated against my own published work specifically so it can't smooth my sentences toward a generic "good writing" average. There's no average in it to smooth toward. Just my corpus. It doesn't generate the underlying argument for me. Nothing does that yet, and nothing should. What it catches is the RLHF hedge-language creeping back in after a draft is written — the sentence that trails off into a comma and an "-ing" clause instead of just ending, the vague "many people" claim that should name someone instead.
I'm not claiming AI writing is solved. I'm saying I spent a week watching smart people argue about a math claim that turned out to be checkable in about ten minutes, and the actual fix for my half of the problem was something I could run on a draft that same afternoon.
AI helped me research, structure, and edit this piece. The arguments, the examples, and the opinions are mine. So is whatever's wrong with them.
Top comments (27)
i'd push back on the 36-pattern checklist - feels like whack-a-mole. avoid one AI tell and three new ones appear. the real issue is AI optimizes for coherence over truth. no pattern list fixes that
Mykola, fair - the list catches tone not truth. Coherence-over-truth is the real problem and no checklist fixes that...
yeah the coherence framing is sharper. a tell-filter catches surface patterns but coherent-sounding misinformation slips right through - the list won't help there at all.
' The same failure, run twice, dressed up as rigor' — brutal and accurate. Half the AI writing advice out there is elaborate machinery for averaging the average.
I like your #4 / #5 split. But I think you left out the gap in the middle — where technical writing crosses into narrative. That's the ground my series lives on.
The stories are fiction. The engineering is not. Every one of them wraps a real technical pattern — benchmark gaming, POC traps, compliance misdirection — inside a story. The RLHF smoothing you described? It hits hardest right where fiction tries to sound technical. Those sentences that want to be a story but come out reading like a cautious corporate blog post.
The editing pass doesn't land evenly across that space, which is what makes it interesting to work in. Technical paragraphs clean up fast — kill the hedge words, tighten the claims, done. Narrative paragraphs don't have a "compiles" signal. What they have instead is pacing. A paragraph that drags: you feel it before you can explain why. That's a feedback loop you can actually work with. Harder than a test suite, sure. But you don't need to build the emotional arc of a 300-page novel, either.
I think the real split isn't #4 vs #5 at all. It's simpler than that. Does your writing have something you can push against? A reader bouncing, a sentence that refuses to tighten — something that tells you you're not there yet. Technical writing has that. Narrative does too, just slower and noisier. The gray area is where you're fighting both at once.
P.S. This reply went through a few rounds of AI polish because it felt wrong to comment on an article about AI writing without being honest about it 🤣
Xulingfeng, the gray zone you're describing isn't a third category. It's evidence for something. Mode collapse doesn't check genre before it hits . your technical-narrative sentences get smoothed at the exact point they'd otherwise commit to a specific engineering claim inside a story frame, because rater-preference training punishes specificity everywhere it shows up, not just in essays. Pacing as a signal is closer to nonfiction's problem than you're giving it credit for, too.
"The reader bounced" is still a human judgment call, not a compiler.
The P.S. actually proves the point — polish and hedge-collapse aren't the same failure, which is why you could run one without the other kicking in. Does the drag you feel correlate with the same sentences your editing pass would flag for hedge-language, or are those two separate tells that just happen to land in the same paragraph?
Mode collapse as the mechanism, not the symptom — yeah that reframes things. I think I was describing what I saw without really getting why, and you just named the why.
Your last question's been bugging me (good way). Quick pass says they're separate tells sharing a paragraph, not the same thing. But "quick pass" is doing a lot of work there — I'd have to actually tag every sentence across a few drafts to know for sure. Might do that.
Have you run that test yourself? Or is this a "I know because I've stared at it long enough" kind of thing?
no, not on fiction. My checklist runs against my own nonfiction corpus nd it catches hedge-language because I can point at a sentence and say that's not how I'd have said it.
I've never had to separate that from pacing because nonfiction doesn't drag the way a scene does. It just goes vague.
So this is closer to "I know because I've stared at my kind of writing long enough" than an actual tagged result.
Your version of the test is the one worth running, since you're the one who'd notice the two tells splitting apart. If you do tag a few drafts, I'd want to know which shows up first in a paragraph that's failing .the hedge, or the drag.
"Not a real tagged result" — honestly I respect that more than if you'd made up a methodology 😂
My bet: hedge lands first, quietly. The drag shows up later when you read the whole paragraph and it just doesn't hit right. One's a typo in the code, the other's the runtime error. You see the error, but the typo was there first.
Just a guess though. I'll tag some old drafts this week. If anything interesting comes out I'll let you know. Consider it a debt. No promises though 🤣
The compile-error/runtime-error split is the useful part . a hedge you can catch reading one sentence at a time, a drag you can only catch reading the paragraph as a whole. That's not a guess dressed up, that's actually a testable order of operations. If it holds, it means the 2 tools need to run at different zoom levels, not just at different times.
No debt tracking on my end — tag them whenever and if the hedge really does show up first every time, that's worth its own comment thread...
This is a great article! I like how you explained that AI writing sounds boring not because of a speed trick, but because it is trying too hard to be safe and nice. Your idea to use a personal checklist to fix the writing and remove the boring parts is very smart. Thank you for sharing this!
I like this way of thinking on multiple levels:
Peter . That's the sharper version of it, actually: no penalty for average is the whole reason average keeps winning...
The speculative-decoding-is-lossless point is the one most threads get wrong, so I'm glad to see it stated plainly: the target model verifies every token, so the output distribution matches ordinary sampling. The acceptance-rate gap you cite for fiction vs code tracks with entropy, high-temperature prose rejects more draft tokens and loses the speedup, not the quality. Do you think the RLHF flattening is measurable on a fixed prompt set, or only once rater preferences are baked in?
Kartik, measurable on a fixed prompt set . That's literally the methodology. Generate several completions per prompt, run distinct n-grams or self-BLEU across them, compare the SFT-only version against the RLHF version. You don't need the reward model or the raters in front of you to see it; the flattening shows up as a property of the output distribution itself, not something that requires knowing what got baked in. "Baked in" is the cause, not a precondition for detecting the symptom.
Are you asking because you're building a diagnostic against your own model's outputs, or because you want to pin down exactly where the causal story stops being testable??
I like the distinction between generation quality and optimization objectives. It feels similar to software engineering if you optimize a system for one KPI, you'll often sacrifice another. Models tuned to be consistently helpful and low-risk naturally converge toward safer outputs, which makes editing and evaluation more important than expecting the first draft to carry originality on its own.
Mateo, that's the right frame, and it's worth pushing one step further: in software, you at least get to pick the KPI and you can see the sacrifice happening in a dashboard. With RLHF, the "KPI" is rater preference in aggregate, which nobody picking a single model output ever sees directly — you just get the flattened result with no visibility into what got traded away to produce it. That's why editing matters more than people expect:
you're not correcting a known trade-off, you're reverse-engineering one that was never surfaced to begin with. Curious whether you've seen the same pattern on the eng side . does optimizing hard for one metric ever get invisible the same way, or does someone always end up naming the trade-off out loud during review?
I agree with the editing argument.
I'd add one more layer:
Before editing, AI needs an accurate understanding of the system it's writing about.
Better understanding produces better first drafts.
RapidKit fair enough, tho 'accurate understanding' is doing a lot of work there.
What does that look like concretely, for you??
Good question.
For me, it means the AI doesn't have to reconstruct the system from scattered files every time it writes.
It starts from an evidence-backed understanding of the workspace: architecture, ownership, dependencies, contracts, impact, and what's verified versus inferred.
That shifts the first draft from educated guesses to informed explanations
still abstract. Name one project where that changed the first draft, and I'm in
I think we're seeing lots of open-source projects solving different parts of the same problem.
_Workspace Intelligence _is our attempt to define the layer that brings those pieces together into an evidence-backed understanding of a software system.
Many companies are starting to recognize the limitations of AI. I even read an article saying Ford had to rehire its development team after laying them off in favor of AI just last week.
Benjamin — real story, though worth noting. it's about production quality control, not writing. Different failure mode than the one here
that is true
Really enjoyed your analysis! It’s eye-opening to learn RLHF mode collapse, not speculative decoding, makes AI writing bland. Your split between nonfiction and fiction struggles gives me a new angle. Totally agree that unique voice and core ideas are still our responsibility as creators.
Some comments may only be visible to logged-in visitors. Sign in to view all comments.