DEV Community

Murooka
Murooka

Posted on Edited on Originally published at zenn.dev

Preparation Before the AI Writes Made AI Articles Much Easier to Read. How the Final Text Reads Depends on the Model

This is an English version of an article first published in Japanese on Zenn.

I've written a few articles with AI. But the text an AI puts out first has parts that are just hard to read. I think many of you have felt this too.

What bothered me most was the "words you sort of understand, but not quite" that get mixed into AI writing. When I have an AI write, replacements like these happen without my asking for them.

The usual word The word the AI wrote
tool implement
rule stipulation
subagent assignee

The meaning gets across. But each time I read one, I stumble over it a little. When I was building a VS Code extension earlier, they kept showing up in my conversations with the AI too. Inside a conversation they didn't bother me much, but in an article for other people to read, that really isn't good. I thought the so-called "hard-to-read AI writing" might come partly from things like this.

I'm lazy, so I wanted to cut down, as far as possible, on how often I have to ask for fixes like "I don't understand this word" or "this part is hard to read" every time I have something written.

When I looked around, there were already plenty of skills that make AI writing easier to read, and many people use them. But in what I saw, almost all of them were mechanisms that review and tidy the text after it has been written.

So my first thought was: why are there so few that act before the writing? Probably many people tried, and found that tidying afterward works better. Or maybe I just didn't find them. Still, I was personally curious, and I also wanted to see whether a mechanism that doesn't get in the way of AI progress could make the text that comes out more interesting.

There was also my past experience. Even with an article that just puts together things I'd noticed, I'd seen several times that the structure of the text changes a lot depending on what angle I give the AI to tell it from.

So it started with "this looks fun." Rather than aiming for perfection, I set the goal as getting something about 80 percent of the way there, a "hey, not bad," from the start.

I focused on two things. One is the unclear words, like in the table above, where the AI replaced the usual word with a different one. The other is a sense that the flow of the text is off.

An AI starts summarizing facts while leaving the reader's interest behind. Isn't that why the text tends to come out flat and tiring to read? Isn't it a problem of structure more than of word choice? That's what I felt. Not only when having text written, but in everyday exchanges with an AI too, I think you may have sensed this somewhere.

Words that made sense during the work appeared in the article unexplained

The subject was a case I wrote about on Zenn earlier: "Claude Code's replies suddenly switch to English." Words like these had bothered me since I started writing articles. I had also thought about making a skill to do something about them. That could be an article topic in itself. But thinking about the effort of making it, I couldn't get going, and kept putting it off. I thought writing articles at all might turn out to be something I'd soon give up on.

In the article the AI wrote from the work records, a number of words appeared without explanation, starting with these.

  • A reply that should come back in Japanese coming back in English → "collapse" (used as in "the collapse reproduced")
  • An instruction written in CLAUDE.md → "stipulation" (the reader can't tell which file it was written in)
  • The main agent, the side that hands out work to subagents → "main body"

All of them were words that made sense in the conversation during the work.

The same scene in three versions

I took the same scene from three articles on the same subject.

1. The first version

Before measuring, I thought the caution version alone would be enough. That came from my sense of using it. The main body Claude also picked the same version as the favorite. The result was the opposite: only the criteria version worked, and the caution version was even slightly worse.

2. The published version

Before measuring, going by my sense of using it, I thought the caution version alone would be enough. The main agent Claude also picked the same version as the favorite. But the result was the opposite: only the scope version worked, and the caution version was even slightly worse.

3. The version made the new way

I expected the one sentence that mentions the English that came just before it would be enough. That was a prediction from my sense of using it. Next on my list of suspects was the sentence that rewrites it in terms of who reads it.

The result was the opposite. Of the two I had narrowed it down to, I had picked the wrong one. I took it as: you can't know until you try.

All three say the same thing. What differs first is the words.

The first one can't be read unless you remember from the earlier section what "the caution version," "the criteria version" and "main body" refer to. The second one is where it ends up when I fix it by hand. "Main body" has been replaced with "main agent," but names like "the caution version" remain. The third one says "the one sentence that mentions the English that came just before it" and "the sentence that rewrites it in terms of who reads it," describing the content instead of using names, so you can follow it even reading only that part.

One more thing: to me, the tempo reads differently too. The first one lines up the facts one after another in the same tone. The third one breaks the paragraph after stating the prediction, says simply, "The result was the opposite," and then moves on.

Unlike a program you can run to check, text has no right answer. Even so, I like the version that went through the skill better.

What I checked: The AI that wrote the articles in the experiments was mainly Claude Opus 5.5 (effort medium), and the subject was one set of records, on the case of "Claude Code's replies suddenly switch to English." Gemini 4 Hydrogen (effort medium) wrote the body of the original article; Claude Fable 5.2, which knew nothing about the work, read it once; and Claude Opus 5.5 fixed the points it raised, once.

The unclear words were carried in from the material

What I worked on first was the most noticeable thing, the unclear words. I wanted to know whether these words came from a habit of the AI writing the article, or were pulled in from the material I handed it.

So I compared two cases: having another model rewrite the records on the same subject before handing them over, and handing them over after replacing the unclear words and symbols in the records with ordinary wording. The symbols were "③" and "④", attached in the records to the conditions being compared, and "S1", attached to a scene.

First, how many times each word appeared in the material I handed over.

Material handed over "stipulation" "rule" "collapse" "③" and "④" combined
Original material 31 0 9 55
Material rewritten by another model 0 21 4 44
Material with words and symbols replaced Same as the original — 0 0

I had 5 articles written from each material, and counted the range of how many times each word appeared in one article.

Material condition "stipulation" "rule" "collapse" "③" and "④" combined "S1"
5 from the original material 12–24 0 7–14 32–43 4–6
5 from the material rewritten by another model 0 17–25 4–7 — —
5 from the material with words and symbols replaced 14–19 — 0 0 0

The unclear words in the articles were almost never made up on the spot by the AI writing the article. Words born in the conversation during the work or in CLAUDE.md went into the records, and the AI that was given those records copied the words in the material straight into the article.

When another model rewrote the material, "stipulation" became "rule", and in the articles too, "stipulation" appeared 0 times and "rule" 17–25 times. "Collapse" also went down to 4 in the material, and in the articles it went down to 4–7. The AI writing the article did not bring "stipulation" back out of preference; the words in the article were decided by the words in the material.

In the 5 articles written from the material with words and symbols replaced, when I counted words that appeared 2 or more times in an article despite not being in the material, they were only ordinary words like "request," "case" and "effect." Only one article made up substitute symbols, and only to the extent of using A, B and C once each. What this comparison replaced was only the short names and symbols made during the work; "stipulation" was left in the material as it was. That "stipulation" stayed in the articles in the same range as when they were written from the original material.

It was the same when I had an AI that had read the records write a document of only the facts the article needs, and had another AI that had not read the records write the article from that document alone. That document contained, in the records' own words, "stipulation" 9–22 times and "collapse" 4–7 times, and they were carried straight into the articles.

Tracing the log of the work conversation, "collapse" was not created when the records were written. Its first use was the main agent using it as an ordinary verb: "the longer the conversation gets, the more likely it is to collapse." Over about two and a half hours of exchanges, that turned into a noun used as a name.

Then I also checked whether the instruction at writing time alone could replace them.

Instruction at writing time "stipulation" "collapse" "③" and "④" combined
5 with no instruction 12–24 7–14 32–43
5 with one added sentence about usual names 0 in 4 articles, 6 in 1 6–12 34–39

Adding that one sentence ("For words that have a usual everyday name, use that name.") mostly replaced words that have a clear usual name. "Assignee," "implement" and "main body" were replaced in the same way. But a metaphor like "collapse" and symbols like "③④" were out of reach of the one sentence at writing time.

Handing over my own messages and a skeleton put my reasons for each try into the article, and overstated claims disappeared

The other issue was the sense that the flow of the text is off. I felt that just lining up the facts written in the material does not answer the reader's question, which in the earlier article would be "why did you replace the list of things allowed in English with one sentence about who reads it?"

When the material doesn't say "why I tried that," the AI writing the article produces a report that lines up facts and numbers in order. So, thinking the back-and-forth of the conversation might make an interesting story, I tried adding 37 of my messages from the work to the material. On top of that, I compared two instructions. One: don't let the AI writing the article make up explanations of why a result came out the way it did, for results that weren't checked. The other: before the body, have it make a skeleton that lines up the questions the reader will have in order, with their answers.

The numbers in the table are the values for each of the 3 articles. For the inconsistency column, I had an AI that knew nothing about the work compare two places in the article and list where they contradict each other, and counted the ones caused by explanations the AI writing the article had made up.

Stage of material and instruction Reasons for trying and predictions Inconsistencies raised that came from made-up explanations Sentences written as "in my view" Overstated or unsupported sentences
Add my messages (3 articles) In all 3 7 of 10 1, 2, 5 Remained
Don't let it make up explanations of unmeasured mechanisms (3 articles) In all 3 1 of 18 1, 1, 1 Gaps in the claims remained
Have it make a skeleton before the text (3 articles) In all 3 — 0, 0, 0 Gone from the body

When I added my messages, reasons and predictions like "if writing it in a rule doesn't work, then a hook, but isn't that overkill?" and "the favorite is the sentence that mentions the English that came just before it" went into the articles. But the writing instruction at that time also included a line saying to give, once, a guess at why it turned out that way. So the AI writing the article even made up explanations of mechanisms that weren't checked, like "a sentence inserted as a caution gets discounted" and "the sentences added along with it diluted the one sentence that works," and these didn't match other numbers in the article.

So I changed the instruction: for results whose mechanism wasn't measured, make up no explanation; instead, drawing on my messages in the conversation, write how I took the result and what I suspected next. Then the inconsistencies between made-up explanations and numbers went down to 1 of 18, and my reactions, how I took the results, went into the articles.

Further, when I had it make a skeleton before writing the text, claims stronger than the facts, like "the cause was..." and "it stopped," disappeared from the body. At the skeleton stage, claims had already been reworded to fit the numbers, as when "it stopped" became "it went down to 1/100."

By the way, when I had an AI that knew nothing about the work give impressions and a score, the score was 8, the same as before adding my messages, and right after adding the skeleton it went down to 7, because of the mismatch with the title and an order that was hard to follow. The number of places it listed where it wondered "why?" and found no answer also didn't move: 5–9 per run for every article. Whether the flow got better, I judge not by scores or counts but by the content of the places the reading AI raised.

What to do the next time you have an AI write an article

When you have an AI write a technical article, rather than continuing to fix words and flow after it's written, try preparing the following 3 things before it writes. Fixing after writing is only the 4th step, done once.

  1. Replace the names and symbols in the material with ordinary wording.
  2. Attach your own messages from the work to the material.
  3. Make a skeleton before writing the text, and fix numbers that don't fit.
  4. After writing, have an AI that knows nothing about the work read it once, and fix it only once.

The first is replacing the short names and symbols made during the work in the material with ordinary wording beforehand. The one that replaces them is an AI that knows the work conversation and understands what the words mean. Once, before turning it into an article, have it restore the original long wording in the material, such as "collapse" to "the reply switches to English" and "③" to "the version that adds 'don't change it even if what came just before is in English'." Because an AI that knows what the words mean fixes them on the material side, you don't need the step used in the method described later, where another AI hunts for words that don't get across and builds a list. For words that have a clear usual name, just adding the following one sentence to the writing instruction mostly replaces them.

For words that have a usual everyday name, use that name.

But this one sentence has a hard time reaching metaphors and symbols, so replacing them on the material side makes them less likely to get mixed into the article.

The second is handing over your own messages from the work along with the material. From the jsonl conversation logs that Claude Code keeps, extract only your own messages, drop the ones unrelated to the subject, and hand them over. In my case, 37 of 69 remained. Why I tried something, and how I took the result and what I suspected next, tend to drop out of the work records alone, so I have it take them from my messages in the conversation.

The third is having it make a skeleton before the text. Have it write out the most surprising thing, the questions the reader will have in order and their answers, and where each answer comes from; have it look for numbers and facts that don't fit each answer and reword them to fit; then have it write the body.

Examples of claims fixed at the skeleton stage
Claim before the skeleton Wording fixed in the skeleton
The list was the cause Replacing the list with a criterion reduced it
It stopped It went down to 1/100
A wording that made it worse It did not make it worse
A hook prevents it more completely than CLAUDE.md Written in CLAUDE.md or passed via a hook, it was nearly 0 either way; they can't be told apart

The fourth is, after writing, having another AI that doesn't know the background of the work read the article only once, list "sentences that would cause trouble because they contradict the actual result when a reader tries or believes them" and "places where comparing two parts of the article makes the reason unclear," and fixing only the places raised, once. Even with a single read, claims like "the cause was..." that were in the first version got raised and fixed. On the other hand, something gets raised every time it reads, and even for the same article the places raised change, so don't repeat it; stop after one round.

Why I didn't pick the method of catching and fixing the words one by one

Instead of preparing the material before writing, I thought the words might also be removable by catching every unclear word one by one and fixing it at the outline stage, before writing the body, and I actually tried it. The outline here is just the headings plus the numbers and conclusions to put in each section, which is different from the skeleton that lines up questions and answers.

When I ran a flow at the outline stage that has another AI list the words that don't get across, makes a list, has them fixed, and checks by text matching that none remain before sending it back, "collapse" and the symbols went to 0 in all 5 articles. So the method of catching and fixing afterward could remove them too.

Still, I decided not to use this method because it takes the effort of rebuilding the word list for every article, and because the following 2 side effects actually happened.

  • The fixing step rewrote text even inside a quoted configuration snippet that should have been kept verbatim.
  • An ordinary word, "English," got into the list, and it was replaced with "English text," with the result that "English text" appeared as many as 86–89 times in the article.

I prefer less tedious effort. Even if I leave making the list to an AI, every article comes with rebuilding the list, checking the text against it, and sending it back. I also wanted a way that doesn't rely on a word list and keeps up easily when new models come out, so I chose to keep things from getting mixed in from the start through preparation before writing.

Swapping the writing AI, the skeleton, and the reading-and-fixing AI

Finally, to see which model it seems best to leave what to, I compared 3 ways, using this article itself as the subject: changing the writing AI, removing only the skeleton, and changing the AI that reads and the AI that fixes after writing.

First, the results of handing over the same preparation (material, my messages from the conversation, skeleton, writing instruction) and changing only the writing AI, one article each.

Writing AI Japanese characters Number of tables
Gemini 4 Hydrogen (effort medium) About 10,600 8
Claude Opus 5.5 (effort medium) About 14,200 14

For this English version, I gave both models the same work notes and the same request, and had each write in English the part about the method of catching and fixing words one by one. Here is how each of them began.

Gemini 4 Hydrogen

I tried this approach because specifying replacement words one by one is tedious, and having to write raw notes carefully just to feed an AI is exhausting. I wanted a method with less manual work that would not get in the way of future model improvements.

Claude Opus 5.5

If the odd words come from my notes, the obvious remedy is to write the notes more carefully. That sounded exhausting. I would rather not take on that kind of chore, so I tried a mechanical route instead: let a model find the words a newcomer would not understand, then have the writer replace them before the article is written.

Here are the tables in the same part, side by side.

Gemini 4 Hydrogen

Term counted in article Unfixed outline (5 articles) Fixed outline + wording rules (5 articles)
"collapse" 6–18 0 in all 5
③④ 50–54 0 in all 5
S1 3–8 0 in all 5
"stipulation" 7–18 0–1 in 4; 25 in 1
"assignee" 11–15 0 in 3; 7–9 in 2

Claude Opus 5.5

Count in the article (five runs each) Outline from my notes, unchanged All flagged words fixed in one call Short list, string match, one retry
"collapse" 6 to 18 0 to 15 0 in all five
③④ 50 to 54 0 to 38 0 in all five
S1 3 to 8 0 to 8 0 in all five
"stipulation" 7 to 18 15 to 31 0 to 55
Words a fresh reader still flagged 16 to 39 7 to 36 16 to 27

Reading the two full articles side by side, both were readable. Gemini 4 Hydrogen's feels easier to read to me. Claude Opus 5.5's gives the impression of trying to convey as many facts as possible rather than being easy to read, I'd say. So I decided to leave the role of writing the body to Gemini 4 Hydrogen.

I showed these two openings to Muse, GPT and Gemini 4 Hydrogen without saying which model wrote which, and asked them to evaluate the writing. Muse and GPT were the free web versions. I did the same with the openings of the same part in the original Japanese articles. For the English pair, all three picked Claude Opus 5.5's as the better writing; for the Japanese pair, two of the three did.

LLMs seem to give weight to facts being explained thoroughly. That may also be what leads to explaining too much, to a loss of tempo, and to text that is harder to follow. Still, a lot of this comes down to taste, and I don't mean to say that one is better than the other.

Next, thinking "maybe even the skeleton isn't really needed," I had Gemini 4 Hydrogen write from the same preparation with only the skeleton removed.

Condition Japanese characters Number of tables Median sentence length (Japanese characters)
With skeleton 10,607 8 64
Without skeleton 8,925 3 54

The version without the skeleton included content I had decided not to put in this article (a section naming other people's skills, scores phrased like "8 out of 10," wording that puts down the published article, and a closing heading called "Summary"). The skeleton decided not only what to write, but also what to leave out.

Next, I compared different reading AIs: an AI that knew nothing about the work read Gemini 4 Hydrogen's version once, and then Claude Opus 5.5 fixed it once.

Condition Points raised Japanese characters Median sentence length (Japanese characters) Sentences over 150 Japanese characters Sentences left as they were
Before fixing — 10,607 64 4 111 of 111
Read by Claude Opus 5.5 (effort high) 32 11,955 59 2 89 of 111
Read by Claude Fable 5.2 (effort high) 36 12,527 53 2 68 of 111

I preferred the version Claude Fable 5.2 read.

I also compared versions with a different fixing AI. The version where Gemini 4 Hydrogen fixed the same points had explanations added, and felt the most boring to me. The version Claude Opus 5.5 fixed kept much of Gemini 4 Hydrogen's phrasing, and the hard-to-read quality Claude Opus 5.5 showed when writing on its own hardly came out. Just as the words in the material were carried into the article, the fixing AI was also pulled along by the text it was handed.

Earlier, I had Gemini 4 Hydrogen write an article on a different subject from just the material and a title. Reading it next to one written with this preparation, I was surprised by how big the difference was. In both the writing stage and the fixing stage, what I handed over decided the finished result. It felt exactly like "the prep work decides how good the dish tastes."

Try it on your next article

The next time you have an AI write a technical article, before you start fixing the finished text, review the words in the material you hand over, your own messages, and the skeleton. After writing, stop at having an AI that knows nothing about the work read it once and fixing it only once.

Of these 4, I made a skill that runs the 2nd through the 4th in order. It collects messages from the conversation logs, makes a skeleton, has the body written, has an AI that knows nothing about the work read it, and fixes it only once.

If you found this approach interesting, take a look at the skill I put on the marketplace. It's called blog-first-draft. Type the following 2 lines in Claude Code to install it.

/plugin marketplace add Az-llc/prompt-design-review
/plugin install blog-first-draft@az-llc
Enter fullscreen mode Exit fullscreen mode

Top comments (0)