I've been building an AI-assisted editorial pipeline for technical writing. Notion cards become markdown drafts in the repo, pass through review, then sync to dev.to.
Last month I shipped a post about the first big fix to my editor-critique reviewer skill: The AI reviewer scored 23/25 and missed the point. The problem was sequence. A score-first pass treated a polished rubric as the first lens and produced QA feedback when I needed editorial feedback. Reordering the skill so analysis precedes scoring fixed that.
I assumed the next improvements would come from rubric tuning. Longer prompts. Another scoring dimension. Sharper checklists.
That assumption was half right. The rubric still matters. But every useful fix after the baseline shared a different shape.
The pattern I kept missing
After I reordered analysis before scoring, reviewer failures kept arriving from different incidents. A critique that agreed with itself too easily. Drafts that grew every revision without getting shorter. A middle section that felt like a second article.
Each time I reached for the same lever: expand the rubric, add a rule, lengthen the prompt.
Incident 1: When the reviewer needs to argue with itself
editor-critique produced decisive scorecards and prioritized feedback, but the report rarely challenged its own conclusions. A draft could earn Ready to sync with medium items left unexamined.
Score-first review had failed because it judged too early. This failure was different: the primary critique could be thorough and still under-falsified.
The fix was another staged pass. After the primary critique drafts, freeze it. Run adversarial review that assumes the primary assessment is wrong until draft-supported counter-evidence proves otherwise. Then synthesize: change the publication recommendation only when falsification is material.
I added adversarial review, synthesis, and canonical report assembly as new skill steps. A follow-up pass tightened adversarial review with an anchor requirement: every counter-evidence bullet must name the frozen primary claim it challenges. No orphan hypotheticals like "title spoils thesis?" when the primary critique already praised title strategy.
Before:
Editorial read-through
→ Score
→ Critique
→ Post report
After:
Editorial read-through
→ Score
→ Primary critique
→ Adversarial review (frozen inputs)
→ Synthesis
→ Post report
That was the first time staging a different kind of reasoning into its own pass beat rubric expansion. Two more failures would repeat the same shape before I stopped treating it as coincidence.
Incident 2: When critique only adds
Self-falsification helped, but drafts were still growing. Investigation while critiquing Upgrades don't have to be a blind trust exercise showed feedback was consistently additive, but not subtractive. editor-critique found missing framing and evidence boundaries reliably. It did not ask what should be removed when new material arrived.
The result was layered drafts: an opening stacked on another opening, the same four-step investigation loop restated in three sections, a mental-model diagram that walked through event flow the prose had already established in the previous section.
The fix was not "be shorter" in the rubric. It was naming another cognitive job in the read-through: subtractive editing. Every paragraph should continue earning its place. Flag existing redundancy and addition-induced redundancy. Pair expansion recommendations with material that would become redundant if adopted.
A companion technique, single-owner ideas, lists 2–4 core ideas and flags when the same idea appears in multiple sections without new evidence. I codified subtractive editing in the skill file along with a test case that catches additive-only critique regressions and a lightweight subtractive pass in the human revision step.
The primary critique still owns expansion. Subtractive editing is a separate observational pass, not a rewrite engine.
Incident 3: When a section becomes its own article
The last failure pushed past critique mechanics into reader cognition. While critiquing draft variants in my editorial workflow, several middle-body sections were technically correct but felt wrong in context. In one draft, an implementation walkthrough interrupted the investigation arc. In another, a full section on validation tooling read like its own mini-article.
The failure mode was narrow: a section stopped advancing the reader's current question and temporarily made another explanatory thread the center of gravity.
Adding a rubric dimension for "section focus" would have been vague. What worked was an observational lens in the editorial read-through step: name the primary thread, name the secondary thread, decide whether to compress, delay, embed later, or leave as-is.
I codified this as a Secondary explanatory thread lens in the skill file. The rubric stayed the same. It simply added a named cognitive job: track whether prose is serving the reader's current question or drifting into a side article.
What stayed constant
Three incidents, three skill changes, one pattern. Across all three, a few constraints held:
- The five-dimension rubric stayed mostly intact.
- Read-only governance did not change: critique still does not write repo files or gate publish.
- Each pass added another observational lens, not another scoring dimension.
- The expensive part was naming the cognitive job precisely enough to operationalize in a skill file.
The recurring mistake was treating undifferentiated reasoning as one pass. Each fix changed the sequence, not the rubric weight. A capable reviewer can read before it scores and still under-read if falsifying primary judgment, displacing redundant prose, and tracking reader focus all compete in the same step.
Before you expand the rubric
- List the failure modes that survived your last sequence fix.
- For each one, name the cognitive job that failed (self-falsification, subtractive editing, reader-focus tracking).
- Stage that job as its own observational pass with a frozen handoff to the next step.
- Expand the rubric only if that observational pass still misses failures in production.
Once editor-critique understood before judging, the remaining improvements came from separating kinds of reasoning into distinct stages, not from a bigger rubric or a longer single pass. I suspect the pattern may generalize beyond editorial critique.
Takeaway: When a reviewer skill plateaus after a sequence fix, ask which cognitive jobs are still sharing one undifferentiated pass. Stage them before you expand the rubric.
If you'd like to see the project behind these workflow experiments, try Codenames AI.
Top comments (3)
This hits close to home. I went through something very similar with my series — kept reaching for the "sharper rubric" lever until I realized the problem wasn't the rubric. The "naming the cognitive job precisely enough" part is exactly where the leverage is. Appreciate you writing this up — validating. 👀
That's exactly the shift I was hoping to capture. I kept treating each failure as a missing rubric rule until I realized they were different kinds of reasoning competing in the same pass. Once the cognitive job had a clear owner, the fixes became much more obvious.
Different kinds of reasoning competing in the same pass" — that's a really clean way to put it. I found that once I gave each "kind of reasoning" its own seat at the table (and told them they're allowed to disagree), the quality bar moved way more than any single pass ever did. Have you experimented with having them disagree on purpose, or do you keep them in sequence?