DEV Community

Cover image for Why My AI Writer Waits 10 Posts Before Learning
Naheel Muhammed
Naheel Muhammed

Posted on Edited on Originally published at naheel.me

Why My AI Writer Waits 10 Posts Before Learning

Generic writing advice is a prior. Your own posts are the evidence. I put that line at the top of my dev-skills README, and it's the reason for the rule I like most in the repo: it won't change a single weight until I've audited 10 posts.

So I built dev-skills: seven agent skills and one 8 KB Python script that write in your voice, audit what landed, and feed the results into the next draft. I don't have a web app or a dashboard here. It's files a coding agent reads.

What I shipped

I pushed dev-skills today. It's MIT licensed, it has 4 stars, and it works with OpenCode, Claude Code, and other agents that load SKILL.md files.

skill what it does
/dev-post Turn one raw idea into a full draft, three titles, and a copy-ready block
/dev-human Strip AI tells from a draft and score it
/dev-comment Three angles and one full comment on someone else's post
/dev-audit Post-mortem on published posts using analytics plus log.md
/dev-learn Apply the audit: promote winning formulas and tags, demote losers
/dev-profile Score a dev.to profile and rewrite the parts that lose points
/dev-repurpose Turn a video, newsletter, or transcript into an article with a canonical URL

The loop I wired up

I keep a post history in ~/.claude/devto/log.md. /dev-post appends the date, title, formula, and tags. /dev-audit fills in engagement metrics later. dev-learn.py reads that table and rewrites the weight files that /dev-post reads on the next draft. Here is the loop as I drew it in the README:

dev-learn.py audit --apply  <--  /dev-audit  <--  analytics + ~/.claude/devto/log.md
        |  promotes winning formulas/tags, demotes losers
        +---------------------->  /dev-post's next draft
Enter fullscreen mode Exit fullscreen mode

I don't call it a feedback loop unless something actually changes. The thing that changes is a number in title-formulas.json or tag-suggestions.json.

Why 10 posts

I set MIN_SAMPLE = 10 in the script. I can't learn a formula from one post, and I don't think one landing tells me much: it could be timing, a lucky tag, or a topic that was hot that week.

Here is the guard, straight from the code:

MIN_SAMPLE = 10
...
if n < data.get("min_sample", MIN_SAMPLE):
    print(f"formulas: only {n} audited posts (need {MIN_SAMPLE}). Skipping weight updates.")
    return
Enter fullscreen mode Exit fullscreen mode

Below 10, I don't let the script touch a weight. I don't care how well your first post did.

What the script actually changes

Above 10 posts, I compute the mean engagement rate from the log. A formula or tag at or above the mean gets +0.1, capped at 2.0. One at or below half the mean gets -0.1, floored at 0.0. When a weight hits zero I move it into archive instead of deleting it, so I keep the record.

I don't write anything unless I pass --apply, so every run is a dry run by default. That means I can read the proposed changes before they land.

What I left manual

I keep slop words in dev-human/slop.json under human review, because they're voice-specific; my script only reports how many replacements are tracked. And I can't run rubric rebalancing in dev-profile yet, because my log doesn't capture follower-growth data. The script prints that limitation instead of pretending it learns from profiles. That's deliberate. I'd rather have a skill say "not enough data" than have it guess.

How I install it

I clone the repo and copy the skill folders into whichever agent I'm using:

git clone https://github.com/naheel0/dev-skills.git
cp -r dev-skills/dev-* ~/.config/opencode/skills/     # OpenCode
cp -r dev-skills/dev-* ~/.claude/skills/              # Claude Code
cp dev-skills/dev-learn.py ~/.config/opencode/skills/
Enter fullscreen mode Exit fullscreen mode

If I install somewhere else, I set DEVTO_SKILLS_DIR so dev-learn.py knows where the skills live.

First run needs ~/.claude/devto/. I create voice.md from three of my own posts before I let /dev-post write anything, because a draft in the wrong voice is worse than no draft. I start log.md with a 14-column header and append one row per post.

The honest part

I have one published post. So the guard is doing exactly what I built it to do: dev-learn.py reads my log, counts one audited post, and prints "only 1 audited posts (need 10). Skipping weight updates." Nothing has been learned yet.

I checked everything in this post against the repo: seven skills, one 8 KB script, MIN_SAMPLE = 10, 4 stars, MIT license.

Disclosure: I used AI to help draft this post. The code and the decisions in the repo are mine.

Source: github.com/naheel0/dev-skills

If you're letting an agent tune your writing weights, you're trusting it with your voice. What sample size would you trust before it starts?

Top comments (0)