DEV Community: untactit

Playwright's fill() threw an error and still inserted the text: the CodeMirror 6 double-paste trap

untactit — Wed, 19 Aug 2026 08:31:34 +0000

Browser automation has a standard reflex: if the action throws, retry it. CodeMirror 6 broke that reflex for us in the most expensive way possible — the throw and the success were the same event.

This is a short field note on the failure, the mechanism as far as we could actually measure it, and the fix that made it structurally impossible to hit again.

The symptom

We were automating a post editor built on CodeMirror 6, filling a long article body with Playwright's fill().

fill() threw a timeout error
Our handler treated the throw as "nothing happened" and retried
The saved article came out with the entire body duplicated

The first fill() had thrown and inserted the content. The retry inserted it again. Every layer of that pipeline behaved reasonably, and the output was still wrong.

What we could verify, and what we could not

CodeMirror 6 manages a contenteditable surface through its own view model — the DOM you see and the editor state behind it are separate structures. fill() inserts text, then verifies the result against its own expectation of the element's value. On a CodeMirror surface, that verification can fail even though the insertion event was fully processed.

That is as far as we measured. We did not chase the internals further, because the useful lesson is not the specific bug — it is that "exception means not executed" is an assumption, and some editors break it.

The fix: one path, then count

We stopped trying to make the retry smarter and instead made double-insertion structurally impossible.

1. Insert through one path that replaces instead of appending.

await editor.click();
await page.keyboard.press('Meta+A');
await page.evaluate(text => {
  document.execCommand('insertText', false, text);
}, body);

Select-all before insert means a retry overwrites the previous attempt instead of stacking on top of it. execCommand is deprecated, but it goes through the browser's editing pipeline — which is what contenteditable frameworks actually listen to — and in practice it is still the most stable way in.

2. Verify by counting, after reload.

We plant a unique marker string at the end of the body, save, reload, and count occurrences:

const count = await page.evaluate(m =>
  document.body.innerText.split(m).length - 1, MARKER);
if (count !== 1) throw new Error(`marker x${count} — duplicated insert`);

The success check moved from "did the call return cleanly" to "does the saved artifact contain exactly one copy." Those are different questions, and only the second one is about reality.

The general shape

For write operations, an exception is not proof of non-execution
Before any retry of a write, re-read the current state — the previous attempt may have partially landed
Non-idempotent operations (append, insert, submit) deserve a marker you can count afterward
Judge success from the saved artifact, not from return values or toasts

The same trap exists anywhere a "submit" or "add row" can throw after the side effect landed. When a write throws, the first question is not why did it fail — it is did it actually fail.

Field note from building untactit, a control plane for the skills, rules, and memory AI agents run on — currently pre-launch. The product applies the same rule: a write counts as done only after the target is read back.

24 of the 30 duplicate groups were not duplicates

untactit — Tue, 18 Aug 2026 23:02:07 +0000

We grouped an asset database by title and found 30 groups of "duplicates." Content hashing dissolved 24 of them. Five were real. One was unjudgeable — because the rows had no content at all.

This is a field note on deduplication judgment, from auditing a store of AI-agent assets (conversations, skills, memories, instruction files) collected across many platforms. The numbers are from a live audit on 2026-08-19.

Title matching is a grouping key, not a judgment

The 30 groups came from exact title matches. That is a fine way to find candidates. It is a terrible way to decide. For each group we judged on four axes instead:

sha256 of the full content — identical bytes end the discussion
Similarity of the stripped body — headers removed, then compared
Provenance — which system claims to have produced the row
Creation-time gap — seconds apart and months apart mean different things

Here is how the 30 groups actually broke down.

5 groups: real duplicates

Identical content, twice in the store. Every one had a mundane cause: the same asset written under a underscore slug and a hyphen slug; the same memory saved once under an English key and once under a localized key; the same body captured under two different provenance IDs.

Real duplicates are boring. That is the point — they are the minority, and they are the only rows you may safely collapse.

6 groups: same title, different asset

Two conversations titled identically, created 45 days apart, 8% body similarity. A 213KB thread and a 27KB thread sharing a name. One pair sat at 85% similarity with a ten-month gap — almost certainly a person pasting an old conversation to continue it, which makes it two assets, not one.

Collapse these and you destroy history. Title reuse is normal human behavior, not corruption.

7 groups: probes that look like twins

Verification probes — synthetic assets written to test a pipeline — differ only by an embedded marker. Two probes from different runs look like duplicates to every fuzzy matcher. They are separate events, and folding them together erases the record of one run.

If your system writes test artifacts into real storage, your dedup logic needs to know what a probe looks like. Ours does now.

12 rows in 4 groups: nothing to judge

The strangest finding: rows whose body was entirely empty — header only. Same titles as real assets, no content. All of them were created inside one window of about 8.7 minutes on a single afternoon, up to five copies of one title, seconds apart.

That pattern is not a duplicate problem. It is the fossil record of a write-path incident — something retried row creation without writing bodies. The dedup lesson: when the only evidence you have is absence, you cannot classify. These rows go to quarantine, not to a merge queue.

The provenance field was lying too

Three pairs shared the same provenance reference while holding completely unrelated content — different platforms, different topics, different everything. The field that was supposed to anchor identity had collisions.

Any pipeline that had trusted provenance alone to say "already collected, skip" would have silently dropped real assets. We found the collisions only because content hashing disagreed with the reference field — two signals, arguing, is what an audit is for.

The scoreboard

Verdict	Groups	Safe action
Real duplicate (byte-identical)	5	Archive one copy
Same title, different asset	6	Keep both
Distinct test probes	7	Keep both, tag as probes
Empty-shell rows	4 (12 rows)	Quarantine, investigate the writer
Provenance collisions	3 pairs	Re-verify at the source

80% of what looked like duplication was not. The uncomfortable inverse also holds: a store that "has no duplicates" by title may be full of byte-identical content under different names. Both errors come from the same root — judging identity by the cheapest available key.

Notes from building untactit — pre-launch — where asset identity across platforms is the whole job. Content hash plus provenance plus read-back verification; never title alone.

The ledger said 763 assets. The platforms said about 400.

untactit — Tue, 18 Aug 2026 23:02:02 +0000

A sync ledger fails silently. Ours said 763 assets. When we stopped trusting it and counted by hand — opening 21 AI platforms in browser tabs, calling each platform's own API from the page context, comparing ID by ID — roughly 400 of those were real.

This is the audit log of that gap. The numbers are from one working account, measured on 2026-08-18. Nothing here is hypothetical.

The setup

The database held every asset collected from the platforms a working account actually uses — conversations, projects, skills, instruction files, memories — across Claude, ChatGPT, Gemini, Grok, Perplexity, Kimi, Qwen and fourteen others. 763 rows total.

The verification method matters more than the tool:

Call the platform's own list API from inside an authenticated page (fetch with credentials: 'include'), paginating to exhaustion.
Cross-check against what the UI actually shows.
Where possible, compare ID sets — intersection and both differences — not counts.

A count that matches can still be two different lists. Only the ID diff tells you what is actually there.

The three ways a ledger lies

The 363-row gap was not one bug. It was three independent failure modes, each of which produces a database that looks healthy.

Failure mode	What happens	Confirmed rows
Ghosts	Deleted on the platform, still alive in the DB	at least 173
Misses	Exists on the platform, never made it into the DB	at least 63
False positives	Platform-provided defaults counted as user assets	at least 109

Note the shape of the problem: ghosts and false positives inflate the ledger, misses deflate it. They partially cancel. A total count is the one metric guaranteed to hide all three.

The three worst cells

A page-size limit became a count. One platform's skill list showed 100 in the DB. The user had installed 4. The other 96 were the platform's public catalog — shown to every user — and the number 100 was simply the API's page-size cap, recorded as if it were a fact about the account.

Example strings became memories. A memory collector stored 2 items for a platform where the user had 8 real saved memories. It had missed all 8 — and stored two lines of the settings page's placeholder text ("e.g. ...") as if they were user data. On the dashboard, this cell looked like one of the working ones.

Files that were never there. 42 instruction rows pointed at local rule directories that did not exist on the machine at all. Nothing had been deleted; the source path had simply never existed on this host. The collector wrote 42 rows anyway.

Why "the sync ran successfully" means nothing

Every one of these rows was written by a run that reported success. That is the actual lesson:

Success of the write says nothing about truth of the read. The collector faithfully stored what it fetched. What it fetched was a catalog page, a placeholder string, a stale cache.
Pagination that stops early looks identical to pagination that finished. 63 real assets were missing because list calls stopped at page one and the UI showed a green check.
Counts drift toward plausible. 763 is a believable number for a heavy user. Nobody questions a believable number.

What actually fixes it

The fix is not a better collector. It is a read-back audit as a first-class operation: after any sync, re-read the platform and diff IDs in both directions, and treat "platform-provided default" as a category your matcher must know about. Run it on a schedule, not once.

If you run agents across several tools, try the count yourself. Pick one platform, list what your ledger thinks is there, then page through the platform's own API to the end. The first diff usually takes ten minutes and is rarely empty.

This audit is part of building untactit, a control plane for the skills, rules, and memory AI agents run on — currently pre-launch. The read-back verification described here is the product lesson: deployment without re-reading the target is how ledgers rot.

AGENTS.md was supposed to end the sprawl. I counted the files on one laptop.

untactit — Sun, 16 Aug 2026 21:57:32 +0000

AGENTS.md solved the format argument. Codex CLI, GitHub Copilot, Cursor, Windsurf, Amp and Devin
all read the same file now. Claude Code still reads CLAUDE.md, Gemini CLI still reads GEMINI.md,
and that exception list keeps getting shorter.

The format was never the expensive part.

I scanned one working laptop this morning. Nothing exotic — one developer, a few years of projects,
the normal agent tooling:

file	copies on disk	distinct contents
`AGENTS.md`	111	19
`CLAUDE.md`	38	19
`GEMINI.md`	2	2
`SKILL.md` (agent skills)	3,073	887

Method: find ~ -maxdepth 7 -type f -name <X>, pruning node_modules, .venv*, site-packages
and Library/Caches. "Distinct contents" is the count of unique md5 hashes. Vendor-installed skill
bundles are included in the SKILL.md row, which is exactly the point — they are on the machine and
an agent can read them.

111 files. 19 different things being said.

Nested files are a feature. A monorepo should scope instructions per package, and the spec says so.
But 19 distinct contents across 111 copies is not scoping. That is copy, edit one copy, forget.

Where the copies come from

1. The hand copy. You write a rule you like. You paste it into the next repo. Six months later you
fix a typo in one of them. Every other copy is now wrong and nothing tells you.

2. The machine boundary. Global config lives outside every repo: ~/.claude/CLAUDE.md,
~/.gemini/GEMINI.md, ~/.config/opencode/AGENTS.md. Two laptops, two versions, and the difference
surfaces as "it behaves differently on my machine" three weeks later.

3. The generator. The common fix is to keep one source file and generate the rest —
AGENTS.md → CLAUDE.md → .cursor/rules/*.mdc → .github/copilot-instructions.md. It works on
the day you run it. It does not survive the next person who edits a generated file directly, because
nothing on disk marks a file as generated, and nothing checks.

I wrote about that failure mode separately:
Generating your agent rules from one file does not stop them drifting.

Where each agent actually looks

Worth having in one table, because the paths are the thing people get wrong:

tool	file it reads
Claude Code	`CLAUDE.md` (project root, subdirectories, `~/.claude/`), `.claude/skills//SKILL.md`, `.claude/agents/.md`
Codex CLI	`AGENTS.md` (root + nested, nearest wins)
GitHub Copilot	`.github/copilot-instructions.md`, scoped `.github/instructions/*.instructions.md`
Cursor	`.cursor/rules/*.mdc` (legacy `.cursorrules`)
Windsurf	`.windsurf/rules/*.md` (legacy `.windsurfrules`)
Gemini CLI	`GEMINI.md` (`~/.gemini/`, workspace root, subdirectories)
opencode	`AGENTS.md`, falls back to `CLAUDE.md`, global at `~/.config/opencode/AGENTS.md`

A script that shows you the spread

agent-drift is a single-file Python script, MIT, no dependencies. It walks a tree, finds every
instruction file the agents above read, groups them by content hash and prints the ones that disagree.

curl -O https://raw.githubusercontent.com/untactit/agent-drift/main/agent_drift.py
python3 agent_drift.py ~

Repo: https://github.com/untactit/agent-drift

It does not fix anything. It tells you how bad it is, which is the part most teams are missing —
you cannot argue for a process change without a number.

What we are building

I work on untactit. One place to hold the skills, rules and memory your agents
run on, review a change once, and have it land in every target without anyone opening a file.

The distinction that matters: a person decides what gets approved. Nothing after that is touched by
hand. Drift is not something we repair afterwards — it stops being produced, because nobody is
editing copies any more.

It is pre-launch. The scanner above is useful without it, and that is deliberate.
If you want the longer version of the argument, it is on the
product page.

If you run the scan on your own machine, I would like to see the numbers. My guess is that the
AGENTS.md row is worse than mine on any team of more than three people.

Why your AI agent instruction files drift across machines

untactit — Thu, 13 Aug 2026 06:35:16 +0000

You write CLAUDE.md once. Six months later there are eighteen of them on your laptop and you cannot say which one is right.

This is not a hypothetical. I scanned my own machine and found exactly that. What follows is why it happens, and why the obvious fix does not fix it.

The copy is created by the tool, not by you

Every agent reads instructions from a path it owns.

Tool	Path
Claude Code	`CLAUDE.md`, `~/.claude/CLAUDE.md`
Cursor	`.cursor/rules/*.mdc`
GitHub Copilot	`.github/copilot-instructions.md`
Codex / others	`AGENTS.md`

You did not decide to keep four copies. You decided to use four tools. The copies came with them.

Drift is not caused by carelessness

Three mechanics produce it, none of which involve anyone being sloppy.

Editing at the point of use. An agent misbehaves mid-task. You fix the instruction file that is open in front of you. That file is one of four, and the fix lands in one.

The checkout is not the unit. You have the same repo cloned twice — one for the main branch, one for a long-lived branch. Both have instruction files. They diverge the moment either is touched.

Home directory rules are invisible. ~/.claude/CLAUDE.md belongs to no repository. It is in no CI. It is where the rules you did not want to argue about in review end up.

Generating from one source does not close it

The natural response is to stop keeping copies: write AGENTS.md once and generate the rest.

I built that. It works. It is agent-fanout — single-file Python, zero dependencies, MIT.

python3 agent_fanout.py

create    CLAUDE.md
create    .cursor/rules/from-agents-md.mdc
create    .github/copilot-instructions.md

Generated files carry a header so nobody edits them directly, and --check in CI turns a hand-edited derivative into a red build.

That covers this repository, where CI runs. Which sounds like everything until you list what falls outside:

Global config in the home directory — no repo, no CI
The window between edits — an agent reads a file the moment it changes; CI notices on push, if there is a push
Repos with no CI — prototypes, throwaway clones, last week's experiment. The place instructions get rewritten most freely and gated least
The unit is the machine, not the repo. The agent runs on a laptop holding many checkouts, several copies of the same repo, and a home directory. Anything scoped to one repository cannot see across that.

So measure, do not only prevent

Prevention is policy. Detection is measurement. Policy gets bypassed in ways you cannot see unless you measure.

agent-drift scans paths rather than repos, and groups by content similarity rather than filename:

python3 agent_drift.py ~/work ~/side-projects

Scanned 47 instruction files.
DRIFT: 2 documents, 5 distinct versions between them.

  claude-code:CLAUDE.md
    6 copies, 3 versions
      9b01aeaa204d  3 files, 406 lines
      2dc3c616c279  2 files, 411 lines

Filename matching is useless here. Unrelated projects having different CLAUDE.md files is correct, and a tool that reports that as drift stops being used. Point it at your whole working directory, not one project — the interesting results cross repository boundaries.

Two jobs, not one

	Scope	Question it answers
agent-fanout	one repository	are the derived files current
agent-drift	whole machine, many paths	where did copies diverge

Generation removes the reason copies exist. Detection catches the copies that exist for reasons you did not anticipate. Doing only one leaves you feeling covered, which is worse than knowing you are not. I walked into that myself.

Where this ends

Honestly, the end state is that neither script is needed — assets do not sit scattered as files on a machine, and one reviewed copy reaches every machine. That is what I am building at untactit, currently pre-launch.

The scripts do not depend on it. Take whichever part is useful.

Generating your agent rules from one file does not stop them drifting

untactit — Tue, 11 Aug 2026 22:51:21 +0000

The obvious fix for "our agent instruction files keep diverging" is to stop maintaining copies. Write AGENTS.md, generate CLAUDE.md and .cursor/rules/*.mdc and .github/copilot-instructions.md from it, done.

I built that. It works. It also does not solve the problem, and the gap between those two statements is worth spelling out, because I only saw it after running the thing on a real machine.

What generation actually covers

python3 agent_fanout.py

create    CLAUDE.md
create    .cursor/rules/from-agents-md.mdc
create    .github/copilot-instructions.md

One source, several derived files, a header on each so nobody edits the derived copy by accident:

<!-- Generated from AGENTS.md by agent-fanout. Do not edit this file. -->

Add it to CI and the pull request that edits CLAUDE.md directly turns the build red:

- run: python3 agent_fanout.py . --check

That covers this repository, on the machines that run CI. Which sounds like everything until you list what it isn't.

What it does not cover

Other repositories. Your team has more than one. Each has its own AGENTS.md, and they were copy-pasted from each other at some point. Generation keeps each repo internally consistent while the repos drift apart from one another.

Global config. Claude Code reads ~/.claude/CLAUDE.md in addition to the project file. Cursor has user-level rules. Those live outside any repository, are never in CI, and are exactly where people put the rule they didn't want to argue about in review.

The window between edits. Generation runs when someone runs it. Between that moment and the next CI run, a derived file can be edited and used. The agent reads it immediately; CI notices on push, if there is a push.

Repos without CI. Prototypes, scratch clones, the repo someone made last Tuesday. Those are where instructions get freely modified, and they are the ones with no gate at all.

Machines, not repositories. The unit that runs an agent is a laptop. A laptop has many checkouts, several of the same repo, and a home directory. Nothing that operates per-repository can see across that.

So you need to look, not just prevent

Prevention is a policy. Detection is a measurement. Policies get bypassed in ways that are invisible until you measure.

python3 agent_drift.py ~/work ~/side-projects

Scanned 47 instruction files.

DRIFT: 2 documents, 5 distinct versions between them.

  claude-code:CLAUDE.md
    6 copies, 3 versions
      9b01aeaa204d  3 files, 406 lines
      2dc3c616c279  2 files, 411 lines

This one scans paths rather than repositories, and groups files by content similarity rather than by filename — two unrelated projects having different CLAUDE.md files is not drift, and reporting it as drift makes the output worthless. (Getting that grouping right took three rewrites; I wrote that part up separately.)

Run it across your whole working directory, not one project. The interesting results are the ones that cross repository boundaries.

The pairing

	Scope	Answers
`agent-fanout`	one repository	"are the derived files current?"
`agent-drift`	whole machine, many paths	"where did copies diverge anyway?"

Generation removes the reason copies exist. Detection catches the copies that exist for reasons you did not anticipate. Neither is redundant, and doing only the first one gives you a false sense of coverage — which is the actual failure mode I want to warn about, because it is the one I walked into.

Both are single-file Python, no dependencies, read-only where it matters, MIT:

agent-fanout — write AGENTS.md once, generate the rest
agent-drift — find the copies that no longer agree

Where this goes

The honest end state is that neither script is necessary, because the assets are not files sitting on laptops at all — they live in one reviewed place and reach every machine without anyone copying anything. That is what I am building at untactit, currently pre-launch.

The scripts stand on their own and don't depend on it. Use them, ignore the rest.

I found 18 versions of the same CLAUDE.md on one laptop

untactit — Tue, 11 Aug 2026 22:29:37 +0000

Your team runs Claude Code, Cursor, and Copilot. Each of them reads a file before it acts — CLAUDE.md, .cursor/rules/*.mdc, .github/copilot-instructions.md. Those files get copied: into another repo, onto another laptop, into someone's scratch directory.

Then somebody edits a copy.

Nothing errors. No warning, no diff, no failing test. The agent on that machine just behaves differently — a rule someone removed six weeks ago, a convention only half the team's agents know about. It shows up as "works on my machine" with nothing to bisect.

I wrote a script to find those files and tell you which copies disagree.

curl -O https://raw.githubusercontent.com/untactit/agent-drift/main/agent_drift.py
python3 agent_drift.py ~/work

Scanned 47 instruction files.

DRIFT: 2 documents, 5 distinct versions between them.

  claude-code:CLAUDE.md
    6 copies, 3 versions
      9b01aeaa204d  3 files, 406 lines
        ~/work/api/CLAUDE.md
        ~/work/web/CLAUDE.md
      2dc3c616c279  2 files, 411 lines
        ~/work/infra/CLAUDE.md

One file, standard library only, Python 3.8+. Reads only — it never writes to the files it scans. MIT.

The interesting part is what I got wrong

The naive version took me ten minutes and was useless. The three fixes are the whole story.

Grouping by filename is wrong

First version: collect every CLAUDE.md, hash them, report the ones that differ.

Ran it on my own machine. It reported 21 files as one document with 18 versions.

Of course it did. Unrelated projects have unrelated instructions. They were never supposed to match. A tool that reports that as drift is noise, and a noisy tool is worse than no tool — you stop reading it by the third run.

Files are the same document when their content says so, not when their filename does.

A bag of words is not enough either

So: normalise whitespace, tokenise, compare with Jaccard similarity, threshold 0.7.

Still one giant cluster. I measured actual pairs to find out why:

012Hki23QBMr ↔ 016ct6suZ14M : 0.000
012Hki23QBMr ↔ 016avoYh97Eu : 0.764

Instruction files written for the same tool share almost all of their vocabulary. claude, skill, agent, file, commit, never, always. Two documents with nothing in common still overlap heavily as word sets.

Switched to 5-word shingles — the set of every consecutive 5-word phrase. Vocabulary can collide; phrasing rarely does. Re-measured: unrelated pairs dropped to 0.000, genuinely forked documents landed at 0.53–0.76. That separation is what you want, and it came from measuring, not from reasoning about it.

Single-linkage clustering chains everything together

Still one cluster. This one is a classic.

Union-find with "similar to any member" means A joins B, B joins C, and now A and C are in the same cluster despite sharing nothing. One weak link and the cluster swallows the tree.

Each cluster now keeps a representative — its largest member — and a file joins only if it resembles that representative directly. Clusters stay coherent regardless of scan order.

Final result on my machine: 15 files, 9 of them 0.53–0.65 similar to the representative. Those really are forks of one template. Correct detection, and the number is small enough to act on.

Put it in CI

- run: python3 agent_drift.py . --fail-on-drift

Exit 1 when drift exists. The build goes red the day two copies of your team's instructions stop matching, instead of you finding out in six months when someone asks why the agent stopped following a rule.

--json gives you the full report if you'd rather post-process it.

What it covers

Claude Code, Codex/AGENTS.md, Cursor, GitHub Copilot, Gemini, Windsurf, Cline, Aider, Continue, Zed. The patterns are one constant at the top of the file — PR or issue if yours is missing.

After you find it

Detection is the easy half. The copies exist for a reason, and until that reason is gone the drift comes back. What removes it is one reviewed source that reaches every machine without anyone pasting anything.

That's the problem I'm building untactit around — it's pre-launch. This script doesn't depend on it and never will.

Repo: github.com/untactit/agent-drift