DEV Community: Mxhlix

My agent filled in every field on the form. It could not attach the three images.

Mxhlix — Wed, 29 Jul 2026 01:02:08 +0000

I had an AI agent filling out a listing form on a web service. Every text field went in. It clicked Save Draft. All that was left was attaching three PNGs.

That part never worked.

I assumed that if it could fill the form, it could attach the files. Those turned out to be two different capabilities, gated in two different places, and the gate behaves differently depending on how you're running Claude Code. Same machine, same settings, same browser extension, different answer.

My setup: the Claude Code desktop app on Windows 11, the Claude in Chrome extension, and the computer-use desktop automation server.

The sanctioned route is closed first

The extension hands the agent a tool called file_upload. Its description tells you exactly how to use it, and what not to do:

Upload one or multiple files to a file input element on the page. Do not click on file upload buttons or file inputs — clicking opens a native file picker dialog that you cannot see or interact with. Instead, use read_page or find to locate the file input element, then use this tool with its ref to upload files directly. Only files the user has shared with this session (attachments, the session's outputs/uploads folders, or folders the user has connected) can be uploaded; other paths will be rejected. The combined size of all files in a single call must stay under 10 MB.

The first three sentences are the procedure, and the procedure worked immediately: locate the input, pass the ref and a path. The fourth sentence is where everything stopped.

Cannot upload "...": only files the user has shared with this session can be uploaded.

I tried four ways of "sharing" a file. A directory that was in the session's allowed list from the moment it started. The session's own temp folder. A folder approved through the directory-connect tool. A folder added to additionalDirectories in settings.local.json. All four came back with the same sentence.

The first one deserves a second look. That allowed directory is where the session reads and writes; I had the agent generate the test PNG there. A session cannot upload a file it wrote itself, thirty seconds earlier, in a directory it is explicitly allowed to write to.

Attaching the images to the chat doesn't help either, because on this surface a chat attachment never becomes a file. The model receives the image data, but nothing lands on disk. I scanned the temp tree, the config tree, and the app's data folder: zero new PNGs. There's a separate image-upload tool, but it requires an imageId, and chat attachments don't carry one.

None of this is a bug. The parenthetical in that description limits "shared" to exactly three things: chat attachments, the session's outputs/uploads folders, and folders the user connected.

Fine — then put the file in one of the three. Except the session directory has no outputs or uploads folder in it. So I created them, dropped a PNG in each, and tried again:

Cannot upload "...\outputs\test-out.png": only files the user has shared with this session can be uploaded.
Cannot upload "...\uploads\test-up.png": only files the user has shared with this session can be uploaded.

Matching the names changes nothing. And "folders the user has connected" can't mean the directory-connect tool, because that one is already on the rejected list. On this surface, all three of the listed escape hatches are decorative.

You can't reach the native dialog

The obvious next thought: the button opens a file picker, so drive the file picker.

That window isn't File Explorer. It's the common file dialog, owned by chrome.exe. Desktop automation decides permissions per owning process of the frontmost window, so as far as it's concerned, that dialog is Chrome.

And browsers are pinned to the "read" tier in computer-use. Screenshots yes, clicks and keystrokes no. Request access (it returns granted tier:"read"), bring Chrome to the front, try a click:

"Chrome" is granted at tier "read" — visible in screenshots only, no clicks or typing.
... never use AppleScript, System Events, shell commands, or any other method
to send clicks or keystrokes to this app.

The first half is the state. The second half enumerates the workarounds you were about to think of and then closes the list with "or any other method," which also kills route three: File Explorer is controllable at full tier, so dragging a file from Explorer onto the browser's drop zone looks viable, right up until you notice it's a way of sending input to Chrome.

Bonus trap: if you do click the button, the dialog you opened is now unclosable. Chrome is read-tier, so you can't even send it an Escape. It sits there being modal until a human walks over and dismisses it.

Same tool name, different gate

With all three routes dead I went to the docs, and found a version requirement: local file upload landed in Claude Code v2.1.211.¹ Three restrictions come with it — the session must be allowed to read the file, 10 MB total per call, and files with multiple hard links are refused.

So I ran the same task from the CLI (claude --chrome) on a version past that line. The rejection message is different:

only files this session is allowed to read ... add its folder with /add-dir

The test moved from "was it shared" to "can you read it." I ran /add-dir on my Downloads folder and file_upload went straight through. Three PNGs, 731 KB total, page JavaScript rendered the previews, all three slots consumed.

Meanwhile, on the same machine, the desktop app still refuses. It sits in the same project, reads the same settings file that the CLI just used, and returns the old "only files the user has shared with this session." Typing the fix into the app gets you this:

/add-dir isn't available in this environment.

The extension is identical in both cases (Claude in Chrome 1.0.81), so the gate isn't in the extension. It's in Claude Code, implemented per surface.

Now the part that is documentation rather than measurement. The published permission rule reads like my case should work:

Permissions: Claude can upload a file only when the session is allowed to read it

Read it, upload it. My measurements say otherwise. But before calling that a spec violation — that page opens by scoping itself to browser automation "from the CLI or the VS Code extension." The desktop app is never mentioned. I had been reading the manual for the surface next to mine.

The desktop app does have its own manual.² It even explains the /add-dir result: commands that open an interactive terminal panel behave differently in the Code tab, and the ones with no argument form answer isn't available in this environment. The message I'd been squinting at was documented behavior all along.

That manual has a browser section too — but it describes the Browser pane, the app's own built-in browser, which is not the Chrome extension. The Browser pane has no file attachment capability at all. The word "upload" does not appear on the page.

And then the same page sends you here:

When you want Claude to act as you in your logged-in sessions, use the Claude in Chrome extension instead, which shares your browser's login state.

Follow that advice and you arrive at the page that opened by telling you it's about the CLI and the VS Code extension. Two help desks, each politely pointing at the other, neither of them saying anything false.

So: not a documented condition being violated. An undocumented surface. It took reading both manuals to see that.

One footnote to save you an hour: if you're in the desktop app and you ask the agent what version you're on, it will shell out to claude --version and report your npm CLI. I got fooled by this. The shell said 2.1.215; the app was actually running the 2.1.219 it ships inside AppData\Roaming\Claude\claude-code\. There's no desktop-side version command, so the only reliable answer is the executable path of the running process.

The clipboard route works

The goal was still reachable, just not through the front door. OS clipboard plus a trusted paste event:

Set-Clipboard -LiteralPath <file> in PowerShell (this puts CF_HDROP on the clipboard)
Install a paste listener on the page
Send a real Ctrl+V through browser automation
Take the File off clipboardData.files, put it into input[type=file].files via DataTransfer, dispatch change

I ran this against the real form's image slot. A 307 KB PNG arrived in clipboardData.files with name, size, and MIME intact, the page's JavaScript accepted it, the preview rendered, and the save went through. No question was asked about where the file came from.

Two things cost me time here. Specifying the modifier as a separate parameter does nothing useful — you get a literal v typed into the page; the key has to be passed as the string ctrl+v. And coordinate clicks land nowhere unless you take a screenshot first, because the coordinate space is defined by the screenshot.

Working and recommendable are different things, and this route has four problems. It routes around the origin restriction on file_upload — a restriction that presumably exists to stop an agent from pushing local files onto the web without anyone looking, which is not the kind of thing you disable because it's in your way. It clobbers the clipboard, so whatever the user had copied is gone, and when an agent does that silently it reads as a malfunction. It depends on how the page is built: fine if the page watches input.files, dead if a custom uploader only listens for drop. And putting CF_HDROP on the clipboard is Windows-specific. On top of all that, it isn't a supported path, so an update could close it without notice.

Which brings me to a question. If you know the sanctioned route on the desktop app, or a way to do this without the four problems above, I'd like to hear it. Two candidates I haven't tested: running claude --chrome in the app's built-in terminal, and using the VS Code extension. Both should land on the CLI side of the gate, but I haven't confirmed either.

What half a day bought

One tool name, file_upload, two behaviors: the CLI accepts anything the session can read, the desktop app interrogates where the file came from and says no. When I design work to hand off to an agent, I'd been treating permissions as a property of the settings file. They're also a property of which binary is running.

Practically, that leaves three options for form work that involves attachments: run it from the CLI, keep a clipboard procedure on hand for the desktop app, or leave the attachment step to a human. My own runbook already said the reliable route was manual. It took half a day of building a fourth route to believe it. A known two-minute manual step beats an unknown fully-automated one, and apparently that's a lesson with a tuition fee.

What I haven't verified

What "folders the user has connected" actually refers to. I've only established that the directory-connect tool's approval isn't it.
Whether chat attachments count as shared on surfaces where they do materialize as files (the browser version, presumably). That's how the description reads, but I haven't tested it.
Which Claude Code version the desktop app bundled on the day I first hit this. The number I recorded then was the npm CLI's, and the app deletes the old version folder when it updates.
Along the way I also tried injecting via a local HTTP server, and hit a case where a fetch from an HTTPS page to http://127.0.0.1 neither succeeds nor errors — it just hangs forever. I suspect the browser's local network access check swallowed it, but I didn't confirm that.

And the big caveat: everything here is bound to a version and a surface. The CLI already passes cleanly. There's a good chance this whole post is a historical document within six months.

Verified July 2026. Environment: Windows 11, Claude Code desktop app (bundled version 2.1.219), npm CLI 2.1.215, Claude in Chrome 1.0.81, computer-use. Desktop-app re-measurement and the clipboard route were both run on 2026-07-26.

Use Claude Code with Chrome, "Upload files to web pages." ↩
Desktop application, sections "Browse external sites," "Choose between the Browser and the Chrome extension," and "What's not available in Desktop." ↩

My coding agent always stopped at 80%. The spec I gave it was why.

Mxhlix — Sun, 26 Jul 2026 01:37:36 +0000

I run about 600 browser-based web tools, each shipped in five locales, all built by Claude Code. For a long time, adding a new one meant handing the agent a spec: design conventions, component rules, forbidden patterns. It would read the spec, produce something plausible, and stop at roughly 80% quality. The last 20% took rounds of me pointing at things.

"Roughly 80%" was a feeling, not a measurement. I only got a number after I changed the setup.

Two things changed. The agent stopped reading specs and started copying production code. And the "build a new tool" skill got split from the "fix a broken tool" skill. Below is what that actually looks like in the repo, with the files and commits it came from.

What replaces the spec

A spec starts drifting from reality the moment you write it. Every small decision made during implementation — this pattern gets that padding, this one case is an exception — never makes it back into the document. Nobody keeps that up. So the agent fills in the unwritten parts its own way, differently every time.

Production code has all of those decisions baked in. So the skill file became a thin wrapper, and the real content moved into a catalog: 263 lines indexing 15 production references across 5 implementation patterns.

The catalog opens with a decision tree. Three questions, plus one for the specialized case:

Q1. Is the output PNG or SVG?
  ├─ SVG only          → §6 Pure SVG
  └─ PNG (primary)     → Q2

Q2. Does the preview composite a user-supplied image (a photo, etc.)?
  ├─ Yes → §3 html2canvas (background image + text overlay)
  │        * If objects are dragged / rotated / edited individually,
  │          use §2 Canvas 2D (ita-bag-badge-layout-calculator) instead
  └─ No  → Q3

Q3. Is text rendering precision decisive (typography-critical)?
       (name badges, price tags, labels, certificates, address cards,
        vertical Japanese text, print offset)
  ├─ Yes (1px of glyph drift is visible) → §2 Canvas 2D direct draw
  └─ No  (just image-ify the DOM)        → §4 dom-to-image-more

Q4 (Specialized). QR / barcode?
  └─ Yes → §5 dedicated library (QRCode.js / JsBarcode)

Whichever branch it lands on has a table of production references. Canvas 2D has nine. Three rows of it:

Reference	Template path	Strengths	DL handler line	Draw core line
`shugibukuro-address-template`	`templates/visual/shugibukuro-address-template.html`	vertical text / rotated glyphs / auto line-size	202	227
`wedding-place-card-maker`	`templates/visual/wedding-place-card-maker.html`	honorifics / font auto-fit / multi-page A4	173	214
`ita-bag-badge-layout-calculator`	`templates/calc/ita-bag-badge-layout-calculator.html`	drag & drop editing / tiling / coverage ratio / one renderer shared by editor and PNG export	994	387

The agent picks two: one to use as the base, one to diff against. It doesn't read either file end to end — the table says where the download handler is and where the render core starts, so it reads around those line numbers. Then it copies the base template as a file and reshapes it around the new tool's data.

Did it work

One tool has before/after numbers, because I instrumented it while validating the approach. The measurement is the gap between where text sits in the DOM preview and where it lands in the downloaded PNG:

broken state:
  scaleFactor: 0.397
  hasJustifyCenter: true
  Canvas vs DOM drift (text top): event 0px / name -7px / role -6px / org -11px

skill applied:
  scaleFactor: 0.733
  hasJustifyCenter: false
  Canvas vs DOM drift: event +0.8px / name +2.5px / role +1px / org +0.8px

Worst-case drift went from 11px to 2.5px. That's the specific class of defect I used to catch by eye and send back.

The honest caveat: this was measured while repairing an existing tool, not while building a new one. The handoff note from that week lists "validate the skill on a from-scratch build" as untested, and it stayed that way. My claim that new builds got better is operational — the correction rounds mostly stopped — not instrumented.

One side effect worth mentioning: the catalog records convergence. Five patterns exist, but 9 of the 15 production references are Canvas 2D. The line in the catalog that says "when in doubt, pick Canvas 2D" is writable only because that 60% is sitting right above it.

The split that lasted 94 minutes

The second decision was separating new construction from repair. I used to explain it as "one skill would bloat." What actually happened is simpler, and the commit timestamps tell it better than I can.

The prototype was a single 83-line skill with build steps and repair steps living together. At 21:11 I split it in two: 71 lines of build, 85 lines of repair. A clean bisection.

At 22:45, 94 minutes later, the repair skill I had just created was demoted. Its whole SKILL.md became qa-audit/references/visual.md, one leaf under a new skill that routes across six symptom categories.

The reason it collapsed that fast is that the two sides branch on different axes. Build branches on implementation pattern: Canvas 2D, html2canvas, dom-to-image, dedicated library, pure SVG. Repair branches on symptom category:

Reference	Open when symptom matches	Status
`references/layout.md`	LayoutPatternValidator flagged, fullscreen button overlap, download / reset / copy button position wrong	✅ Implemented
`references/visual.md`	Image-export tool: preview ≠ downloaded image, clipped text, filename `----.png` for CJK input, scaler off-center	✅ Implemented
`references/i18n.md`	FR / PT translation reads as machine-translated, hardcoded JA/EN UI strings	⏳ Stub
`references/interactive.md`	Reset button doesn't reset, button state stuck, tap feedback missing on mobile	⏳ Stub

Those axes are orthogonal. Layout breakage and bad translations happen regardless of whether the tool draws with Canvas 2D or html2canvas. And "how to build a Canvas 2D tool" has no entry point until someone reports a symptom, which is exactly what never happens on a new build. Keep both in one skill and every read costs the agent a pass over the axis it doesn't care about. The build side is 82 lines today. The repair side is 495 (85 in SKILL.md, 410 across six reference files). Merged, every new tool would have walked past all of it.

Unbuilt categories are stubs that point at the canonical doc instead of describing a procedure, so the agent can't invent one that doesn't exist. The promotion rule got written into the same commit: a stub becomes a real reference when the same bug recurs across two or more tools and the existing pointer doesn't already contain the fix.

Skills advise. Hooks enforce.

A skill is a document. It has no power. The agent skims.

So the violations that must never ship are blocked at write time. Here's one. If flex-1 or min-h-[Npx] survives on a preview element, it silently overrides the height JS sets on that element. It shows up two ways: empty checkerboard padding above and below the preview, or the inner card punching out through the bottom of the tool container. Same root cause, and from the symptom you would not guess they're related.

Writing "delete this class" in the skill did not stop it. It came back on a second tool. Then this went in:

# [P-068] preview-stage / preview-wrapper has flex-1 or min-h-[N] while JS controls height
# Past recurrences: conference-name-badge-maker (Wave 9), perspective-grid-generator (2026-05-27).
if echo "$CONTENT" | grep -qE 'id="(preview-stage|preview-wrapper|preview-area|preview-container)"[^>]*(flex-1|min-h-\[)'; then
  VIOLATIONS="${VIOLATIONS}\n⚠ P-068: ... Canonical class is 'relative w-full overflow-hidden' and let JS own the height."
fi

(Warning text trimmed. The recurrence list stays in the comment because it's the evidence for promoting this from advice to a block.)

One grep. That stopped it. The build skill also grew a line telling the agent to strip the class if it finds it in the reference it's copying from, because references are not automatically clean.

"It's in the skill but the agent ignored it" is not a writing problem. It's what happens when you expect enforcement from advice.

What it costs

Reference quality is the output ceiling. You still have to build the first one properly, by hand, with the taste applied. That is more up-front work than writing a spec, not less.

The catalog needs maintenance. Line numbers drift when a template gets reworked, so the catalog declares a tolerance: ±5 lines, update only after a major rewrite. That doesn't save you from the other failure. While pulling quotes for this post I found that the skill file still summarizes Canvas 2D as having 8 references while the catalog itself lists 9. I added a reference, updated the catalog, and never touched the summary table in the skill.

Which is the part I'd push back on if someone else wrote this post. Production code doesn't rot while it keeps running. The index pointing at it rots like any other document. What reference-driven buys you is that the rot is now visible and cheap — a stale line number sends the agent five lines off target, instead of a stale rule sending it down a design that no tool has used in months.

And this only pays off because I'm mass-producing. There's no reason to build a catalog for a project with one tool in it. Somewhere around the third one, when you catch yourself thinking "I wrote this already," is when it's worth naming the first one a reference.

Verified April–June 2026. Environment: Claude Code with custom skills, running 600+ web tools across 5 locales. Line counts and code excerpts taken from the live files in July 2026.

4 ways canvas text rendering breaks in multilingual apps (that en/ja testing will never catch)

Mxhlix — Thu, 23 Jul 2026 00:47:52 +0000

I run a large fleet of "preview it, then download it as PNG" web tools — name tags, certificate generators, price cards, badges — in five languages: Japanese, English, Spanish, French, Portuguese.

Canvas 2D text rendering looks correct as long as you only test Japanese and English.

It breaks when you run es/fr/pt through it.

After stepping on these repeatedly, the failures collapse into four patterns.

The premise: Latin languages run 1.4–2× longer than Japanese

Design data first. The same label, measured across five locales:

Example	ja	en	es	fr	pt
Tool name	22 chars	35	62	48	50
"Standard" button	4	8	10	20	12

Rule of thumb: fr/pt come out 1.4–1.7× longer than ja; es can balloon to nearly 3×. A font size and maxWidth tuned to fit Japanese will not fit the Latin locales. All four failure patterns grow from this.

Pattern 1: hand-rolled wrapping via `text.split(/\s+/)` collapses on CJK

The classic snippet — split on spaces, wrap word by word — does nothing for Japanese or Chinese, where words aren't space-delimited. An entire sentence becomes one unbreakable token and clips at the canvas edge.

Test with real Japanese input and check that the final line renders to its last character. "Most of it showed up" is not a pass.

Pattern 2: an ASCII-only tokenizer splits words at accented characters

Fix pattern 1 with a character-class tokenizer like [A-Za-z0-9'\-_] and you've traded one regression for another: ç é ã ó ñ aren't in that class, so produção fragments into produ / ç / ão mid-word.

An English test will never catch this.

Generate actual PNGs with fr/es/pt samples and eyeball the area around accented characters. I never found another detection method — string-comparison tests can't see a rendering-level split.

Pattern 3: the important word at the end vanishes into "…"

Since fr/pt run 1.4–1.7× longer than the ja the layout was tuned for, text overflows its two lines and gets ellipsized. The cruel part: what disappears is the tail of the phrase — often the semantically critical word, like production / produção.

The fix was splitting fields into two classes. Fields that may truncate (descriptions) wrap-and-ellipsize. Fields that must never truncate (titles, names) get auto-fit: measure on an offscreen canvas with measureText, search for a font size that fits, render at that size. A title with an ellipsis is a broken deliverable, so never mix the two strategies.

Pattern 4: the preview and the downloaded PNG break lines in different places

The sneakiest one. Implement pattern 3's auto-fit only on the canvas side while the DOM preview keeps its fixed font size, and the line breaks you see on screen won't match the line breaks in the PNG. From the user's perspective, the tool's core promise — "download what you see" — is broken.

The fix: centralize the auto-fit font-size decision in one place (the offscreen canvas measurement) and apply the resulting size to both the canvas render and the preview DOM's style.fontSize. My pass condition: preview line count equals PNG line count, in all five locales.

Pre-release checklist

For every new image-output tool:

[ ] ja: CJK renders to the last character of the final line, no edge clipping
[ ] fr / es / pt: generate real PNGs; no word splits around ç é ã ó ñ
[ ] fr / pt: no ellipsis eating the important trailing words
[ ] all five locales: preview (DOM) and downloaded PNG agree on line breaks and line count

Declaring "multilingual support: done" after checking only en and ja was the single biggest trap.

Limits and caveats

The character-count ratios are measured from my own fleet (utility-tool names, descriptions, UI labels). Different content genres will shift the ratios.

On automation limits: patterns 2 and 4 resist static analysis entirely. Generating PNGs and looking at them was the only detection that worked. Five locales × generation checks is a real cost — whatever you skip surfaces in production instead.

Verified: April–May 2026. Environment: Canvas 2D API / vanilla JS / ja·en·es·fr·pt locales in production.

Giving an AI QA agent a Bayesian prior: predicting where the bugs are before looking

Mxhlix — Mon, 20 Jul 2026 00:04:31 +0000

Hand an AI agent a QA checklist and here's what you get: every item inspected top to bottom with perfectly uniform attention. And with attention spread uniformly, the places that actually break get missed.

Running QA across 600+ multilingual web tools with AI agents, I ended up at a design I'd describe as giving QA a prior probability. The entire implementation is markdown files.

The problem: static checklists distribute attention evenly

A veteran human QA engineer doesn't read a checklist evenly. They carry a heuristic — "this type of tool usually breaks here" — and aim their attention at the suspicious spots first.

Give an AI a checklist (mine catalogs 70+ known violation patterns) and that gradient of attention doesn't exist. It dutifully verifies everything, burns context doing it, and still walks past the frequent offenders.

The fix: accumulate violation history, predict before inspecting

What I did is almost embarrassingly simple.

Every violation found in QA gets logged to a knowledge base (a markdown file): which category of tool, which violation pattern. That's the accumulation side.

Then at QA start, before touching the checklist, the agent predicts the top 3–5 likely violations from the tool's category × historical frequency. My category-level predictions look like:

Calculators: overexposed "real-time calculation" marketing copy / missing unit toggles / broken output-card layout
Visual tools: initialization flicker / preview overflow / export-element size breakage
Text tools: multilingual label overflow / hardcoded UI strings

The predictions miss sometimes. Even so, just aiming the first pass of visual attention at "places that actually broke before" cut the misses noticeably. In Bayesian terms: stop starting QA from a uniform distribution; start from a prior.

Two designs that compounded the effect

Kill everything grep can catch before visual QA starts

Some violation patterns are mechanically detectable: banned words in visible copy, inline onclick, hardcoded UI strings. Run those as one batch grep sweep before any visual inspection. The expensive check (eyes on pixels) should only cover what eyes alone can catch. Cheap checks first, expensive checks second — obvious sequencing that AI agents skip unless you force it.

Separate check and fix into different sessions and branches

Let the QA agent fix things in the same session and you contaminate the verdict: the desire to have "fixed it" bleeds into the pass/fail judgment. I once got a QA record where "PASS" and a list of unresolved issues coexisted in the same file.

Since then: the check role detects and reports, and never touches code. Fixes happen in a separate intent, separate branch, separate session. If the same finding survives three rounds, escalate to the human instead of continuing to fix (shelving is a human-only decision).

Worth noting: the "pass + unresolved issues" record corruption may be less likely with today's tooling. But the design principle — QA objectivity is guaranteed by not holding the fix permissions — stays valid after any specific cause is patched.

Implementation cost

No special tools. The violation catalog is one markdown file. The KB (violation history) is one markdown file, layered into always-read and dig-when-needed sections. Prediction and routing live in the skill file (the procedure doc) handed to the agent.

So: it's all just markdown. No training, no external database. One extra step — "consult history, predict" — inserted into the procedure. That one step made the QA qualitatively different from a static checklist.

Limits and caveats

A prior needs accumulation. Early on the KB is empty and this does nothing; it starts working after a few dozen QA runs of logged violations.

Sloppy categories produce sloppy predictions. I started with roughly "calculator / visual / text" granularity, and that was enough.

There's a decay risk in trusting predictions too much — the agent stops looking beyond the predicted 3–5. The predictions only reorder attention; the full sweep never gets skipped.

Verified: April–June 2026. Environment: Claude Code + custom skills/knowledge base (markdown) / QA across a multilingual web-tool fleet.

My AI QA agent said "all features working." The canvas was blank. Here's what it was actually seeing.

Mxhlix — Fri, 17 Jul 2026 02:12:49 +0000

Back when I first delegated QA to an AI agent, it signed off on a tool with "all features working, pass." I opened the tool myself. The canvas was blank.

The AI wasn't lying. In the environment it was looking at, the tool genuinely appeared to work.

I run visual QA across a large fleet of web tools using Claude + Chrome MCP, and this class of false positive traced back to exactly two causes.

Cause 1: requestAnimationFrame stops in hidden tabs

Chrome MCP typically operates in hidden (background) tabs. And browsers throttle requestAnimationFrame aggressively in hidden tabs to save power.

QA a canvas or animation feature in that environment and here's what happens: the JS executes, no errors fire, event handlers respond — and the render finishes at zero frames. The AI looks at a screenshot plus clean JS results and concludes "animation started, no errors, pass." Code executing and pixels rendering are different events, and a hidden tab erases the distinction.

Reproduction test before publishing (July 10, 2026)

I didn't want this to be a "back in my day" post, so I re-ran the measurement right before publishing. Opened a tab via Chrome MCP, ran an rAF counter:

Measurement	Result
document.visibilityState	hidden
rAF fires in 3.37s	0
setInterval(100ms) fires, same window	4 (expected: 33)
setTimeout(3000ms) actual delay	3373ms

rAF wasn't throttled. It was stopped — zero frames.

And as a bonus finding: setInterval was decimated to roughly 1/8 of its expected rate, and even setTimeout drifted. So it's not just rAF-based animation; timer-driven logic in general cannot be trusted in a hidden tab.

Fixes

Open tabs with active: true so they're visible
Trigger the interaction in a visible tab, wait a few seconds, then screenshot and confirm the rendered output actually exists
If you must evaluate in a hidden tab, the report has to say "dynamic rendering not visually observed." Never silently pass it

Cause 2: confusing "healthy JS state" with "the feature works"

The second trap shows up in AI QA reports as pass rationale like this:

onclick wiring confirmed. Zero JS errors. Library loaded. → PASS

All of that describes code-path health, not feature behavior. Wiring can be correct while nothing renders (Cause 1). Errors can be absent while the downloaded file is empty.

My fix: grep the template for dynamic features first, then require a behavior check for every hit.

Found in code	Required behavior check
`<canvas>`	Interact in a visible tab → screenshot after a delay shows rendered content
`<input type="file">`	Upload a dummy file → preview src is non-empty
`download` attr / `toBlob`	Press the button → observe Blob creation
`mousedown` / `touchstart`	Fire the event → state/transform actually changed
`requestAnimationFrame`	Transform values differ after time passes

QA reports must include a feature-coverage table, and "exists in code but behavior unverified" can never be graded as pass. That's the whole rule.

That alone cut the false positives dramatically.

Bonus: screenshot-vs-DOM contradiction checks

A side benefit of visual QA: cross-checking what the code claims against what the pixels show. "The class is text-secondary (cyan) but it renders gray" — cascade and override bugs get caught with evidence attached. Humans tend to stall at "the color feels off somehow."

Limits and caveats

Hidden-tab throttling is a browser power-saving behavior, so Chrome could change it. My operating principle isn't "hidden tabs are bad" — it's "dynamic rendering must be verified in a visible state."

Behavior checks cost time and tokens. Running them on every feature of every tool is too heavy; I only require them for tools where the grep finds dynamic features.

Verified: April–June 2026 in production, re-reproduced July 10, 2026. Environment: Claude + Chrome MCP / Windows 11.

Automating an app with no DOM: driving Flutter/canvas editors with coordinates only

Mxhlix — Tue, 14 Jul 2026 00:35:36 +0000

In my last post I said that for normal HTML pages, element-based automation (find / read_page) beats coordinates every time. This post is about the apps where that advice is useless.

Flutter Web apps. Canvas-rendered editors. Every button and panel you can see on screen doesn't exist in the DOM — it's all pixels painted onto a single canvas. find returns nothing. read_page's accessibility tree is effectively empty.

I got Claude to drive the Rive editor (an animation tool built with Flutter) all the way through selecting assets and exporting them. Here's the procedure that survived contact with reality.

Step zero: confirm you're actually in this situation

Coordinate automation is fragile, so you should only accept it after ruling out the alternative. The test is quick: run read_page. If the visible UI has almost no corresponding nodes, you're looking at a canvas-rendered app, and coordinates are the only interface you have.

The four rules

1. Wait for the window size to settle before anything else

Same failure mode as my previous post: right after load, the viewport hasn't reached its final width (I measured 1664→1920 over 2–3 seconds), and clicks based on an early screenshot land to the right of the target. Read innerWidth via javascript_tool twice; only proceed when two consecutive reads match. But matching innerWidth alone isn't enough — also confirm devicePixelRatio hasn't changed since the screenshot you're about to act on (a follow-up to my previous post surfaced this: when DPI or scaling changes, the whole coordinate space rescales the same way, but the new values stabilize immediately, so an innerWidth-only check can't catch it). Canvas apps deserve extra paranoia here, because there is no element-based fallback when a click misses.

2. Read text by zooming, not by extracting

Text painted on canvas can't be pulled out of the DOM. To read a menu item or panel label, zoom into that region and read the enlarged screenshot as an image. Full-page screenshots make small text easy to misread — make "zoom before you read" a reflex.

3. Demand visual evidence after every click

A missed coordinate click produces no error. Nothing. And if you proceed without noticing, every subsequent step operates on a false premise.

So: click → screenshot → confirm the visual evidence of selection (in Rive, the blue highlight; in other apps, a selection outline or color change) → only then continue. No evidence, no progress — re-derive the coordinates and retry.

4. Build double-clicks from two rapid single clicks

The double_click action can exceed the app's double-click threshold and degrade into two single clicks. Canvas editors love double-click-to-enter-hierarchy interactions, so this will block you. Fire two left_clicks at the same coordinates inside one browser_batch — the gap tightens enough to register.

A real run (Rive editor)

Putting it together, "select an asset and export it" looks like this:

Open the editor → wait until innerWidth stabilizes
Full screenshot → locate the target panel
Rapid double-fire on the object → screenshot → confirm blue highlight
Click the menu coordinates → zoom the menu region → read the item labels
Click Export → screenshot → confirm the dialog appeared

Every step has a verification wedged after it, so the operation count is easily double what a human would need.

Slow, but it keeps moving.

Chasing speed with coordinate automation produces chains of unnoticed misses, and those cost far more time than the checks do. That was the actual lesson of running this in production.

Limits and caveats

This approach assumes a stable screen layout. Pages that reflow constantly (ads, notifications) degrade coordinate reliability even further.

Any specific app's UI layout changes with updates — Rive's included. What transfers is the procedure, not the pixels: settle-wait → zoom-read → evidence-check → rapid double-fire.

Verified on Windows / 1920×1080. Different DPI scaling may change coordinate behavior.

Verified: June 2026. Environment: Windows 11 / Chrome + Claude in Chrome extension / Rive editor (Flutter/canvas).

Claude in Chrome keeps missing clicks. I measured why — there are exactly two causes.

Mxhlix — Sat, 11 Jul 2026 05:43:04 +0000

If you use Claude in Chrome (Chrome MCP) for browser automation, you have probably hit some of these:

Clicks land next to the target, not on it
The pointer drifts right of the element, and retrying never fixes it
You take a screenshot, and moments later the page dimensions have changed
Double-clicks don't enter edit mode — you get a row selection instead

I run 600+ web tools built and QA'd through AI-driven browser automation, so I hit these constantly. I burned hours suspecting the page elements. They were fine.

The real problem: the assumptions behind coordinate-based clicking were breaking. It comes down to exactly two causes.

Cause 1: the window width settles late, so coordinates drift right

Right after a page loads, innerWidth can report a smaller value than the real window. On my machine it started at 1664. Even after document.readyState hits complete, it keeps widening for 2–3 seconds until it reaches the true width (1920). During that window, outerWidth / outerHeight sometimes return 0.

Now take a screenshot during that settling period and click using its coordinates. The page has widened since the capture, so your click lands to the right of the target. 1664→1920 is a 1.15× stretch — the further right the element, the bigger the miss.

That's the whole mystery: "the page changed size after my screenshot" and "the pointer keeps drifting" are the same bug.

This is not limited to exotic canvas apps. Ordinary HTML pages with lazy loading or dynamic layout do it too. Some of my "this input just won't click" sessions were nothing more than coordinates swinging at empty air.

How to diagnose it

Read the viewport with javascript_tool and check whether it has stabilized:

({ url: location.href, readyState: document.readyState,
   dpr: window.devicePixelRatio,
   innerW: window.innerWidth, innerH: window.innerHeight,
   outerW: window.outerWidth, outerH: window.outerHeight })

Four rules. If outerW is 0, the window size hasn't settled — do not interact yet. Read again after a moment; if innerW changed, it's still settling — keep waiting. Two consecutive identical innerW readings with a non-zero outerW and the size is clear. The fourth rule guards against a different failure entirely: dpr has to match whatever it was when you took your screenshot. Drag the window onto a monitor with different DPI scaling and every coordinate rescales — same math as the 1664→1920 stretch — except nothing ever looks unsettled, because the new values are correct and stable from the very first read. An innerW-only check waves it straight through. (Pointed out by @skillselion in the comments — see the update note below.)

Cause 2: `double_click` fires too slowly and becomes two single clicks

The computer tool's double_click action can leave a gap between the two clicks that exceeds the app's double-click threshold. The app interprets it as two single clicks — so instead of entering text-edit mode, you get a row selection or nothing.

The nasty part: the tool believes it sent a double-click. The logs look correct.

You cannot see this failure in any log.

Fixes

Plan A (preferred): stop using coordinates, use element references

On a normal HTML page this is the reliable path. Drop coordinate clicks entirely. Use find (natural-language element search) or read_page (accessibility tree) to get element references, then click and fill through those references (form_input for values). Element references survive layout shifts.

They're not a free pass, though. A ref is a pointer to one specific node and nothing more — it knows nothing about where that node sits on the page or what it currently means to the user. Replace the node in a rerender and the ref dies with an explicit error. Move the node, or let it take on a different role, and the ref keeps right on working: it reports success and clicks whatever that node has become, wherever it now lives. That second case is a silent miss with a clean log. Both reproduce in plain vanilla JS, so there's no framework magic involved — a React rerender just does the replacing for you. The fix is the same either way: treat refs as disposable. Grab one right before you use it, use it once, and never carry it across a navigation or a form submit (credit to @skillselion in the comments for the single-use rule). One more caveat I'd rather admit than bury: I've watched a perfectly fresh ref click no-op silently while the window was still settling, which tells me ref clicks get resolved to coordinates somewhere under the hood. Plan A shrinks the coordinate problem. It does not make it go away.

The exception: canvas/Flutter apps (the Rive editor, for example). Their UI doesn't exist as HTML elements, so find and read_page return nothing. Then Plan B is your only option.

Plan B: if you must use coordinates, pair "wait for settle" with "rapid double-fire"

Wait for the size to settle before doing anything — use the diagnostic snippet above until innerW stabilizes, then take your screenshot
Don't let time pass between screenshot and click. Capture → click on those fresh coordinates, as one unit. If anything intervened, re-capture
Never use double_click. Build your own with two rapid left_clicks at the same coordinates via browser_batch — the gap gets tight enough to pass the threshold:

browser_batch([
  { name: "computer", input: { action: "left_click", coordinate: [x, y], tabId } },
  { name: "computer", input: { action: "left_click", coordinate: [x, y], tabId } },
])

After every click, zoom into the target and verify the expected state change (selection, edit mode, highlight) before moving on. One unnoticed miss desyncs everything after it

Cheat sheet

Symptom	First suspect
Clicks miss	Size not settled (Cause 1) — wait
Double-click doesn't work	Switch to rapid double-fire (Cause 2)
Stuck on a normal HTML page	Switch to Plan A (element refs)
Ref click errors out or hits the wrong element	Stale ref — re-find immediately before the interaction (refs are single-use)
Canvas/Flutter app	Plan B only — there are no elements

Limits and caveats

The numbers here (1664→1920, 2–3 seconds) are from my setup (Windows, 1920×1080). Display config and DPI scaling will change them. Take the principle, not the numbers: wait for settle before you capture.

Claude in Chrome ships updates frequently. If the tool ever starts waiting for size-settle itself, this whole workaround becomes unnecessary. And coordinate-based automation is inherently fragile — whenever element references are available, they win.

Update (2026-07-11)

@skillselion left a comment below that made this article better, and I've revised Plan A and the settle rules accordingly. I didn't take either point on faith — both got re-verified in plain vanilla JS before I touched the text. Refs really do track node identity and nothing else: kill the node and the ref dies loud, move it and the ref follows it silently into whatever it has become. And the innerW-only settle check really is blind to DPI changes — by design, since a rescaled-but-stable viewport is indistinguishable from a settled one if all you compare is innerW. I couldn't physically test the cross-monitor move (one display here), but my own setup argued the point for me: the same physical 1920px screen that settled at innerWidth 1920 in June now reports 1745 at a fractional dpr of 1.1. The full back-and-forth is in the comments.

Verified: June 2026; Plan A and settle conditions re-verified July 2026 (dpr 1.1 / 110% scaling environment). Environment: Windows 11 / Chrome + Claude in Chrome extension / 1920×1080, including automation of the Rive editor (Flutter/canvas).

DEV Community: Mxhlix

My agent filled in every field on the form. It could not attach the three images.

The sanctioned route is closed first

You can't reach the native dialog

Same tool name, different gate

The clipboard route works

What half a day bought

What I haven't verified

My coding agent always stopped at 80%. The spec I gave it was why.

What replaces the spec

Did it work

The split that lasted 94 minutes

Skills advise. Hooks enforce.

What it costs

4 ways canvas text rendering breaks in multilingual apps (that en/ja testing will never catch)

The premise: Latin languages run 1.4–2× longer than Japanese

Pattern 1: hand-rolled wrapping via text.split(/\s+/) collapses on CJK

Pattern 2: an ASCII-only tokenizer splits words at accented characters

Pattern 3: the important word at the end vanishes into "…"

Pattern 4: the preview and the downloaded PNG break lines in different places

Pre-release checklist

Limits and caveats

Giving an AI QA agent a Bayesian prior: predicting where the bugs are before looking

The problem: static checklists distribute attention evenly

The fix: accumulate violation history, predict before inspecting

Two designs that compounded the effect

Kill everything grep can catch before visual QA starts

Separate check and fix into different sessions and branches

Implementation cost

Limits and caveats

My AI QA agent said "all features working." The canvas was blank. Here's what it was actually seeing.

Cause 1: requestAnimationFrame stops in hidden tabs

Reproduction test before publishing (July 10, 2026)

Fixes

Cause 2: confusing "healthy JS state" with "the feature works"

Bonus: screenshot-vs-DOM contradiction checks

Limits and caveats

Automating an app with no DOM: driving Flutter/canvas editors with coordinates only

Step zero: confirm you're actually in this situation

The four rules

1. Wait for the window size to settle before anything else

2. Read text by zooming, not by extracting

3. Demand visual evidence after every click

4. Build double-clicks from two rapid single clicks

A real run (Rive editor)

Limits and caveats

Claude in Chrome keeps missing clicks. I measured why — there are exactly two causes.

Cause 1: the window width settles late, so coordinates drift right

How to diagnose it

Cause 2: double_click fires too slowly and becomes two single clicks

Fixes

Plan A (preferred): stop using coordinates, use element references

Plan B: if you must use coordinates, pair "wait for settle" with "rapid double-fire"

Cheat sheet

Limits and caveats

Update (2026-07-11)

Pattern 1: hand-rolled wrapping via `text.split(/\s+/)` collapses on CJK

Cause 2: `double_click` fires too slowly and becomes two single clicks