Payload CMS has a search box in its admin UI. Type a query in Japanese, press Enter to confirm the kanji your IME just converted, and the search fires on the partial string that was still mid-conversion, not the query you meant to type. The Enter that closes an IME conversion and the Enter that submits a form are, to the browser, the exact same keydown event. Nothing in SearchInput's handler checked which one it was looking at.
That's payloadcms/payload#17138, one line: skip the handler when event.nativeEvent?.isComposing is true. I've filed close to identical fixes in dozens of other projects. Not similar bugs. The identical bug, in code that has never seen the others.
I keep a corpus of these. Right now it holds 129 documented bugs across 120 open-source libraries. 115 of those are pull requests I opened myself, 65 already merged; the other 14 cite an existing report someone else filed. Every entry links to a real GitHub PR or issue, and the build fails on any entry whose link isn't one. After sorting the categories for a while, the 129 collapse into three broken assumptions, not twelve.
a keystroke is a submit
This is the Payload bug, and it's the largest bucket by a wide margin: 40 of the 129 entries, 31 percent, more than any other single category in the corpus. It shows up in React, Vue, Svelte, Angular, and every headless component library that lets you bind a handler to Enter. CopilotKit/CopilotKit#5764 has the same shape in an Angular chat widget: the composer submitted the message on the Enter that confirms an IME composition instead of waiting for the real one. Different framework, different UI, same missing isComposing check. The fix is always one line. The bug survives review because most contributors, and every CI runner, type in a language where a keystroke and a finished character are the same thing.
a character is a byte
ratatui's barchart centers a value label by measuring its display width, correctly, in the sibling function right next to it, and by its UTF-8 byte length in this one. "δΈζ" is 4 columns wide on a terminal but 6 bytes long, so the centering math undershoots and the label lands left of where it should sit (ratatui/ratatui#2625). A CJK character isn't one byte, isn't always two bytes, and isn't always one terminal column either, and code that treats those as interchangeable drifts the moment real text shows up.
pdf.js has a version of the same mistake one layer down. Font.prototype.encodeString silently drops the character that follows U+FFFE or U+FFFF when saving or printing a PDF, because its surrogate-pair guard treats those two single code units as if they started a surrogate pair, then swallows the next character as the pair's second half (mozilla/pdf.js#21538). The fix is a tighter range check. It's the same family as ratatui's bug: code written against "how many units is this" instead of "what does this actually represent."
text is left-to-right
out-of-character exists specifically to catch invisible and abusable Unicode. It already stripped the implicit bidi marks (U+200E, U+200F, U+061C), and let the explicit bidi embedding, override, and isolate controls (U+202A-U+202E, U+2066-U+2069) pass straight through untouched (spencermountain/out-of-character#90). Those are the characters behind Trojan Source (CVE-2021-42574): reorder how a line renders without changing a single byte of what's actually there. A library built to catch this class of character missed nine of the code points in its own job description.
bangumi/server-private had the narrower version of the same gap: its printable-character check matched the older bidi marks but not the directional isolates U+2066-U+2069, so a string made entirely of isolate controls passed validation as ordinary text (bangumi/server-private#1700). Bidi and control characters are one of the smaller categories in the corpus, 4 of 129, but it's the one where "harmless-looking bug" and "supply-chain attack vector" turn out to be the same finding.
using the corpus
Everything above comes straight out of data/corpus.json: one object per bug, with the category, the symptom, a minimal repro, and the fix, checked against the live GitHub API so a closed or reverted PR can't sit there uncorrected. A companion repo, cjk-agent-fixtures, turns the repros into CI fixtures, so a regression on any of these three assumptions gets caught before it ships again instead of waiting for someone with a Japanese keyboard to notice by hand.
None of the twelve categories in the corpus are actually twelve problems. They're three, wearing whatever framework or language happened to be lying around. If your test suite only ever types in English, at least one of them is sitting in your codebase right now. The corpus is the fastest way to check which.
Top comments (0)