<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:dc="http://purl.org/dc/elements/1.1/">
  <channel>
    <title>DEV Community: Lovora</title>
    <description>The latest articles on DEV Community by Lovora (@lovora).</description>
    <link>https://dev.to/lovora</link>
    <image>
      <url>https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F4256442%2Fa769a2cc-49f0-447e-926c-8f37c316b0c6.png</url>
      <title>DEV Community: Lovora</title>
      <link>https://dev.to/lovora</link>
    </image>
    <atom:link rel="self" type="application/rss+xml" href="https://dev.to/feed/lovora"/>
    <language>en</language>
    <item>
      <title>JavaScript's \b Is ASCII-Only: How It Silently Broke Our Rules in Six Languages</title>
      <dc:creator>Lovora</dc:creator>
      <pubDate>Fri, 20 Nov 2026 22:58:13 +0000</pubDate>
      <link>https://dev.to/lovora/javascripts-b-is-ascii-only-how-it-silently-broke-our-rules-in-six-languages-5bk5</link>
      <guid>https://dev.to/lovora/javascripts-b-is-ascii-only-how-it-silently-broke-our-rules-in-six-languages-5bk5</guid>
      <description>&lt;p&gt;We run a chat application used in nine languages, and a surprising amount of its behaviour depends on regular expressions: recognising that a user said "yes", detecting that a message asks for a photo, catching a stated age in the safety layer. In one month we found the same defect three times, in three places, each time without an error, a log line or a failing test. A rule simply never fired.&lt;/p&gt;

&lt;p&gt;The cause: &lt;strong&gt;in JavaScript, &lt;code&gt;\b&lt;/code&gt; does not know about Unicode.&lt;/strong&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  The behaviour
&lt;/h2&gt;

&lt;p&gt;A word boundary &lt;code&gt;\b&lt;/code&gt; is a position between a character in &lt;code&gt;[A-Za-z0-9_]&lt;/code&gt; and one that is not. That definition is fixed: the &lt;code&gt;u&lt;/code&gt; and &lt;code&gt;v&lt;/code&gt; flags do not change it. Every accented Latin letter, and every letter of every other script, counts as a non-word character.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight javascript"&gt;&lt;code&gt;&lt;span class="o"&gt;/&lt;/span&gt;\&lt;span class="nf"&gt;b&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;si&lt;/span&gt;&lt;span class="o"&gt;|&lt;/span&gt;&lt;span class="n"&gt;s&lt;/span&gt;ì&lt;span class="p"&gt;)&lt;/span&gt;\&lt;span class="n"&gt;b&lt;/span&gt;&lt;span class="o"&gt;/&lt;/span&gt;&lt;span class="n"&gt;i&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;test&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s1"&gt;'si'&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;      &lt;span class="c1"&gt;// true&lt;/span&gt;
&lt;span class="o"&gt;/&lt;/span&gt;\&lt;span class="nf"&gt;b&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;si&lt;/span&gt;&lt;span class="o"&gt;|&lt;/span&gt;&lt;span class="n"&gt;s&lt;/span&gt;ì&lt;span class="p"&gt;)&lt;/span&gt;\&lt;span class="n"&gt;b&lt;/span&gt;&lt;span class="o"&gt;/&lt;/span&gt;&lt;span class="n"&gt;i&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;test&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s1"&gt;'sì'&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;      &lt;span class="c1"&gt;// false: nothing after ì is a boundary&lt;/span&gt;
&lt;span class="o"&gt;/&lt;/span&gt;\&lt;span class="nf"&gt;b&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;yes&lt;/span&gt;&lt;span class="o"&gt;|&lt;/span&gt;はい&lt;span class="p"&gt;)&lt;/span&gt;\&lt;span class="n"&gt;b&lt;/span&gt;&lt;span class="o"&gt;/&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;test&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s1"&gt;'はい'&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;   &lt;span class="c1"&gt;// false&lt;/span&gt;
&lt;span class="o"&gt;/&lt;/span&gt;\&lt;span class="n"&gt;b&lt;/span&gt;â&lt;span class="n"&gt;ge&lt;/span&gt;\&lt;span class="n"&gt;b&lt;/span&gt;&lt;span class="o"&gt;/&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;test&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s1"&gt;'âge'&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;          &lt;span class="c1"&gt;// false: nothing before â is a boundary either&lt;/span&gt;
&lt;span class="o"&gt;/&lt;/span&gt;\&lt;span class="nf"&gt;b&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;ok&lt;/span&gt;&lt;span class="o"&gt;|&lt;/span&gt;好的&lt;span class="p"&gt;)&lt;/span&gt;\&lt;span class="n"&gt;b&lt;/span&gt;&lt;span class="o"&gt;/&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;test&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s1"&gt;'好的！'&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;  &lt;span class="c1"&gt;// false&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;An alternative that ends with a non-ASCII character can never be followed by &lt;code&gt;\b&lt;/code&gt; at the end of a string or before a space. The pattern reads correctly, compiles, and matches the ASCII alternatives next to it, which is exactly why nobody notices.&lt;/p&gt;

&lt;h2&gt;
  
  
  Three incidents in one codebase
&lt;/h2&gt;

&lt;ol&gt;
&lt;li&gt;
&lt;strong&gt;"Yes" in Arabic, Chinese, Japanese and Russian.&lt;/strong&gt; The pattern that recognises a short "yes" had been extended with نعم, 好的, はい and да. None of them had ever matched.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Accented Latin.&lt;/strong&gt; &lt;em&gt;sì&lt;/em&gt; (Italian) and &lt;em&gt;sí&lt;/em&gt; (Spanish), the normal way to say yes in two of our languages, had never been recognised. The unaccented &lt;em&gt;si&lt;/em&gt; worked, so every quick test looked fine.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;A safety rule.&lt;/strong&gt; A deterministic rule that catches stated ages under 18 ended with &lt;code&gt;\b&lt;/code&gt; after the Chinese, Japanese and Arabic words for "years old", so in those scripts it could not fire. (Our other moderation layers, including an independent classifier that does not use these patterns, were not affected.)&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;Six of our nine languages were affected somewhere.&lt;/p&gt;

&lt;h2&gt;
  
  
  The fix
&lt;/h2&gt;

&lt;p&gt;Replace &lt;code&gt;\b&lt;/code&gt; with lookarounds over Unicode property classes, and add the &lt;code&gt;u&lt;/code&gt; flag:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight javascript"&gt;&lt;code&gt;&lt;span class="c1"&gt;// before&lt;/span&gt;
&lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="no"&gt;YES&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="o"&gt;/&lt;/span&gt;\&lt;span class="nf"&gt;b&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;yes&lt;/span&gt;&lt;span class="o"&gt;|&lt;/span&gt;&lt;span class="n"&gt;si&lt;/span&gt;&lt;span class="o"&gt;|&lt;/span&gt;&lt;span class="n"&gt;s&lt;/span&gt;ì&lt;span class="o"&gt;|&lt;/span&gt;&lt;span class="n"&gt;s&lt;/span&gt;í&lt;span class="o"&gt;|&lt;/span&gt;&lt;span class="n"&gt;oui&lt;/span&gt;&lt;span class="o"&gt;|&lt;/span&gt;&lt;span class="n"&gt;ja&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;\&lt;span class="n"&gt;b&lt;/span&gt;&lt;span class="o"&gt;/&lt;/span&gt;&lt;span class="n"&gt;i&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;

&lt;span class="c1"&gt;// after: a "word character" is any letter or digit in any script&lt;/span&gt;
&lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="no"&gt;B_START&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="s1"&gt;'(?&amp;lt;![\\p{L}\\p{N}_])'&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="no"&gt;B_END&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="s1"&gt;'(?![\\p{L}\\p{N}_])'&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="no"&gt;YES&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="k"&gt;new&lt;/span&gt; &lt;span class="nb"&gt;RegExp&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sb"&gt;`${B_START}(yes|si|sì|sí|oui|ja)${B_END}`&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;'iu'&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;

&lt;span class="no"&gt;YES&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;test&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s1"&gt;'sì'&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;   &lt;span class="c1"&gt;// true&lt;/span&gt;
&lt;span class="no"&gt;YES&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;test&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s1"&gt;'sì!'&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;  &lt;span class="c1"&gt;// true&lt;/span&gt;
&lt;span class="no"&gt;YES&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;test&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s1"&gt;'sìx'&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;  &lt;span class="c1"&gt;// false&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;&lt;strong&gt;Chinese and Japanese need no trailing boundary at all.&lt;/strong&gt; They do not put spaces between words, so a correct match is usually followed directly by another letter: &lt;code&gt;15歳です&lt;/code&gt; ("I am 15") has no break after 歳, and even the Unicode lookahead would reject it. Tokens in these scripts are specific enough to match on their own.&lt;/p&gt;

&lt;h2&gt;
  
  
  Finding it automatically
&lt;/h2&gt;

&lt;p&gt;Reading the patterns does not work: they look right. A mechanical check does. This script scans a source tree for the two shapes of the bug:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight javascript"&gt;&lt;code&gt;&lt;span class="c1"&gt;// find-ascii-boundaries.mjs — usage: node find-ascii-boundaries.mjs src&lt;/span&gt;
&lt;span class="k"&gt;import&lt;/span&gt; &lt;span class="n"&gt;fs&lt;/span&gt; &lt;span class="k"&gt;from&lt;/span&gt; &lt;span class="s1"&gt;'node:fs'&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;span class="k"&gt;import&lt;/span&gt; &lt;span class="n"&gt;path&lt;/span&gt; &lt;span class="k"&gt;from&lt;/span&gt; &lt;span class="s1"&gt;'node:path'&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;

&lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="n"&gt;nonAscii&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;ch&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="n"&gt;ch&lt;/span&gt; &lt;span class="o"&gt;&amp;amp;&amp;amp;&lt;/span&gt; &lt;span class="n"&gt;ch&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;codePointAt&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;&amp;gt;&lt;/span&gt; &lt;span class="mi"&gt;127&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;

&lt;span class="kd"&gt;function&lt;/span&gt;&lt;span class="o"&gt;*&lt;/span&gt; &lt;span class="nf"&gt;files&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;dir&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
  &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="n"&gt;e&lt;/span&gt; &lt;span class="k"&gt;of&lt;/span&gt; &lt;span class="n"&gt;fs&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;readdirSync&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;dir&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="n"&gt;withFileTypes&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kc"&gt;true&lt;/span&gt; &lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="n"&gt;p&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;path&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;join&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;dir&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;e&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;name&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
    &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;e&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;isDirectory&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;&amp;amp;&amp;amp;&lt;/span&gt; &lt;span class="n"&gt;e&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;name&lt;/span&gt; &lt;span class="o"&gt;!==&lt;/span&gt; &lt;span class="s1"&gt;'node_modules'&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="k"&gt;yield&lt;/span&gt;&lt;span class="o"&gt;*&lt;/span&gt; &lt;span class="nf"&gt;files&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;p&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
    &lt;span class="k"&gt;else&lt;/span&gt; &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="o"&gt;/&lt;/span&gt;\&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;m&lt;/span&gt;&lt;span class="o"&gt;?&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="n"&gt;jt&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="n"&gt;sx&lt;/span&gt;&lt;span class="o"&gt;?&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;$&lt;span class="o"&gt;/&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;test&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;e&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;name&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="k"&gt;yield&lt;/span&gt; &lt;span class="n"&gt;p&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
  &lt;span class="p"&gt;}&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;

&lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="n"&gt;f&lt;/span&gt; &lt;span class="k"&gt;of&lt;/span&gt; &lt;span class="nf"&gt;files&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nb"&gt;process&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;argv&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="mi"&gt;2&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt; &lt;span class="o"&gt;||&lt;/span&gt; &lt;span class="s1"&gt;'.'&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
  &lt;span class="n"&gt;fs&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;readFileSync&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;f&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;'utf8'&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;split&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s1"&gt;'\n'&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;forEach&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;line&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;i&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="c1"&gt;// 1) an alternative ending in a non-ASCII char right before ")\b"&lt;/span&gt;
    &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="n"&gt;m&lt;/span&gt; &lt;span class="k"&gt;of&lt;/span&gt; &lt;span class="n"&gt;line&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;matchAll&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="o"&gt;/&lt;/span&gt;\&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="o"&gt;^&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="o"&gt;*&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;\&lt;span class="p"&gt;)&lt;/span&gt;\\&lt;span class="n"&gt;b&lt;/span&gt;&lt;span class="o"&gt;/&lt;/span&gt;&lt;span class="n"&gt;g&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
      &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="n"&gt;alt&lt;/span&gt; &lt;span class="k"&gt;of&lt;/span&gt; &lt;span class="n"&gt;m&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="mi"&gt;1&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;split&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s1"&gt;'|'&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
        &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nf"&gt;nonAscii&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;alt&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;pop&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="nb"&gt;console&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;log&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sb"&gt;`${f}:${i + 1} "${alt}" before )\\b`&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
    &lt;span class="c1"&gt;// 2) \b written directly next to a non-ASCII character&lt;/span&gt;
    &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="o"&gt;/&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="o"&gt;^&lt;/span&gt;\&lt;span class="n"&gt;x00&lt;/span&gt;&lt;span class="o"&gt;-&lt;/span&gt;\&lt;span class="n"&gt;x7F&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;\\&lt;span class="n"&gt;b&lt;/span&gt;&lt;span class="o"&gt;|&lt;/span&gt;\\&lt;span class="n"&gt;b&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="o"&gt;^&lt;/span&gt;\&lt;span class="n"&gt;x00&lt;/span&gt;&lt;span class="o"&gt;-&lt;/span&gt;\&lt;span class="n"&gt;x7F&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="o"&gt;/&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;test&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;line&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="nb"&gt;console&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;log&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sb"&gt;`${f}:${i + 1} \\b touches non-ASCII`&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
  &lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;In our codebase it found one more case, in an unrelated rule, after all the manual fixes were considered complete.&lt;/p&gt;

&lt;h2&gt;
  
  
  What we changed in how we work
&lt;/h2&gt;

&lt;p&gt;Every multilingual pattern is now tested with one real sentence per language before it counts as working, including the accented and non-Latin forms. If your app speaks anything other than English, run the script above on it: it takes a second, and the bugs it finds are the silent kind.&lt;/p&gt;

&lt;p&gt;&lt;em&gt;Originally published as a research note at &lt;a href="https://lovora.ai/research/unicode-word-boundaries" rel="noopener noreferrer"&gt;lovora.ai/research/unicode-word-boundaries&lt;/a&gt; (CC BY 4.0).&lt;/em&gt;&lt;/p&gt;

</description>
      <category>javascript</category>
      <category>regex</category>
      <category>unicode</category>
      <category>i18n</category>
    </item>
    <item>
      <title>Long-term memory for an AI companion without a vector database: 40 messages verbatim + 15 distilled facts</title>
      <dc:creator>Lovora</dc:creator>
      <pubDate>Fri, 20 Nov 2026 22:55:10 +0000</pubDate>
      <link>https://dev.to/lovora/long-term-memory-for-an-ai-companion-without-a-vector-database-40-messages-verbatim-15-distilled-11gf</link>
      <guid>https://dev.to/lovora/long-term-memory-for-an-ai-companion-without-a-vector-database-40-messages-verbatim-15-distilled-11gf</guid>
      <description>&lt;p&gt;I run &lt;a href="https://lovora.ai" rel="noopener noreferrer"&gt;Lovora&lt;/a&gt;, an AI companion service for adults in nine languages. The number one thing people ask an AI companion is "will she remember me?", and "memory" turns out to mean two different things. Here is the design we use, and why.&lt;/p&gt;

&lt;h2&gt;
  
  
  Two layers, two jobs
&lt;/h2&gt;

&lt;p&gt;&lt;strong&gt;1. The recent conversation, word for word.&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;On every reply the model receives the last 40 messages of the chat, both sides, exactly as written. In voice calls it is the last 24, because spoken turns need to be answered faster and the prompt has to stay small.&lt;/p&gt;

&lt;p&gt;This layer is what keeps a scene or an argument coherent. A summary would lose the exact wording, the jokes and the thing she promised two messages ago, which is exactly what people notice in a conversation.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;2. Long-term facts about the user.&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;Every 5 messages the user sends, a separate pass reads the latest part of the conversation and updates a short list of lasting facts: at most 15 short facts, about 1,500 characters. That list goes into the prompt of every later reply, in chat and in calls.&lt;/p&gt;

&lt;p&gt;That is the layer that lets her ask about the job interview you mentioned last week, hundreds of messages later.&lt;/p&gt;

&lt;h2&gt;
  
  
  What goes into the fact list (and what does not)
&lt;/h2&gt;

&lt;p&gt;The distillation pass is instructed to keep facts about &lt;strong&gt;the user&lt;/strong&gt;, never about the character, whose personality and backstory are fixed:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;name, age, job and where they live, if shared;&lt;/li&gt;
&lt;li&gt;interests, likes and dislikes;&lt;/li&gt;
&lt;li&gt;relationship, family and pet details;&lt;/li&gt;
&lt;li&gt;plans and events mentioned, and how they feel;&lt;/li&gt;
&lt;li&gt;stated preferences, including intimate ones;&lt;/li&gt;
&lt;li&gt;anything they explicitly asked her to remember.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Small talk is discarded. A fact that is contradicted later is updated or dropped, not appended: the list is rewritten, not grown. The character is told to use what she remembers naturally, by asking about it or referring to it, and never to recite it as a list. Reciting is the fastest way to make a companion feel like a database.&lt;/p&gt;

&lt;h2&gt;
  
  
  Why not embeddings and retrieval?
&lt;/h2&gt;

&lt;p&gt;We chose not to retrieve past messages by similarity. For a companion it has three problems:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;
&lt;strong&gt;Relevance is not similarity.&lt;/strong&gt; The message that matters ("my mother is in hospital") is rarely the one most similar to the current message ("good morning").&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Retrieved fragments arrive without context&lt;/strong&gt;, so the model quotes them oddly or at the wrong moment.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;It grows forever.&lt;/strong&gt; A capped, rewritten list of 15 facts costs the same on message 10 and message 10,000, and it is easy to inspect, correct and delete.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;A small fixed budget forces the distillation step to decide what matters, which is exactly the judgment retrieval skips.&lt;/p&gt;

&lt;h2&gt;
  
  
  Scope and privacy
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Per companion.&lt;/strong&gt; Each companion has her own conversation and her own memory; what you tell one is not known to another.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Deletable.&lt;/strong&gt; Deleting the account erases the conversations and what she remembers (&lt;a href="https://lovora.ai/account-and-data" rel="noopener noreferrer"&gt;what deletion erases&lt;/a&gt;).&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Model-agnostic.&lt;/strong&gt; Both layers are plain text in the prompt, so they survive a model change.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  What it costs
&lt;/h2&gt;

&lt;p&gt;The verbatim window is the expensive part: 40 messages is a real share of the context on every reply. The fact pass is cheap: it runs once every 5 user messages, as a separate call on a short excerpt.&lt;/p&gt;

&lt;p&gt;If you are building anything conversational (support bots, tutors, companions), I would start here: a verbatim window sized to your latency budget, plus a small, rewritten list of facts about the user. Retrieval can come later if you ever need it.&lt;/p&gt;

&lt;p&gt;The full, current description is in our docs: &lt;a href="https://lovora.ai/memory" rel="noopener noreferrer"&gt;How memory works on Lovora&lt;/a&gt;.&lt;/p&gt;

</description>
      <category>ai</category>
      <category>llm</category>
      <category>architecture</category>
      <category>webdev</category>
    </item>
  </channel>
</rss>
