<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:dc="http://purl.org/dc/elements/1.1/">
  <channel>
    <title>DEV Community: Chanhyoung Lee</title>
    <description>The latest articles on DEV Community by Chanhyoung Lee (@chanhyounglee).</description>
    <link>https://dev.to/chanhyounglee</link>
    <image>
      <url>https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F4248348%2Fb41cbacd-8f7b-46dc-89a1-6ae0f41c95de.jpg</url>
      <title>DEV Community: Chanhyoung Lee</title>
      <link>https://dev.to/chanhyounglee</link>
    </image>
    <atom:link rel="self" type="application/rss+xml" href="https://dev.to/feed/chanhyounglee"/>
    <language>en</language>
    <item>
      <title>Checking every number an LLM writes against the source file</title>
      <dc:creator>Chanhyoung Lee</dc:creator>
      <pubDate>Mon, 16 Nov 2026 19:27:30 +0000</pubDate>
      <link>https://dev.to/chanhyounglee/checking-every-number-an-llm-writes-against-the-source-file-2obo</link>
      <guid>https://dev.to/chanhyounglee/checking-every-number-an-llm-writes-against-the-source-file-2obo</guid>
      <description>&lt;p&gt;LabChure turns uploaded documents (PDF, PPTX, DOCX, HWP, images and more) into a web brochure with its own URL.&lt;/p&gt;

&lt;p&gt;Getting the model to write the page was the easy part. Numbers were the hard part. A model that rewrites a spec sheet will sometimes turn "12 kW" into "15 kW", and the page still reads fine.&lt;/p&gt;

&lt;p&gt;So we added a gate. Before a page can be published, code checks its numbers against the uploaded files.&lt;/p&gt;

&lt;h2&gt;
  
  
  What the code checks
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;Numbers with a unit: mm, kg, kWh, Hz, rpm, °C, %, inch and similar&lt;/li&gt;
&lt;li&gt;Prices: ₩ $ € ¥ £, or amounts with KRW / USD&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Both sides are normalized first: Unicode NFKC, lowercase, no spaces, no thousands separators. So "1,200 mm" and "1200mm" are the same token.&lt;/p&gt;

&lt;p&gt;Matching has two steps:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;The passage that section cites&lt;/li&gt;
&lt;li&gt;If it's not there, every file you uploaded&lt;/li&gt;
&lt;/ol&gt;

&lt;h2&gt;
  
  
  What happens on a mismatch
&lt;/h2&gt;

&lt;p&gt;The page goes to "needs review" and the publish API returns 409. The editor shows the token: &lt;code&gt;Not found in your documents: 15 kW&lt;/code&gt;. You fix the text or press "Publish anyway". Nothing passes silently.&lt;/p&gt;

&lt;p&gt;After an AI edit, the check runs again on its own. After a manual edit, the page goes back to "needs review" until you run the check again or press "Publish anyway". After a full generation, if the first review flags sections, the model gets one try to fix those sections, then they're reviewed again.&lt;/p&gt;

&lt;h2&gt;
  
  
  What the code doesn't check
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;Dates and years&lt;/li&gt;
&lt;li&gt;Numbers without a unit&lt;/li&gt;
&lt;li&gt;Short units like g, m, W, V and lb&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;For these, a second model pass reads each section next to its sources. It catches a lot, but it's still a model.&lt;/p&gt;

&lt;h2&gt;
  
  
  Bugs that shaped the regex
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;The source said "120 C", the page said "120 °C". The gate called it a mismatch. Now the degree sign is dropped before matching, and a bare C after a number counts as Celsius.&lt;/li&gt;
&lt;li&gt;The source said "19-inch", the Korean copy said "19인치". In the source files, a hyphen between a number and a unit is now read as a space, and 인치 maps to inch.&lt;/li&gt;
&lt;li&gt;Test certificates put the unit in the column header: &lt;code&gt;Mass Fraction (%)&lt;/code&gt;, with &lt;code&gt;17.803 ± 0.099&lt;/code&gt; in the cell. Our regex looks for a unit after the number, so it matched nothing, and the composition values were dropped from the page. We added a separate header pattern instead of loosening the main regex, because the gate shares that regex.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  Scanned PDFs
&lt;/h2&gt;

&lt;p&gt;Scanned pages and text inside images are read page by page. If a page can't be read, the file list says &lt;code&gt;Read X of Y pages&lt;/code&gt;, and that page isn't used. We don't guess.&lt;/p&gt;

&lt;h2&gt;
  
  
  Retrieval
&lt;/h2&gt;

&lt;p&gt;There's no separate vector database. Search runs inside our SQL database: vector search plus full-text keyword search, merged with RRF (k=60). Each section keeps the passages it cites, and the gate checks those first.&lt;/p&gt;

&lt;h2&gt;
  
  
  Known limits
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;You need an account to build a page.&lt;/li&gt;
&lt;li&gt;Tables in PDFs can come out with rows misaligned.&lt;/li&gt;
&lt;li&gt;The model pass on dates and plain numbers can miss things.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  Question
&lt;/h2&gt;

&lt;p&gt;If you've built something that checks LLM output against source documents, where did it break? For us it's documents without a clear structure.&lt;/p&gt;

&lt;p&gt;A published example: &lt;a href="https://labchure.site/itemwright-digital-sat" rel="noopener noreferrer"&gt;https://labchure.site/itemwright-digital-sat&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;&lt;em&gt;I used AI (Claude) to help write this post.&lt;/em&gt;&lt;/p&gt;

</description>
      <category>showdev</category>
      <category>ai</category>
      <category>webdev</category>
      <category>dotnet</category>
    </item>
  </channel>
</rss>
