<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:dc="http://purl.org/dc/elements/1.1/">
  <channel>
    <title>DEV Community: bridalbreak</title>
    <description>The latest articles on DEV Community by bridalbreak (@bridalbreak).</description>
    <link>https://dev.to/bridalbreak</link>
    <image>
      <url>https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F4250196%2F0b02a7af-55da-4f6a-ae58-0bfe5ad748d6.png</url>
      <title>DEV Community: bridalbreak</title>
      <link>https://dev.to/bridalbreak</link>
    </image>
    <atom:link rel="self" type="application/rss+xml" href="https://dev.to/feed/bridalbreak"/>
    <language>en</language>
    <item>
      <title>Running a 1.5M-Parameter NLP Model in WebAssembly Under 10ms</title>
      <dc:creator>bridalbreak</dc:creator>
      <pubDate>Tue, 17 Nov 2026 15:58:40 +0000</pubDate>
      <link>https://dev.to/bridalbreak/running-a-15m-parameter-nlp-model-in-webassembly-under-10ms-1677</link>
      <guid>https://dev.to/bridalbreak/running-a-15m-parameter-nlp-model-in-webassembly-under-10ms-1677</guid>
      <description>&lt;p&gt;I got tired of waiting 500ms every time I typed something into my app's command bar.&lt;/p&gt;

&lt;p&gt;When you just want to type "Algorithms exam tomorrow at 9am" and hit Enter, sending a request to a cloud LLM feels broken. You feel the network lag, it fails on bad campus Wi-Fi, and paying for API calls just to figure out if a sentence is a calendar event or a to-do item makes no sense.&lt;/p&gt;

&lt;p&gt;So for Stedy&lt;br&gt;
, I ripped the cloud call out of the hot path and moved classification into the browser.&lt;/p&gt;

&lt;p&gt;The setup is pretty simple:&lt;/p&gt;

&lt;p&gt;Deterministic TypeScript code extracts dates, time ranges, and known course names.&lt;br&gt;
ADS-1.5M, a small INT8-quantized model (~1.5M parameters), runs in WebAssembly via onnxruntime-web/wasm to classify the intent (EVENT, TASK, or NOTE) and category.&lt;/p&gt;

&lt;p&gt;Once the .int8.onnx file and vocab.json are cached in CacheStorage, a forward pass takes less than 10ms locally:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight javascript"&gt;&lt;code&gt;&lt;span class="k"&gt;import&lt;/span&gt; &lt;span class="o"&gt;*&lt;/span&gt; &lt;span class="k"&gt;as&lt;/span&gt; &lt;span class="n"&gt;ort&lt;/span&gt; &lt;span class="k"&gt;from&lt;/span&gt; &lt;span class="s2"&gt;"onnxruntime-web/wasm"&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="n"&gt;session&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="k"&gt;await&lt;/span&gt; &lt;span class="n"&gt;ort&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nc"&gt;InferenceSession&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;create&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
  &lt;span class="s2"&gt;"/models/intent-classifier.int8.onnx"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="n"&gt;executionProviders&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"wasm"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt; &lt;span class="p"&gt;}&lt;/span&gt;
&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="n"&gt;feeds&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
  &lt;span class="n"&gt;input_ids&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="k"&gt;new&lt;/span&gt; &lt;span class="n"&gt;ort&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nc"&gt;Tensor&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;"int64"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;tokenIds&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="mi"&gt;1&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;tokenIds&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;length&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="n"&gt;results&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="k"&gt;await&lt;/span&gt; &lt;span class="n"&gt;session&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;run&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;feeds&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Is a 1.5M model as smart as a frontier LLM? Obviously not. If the local confidence score is too low, it still falls back to an API call.&lt;/p&gt;

&lt;p&gt;But for 90% of daily inputs, running a tiny quantized model in WASM makes the whole app feel instant, works completely offline, and keeps user notes on their own device.&lt;/p&gt;

&lt;p&gt;If you're curious to test it out, it's live inside &lt;a href="https://stedy.me" rel="noopener noreferrer"&gt;Stedy&lt;/a&gt;&lt;/p&gt;

</description>
      <category>webassembly</category>
      <category>typescript</category>
      <category>machinelearning</category>
      <category>webdev</category>
    </item>
  </channel>
</rss>
