<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:dc="http://purl.org/dc/elements/1.1/">
  <channel>
    <title>DEV Community: hamdallah jodah</title>
    <description>The latest articles on DEV Community by hamdallah jodah (@hamdallah_jodah).</description>
    <link>https://dev.to/hamdallah_jodah</link>
    <image>
      <url>https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F4057277%2F29076dff-bd3b-4792-aa17-82d2ebb99939.jpg</url>
      <title>DEV Community: hamdallah jodah</title>
      <link>https://dev.to/hamdallah_jodah</link>
    </image>
    <atom:link rel="self" type="application/rss+xml" href="https://dev.to/feed/hamdallah_jodah"/>
    <language>en</language>
    <item>
      <title>BatchFlow: sum, filter and group a huge CSV in PHP without building arrays</title>
      <dc:creator>hamdallah jodah</dc:creator>
      <pubDate>Fri, 13 Nov 2026 08:37:52 +0000</pubDate>
      <link>https://dev.to/hamdallah_jodah/batchflow-sum-filter-and-group-a-huge-csv-in-php-without-building-arrays-44g8</link>
      <guid>https://dev.to/hamdallah_jodah/batchflow-sum-filter-and-group-a-huge-csv-in-php-without-building-arrays-44g8</guid>
      <description>&lt;p&gt;I just released &lt;strong&gt;BatchFlow 1.0.0&lt;/strong&gt;, a PHP extension written in C. It scans CSV files and streams, filters rows, counts, sums and groups in native code, and returns only the result to PHP.&lt;/p&gt;

&lt;h2&gt;
  
  
  The loop we all write
&lt;/h2&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight php"&gt;&lt;code&gt;&lt;span class="nv"&gt;$totals&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;span class="nv"&gt;$h&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nf"&gt;fopen&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nv"&gt;$path&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;'rb'&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;span class="nf"&gt;fgetcsv&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nv"&gt;$h&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;span class="k"&gt;while&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nv"&gt;$row&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nf"&gt;fgetcsv&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nv"&gt;$h&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;!==&lt;/span&gt; &lt;span class="kc"&gt;false&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="nv"&gt;$totals&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="nv"&gt;$row&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nv"&gt;$totals&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="nv"&gt;$row&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt; &lt;span class="o"&gt;??&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;+&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="kt"&gt;int&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="nv"&gt;$row&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="mi"&gt;2&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Every row becomes a PHP array, even when you only want a few totals.&lt;/p&gt;

&lt;h2&gt;
  
  
  The BatchFlow version
&lt;/h2&gt;

&lt;p&gt;The examples use this &lt;code&gt;sales.csv&lt;/code&gt;:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;region,product,units,price
north,apple,10,1.5
south,pear,4,2.25
north,pear,7,2.25
east,apple,3,1.5
south,apple,8,1.5
north,fig,,4
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;





&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight php"&gt;&lt;code&gt;&lt;span class="k"&gt;use&lt;/span&gt; &lt;span class="nc"&gt;BatchFlow&lt;/span&gt;\&lt;span class="nc"&gt;Csv&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;

&lt;span class="nv"&gt;$schema&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s1"&gt;'region'&lt;/span&gt; &lt;span class="o"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="s1"&gt;'string'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;'product'&lt;/span&gt; &lt;span class="o"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="s1"&gt;'string'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;'units'&lt;/span&gt; &lt;span class="o"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="s1"&gt;'int64'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;'price'&lt;/span&gt; &lt;span class="o"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="s1"&gt;'float64'&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;span class="nv"&gt;$query&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nc"&gt;Csv&lt;/span&gt;&lt;span class="o"&gt;::&lt;/span&gt;&lt;span class="nf"&gt;scan&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nv"&gt;$path&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nv"&gt;$schema&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;

&lt;span class="k"&gt;echo&lt;/span&gt; &lt;span class="nv"&gt;$query&lt;/span&gt;&lt;span class="o"&gt;-&amp;gt;&lt;/span&gt;&lt;span class="nb"&gt;count&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"\n"&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;span class="k"&gt;echo&lt;/span&gt; &lt;span class="nv"&gt;$query&lt;/span&gt;&lt;span class="o"&gt;-&amp;gt;&lt;/span&gt;&lt;span class="nf"&gt;sum&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s1"&gt;'units'&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"\n"&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;span class="k"&gt;echo&lt;/span&gt; &lt;span class="nv"&gt;$query&lt;/span&gt;&lt;span class="o"&gt;-&amp;gt;&lt;/span&gt;&lt;span class="nf"&gt;avg&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s1"&gt;'units'&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"\n"&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;span class="k"&gt;echo&lt;/span&gt; &lt;span class="nb"&gt;json_encode&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nv"&gt;$query&lt;/span&gt;&lt;span class="o"&gt;-&amp;gt;&lt;/span&gt;&lt;span class="nf"&gt;groupSum&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s1"&gt;'region'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;'units'&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"\n"&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;





&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;6
32
6.4
[["north",3,17],["south",2,12],["east",1,3]]
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;An empty field is &lt;code&gt;null&lt;/code&gt; and is skipped by &lt;code&gt;sum&lt;/code&gt;, &lt;code&gt;min&lt;/code&gt;, &lt;code&gt;max&lt;/code&gt; and &lt;code&gt;avg&lt;/code&gt;. &lt;code&gt;groupSum&lt;/code&gt; returns &lt;code&gt;[key, rows, sum]&lt;/code&gt; in first-seen order.&lt;/p&gt;

&lt;h2&gt;
  
  
  Filters and rows
&lt;/h2&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight php"&gt;&lt;code&gt;&lt;span class="nv"&gt;$apples&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nc"&gt;Csv&lt;/span&gt;&lt;span class="o"&gt;::&lt;/span&gt;&lt;span class="nf"&gt;scan&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nv"&gt;$path&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nv"&gt;$schema&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="o"&gt;-&amp;gt;&lt;/span&gt;&lt;span class="nf"&gt;whereEq&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s1"&gt;'product'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;'apple'&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="o"&gt;-&amp;gt;&lt;/span&gt;&lt;span class="nf"&gt;whereGte&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s1"&gt;'units'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;5&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;

&lt;span class="k"&gt;echo&lt;/span&gt; &lt;span class="nv"&gt;$apples&lt;/span&gt;&lt;span class="o"&gt;-&amp;gt;&lt;/span&gt;&lt;span class="nb"&gt;count&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"\n"&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;span class="k"&gt;echo&lt;/span&gt; &lt;span class="nb"&gt;json_encode&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nv"&gt;$apples&lt;/span&gt;&lt;span class="o"&gt;-&amp;gt;&lt;/span&gt;&lt;span class="nf"&gt;toArray&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"\n"&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;





&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;2
[{"region":"north","product":"apple","units":10,"price":1.5},{"region":"south","product":"apple","units":8,"price":1.5}]
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Filters are immutable and chainable: &lt;code&gt;whereEq&lt;/code&gt;, &lt;code&gt;whereNe&lt;/code&gt;, &lt;code&gt;whereGt&lt;/code&gt;, &lt;code&gt;whereGte&lt;/code&gt;, &lt;code&gt;whereLt&lt;/code&gt;, &lt;code&gt;whereLte&lt;/code&gt;, &lt;code&gt;whereIn&lt;/code&gt;.&lt;/p&gt;

&lt;h2&gt;
  
  
  Many aggregates in one pass
&lt;/h2&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight php"&gt;&lt;code&gt;&lt;span class="k"&gt;echo&lt;/span&gt; &lt;span class="nb"&gt;json_encode&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nv"&gt;$query&lt;/span&gt;&lt;span class="o"&gt;-&amp;gt;&lt;/span&gt;&lt;span class="nf"&gt;groupAgg&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s1"&gt;'region'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;'product'&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s1"&gt;'sum'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;'units'&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s1"&gt;'max'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;'price'&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"\n"&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;





&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;[["north","apple",1,10,1.5],["south","pear",1,4,2.25],["north","pear",1,7,2.25],["east","apple",1,3,1.5],["south","apple",1,8,1.5],["north","fig",1,0,4]]
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;One to four keys and up to 16 aggregates.&lt;/p&gt;

&lt;h2&gt;
  
  
  Streams and cursors
&lt;/h2&gt;

&lt;p&gt;The schema lists every column of the file, in file order. This example uses a second, two-column file.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight php"&gt;&lt;code&gt;&lt;span class="nv"&gt;$path2&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nf"&gt;sys_get_temp_dir&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;.&lt;/span&gt; &lt;span class="s1"&gt;'/units.csv'&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;span class="nf"&gt;file_put_contents&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nv"&gt;$path2&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"region,units\nnorth,10\nsouth,4\nnorth,7\neast,3\nsouth,8\n"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;

&lt;span class="nv"&gt;$stream&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nf"&gt;fopen&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nv"&gt;$path2&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;'rb'&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;span class="nv"&gt;$total&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nc"&gt;Csv&lt;/span&gt;&lt;span class="o"&gt;::&lt;/span&gt;&lt;span class="nf"&gt;scanStream&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nv"&gt;$stream&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s1"&gt;'region'&lt;/span&gt; &lt;span class="o"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="s1"&gt;'string'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;'units'&lt;/span&gt; &lt;span class="o"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="s1"&gt;'int64'&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="o"&gt;-&amp;gt;&lt;/span&gt;&lt;span class="nf"&gt;sum&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s1"&gt;'units'&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;span class="k"&gt;echo&lt;/span&gt; &lt;span class="nv"&gt;$total&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"\n"&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;

&lt;span class="nv"&gt;$cursor&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nc"&gt;Csv&lt;/span&gt;&lt;span class="o"&gt;::&lt;/span&gt;&lt;span class="nf"&gt;scan&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nv"&gt;$path2&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s1"&gt;'region'&lt;/span&gt; &lt;span class="o"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="s1"&gt;'string'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;'units'&lt;/span&gt; &lt;span class="o"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="s1"&gt;'int64'&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="o"&gt;-&amp;gt;&lt;/span&gt;&lt;span class="nf"&gt;cursor&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;span class="k"&gt;while&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nv"&gt;$rows&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nv"&gt;$cursor&lt;/span&gt;&lt;span class="o"&gt;-&amp;gt;&lt;/span&gt;&lt;span class="nf"&gt;fetch&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mi"&gt;2&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="k"&gt;echo&lt;/span&gt; &lt;span class="nb"&gt;json_encode&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nv"&gt;$rows&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"\n"&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;span class="nv"&gt;$cursor&lt;/span&gt;&lt;span class="o"&gt;-&amp;gt;&lt;/span&gt;&lt;span class="nf"&gt;close&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;





&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;32
[{"region":"north","units":10},{"region":"south","units":4}]
[{"region":"north","units":7},{"region":"east","units":3}]
[{"region":"south","units":8}]
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;&lt;code&gt;scanStream&lt;/code&gt; reads any PHP stream. A cursor returns rows in batches, so memory stays flat.&lt;/p&gt;

&lt;h2&gt;
  
  
  Load once with Table
&lt;/h2&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight php"&gt;&lt;code&gt;&lt;span class="k"&gt;use&lt;/span&gt; &lt;span class="nc"&gt;BatchFlow&lt;/span&gt;\&lt;span class="nc"&gt;Table&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;

&lt;span class="nv"&gt;$table&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nc"&gt;Table&lt;/span&gt;&lt;span class="o"&gt;::&lt;/span&gt;&lt;span class="nf"&gt;fromCsv&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nv"&gt;$path&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nv"&gt;$schema&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;span class="k"&gt;echo&lt;/span&gt; &lt;span class="nb"&gt;json_encode&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nv"&gt;$table&lt;/span&gt;&lt;span class="o"&gt;-&amp;gt;&lt;/span&gt;&lt;span class="nf"&gt;whereEq&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s1"&gt;'product'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;'apple'&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="o"&gt;-&amp;gt;&lt;/span&gt;&lt;span class="nf"&gt;groupSum&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s1"&gt;'region'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;'units'&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"\n"&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;





&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;[["north",1,10],["east",1,3],["south",1,8]]
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The file is read once and kept as typed columns. Repeated queries do not read it again.&lt;/p&gt;

&lt;h2&gt;
  
  
  Errors you can handle
&lt;/h2&gt;

&lt;p&gt;For a file with the lines &lt;code&gt;region,units&lt;/code&gt;, &lt;code&gt;north,10&lt;/code&gt; and &lt;code&gt;south,many&lt;/code&gt;, and the schema &lt;code&gt;['region' =&amp;gt; 'string', 'units' =&amp;gt; 'int64']&lt;/code&gt;, &lt;code&gt;sum('units')&lt;/code&gt; throws:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;BatchFlow\CsvException: row 2 (line 3, offset 22): column "units": expected int64, got malformed value
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Messages carry row, line, offset and column. They never carry record bytes, so they are safe to log. BatchFlow checks &lt;code&gt;memory_limit&lt;/code&gt; before it allocates, so a result that is too big is a &lt;code&gt;LimitException&lt;/code&gt; and not a fatal error (best effort for small blocks on a fragmented heap; see the threat model).&lt;/p&gt;

&lt;h2&gt;
  
  
  Benchmarks (with the conditions)
&lt;/h2&gt;

&lt;p&gt;One machine: Intel Xeon Platinum 8358 (Linux x86-64, AVX2), PHP 8.3.36 NTS. Files of 1,000,000 rows. One core per timed job, nothing else on it. Each range covers all tested cases, in a long-running process (warm) and a new process (fresh). "Fastest JIT'd PHP loop" means PHP with opcache and the tracing JIT on. Results are checked to be identical to the PHP loops.&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;What&lt;/th&gt;
&lt;th&gt;Result&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Group-by (&lt;code&gt;groupCount&lt;/code&gt;, &lt;code&gt;groupSum&lt;/code&gt;, 10 to 100,000 keys) vs the fastest JIT'd PHP loop&lt;/td&gt;
&lt;td&gt;3.7x to 13.6x faster&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Group-by vs DuckDB CLI, one thread&lt;/td&gt;
&lt;td&gt;2.6x to 5.5x faster&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Streaming sum vs the fastest JIT'd PHP loop&lt;/td&gt;
&lt;td&gt;5.2x to 15.4x faster. Peak memory 0.84 times that of the streaming PHP loop (about 16% lower), constant at 23,280 KB in every case&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;
&lt;code&gt;toArray&lt;/code&gt; vs the fastest PHP CSV loader&lt;/td&gt;
&lt;td&gt;2.8x to 7.5x faster&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;h2&gt;
  
  
  Where it loses
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;On group-by with 10,000,000 rows where most rows are kept, 4-thread DuckDB on four cores is faster: it takes 0.75 to 0.88 of BatchFlow's time. BatchFlow has no threads. On the other 10,000,000-row group-by cases it ties (0.97 to 1.00).&lt;/li&gt;
&lt;li&gt;At 100,000 rows, one fresh-process cell was only 2.7x faster than the PHP loop. The 3x claim is for 1,000,000 rows and up.&lt;/li&gt;
&lt;li&gt;No arm64 speed claim. The arm64 build is tested for correct results only.&lt;/li&gt;
&lt;li&gt;ZTS is not supported.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  What it is NOT
&lt;/h2&gt;

&lt;p&gt;Not a CSV writer, not a general CSV parsing library, and not a database. No joins, no SQL. It reads CSV and answers questions about it.&lt;/p&gt;

&lt;h2&gt;
  
  
  What is next
&lt;/h2&gt;

&lt;p&gt;JSON input (NDJSON and top-level JSON arrays) with the same API is coming soon, in the next release. No speed or memory claim for it yet.&lt;/p&gt;

&lt;h2&gt;
  
  
  Install
&lt;/h2&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;pie install hamdallah90/batchflow
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;On minimal images such as &lt;code&gt;php:8.3-cli&lt;/code&gt;, install &lt;code&gt;libtool&lt;/code&gt; first. Then add &lt;code&gt;extension=batchflow&lt;/code&gt; to &lt;code&gt;php.ini&lt;/code&gt;. Or build from source: &lt;code&gt;phpize &amp;amp;&amp;amp; ./configure --enable-batchflow &amp;amp;&amp;amp; make &amp;amp;&amp;amp; sudo make install&lt;/code&gt;.&lt;/p&gt;

&lt;p&gt;PHP 8.3, 8.4 and 8.5, non-thread-safe. Linux x86-64 and arm64, and macOS. MIT license.&lt;/p&gt;

&lt;p&gt;Repo: &lt;a href="https://github.com/hamdallah90/php-batchflow" rel="noopener noreferrer"&gt;https://github.com/hamdallah90/php-batchflow&lt;/a&gt; (issues and stars welcome)&lt;/p&gt;

&lt;p&gt;Release notes: &lt;a href="https://github.com/hamdallah90/php-batchflow/releases/tag/v1.0.0" rel="noopener noreferrer"&gt;https://github.com/hamdallah90/php-batchflow/releases/tag/v1.0.0&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Full results and method: &lt;a href="https://github.com/hamdallah90/php-batchflow/blob/main/benchmarks/RESULTS.md" rel="noopener noreferrer"&gt;&lt;code&gt;benchmarks/RESULTS.md&lt;/code&gt;&lt;/a&gt;&lt;/p&gt;

</description>
      <category>php</category>
      <category>performance</category>
      <category>opensource</category>
      <category>csv</category>
    </item>
    <item>
      <title>JsonFuse: filter and project a huge NDJSON file in PHP with one call</title>
      <dc:creator>hamdallah jodah</dc:creator>
      <pubDate>Wed, 11 Nov 2026 19:21:15 +0000</pubDate>
      <link>https://dev.to/hamdallah_jodah/jsonfuse-filter-and-project-a-huge-ndjson-file-in-php-with-one-call-3hcc</link>
      <guid>https://dev.to/hamdallah_jodah/jsonfuse-filter-and-project-a-huge-ndjson-file-in-php-with-one-call-3hcc</guid>
      <description>&lt;p&gt;I just released &lt;strong&gt;JsonFuse 1.0.0&lt;/strong&gt;, a native PHP extension for one narrow job: read NDJSON from a stream, validate each record, filter it, keep some fields, and write NDJSON to another stream. All in one call. 🚀&lt;/p&gt;

&lt;h2&gt;
  
  
  The loop we all write
&lt;/h2&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight php"&gt;&lt;code&gt;&lt;span class="k"&gt;while&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nv"&gt;$line&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nf"&gt;fgets&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nv"&gt;$in&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;!==&lt;/span&gt; &lt;span class="kc"&gt;false&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="nv"&gt;$row&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nb"&gt;json_decode&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nv"&gt;$line&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="kc"&gt;true&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
    &lt;span class="c1"&gt;// ...hand-written checks for field presence and types...&lt;/span&gt;
    &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nv"&gt;$row&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s1"&gt;'event'&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt; &lt;span class="o"&gt;??&lt;/span&gt; &lt;span class="kc"&gt;null&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;!==&lt;/span&gt; &lt;span class="s1"&gt;'purchase'&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="k"&gt;continue&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
    &lt;span class="nf"&gt;fwrite&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nv"&gt;$out&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nb"&gt;json_encode&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s1"&gt;'user_id'&lt;/span&gt; &lt;span class="o"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="nv"&gt;$row&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s1"&gt;'user_id'&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;'amount'&lt;/span&gt; &lt;span class="o"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="nv"&gt;$row&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s1"&gt;'amount'&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;.&lt;/span&gt; &lt;span class="s2"&gt;"\n"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Every line becomes a full PHP array, even when you drop it a moment later.&lt;/p&gt;

&lt;h2&gt;
  
  
  The one-call version
&lt;/h2&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight php"&gt;&lt;code&gt;&lt;span class="k"&gt;use&lt;/span&gt; &lt;span class="nc"&gt;JsonFuse&lt;/span&gt;\&lt;span class="nc"&gt;Plan&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;span class="k"&gt;use&lt;/span&gt; &lt;span class="nc"&gt;JsonFuse&lt;/span&gt;\&lt;span class="nc"&gt;Runner&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;

&lt;span class="nv"&gt;$plan&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nc"&gt;Plan&lt;/span&gt;&lt;span class="o"&gt;::&lt;/span&gt;&lt;span class="nf"&gt;compile&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;
    &lt;span class="s1"&gt;'fields'&lt;/span&gt; &lt;span class="o"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s1"&gt;'user_id'&lt;/span&gt; &lt;span class="o"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="s1"&gt;'int64'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;'amount'&lt;/span&gt; &lt;span class="o"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="s1"&gt;'float64'&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="s1"&gt;'where'&lt;/span&gt;  &lt;span class="o"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s1"&gt;'event'&lt;/span&gt; &lt;span class="o"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="s1"&gt;'purchase'&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;

&lt;span class="nv"&gt;$in&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nf"&gt;fopen&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s1"&gt;'php://memory'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;'w+b'&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;span class="nf"&gt;fwrite&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nv"&gt;$in&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="o"&gt;&amp;lt;&amp;lt;&amp;lt;&lt;/span&gt;&lt;span class="s1"&gt;'NDJSON'&lt;/span&gt;
&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="s2"&gt;"event"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="s2"&gt;"view"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="s2"&gt;"user_id"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="mi"&gt;1&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="s2"&gt;"amount"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;
&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="s2"&gt;"event"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="s2"&gt;"purchase"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="s2"&gt;"user_id"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="mi"&gt;2&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="s2"&gt;"amount"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="mf"&gt;19.99&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="s2"&gt;"currency"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="s2"&gt;"EUR"&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;
&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="s2"&gt;"event"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="s2"&gt;"purchase"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="s2"&gt;"user_id"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="mi"&gt;3&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="s2"&gt;"amount"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="mi"&gt;5&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;

&lt;span class="no"&gt;NDJSON&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;span class="nf"&gt;rewind&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nv"&gt;$in&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;span class="nv"&gt;$out&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nf"&gt;fopen&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s1"&gt;'php://output'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;'wb'&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;

&lt;span class="nv"&gt;$stats&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nc"&gt;Runner&lt;/span&gt;&lt;span class="o"&gt;::&lt;/span&gt;&lt;span class="nf"&gt;process&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nv"&gt;$in&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nv"&gt;$out&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nv"&gt;$plan&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;span class="nf"&gt;printf&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;"read %d, matched %d, emitted %d, wrote %d bytes\n"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="nv"&gt;$stats&lt;/span&gt;&lt;span class="o"&gt;-&amp;gt;&lt;/span&gt;&lt;span class="n"&gt;recordsRead&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nv"&gt;$stats&lt;/span&gt;&lt;span class="o"&gt;-&amp;gt;&lt;/span&gt;&lt;span class="n"&gt;recordsMatched&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nv"&gt;$stats&lt;/span&gt;&lt;span class="o"&gt;-&amp;gt;&lt;/span&gt;&lt;span class="n"&gt;recordsEmitted&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nv"&gt;$stats&lt;/span&gt;&lt;span class="o"&gt;-&amp;gt;&lt;/span&gt;&lt;span class="n"&gt;bytesWritten&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;





&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;{"user_id":2,"amount":19.99}
{"user_id":3,"amount":5.0}
read 3, matched 2, emitted 2, wrote 56 bytes
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h2&gt;
  
  
  Filters: ranges, lists, nested keys
&lt;/h2&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight php"&gt;&lt;code&gt;&lt;span class="nv"&gt;$plan&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nc"&gt;Plan&lt;/span&gt;&lt;span class="o"&gt;::&lt;/span&gt;&lt;span class="nf"&gt;compile&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;
    &lt;span class="s1"&gt;'fields'&lt;/span&gt; &lt;span class="o"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s1"&gt;'order'&lt;/span&gt; &lt;span class="o"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="s1"&gt;'int64'&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="s1"&gt;'where'&lt;/span&gt;  &lt;span class="o"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;
        &lt;span class="s1"&gt;'event'&lt;/span&gt;   &lt;span class="o"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="s1"&gt;'purchase'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="s1"&gt;'amount'&lt;/span&gt;  &lt;span class="o"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s1"&gt;'&amp;gt;='&lt;/span&gt; &lt;span class="o"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="mi"&gt;10&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;'&amp;lt;'&lt;/span&gt; &lt;span class="o"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="mf"&gt;1000.0&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="s1"&gt;'day'&lt;/span&gt;     &lt;span class="o"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s1"&gt;'&amp;gt;='&lt;/span&gt; &lt;span class="o"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="s1"&gt;'2026-11-01'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;'&amp;lt;'&lt;/span&gt; &lt;span class="o"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="s1"&gt;'2026-12-01'&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="s1"&gt;'country'&lt;/span&gt; &lt;span class="o"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s1"&gt;'path'&lt;/span&gt; &lt;span class="o"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s1"&gt;'address'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;'country'&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;'in'&lt;/span&gt; &lt;span class="o"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s1"&gt;'DE'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;'FR'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;'NL'&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;All entries must match (AND). Numbers compare by exact value. Strings compare as bytes, with no collation.&lt;/p&gt;

&lt;h2&gt;
  
  
  Nullable fields, nested paths and defaults
&lt;/h2&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight php"&gt;&lt;code&gt;&lt;span class="nv"&gt;$plan&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nc"&gt;Plan&lt;/span&gt;&lt;span class="o"&gt;::&lt;/span&gt;&lt;span class="nf"&gt;compile&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s1"&gt;'fields'&lt;/span&gt; &lt;span class="o"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;
    &lt;span class="s1"&gt;'id'&lt;/span&gt;       &lt;span class="o"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="s1"&gt;'int64'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="s1"&gt;'city'&lt;/span&gt;     &lt;span class="o"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s1"&gt;'type'&lt;/span&gt; &lt;span class="o"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="s1"&gt;'string?'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;'path'&lt;/span&gt; &lt;span class="o"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s1"&gt;'address'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;'city'&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;'default'&lt;/span&gt; &lt;span class="o"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="kc"&gt;null&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="s1"&gt;'tier'&lt;/span&gt;     &lt;span class="o"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s1"&gt;'type'&lt;/span&gt; &lt;span class="o"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="s1"&gt;'string'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;'default'&lt;/span&gt; &lt;span class="o"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="s1"&gt;'free'&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="s1"&gt;'discount'&lt;/span&gt; &lt;span class="o"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s1"&gt;'type'&lt;/span&gt; &lt;span class="o"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="s1"&gt;'float64'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;'default'&lt;/span&gt; &lt;span class="o"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;





&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;{"id":1,"city":"Lyon","tier":"pro","discount":2.5}
{"id":2,"city":null,"tier":"free","discount":0.0}
{"id":3,"city":null,"tier":"team","discount":0.0}
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;A missing key writes the default. A JSON &lt;code&gt;null&lt;/code&gt; is never replaced by a default. The output stays flat.&lt;/p&gt;

&lt;h2&gt;
  
  
  Compressed input works (any PHP stream does)
&lt;/h2&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight php"&gt;&lt;code&gt;&lt;span class="nv"&gt;$in&lt;/span&gt;  &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nf"&gt;fopen&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;"compress.zlib://$dir/logs.ndjson.gz"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;'rb'&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;span class="nv"&gt;$out&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nf"&gt;fopen&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s1"&gt;'php://stdout'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;'wb'&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;span class="nc"&gt;Runner&lt;/span&gt;&lt;span class="o"&gt;::&lt;/span&gt;&lt;span class="nf"&gt;process&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nv"&gt;$in&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nv"&gt;$out&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nv"&gt;$plan&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h2&gt;
  
  
  Errors you can handle
&lt;/h2&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight php"&gt;&lt;code&gt;&lt;span class="k"&gt;try&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="nc"&gt;Runner&lt;/span&gt;&lt;span class="o"&gt;::&lt;/span&gt;&lt;span class="nf"&gt;process&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nv"&gt;$in&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nv"&gt;$out&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nv"&gt;$plan&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt; &lt;span class="k"&gt;catch&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nc"&gt;RecordException&lt;/span&gt; &lt;span class="nv"&gt;$e&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="nf"&gt;printf&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;"record: %s, field %s, line %d, record %d, byte offset %d\n"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="nv"&gt;$e&lt;/span&gt;&lt;span class="o"&gt;-&amp;gt;&lt;/span&gt;&lt;span class="nf"&gt;getReason&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nv"&gt;$e&lt;/span&gt;&lt;span class="o"&gt;-&amp;gt;&lt;/span&gt;&lt;span class="nf"&gt;getField&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nv"&gt;$e&lt;/span&gt;&lt;span class="o"&gt;-&amp;gt;&lt;/span&gt;&lt;span class="nf"&gt;getInputLine&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nv"&gt;$e&lt;/span&gt;&lt;span class="o"&gt;-&amp;gt;&lt;/span&gt;&lt;span class="nf"&gt;getRecordNumber&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nv"&gt;$e&lt;/span&gt;&lt;span class="o"&gt;-&amp;gt;&lt;/span&gt;&lt;span class="nf"&gt;getByteOffset&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;





&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;record: type_mismatch, field id, line 3, record 3, byte offset 18
emitted before the error: 2
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Every exception has a stable &lt;code&gt;getReason()&lt;/code&gt; code. Messages never contain record bytes, so they are safe to log. Records written before the bad one stay in the output. JsonFuse does not roll back, so for all-or-nothing, write to a temp file and &lt;code&gt;rename()&lt;/code&gt; after success.&lt;/p&gt;

&lt;h2&gt;
  
  
  Benchmarks (with the conditions)
&lt;/h2&gt;

&lt;p&gt;Intel Xeon Platinum 8358, x86-64, PHP 8.3.36 NTS, opcache and JIT on, shared host, pre-release build. Full pipeline, byte-identical output checked. Median of 5 to 10 runs.&lt;/p&gt;

&lt;p&gt;1 million narrow records, 10% match (isolated re-run):&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Implementation&lt;/th&gt;
&lt;th&gt;Median&lt;/th&gt;
&lt;th&gt;JsonFuse is&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;JsonFuse&lt;/td&gt;
&lt;td&gt;245.2 ms&lt;/td&gt;
&lt;td&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;
&lt;code&gt;fastjson&lt;/code&gt; loop&lt;/td&gt;
&lt;td&gt;522.8 ms&lt;/td&gt;
&lt;td&gt;2.13x faster&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;PECL simdjson loop&lt;/td&gt;
&lt;td&gt;613.8 ms&lt;/td&gt;
&lt;td&gt;2.50x faster&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;
&lt;code&gt;json_decode&lt;/code&gt; loop&lt;/td&gt;
&lt;td&gt;973.5 ms&lt;/td&gt;
&lt;td&gt;3.97x faster&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;ul&gt;
&lt;li&gt;vs &lt;code&gt;json_decode&lt;/code&gt;: 3.55x to 5.06x at 100k records and above.&lt;/li&gt;
&lt;li&gt;vs &lt;code&gt;fastjson&lt;/code&gt; (fastest baseline everywhere): 2.08x to 2.63x on narrow records at 100k+. &lt;strong&gt;Wide records (100 fields): only 1.53x to 1.71x.&lt;/strong&gt; Short inputs (10k): 1.49x to 2.15x.&lt;/li&gt;
&lt;li&gt;vs JSON Machine, 100k records: 21.5x to 23.8x.&lt;/li&gt;
&lt;li&gt;Memory: every implementation sits near 25 to 28 MiB. No memory win is claimed.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Not measured: cold cache, parallel workers, opcache off, non-x86-64. Method and raw data: &lt;a href="https://github.com/hamdallah90/php-jsonfuse/blob/main/benchmarks/RESULTS.md" rel="noopener noreferrer"&gt;&lt;code&gt;benchmarks/RESULTS.md&lt;/code&gt;&lt;/a&gt;.&lt;/p&gt;

&lt;h2&gt;
  
  
  What it is NOT
&lt;/h2&gt;

&lt;p&gt;Not a general JSON library: no &lt;code&gt;decode&lt;/code&gt;, no &lt;code&gt;encode&lt;/code&gt;, no iterator, no JSONPath. NDJSON in, flat NDJSON out. For general use, take &lt;code&gt;json_decode&lt;/code&gt; or &lt;a href="https://github.com/iliaal/fastjson" rel="noopener noreferrer"&gt;&lt;code&gt;iliaal/fastjson&lt;/code&gt;&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;What it gives you is a &lt;strong&gt;hard memory bound&lt;/strong&gt;: memory depends on &lt;code&gt;maxRecordBytes&lt;/code&gt; (default 1 MiB), not on file size. In my tests, 2 GiB of 1 MiB records peaked at 27.2 to 27.4 MiB RSS, and one 64 MiB line was refused at the cap after reading 0 records.&lt;/p&gt;

&lt;h2&gt;
  
  
  Quality checks
&lt;/h2&gt;

&lt;p&gt;PHP 8.3 to 8.6, NTS and ZTS. Linux x86-64 and aarch64, macOS arm64. CI on every platform. ASan and UBSan, a libFuzzer harness, and ThreadSanitizer on a multi-thread harness. 🧪&lt;/p&gt;

&lt;h2&gt;
  
  
  Install
&lt;/h2&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;pie install hamdallah90/jsonfuse
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Or from source: &lt;code&gt;phpize &amp;amp;&amp;amp; ./configure --enable-jsonfuse &amp;amp;&amp;amp; make &amp;amp;&amp;amp; sudo make install&lt;/code&gt;, then &lt;code&gt;extension=jsonfuse&lt;/code&gt; in &lt;code&gt;php.ini&lt;/code&gt;.&lt;/p&gt;

&lt;p&gt;Repo: &lt;a href="https://github.com/hamdallah90/php-jsonfuse" rel="noopener noreferrer"&gt;https://github.com/hamdallah90/php-jsonfuse&lt;/a&gt; (stars and issues welcome) ⭐&lt;/p&gt;

&lt;p&gt;Release notes: &lt;a href="https://github.com/hamdallah90/php-jsonfuse/releases/tag/v1.0.0" rel="noopener noreferrer"&gt;https://github.com/hamdallah90/php-jsonfuse/releases/tag/v1.0.0&lt;/a&gt;&lt;/p&gt;

</description>
      <category>php</category>
      <category>performance</category>
      <category>opensource</category>
      <category>json</category>
    </item>
    <item>
      <title>I Chased uWebSockets—and Found a 14-Byte Performance Cliff</title>
      <dc:creator>hamdallah jodah</dc:creator>
      <pubDate>Fri, 31 Jul 2026 22:54:09 +0000</pubDate>
      <link>https://dev.to/hamdallah_jodah/i-chased-uwebsockets-and-found-a-14-byte-performance-cliff-54ap</link>
      <guid>https://dev.to/hamdallah_jodah/i-chased-uwebsockets-and-found-a-14-byte-performance-cliff-54ap</guid>
      <description>&lt;p&gt;&lt;em&gt;Building ramjet-ws-js: a Rust-backed WebSocket path for Node.js that wins where it matters, admits where it loses, and survived 584 million verified round-trips.&lt;/em&gt;&lt;/p&gt;

&lt;p&gt;The benchmark result I learned the most from was not a win.&lt;/p&gt;

&lt;p&gt;At exactly 64 KiB, my WebSocket server was 8.42% slower than uWebSockets.js. It did not crash. It did not corrupt the message. It simply crossed an invisible boundary and became slower.&lt;/p&gt;

&lt;p&gt;The reason was fourteen bytes.&lt;/p&gt;

&lt;p&gt;That small discovery ended up explaining the entire project better than any headline performance number.&lt;/p&gt;

&lt;h2&gt;
  
  
  What I was building
&lt;/h2&gt;

&lt;p&gt;&lt;code&gt;ramjet-ws-js&lt;/code&gt; is a JavaScript WebSocket API backed by a Rust networking core.&lt;/p&gt;

&lt;p&gt;The normal JavaScript message path looks roughly like this:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;Socket → Rust decoder → V8/JavaScript callback → Rust → Socket
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;That design is flexible. A handler can run any JavaScript logic it wants.&lt;/p&gt;

&lt;p&gt;But flexibility has a cost. Every message crosses the Rust-to-JavaScript boundary, creates or transfers a JavaScript buffer, invokes a callback, then sends the response back into Rust.&lt;/p&gt;

&lt;p&gt;For handlers whose behavior is already known, that journey is unnecessary.&lt;/p&gt;

&lt;p&gt;So I added an opt-in native execution path:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight javascript"&gt;&lt;code&gt;&lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="nx"&gt;App&lt;/span&gt; &lt;span class="p"&gt;}&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nf"&gt;require&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;ramjet-ws&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;

&lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;app&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nc"&gt;App&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;

&lt;span class="nx"&gt;app&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;ws&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;/*&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
  &lt;span class="na"&gt;nativeEcho&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kc"&gt;true&lt;/span&gt;
&lt;span class="p"&gt;})&lt;/span&gt;

&lt;span class="nx"&gt;app&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;listen&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mi"&gt;9001&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;With &lt;code&gt;nativeEcho&lt;/code&gt;, the data path becomes:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;Socket → Rust decode and unmask → Rust echo → Socket
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Open and close events can still reach JavaScript, but message frames stay inside the native reactor.&lt;/p&gt;

&lt;p&gt;This is not a trick that makes arbitrary JavaScript 46% faster. It is a different execution model: declare simple behavior in advance, then let the native runtime execute it without visiting V8 for every message.&lt;/p&gt;

&lt;p&gt;In one controlled comparison, the optimized JavaScript callback path reached 400,858 requests per second. The declared native path reached 587,912.&lt;/p&gt;

&lt;p&gt;That was a &lt;strong&gt;46.7% improvement inside the same application&lt;/strong&gt;.&lt;/p&gt;

&lt;h2&gt;
  
  
  More than simply removing a callback
&lt;/h2&gt;

&lt;p&gt;Avoiding JavaScript was only the beginning.&lt;/p&gt;

&lt;p&gt;For eligible frames, Ramjet reuses the pooled receive buffer for the response. It unmasks the client payload in place, inserts the server response header, and submits the same buffer for writing.&lt;/p&gt;

&lt;p&gt;When multiple frames arrive together, the reactor processes them as one fused batch. Unmasking and compaction happen in one pass, and the replies are corked into one write.&lt;/p&gt;

&lt;p&gt;Tiny messages behave differently. For a single small read, the compact streaming encoder was faster than the fused machinery, so the path switches adaptively.&lt;/p&gt;

&lt;p&gt;The generic JavaScript path also received several improvements:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Ordered event batching instead of one native callback per event&lt;/li&gt;
&lt;li&gt;Coalesced wakeups between JavaScript and the reactor&lt;/li&gt;
&lt;li&gt;Per-connection atomics instead of a global lock and lookup&lt;/li&gt;
&lt;li&gt;Size-aware buffer ownership&lt;/li&gt;
&lt;li&gt;Exact outbound accounting and bounded backpressure&lt;/li&gt;
&lt;li&gt;Explicit handling for retained, mutated, and repeatedly sent Node.js buffers&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;The result is not one clever optimization. It is a collection of small decisions that keep unnecessary work out of the hot path.&lt;/p&gt;

&lt;h2&gt;
  
  
  Benchmarking without lying to myself
&lt;/h2&gt;

&lt;p&gt;Performance testing becomes marketing very easily.&lt;/p&gt;

&lt;p&gt;To avoid that, I tested Ramjet against uWebSockets.js, which is backed by uSockets and is one of the strongest WebSocket baselines available.&lt;/p&gt;

&lt;p&gt;Both servers ran on the same AWS C7i.large:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Two logical CPUs on one physical Xeon Platinum 8488C core&lt;/li&gt;
&lt;li&gt;Linux 7.0&lt;/li&gt;
&lt;li&gt;Node.js 22.16.0&lt;/li&gt;
&lt;li&gt;Server pinned to CPU 0&lt;/li&gt;
&lt;li&gt;Client pinned to CPU 1&lt;/li&gt;
&lt;li&gt;Compression disabled&lt;/li&gt;
&lt;li&gt;Same 32 MiB maximum-message configuration&lt;/li&gt;
&lt;li&gt;One server running at a time&lt;/li&gt;
&lt;li&gt;Warm-up before every measured run&lt;/li&gt;
&lt;li&gt;Mirrored execution order&lt;/li&gt;
&lt;li&gt;Three measured runs, reported as medians&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;The benchmark client did not merely count responses. It verified the opcode, payload size, and exact echoed bytes. A fast incorrect response counted as a failure, not a result.&lt;/p&gt;

&lt;h2&gt;
  
  
  Where Ramjet was strongest
&lt;/h2&gt;

&lt;p&gt;The most convincing result came from small pipelined traffic.&lt;/p&gt;

&lt;p&gt;With 200 connections, pipeline depth eight, and frames written in bursts:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Ramjet: &lt;strong&gt;599,829 requests/second&lt;/strong&gt;
&lt;/li&gt;
&lt;li&gt;uWebSockets.js: &lt;strong&gt;518,658 requests/second&lt;/strong&gt;
&lt;/li&gt;
&lt;li&gt;Ramjet throughput advantage: &lt;strong&gt;15.65%&lt;/strong&gt;
&lt;/li&gt;
&lt;li&gt;Ramjet p50 latency: &lt;strong&gt;14.38% lower&lt;/strong&gt;
&lt;/li&gt;
&lt;li&gt;Ramjet p99 latency: &lt;strong&gt;13.28% lower&lt;/strong&gt;
&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;With 50 connections and one 64-byte message in flight, the result was much closer:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Ramjet: &lt;strong&gt;169,468 requests/second&lt;/strong&gt;
&lt;/li&gt;
&lt;li&gt;uWebSockets.js: &lt;strong&gt;166,462 requests/second&lt;/strong&gt;
&lt;/li&gt;
&lt;li&gt;Ramjet advantage: &lt;strong&gt;1.81%&lt;/strong&gt;
&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;That distinction matters. The design is especially strong when frames arrive in batches and the reactor can fuse work, reduce submissions, and reuse buffers.&lt;/p&gt;

&lt;p&gt;The payload sweep produced a more interesting picture:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Payload&lt;/th&gt;
&lt;th&gt;Ramjet&lt;/th&gt;
&lt;th&gt;uWebSockets.js&lt;/th&gt;
&lt;th&gt;Difference&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;1 KiB&lt;/td&gt;
&lt;td&gt;131,248 msg/s&lt;/td&gt;
&lt;td&gt;129,985 msg/s&lt;/td&gt;
&lt;td&gt;Ramjet +0.97%&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;4 KiB&lt;/td&gt;
&lt;td&gt;91,773 msg/s&lt;/td&gt;
&lt;td&gt;91,183 msg/s&lt;/td&gt;
&lt;td&gt;Ramjet +0.65%&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;8 KiB&lt;/td&gt;
&lt;td&gt;72,597 msg/s&lt;/td&gt;
&lt;td&gt;69,358 msg/s&lt;/td&gt;
&lt;td&gt;Ramjet +4.67%&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;64 KiB&lt;/td&gt;
&lt;td&gt;16,755 msg/s&lt;/td&gt;
&lt;td&gt;18,294 msg/s&lt;/td&gt;
&lt;td&gt;Ramjet −8.42%&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;128 KiB&lt;/td&gt;
&lt;td&gt;9,057 msg/s&lt;/td&gt;
&lt;td&gt;8,782 msg/s&lt;/td&gt;
&lt;td&gt;Ramjet +3.14%&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;256 KiB&lt;/td&gt;
&lt;td&gt;4,463 msg/s&lt;/td&gt;
&lt;td&gt;3,765 msg/s&lt;/td&gt;
&lt;td&gt;Ramjet +18.55%&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;512 KiB&lt;/td&gt;
&lt;td&gt;1,784 msg/s&lt;/td&gt;
&lt;td&gt;1,729 msg/s&lt;/td&gt;
&lt;td&gt;Ramjet +3.23%&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;1 MiB&lt;/td&gt;
&lt;td&gt;811 msg/s&lt;/td&gt;
&lt;td&gt;865 msg/s&lt;/td&gt;
&lt;td&gt;Ramjet −6.19%&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;2 MiB&lt;/td&gt;
&lt;td&gt;387 msg/s&lt;/td&gt;
&lt;td&gt;420 msg/s&lt;/td&gt;
&lt;td&gt;Ramjet −7.83%&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;Ramjet is not universally faster. No honest benchmark says that.&lt;/p&gt;

&lt;p&gt;It is particularly strong for small pipelined traffic and payloads through roughly 512 KiB. uWebSockets.js currently leads at one and two MiB, where Ramjet's full-message reassembly and copying become more expensive.&lt;/p&gt;

&lt;h2&gt;
  
  
  The fourteen-byte problem
&lt;/h2&gt;

&lt;p&gt;A 64 KiB payload contains exactly 65,536 bytes.&lt;/p&gt;

&lt;p&gt;But a masked WebSocket frame carrying that payload also needs a 14-byte header. The actual client frame is therefore 65,550 bytes.&lt;/p&gt;

&lt;p&gt;Ramjet's pooled receive buffer is 65,536 bytes.&lt;/p&gt;

&lt;p&gt;The frame misses the whole-buffer fast path by fourteen bytes.&lt;/p&gt;

&lt;p&gt;The connection still works. Ramjet safely falls back to streaming and reassembly. But it loses the in-place buffer reuse that makes nearby payload sizes fast.&lt;/p&gt;

&lt;p&gt;I tested a bounded "tail stitch" workaround that joined the missing bytes onto the pooled buffer. It sounded clever.&lt;/p&gt;

&lt;p&gt;It was slower.&lt;/p&gt;

&lt;p&gt;So I removed it.&lt;/p&gt;

&lt;p&gt;That is an underrated part of performance engineering: a clever optimization that loses in a controlled benchmark is not clever enough to keep.&lt;/p&gt;

&lt;p&gt;Interestingly, the performance did not continue falling after the boundary. Ramjet moved ahead again at 128 KiB and produced its largest payload advantage at 256 KiB.&lt;/p&gt;

&lt;p&gt;The 64 KiB result was a boundary effect, not a general inability to process large messages.&lt;/p&gt;

&lt;h2&gt;
  
  
  Speed without stability is only a demo
&lt;/h2&gt;

&lt;p&gt;After the short benchmarks, I left the same server process under mixed saturation for thirty minutes.&lt;/p&gt;

&lt;p&gt;Every minute, 95 connections were created across four concurrent workloads:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;50 connections sending 64-byte frames with pipeline depth eight&lt;/li&gt;
&lt;li&gt;30 connections sending 1 KiB messages&lt;/li&gt;
&lt;li&gt;10 connections sending 128 KiB messages&lt;/li&gt;
&lt;li&gt;Five connections sending 256 KiB messages&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;The clients ran for 58 seconds, disconnected, and reconnected for the next cycle.&lt;/p&gt;

&lt;p&gt;The final result:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;&lt;strong&gt;584,779,917 exact round-trips&lt;/strong&gt;&lt;/li&gt;
&lt;li&gt;Approximately &lt;strong&gt;833 GiB of application payload&lt;/strong&gt; across both directions&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;2,850 connection sessions&lt;/strong&gt;&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;120 out of 120 workload runs successful&lt;/strong&gt;&lt;/li&gt;
&lt;li&gt;Zero payload mismatches&lt;/li&gt;
&lt;li&gt;Zero client failures&lt;/li&gt;
&lt;li&gt;Zero server crashes&lt;/li&gt;
&lt;li&gt;Zero unexpected timeouts&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;The original server process remained alive for the entire test.&lt;/p&gt;

&lt;p&gt;Memory told the same story:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Initial RSS: 44.9 MiB&lt;/li&gt;
&lt;li&gt;Final RSS: 55.8 MiB&lt;/li&gt;
&lt;li&gt;Maximum observed RSS: 56.7 MiB&lt;/li&gt;
&lt;li&gt;Process high-water mark: 57.9 MiB&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;The memory high-water mark was established during the initial allocator warm-up and did not continue climbing.&lt;/p&gt;

&lt;p&gt;File descriptors moved from 24 at rest to 119 under load and returned to 24 after every reconnect cycle.&lt;/p&gt;

&lt;p&gt;That repeatable &lt;code&gt;24 → 119 → 24&lt;/code&gt; pattern was more meaningful to me than a single high requests-per-second number. It showed that connections and their resources were actually being retired.&lt;/p&gt;

&lt;p&gt;Median performance during the mixed soak was:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Workload&lt;/th&gt;
&lt;th&gt;Throughput&lt;/th&gt;
&lt;th&gt;p50&lt;/th&gt;
&lt;th&gt;p99&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;64-byte burst&lt;/td&gt;
&lt;td&gt;297,474 msg/s&lt;/td&gt;
&lt;td&gt;1.26 ms&lt;/td&gt;
&lt;td&gt;2.83 ms&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;1 KiB&lt;/td&gt;
&lt;td&gt;36,081 msg/s&lt;/td&gt;
&lt;td&gt;0.78 ms&lt;/td&gt;
&lt;td&gt;2.09 ms&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;128 KiB&lt;/td&gt;
&lt;td&gt;1,011 msg/s&lt;/td&gt;
&lt;td&gt;9.45 ms&lt;/td&gt;
&lt;td&gt;14.67 ms&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;256 KiB&lt;/td&gt;
&lt;td&gt;258 msg/s&lt;/td&gt;
&lt;td&gt;19.10 ms&lt;/td&gt;
&lt;td&gt;26.30 ms&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;These numbers are lower than the isolated benchmarks because four clients were sharing the same client CPU simultaneously. The point of the soak was stability under contention, not producing another leaderboard result.&lt;/p&gt;

&lt;h2&gt;
  
  
  The limits are intentional—and visible
&lt;/h2&gt;

&lt;p&gt;The decoder currently accepts messages up to 32 MiB, but native echo has a four MiB per-connection outbound safety budget.&lt;/p&gt;

&lt;p&gt;Because a large server-to-client WebSocket frame needs a 10-byte header, the largest native echo payload is:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;4,194,304 − 10 = 4,194,294 bytes
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;I tested the boundary directly:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;A 4,194,294-byte payload succeeded.&lt;/li&gt;
&lt;li&gt;A 4,194,295-byte payload was rejected with WebSocket close code &lt;code&gt;1008&lt;/code&gt;.&lt;/li&gt;
&lt;li&gt;An exact four MiB payload was also rejected.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;This is not a parser bug. It is backpressure doing what it was designed to do.&lt;/p&gt;

&lt;p&gt;Simply raising the limit would be easy, but it would allow slow clients to pin much more memory. A better future design is to allow one oversized in-flight message when the normal queue is empty, then stream it through multiple pooled buffers using vectored writes.&lt;/p&gt;

&lt;p&gt;That would preserve bounded backpressure while supporting larger messages efficiently.&lt;/p&gt;

&lt;h2&gt;
  
  
  Verification beyond benchmarks
&lt;/h2&gt;

&lt;p&gt;The final artifact passed:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;cargo fmt --check
cargo clippy --all-targets -- -D warnings
cargo test
npm run build
npm test
git diff --check
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The integration suite covers native text and binary echo, pipelined ordering, 64 KiB and 128 KiB messages, close handling, backpressure, retained buffers, mutated buffers, repeated sends, handle cleanup, and connection leaks.&lt;/p&gt;

&lt;p&gt;The exact Linux artifact used for the final tests had this SHA-256:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;8b2d01e361349d1a7401d91c39955037d31d059219c442a3fd6c7566ff114752
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;This still does not prove every production workload. The benchmark ran over loopback, which isolates runtime and CPU performance rather than real network conditions. Cross-machine latency, packet loss, TLS, proxies, and unpredictable application behavior deserve their own tests.&lt;/p&gt;

&lt;p&gt;But it proves something useful: the optimized path is fast, exact, bounded, and stable under sustained pressure.&lt;/p&gt;

&lt;h2&gt;
  
  
  Where I stop—and where I go next
&lt;/h2&gt;

&lt;p&gt;I am stopping the echo optimization here.&lt;/p&gt;

&lt;p&gt;Not because it is perfect, but because the evidence says the next unit of work belongs somewhere else.&lt;/p&gt;

&lt;p&gt;The more valuable next step is an opt-in native pub/sub execution path and a thread-per-core worker API. Those features bring the same "declare behavior, skip unnecessary runtime boundaries" idea into real applications.&lt;/p&gt;

&lt;p&gt;If production workloads regularly carry multi-megabyte messages, the vectored streaming path moves higher on the list. Otherwise, rewriting the large-message architecture to win another synthetic benchmark would add complexity without earning its place.&lt;/p&gt;

&lt;p&gt;The best performance result in this project was not 599,829 requests per second.&lt;/p&gt;

&lt;p&gt;It was knowing which optimization to remove, which limit to keep, and when to stop.&lt;/p&gt;

&lt;p&gt;Links:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;&lt;a href="https://github.com/sofelia-ai/ramjet-ws-js" rel="noopener noreferrer"&gt;https://github.com/sofelia-ai/ramjet-ws-js&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="https://github.com/sofelia-ai/ramjet" rel="noopener noreferrer"&gt;https://github.com/sofelia-ai/ramjet&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="https://crates.io/crates/ramjet" rel="noopener noreferrer"&gt;https://crates.io/crates/ramjet&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="https://crates.io/crates/ramjet-ws" rel="noopener noreferrer"&gt;https://crates.io/crates/ramjet-ws&lt;/a&gt;&lt;/li&gt;
&lt;/ol&gt;

</description>
      <category>rust</category>
      <category>node</category>
      <category>websockets</category>
      <category>performance</category>
    </item>
  </channel>
</rss>
