<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:dc="http://purl.org/dc/elements/1.1/">
  <channel>
    <title>DEV Community: lowspecai</title>
    <description>The latest articles on DEV Community by lowspecai (@lowspecai).</description>
    <link>https://dev.to/lowspecai</link>
    <image>
      <url>https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F4201373%2F38218c82-88e3-4889-ba67-3c63e61ee532.png</url>
      <title>DEV Community: lowspecai</title>
      <link>https://dev.to/lowspecai</link>
    </image>
    <atom:link rel="self" type="application/rss+xml" href="https://dev.to/feed/lowspecai"/>
    <language>en</language>
    <item>
      <title>The cheapest SSD that won't die in a NAS</title>
      <dc:creator>lowspecai</dc:creator>
      <pubDate>Fri, 23 Oct 2026 16:44:40 +0000</pubDate>
      <link>https://dev.to/lowspecai/the-cheapest-ssd-that-wont-die-in-a-nas-1phg</link>
      <guid>https://dev.to/lowspecai/the-cheapest-ssd-that-wont-die-in-a-nas-1phg</guid>
      <description>&lt;p&gt;Most home NAS builds do not die of a bad drive. They die of a cheap drive in the wrong slot, on a machine that loses power without warning.&lt;/p&gt;

&lt;p&gt;I run two machines from 2011 as NAS boxes. One is a laptop board: Celeron B815, two cores, 4 GB of DDR3, a 100 Mbps port, a 256 GB SATA SSD and a 320 GB 5400 rpm drive. The other is a mini PC with an i5-2520M, 8 GB, gigabit, a 128 GB SSD and a 3 TB WD Red Plus. Neither is fast. Both have been through enough failures that the drive advice I would give now is the opposite of what the spec sheets sell.&lt;/p&gt;

&lt;p&gt;This is the short version of &lt;a href="https://lowspechq.com/best/cheap-ssd-for-nas/" rel="noopener noreferrer"&gt;a longer guide I keep on my own site&lt;/a&gt;, with the numbers measured on those two boxes.&lt;/p&gt;

&lt;h2&gt;
  
  
  The endurance maths nobody needs
&lt;/h2&gt;

&lt;p&gt;A 500 GB TLC drive rated at 300 TBW. A busy home NAS writes maybe 5 TB a year — Immich thumbnails, container logs, a backup target, some downloads. 300 ÷ 5 = 60 years. Even with 3× write amplification from a copy-on-write filesystem you are still past 20.&lt;/p&gt;

&lt;p&gt;TBW is a warranty term, not a lifespan prediction. The 256 GB drive in the small box reports &lt;strong&gt;8 power-on hours, 0 reallocated sectors, wear-levelling count 0&lt;/strong&gt;. Nothing about its flash is anywhere near tired. What it lacks is any protection against being unplugged mid-write.&lt;/p&gt;

&lt;p&gt;So stop shopping on endurance. It is not the constraint.&lt;/p&gt;

&lt;h2&gt;
  
  
  What actually kills them
&lt;/h2&gt;

&lt;p&gt;&lt;strong&gt;Power loss on a drive with no power-loss protection.&lt;/strong&gt; Capacitors that flush the write cache on power loss exist only on enterprise drives. Consumer drives — including the expensive ones — do not have them. A DRAM-less drive keeps its mapping table in an on-chip buffer, and losing that mid-write is how you get a filesystem that needs a rebuild instead of a reboot. On a single-drive box that is an afternoon. On an array it can take the whole thing.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;The SLC cache running out.&lt;/strong&gt; Cheap QLC drives benchmark beautifully for the first few gigabytes, then collapse to tens of megabytes per second once the pseudo-SLC cache is exhausted. That is slower than the hard drive you were replacing. Copying 40 GB of photos onto a fresh QLC drive is exactly the workload that finds this.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Heat.&lt;/strong&gt; A 2.5" SATA drive with any airflow at all is fine — mine sits at 45 °C. An M.2 stick bolted behind a drive cage with no airflow will throttle silently and then start dropping off the bus.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;The bridge, not the drive.&lt;/strong&gt; Cheap USB enclosures and add-in SATA controllers fail more often than the disks inside them. The 3 TB drive in the larger box accumulates interface error counters while completely idle, at roughly 0.1 per second, and the count does not grow with I/O volume. That is firmware or link behaviour, not a dying platter. When a drive misbehaves, suspect the cable, the enclosure and the controller before you blame the NAND.&lt;/p&gt;

&lt;h2&gt;
  
  
  What I measured
&lt;/h2&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Measurement&lt;/th&gt;
&lt;th&gt;Number&lt;/th&gt;
&lt;th&gt;What it tells you&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Sequential read, 256 GB SATA SSD (&lt;code&gt;hdparm -t&lt;/code&gt;)&lt;/td&gt;
&lt;td&gt;506 MB/s&lt;/td&gt;
&lt;td&gt;the drive is nowhere near the bottleneck&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Sequential write, 3 TB WD Red Plus (&lt;code&gt;dd&lt;/code&gt;, &lt;code&gt;conv=fsync&lt;/code&gt;)&lt;/td&gt;
&lt;td&gt;107 MB/s&lt;/td&gt;
&lt;td&gt;a good 5400 rpm-class platter&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Buffered read, 320 GB 5400 rpm notebook drive&lt;/td&gt;
&lt;td&gt;88 MB/s&lt;/td&gt;
&lt;td&gt;what the SSD replaced&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;SMB transfer over the onboard 100 Mbps port&lt;/td&gt;
&lt;td&gt;11.4 MB/s up, 11.3 MB/s down&lt;/td&gt;
&lt;td&gt;the real ceiling of that machine&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Web UI first load, before and after the SSD swap&lt;/td&gt;
&lt;td&gt;24 s → 0.25 s&lt;/td&gt;
&lt;td&gt;a 100× win with nothing to do with bandwidth&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;Read that table twice. The SSD moves data about 45 times faster than the network port feeding it. A drive with 10% more sequential bandwidth would have changed nothing anyone could perceive.&lt;/p&gt;

&lt;p&gt;The 100× improvement came from queue depth, not throughput. The 5400 rpm drive choked on a few dozen concurrent small reads — the JavaScript, the CSS, the fonts — and each one waited behind the last. An SSD serves them in parallel. That is the whole mechanism, and it is why the cheapest drive with a DRAM cache already delivers it.&lt;/p&gt;

&lt;p&gt;Practical consequence: once you are on an SSD, paying more for a faster SSD buys nothing measurable on a home NAS. Pay for the two things that do not show up in a benchmark — a DRAM cache and a printed TBW — and stop there.&lt;/p&gt;

&lt;h2&gt;
  
  
  Five lines on the spec sheet
&lt;/h2&gt;

&lt;ol&gt;
&lt;li&gt;
&lt;strong&gt;DRAM cache.&lt;/strong&gt; The datasheet says "DRAM" or names a cache size. If it says nothing, assume none. DRAM-less drives lean harder on the host and degrade faster as they fill.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;TLC, not QLC.&lt;/strong&gt; Capacity per dollar is the QLC pitch and it is the wrong trade for a machine that receives bulk writes. Shortcut: at the same capacity, a QLC drive's TBW is usually about half of a TLC drive's.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;A TBW number that is actually printed.&lt;/strong&gt; A manufacturer that omits it is telling you something.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Power-loss protection — or an admission that there is none.&lt;/strong&gt; Consumer drives do not have it. This line decides whether you need a UPS, not whether you need a pricier drive.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;SMART attributes that are exposed.&lt;/strong&gt; A decent drive reports reallocated sectors, pending sectors, uncorrectable errors, UDMA CRC errors and total bytes written. Cheap USB bridges hide all of them, so you cannot see a drive dying until it is dead.&lt;/li&gt;
&lt;/ol&gt;

&lt;h2&gt;
  
  
  The configuration I actually run
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;System disk:&lt;/strong&gt; a small TLC SATA SSD with a DRAM cache, 240–512 GB. OS, container images, databases. Nothing else.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Data disk:&lt;/strong&gt; a mechanical drive, CMR not SMR, btrfs or ext4. Photos, media, documents, backups.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;The SSD never holds cold data.&lt;/strong&gt; A drive that is only read from does not need to be fast, and a platter gives far more capacity per unit of money and per unit of risk.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Check for SMR before buying a big 2.5" drive.&lt;/strong&gt; Many high-capacity portable drives are shingled and fall out of an array during a rebuild — exactly when you cannot afford it.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Unglamorous, and it is what both boxes do. The SSD absorbs everything random and small; the platter absorbs everything sequential and large.&lt;/p&gt;

&lt;h2&gt;
  
  
  Three commands before you trust a drive
&lt;/h2&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;&lt;span class="c"&gt;# 1. sequential read, bypassing the page cache&lt;/span&gt;
&lt;span class="nb"&gt;sudo &lt;/span&gt;hdparm &lt;span class="nt"&gt;-t&lt;/span&gt; &lt;span class="nt"&gt;--direct&lt;/span&gt; /dev/sdX

&lt;span class="c"&gt;# 2. the SMART attributes that matter&lt;/span&gt;
&lt;span class="nb"&gt;sudo &lt;/span&gt;smartctl &lt;span class="nt"&gt;-A&lt;/span&gt; &lt;span class="nt"&gt;-d&lt;/span&gt; sat /dev/sdX

&lt;span class="c"&gt;# 3. 4K random read - the closest thing to a real NAS workload&lt;/span&gt;
&lt;span class="nb"&gt;sudo &lt;/span&gt;fio &lt;span class="nt"&gt;--name&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;r4k &lt;span class="nt"&gt;--rw&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;randread &lt;span class="nt"&gt;--bs&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;4k &lt;span class="nt"&gt;--iodepth&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;32 &lt;span class="nt"&gt;--size&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;1G &lt;span class="nt"&gt;--runtime&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;30 &lt;span class="nt"&gt;--filename&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;/dev/sdX
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;I have run the first two on my own drives. I have not run the third on the drive that is in production, so I am not going to invent a number for it — the command is the honest part of that section.&lt;/p&gt;

&lt;p&gt;What matters in the SMART table:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;5 Reallocated_Sector_Ct&lt;/strong&gt; — anything above zero, stop putting new data on it.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;187 Reported_Uncorrect&lt;/strong&gt; and &lt;strong&gt;197 Current_Pending_Sector&lt;/strong&gt; — same rule.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;198 Offline_Uncorrectable&lt;/strong&gt; — same rule.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;199 UDMA_CRC_Error_Count&lt;/strong&gt; — climbing means the cable, backplane or bridge, not the NAND. Replace the cheap part first.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;241 Total_LBAs_Written&lt;/strong&gt; — a sanity check on how much life has actually been used.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  The cheapest reliability upgrade is not an SSD
&lt;/h2&gt;

&lt;p&gt;The larger box died three times in ten days. The drive was not the cause. The kernel is configured not to reboot on a panic, and the hardware watchdog is disabled, so when the machine hangs it hangs forever instead of recovering.&lt;/p&gt;

&lt;p&gt;The third time, the smart plug it is plugged into reported 40 W. Idle is 20 W. Power was still flowing — the machine had not lost power, it had stopped responding, and I cut the plug to restart it from another room.&lt;/p&gt;

&lt;p&gt;Every one of those events was an unclean shutdown in the middle of writes to an SSD that has no power-loss protection and never will. So the arithmetic is simple: a cheap drive plus a UPS is more reliable than an expensive drive without one. If your budget covers only one, buy the UPS.&lt;/p&gt;

&lt;p&gt;If you cannot buy a UPS, at least do this: enable "Restore on AC Power Loss" in the BIOS, and put the box on a plug you can switch remotely. A laptop used as a NAS has this built in — mine has a 43 Wh battery that turns a power cut into a non-event.&lt;/p&gt;

&lt;h2&gt;
  
  
  What I would not buy
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;No-name drives with no TBW and no SMART.&lt;/strong&gt; You cannot monitor what you cannot see.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;QLC marketed on cache speed.&lt;/strong&gt; The benchmark is real; the workload that follows it is not the one they tested.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Used enterprise drives with 50,000 power-on hours.&lt;/strong&gt; The remaining life is unknowable and the seller knows it.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;A USB enclosure as permanent storage.&lt;/strong&gt; Controllers drop off the bus, and a btrfs array that loses a device mid-write is a bad afternoon.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;An M.2 drive with no airflow&lt;/strong&gt; in a case designed for platters.&lt;/li&gt;
&lt;/ul&gt;




&lt;p&gt;&lt;em&gt;Numbers above were measured on the two machines described at the top; anything from a vendor spec sheet is labelled as such. Original, with the full method and the failing-drive history: &lt;a href="https://lowspechq.com/best/cheap-ssd-for-nas/" rel="noopener noreferrer"&gt;lowspechq.com&lt;/a&gt;. That page carries one clearly labelled affiliate link to an SSD listing; nothing here is sponsored, and no seller saw this text before you did.&lt;/em&gt;&lt;/p&gt;

</description>
      <category>nas</category>
      <category>hardware</category>
      <category>storage</category>
      <category>selfhosted</category>
    </item>
    <item>
      <title>Tokens per second is a memory bandwidth number. Measure it in 2 seconds.</title>
      <dc:creator>lowspecai</dc:creator>
      <pubDate>Fri, 23 Oct 2026 03:49:06 +0000</pubDate>
      <link>https://dev.to/lowspecai/tokens-per-second-is-a-memory-bandwidth-number-measure-it-in-2-seconds-1omf</link>
      <guid>https://dev.to/lowspecai/tokens-per-second-is-a-memory-bandwidth-number-measure-it-in-2-seconds-1omf</guid>
      <description>&lt;p&gt;A 40B model that does 125 tok/s on a laptop, and a 7B model that does 1.5 tok/s on a 2011 CPU with the same amount of RAM. Same ballpark of memory, ~80× difference in speed. Almost every explanation you will read for that gap is wrong, and the wrong explanation costs people money.&lt;/p&gt;

&lt;p&gt;Here is the number that actually decides it: &lt;strong&gt;how many bytes your machine can read per second.&lt;/strong&gt; Not cores. Not GHz. Not TOPS.&lt;/p&gt;

&lt;h2&gt;
  
  
  The formula
&lt;/h2&gt;

&lt;p&gt;Generating one token with a dense model requires reading the entire set of weights once. Every token. There is no caching trick around it — the weights do not fit in cache, so each token is one full sweep of DRAM.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;tokens/second  ≈  memory bandwidth  ÷  bytes of weights read per token
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;That is the whole ceiling. Everything else — kernel quality, quantization, &lt;code&gt;--threads&lt;/code&gt; — decides how close you get to it, never how far above it you can go.&lt;/p&gt;

&lt;h2&gt;
  
  
  I measured the bandwidth on my own box
&lt;/h2&gt;

&lt;p&gt;The machine is an Intel i5-2520M from 2011 running as a NAS. 8 GB of DDR3-1600, &lt;strong&gt;one&lt;/strong&gt; DIMM, &lt;code&gt;nproc&lt;/code&gt; 4. Not a benchmark rig; it is the thing serving files in the other room.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;numpy&lt;/span&gt; &lt;span class="k"&gt;as&lt;/span&gt; &lt;span class="n"&gt;np&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;time&lt;/span&gt;

&lt;span class="n"&gt;mb&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="mi"&gt;512&lt;/span&gt;
&lt;span class="n"&gt;a&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;np&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;ones&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;mb&lt;/span&gt; &lt;span class="o"&gt;*&lt;/span&gt; &lt;span class="mi"&gt;1024&lt;/span&gt; &lt;span class="o"&gt;*&lt;/span&gt; &lt;span class="mi"&gt;1024&lt;/span&gt; &lt;span class="o"&gt;//&lt;/span&gt; &lt;span class="mi"&gt;4&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;dtype&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="n"&gt;np&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;float32&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;

&lt;span class="n"&gt;a&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;sum&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;                                  &lt;span class="c1"&gt;# fault the pages in first
&lt;/span&gt;&lt;span class="n"&gt;ts&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;[]&lt;/span&gt;
&lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;_&lt;/span&gt; &lt;span class="ow"&gt;in&lt;/span&gt; &lt;span class="nf"&gt;range&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mi"&gt;5&lt;/span&gt;&lt;span class="p"&gt;):&lt;/span&gt;
    &lt;span class="n"&gt;t&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;time&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;perf_counter&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;
    &lt;span class="n"&gt;a&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;sum&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;                              &lt;span class="c1"&gt;# read-only, no write traffic
&lt;/span&gt;    &lt;span class="n"&gt;ts&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;append&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;time&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;perf_counter&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt; &lt;span class="o"&gt;-&lt;/span&gt; &lt;span class="n"&gt;t&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;

&lt;span class="nf"&gt;print&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sa"&gt;f&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="si"&gt;{&lt;/span&gt;&lt;span class="n"&gt;mb&lt;/span&gt; &lt;span class="o"&gt;/&lt;/span&gt; &lt;span class="mi"&gt;1024&lt;/span&gt; &lt;span class="o"&gt;/&lt;/span&gt; &lt;span class="nf"&gt;min&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;ts&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="si"&gt;:&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="mi"&gt;2&lt;/span&gt;&lt;span class="n"&gt;f&lt;/span&gt;&lt;span class="si"&gt;}&lt;/span&gt;&lt;span class="s"&gt; GB/s read&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Results:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Array&lt;/th&gt;
&lt;th&gt;Read-only (&lt;code&gt;sum&lt;/code&gt;)&lt;/th&gt;
&lt;th&gt;Copy (&lt;code&gt;copyto&lt;/code&gt;)&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;256 MB&lt;/td&gt;
&lt;td&gt;8.98 GB/s&lt;/td&gt;
&lt;td&gt;4.76 GB/s&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;512 MB&lt;/td&gt;
&lt;td&gt;9.04 GB/s&lt;/td&gt;
&lt;td&gt;4.74 GB/s&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;Three things worth noticing.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;First, the copy number is not the bandwidth number.&lt;/strong&gt; A copy reads each byte &lt;em&gt;and&lt;/em&gt; writes it, so it moves twice the traffic for the same wall time. 4.7 GB/s of copying is ~9.5 GB/s of DRAM traffic — which is the same wall the read pass keeps hitting. If you benchmark with &lt;code&gt;memcpy&lt;/code&gt; and compare it directly against a model's tok/s, you will be off by a factor of two and conclude your machine is magically twice as fast as physics allows. Use a read-only pass.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Second, 9.0 GB/s against a theoretical 12.8 GB/s for single-channel DDR3-1600 is 70% of peak.&lt;/strong&gt; That is a healthy result. Old hardware is not secretly broken; it is just narrow.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Third, adding threads does not add bandwidth.&lt;/strong&gt; Same 256 MB arrays, one thread per array, min of five runs:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Threads&lt;/th&gt;
&lt;th&gt;Aggregate&lt;/th&gt;
&lt;th&gt;Per thread&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;1&lt;/td&gt;
&lt;td&gt;8.8 GB/s&lt;/td&gt;
&lt;td&gt;8.8 GB/s&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;2&lt;/td&gt;
&lt;td&gt;10.2 GB/s&lt;/td&gt;
&lt;td&gt;5.1 GB/s&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;4&lt;/td&gt;
&lt;td&gt;10.6 GB/s&lt;/td&gt;
&lt;td&gt;2.7 GB/s&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;It plateaus at ~10.5 GB/s and stops. On a dual-channel board four threads would have gone well past that — two channels of DDR3-1600 is 25.6 GB/s of theoretical peak, and you would expect to reach most of the way there. I get about 40% of it. So: one DIMM, and the DIMM is the ceiling, not the core count. Confirm it on your own box with &lt;code&gt;sudo dmidecode -t memory | grep -E "Size|Locator|Speed|Rank"&lt;/code&gt; before you spend anything.&lt;/p&gt;

&lt;h2&gt;
  
  
  What that predicts, and what it does not
&lt;/h2&gt;

&lt;p&gt;Plug the measured 9.0 GB/s in:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Weights per token&lt;/th&gt;
&lt;th&gt;Theoretical ceiling&lt;/th&gt;
&lt;th&gt;What you should expect in practice&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;4.4 GB (7B at Q4_K_M)&lt;/td&gt;
&lt;td&gt;2.0 tok/s&lt;/td&gt;
&lt;td&gt;1.2 – 1.6 tok/s&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;8.0 GB (13B at Q4)&lt;/td&gt;
&lt;td&gt;1.1 tok/s&lt;/td&gt;
&lt;td&gt;0.7 – 0.9 tok/s&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;23 GB (40B at Q4)&lt;/td&gt;
&lt;td&gt;0.39 tok/s&lt;/td&gt;
&lt;td&gt;0.24 – 0.31 tok/s, and it will not fit in 8 GB&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;Real-world runs land at 60–80% of the ceiling. If you are seeing 1.5 tok/s from a 7B Q4 model on hardware like this, your setup is fine and there is nothing left to tune. If you are seeing 0.2 tok/s, something is wrong — usually it is swapping, not compute. On an 8 GB machine with 5.1 GB actually available, a 4.4 GB model plus the OS is right at the edge; the moment the kernel starts paging, you are no longer measuring DRAM bandwidth, you are measuring your SSD.&lt;/p&gt;

&lt;h2&gt;
  
  
  Now the 125 tok/s claim
&lt;/h2&gt;

&lt;p&gt;I have not run Flint 40B, so I am not going to tell you it is fake. I am going to tell you what has to be true for the number to be real.&lt;/p&gt;

&lt;p&gt;At 23 GB of weights per token, 125 tok/s requires &lt;strong&gt;2,875 GB/s&lt;/strong&gt;. Nothing portable has that. The fastest laptop memory shipping today is in the several-hundred-GB/s range. So a 40B model at 125 tok/s is not reading 23 GB per token — which means most of its parameters are not being touched for a given token. That is what mixture-of-experts buys you: 40B parameters resident, a couple of billion active. Divide the active set instead of the whole model and the arithmetic works out fine.&lt;/p&gt;

&lt;p&gt;This is the actual shift in 2026's local models, and it is worth internalising: &lt;strong&gt;parameter count stopped being the thing that sets your speed. Active bytes per token is.&lt;/strong&gt; A "40B" MoE can be several times faster than a "13B" dense model while using more RAM. Benchmarks that report only total parameters tell you nothing about tok/s.&lt;/p&gt;

&lt;h2&gt;
  
  
  The table you actually want
&lt;/h2&gt;

&lt;p&gt;Bandwidth by machine, and the ceiling it implies. Only the first row is measured here; the rest are stated figures with a 75–85% efficiency factor applied, which is why they say "est."&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Memory&lt;/th&gt;
&lt;th&gt;Stated&lt;/th&gt;
&lt;th&gt;Effective (est.)&lt;/th&gt;
&lt;th&gt;7B Q4&lt;/th&gt;
&lt;th&gt;40B Q4&lt;/th&gt;
&lt;th&gt;40B MoE, ~2 GB active&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;DDR3-1600, 1 channel — &lt;strong&gt;measured here&lt;/strong&gt;
&lt;/td&gt;
&lt;td&gt;12.8 GB/s&lt;/td&gt;
&lt;td&gt;9.0 GB/s&lt;/td&gt;
&lt;td&gt;2.0&lt;/td&gt;
&lt;td&gt;0.39&lt;/td&gt;
&lt;td&gt;4.5&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;DDR3-1600, 2 channels&lt;/td&gt;
&lt;td&gt;25.6 GB/s&lt;/td&gt;
&lt;td&gt;~19 GB/s&lt;/td&gt;
&lt;td&gt;4.3&lt;/td&gt;
&lt;td&gt;0.83&lt;/td&gt;
&lt;td&gt;9.5&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;DDR4-3200, 2 channels&lt;/td&gt;
&lt;td&gt;51.2 GB/s&lt;/td&gt;
&lt;td&gt;~38 GB/s&lt;/td&gt;
&lt;td&gt;8.6&lt;/td&gt;
&lt;td&gt;1.7&lt;/td&gt;
&lt;td&gt;19&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Laptop LPDDR5&lt;/td&gt;
&lt;td&gt;~120 GB/s&lt;/td&gt;
&lt;td&gt;~90 GB/s&lt;/td&gt;
&lt;td&gt;20&lt;/td&gt;
&lt;td&gt;3.9&lt;/td&gt;
&lt;td&gt;45&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Discrete GPU, 288 GB/s&lt;/td&gt;
&lt;td&gt;288 GB/s&lt;/td&gt;
&lt;td&gt;~245 GB/s&lt;/td&gt;
&lt;td&gt;56&lt;/td&gt;
&lt;td&gt;11&lt;/td&gt;
&lt;td&gt;122&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;Read this table before you buy anything. Then read it again.&lt;/p&gt;

&lt;h2&gt;
  
  
  What it changes about buying hardware
&lt;/h2&gt;

&lt;p&gt;Three practical consequences, in order of how much money they save.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;1. The cheapest upgrade on an old box is a second DIMM.&lt;/strong&gt; My machine is single-channel. Adding a matching stick roughly doubles the read bandwidth, and therefore roughly doubles tok/s — for a part that costs less than a month of coffee. Check first:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;&lt;span class="nb"&gt;sudo &lt;/span&gt;dmidecode &lt;span class="nt"&gt;-t&lt;/span&gt; memory | &lt;span class="nb"&gt;grep&lt;/span&gt; &lt;span class="nt"&gt;-E&lt;/span&gt; &lt;span class="s2"&gt;"Size|Locator|Speed|Rank"&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;One "Size: 8 GB" line means single channel. Two matched lines means dual. This is a bigger lever than any CPU you can put in the socket.&lt;/p&gt;

&lt;p&gt;The caveat, and it is the reason this advice does not help me: if both slots are already populated you cannot add a channel, only replace the pair with larger sticks — same bandwidth, more capacity. Bandwidth is a property of the channel count, not the DIMM size. So the question to answer before shopping is &lt;em&gt;how many slots are free&lt;/em&gt;, and &lt;code&gt;dmidecode&lt;/code&gt; answers it in the same output.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;2. A CPU upgrade buys you almost nothing for generation.&lt;/strong&gt; My CPU is thermally degraded — it hits its 86 °C throttle line about three seconds into a sustained load and settles at 1.7–2.0 GHz, &lt;em&gt;below&lt;/em&gt; its 2.5 GHz base clock. It has already given up 20–30% of its rated clock. Token generation does not care, because the cores are idle waiting on DRAM. Prefill (processing your prompt) is compute-bound and does care; the first ten seconds of a long prompt will get faster. Steady-state chatting will not.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;3. If you want 40B-class dense models at conversational speed, you are buying bandwidth.&lt;/strong&gt; That means a machine with wide memory — an Apple Silicon laptop, or a GPU with fast VRAM. Not a faster CPU, not more cores, not more RAM of the same narrow kind. More RAM lets you &lt;em&gt;fit&lt;/em&gt; a bigger model; it does not make it faster.&lt;/p&gt;

&lt;p&gt;That is the whole thing. Measure your read bandwidth, divide by the size of the weights, and you know what your hardware is going to do before you download 20 GB over your home connection.&lt;/p&gt;

&lt;h2&gt;
  
  
  Reproduce it
&lt;/h2&gt;

&lt;p&gt;The snippet above is the entire method — &lt;code&gt;numpy&lt;/code&gt;, five runs, minimum. It takes two seconds on any machine and it is more informative than any spec sheet.&lt;/p&gt;

&lt;p&gt;If you want the shopping side of this — which used mini PCs actually have two DIMM slots and what they cost — I keep that list here: &lt;a href="https://lowspechq.com/best/mini-pc-for-ollama-under-300/" rel="noopener noreferrer"&gt;Best mini PC for running Ollama under $300&lt;/a&gt;. I also wrote up &lt;a href="https://lowspechq.com/best/cheap-ssd-for-nas/" rel="noopener noreferrer"&gt;which cheap SSDs survive in a NAS&lt;/a&gt;, because the "it started swapping" failure above is exactly how people kill a consumer drive.&lt;/p&gt;




&lt;p&gt;&lt;em&gt;How this was made: drafted with AI assistance, which I am stating up front rather than leaving you to guess. The memory benchmark was run by me on the machine described above, and every number in the first two tables is a command you can re-run in two seconds. The Flint 40B figures are from the vendor's announcement — I have not run that model, and the arithmetic in that section is presented as arithmetic, not as a benchmark. The two links at the end point to my own site, which carries affiliate links that are labelled as such on the page.&lt;/em&gt;&lt;/p&gt;

</description>
      <category>localai</category>
      <category>hardware</category>
      <category>performance</category>
      <category>ollama</category>
    </item>
  </channel>
</rss>
