<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:dc="http://purl.org/dc/elements/1.1/">
  <channel>
    <title>DEV Community: Jamie Duncan</title>
    <description>The latest articles on DEV Community by Jamie Duncan (@jamie_duncan_3738b0805ed5).</description>
    <link>https://dev.to/jamie_duncan_3738b0805ed5</link>
    <image>
      <url>https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F4254909%2F77d39669-49e3-4897-9dd9-57a21485b30f.jpg</url>
      <title>DEV Community: Jamie Duncan</title>
      <link>https://dev.to/jamie_duncan_3738b0805ed5</link>
    </image>
    <atom:link rel="self" type="application/rss+xml" href="https://dev.to/feed/jamie_duncan_3738b0805ed5"/>
    <language>en</language>
    <item>
      <title>My Dark Factory Reinvented Sprint Planning</title>
      <dc:creator>Jamie Duncan</dc:creator>
      <pubDate>Fri, 20 Nov 2026 06:24:04 +0000</pubDate>
      <link>https://dev.to/jamie_duncan_3738b0805ed5/my-dark-factory-reinvented-sprint-planning-26dm</link>
      <guid>https://dev.to/jamie_duncan_3738b0805ed5/my-dark-factory-reinvented-sprint-planning-26dm</guid>
      <description>&lt;p&gt;&lt;em&gt;Originally published at &lt;a href="https://jduncan.io/blog/2026-11-20-dark-factory-sprint-planning/" rel="noopener noreferrer"&gt;jduncan.io&lt;/a&gt;.&lt;/em&gt;&lt;/p&gt;

&lt;p&gt;As of last week, I had pushed roughly &lt;a href="https://jduncan.io/blog/2026-11-12-forty-billion-tokens-dark-factory-rework/" rel="noopener noreferrer"&gt;40 billion tokens through Leaf&lt;/a&gt;, my local Level 5 dark software factory. Once the core build and review loops felt steady, I couldn't resist poking at what else an unattended fleet could catch on its own.&lt;/p&gt;

&lt;p&gt;First, I added an &lt;strong&gt;SRE loop&lt;/strong&gt;. Its job was to watch application health and performance over time: reading end-to-end test results, repository-declared performance benchmarks, and chronic CI/CD failures, then filing bugs whenever performance envelopes or reliability started drifting the wrong way.&lt;/p&gt;

&lt;p&gt;That ran alongside my &lt;a href="https://jduncan.io/blog/2026-11-06-teaching-my-dark-factory-to-dream/" rel="noopener noreferrer"&gt;daily dreaming loop&lt;/a&gt;, which scans 30-day swaths of telemetry verdicts to catch slow-burn patterns that single-room Analysts miss, plus the single-room Analysts themselves and the Improve loop that looked for dead code and measured code complexity to make the next agent find the right work easier.&lt;/p&gt;

&lt;p&gt;Within 24 hours, the factory handed me a surprise I didn't see coming.&lt;/p&gt;

&lt;p&gt;Telemetry, SRE, and dreaming were all finding the &lt;strong&gt;same issues from different angles&lt;/strong&gt;. Same root causes, just three different points of view on the same symptom.&lt;/p&gt;

&lt;p&gt;That's normal in the real world. What wasn't normal was how my factory handled it. Because my planning room picked up issues one at a time, the factory would grab SRE's ticket, plan a fix, and ship it. Ten minutes later, it would grab Telemetry's ticket for the same underlying flaw and patch it from a slightly different angle.&lt;/p&gt;

&lt;p&gt;It turned into a mess fast.&lt;/p&gt;

&lt;p&gt;I needed a process that could look across all the open issues for a repository, collate them, strip out the duplicates, set dependencies, and organize them into a coherent batch before any planning room opened.&lt;/p&gt;

&lt;p&gt;Oh no. &lt;strong&gt;I needed a sprint planning meeting.&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;What have I done?&lt;/p&gt;

&lt;h2&gt;
  
  
  Agents rediscovered agile ceremony
&lt;/h2&gt;

&lt;p&gt;My first reaction was to laugh at myself. I spent months trying to strip unnecessary human ceremony out of agent workflows, and the factory's own failure mode walked me right back into backlog grooming.&lt;/p&gt;

&lt;p&gt;When you stop and think about the mechanics, though, it makes complete sense.&lt;/p&gt;

&lt;p&gt;Every loop was doing its job faithfully. The SRE agent's job wasn't to track performance across the repository; its job was to notice a drifting benchmark or a flaky deploy and file a clean defect report. The Telemetry Analyst's job was to explain why a pod burned 95 turns. The Dreamer's job was to connect three slow rooms over the last month.&lt;/p&gt;

&lt;p&gt;Their goal wasn't cross-team coordination. Their goal was to report from their specific point of view.&lt;/p&gt;

&lt;p&gt;Human engineering teams have the exact same problem. SRE files a latency alert, QA files a flaky test bug, and a developer files a tech-debt ticket. All three point at the same overloaded query. Sprint planning exists in human orgs because &lt;strong&gt;specialist roles naturally generate overlapping symptoms&lt;/strong&gt;, and someone has to synthesize them before engineers start cutting branches.&lt;/p&gt;

&lt;p&gt;Good process isn't always bureaucracy.&lt;br&gt;&lt;br&gt;
Sometimes it's just the shape of the problem.&lt;br&gt;
Sometimes we have to sit and stare at the stickies on the whiteboard.&lt;/p&gt;
&lt;h2&gt;
  
  
  Building the triage loop
&lt;/h2&gt;

&lt;p&gt;So on Monday, I built a &lt;strong&gt;Triage loop&lt;/strong&gt; right in front of the planning room.&lt;/p&gt;

&lt;p&gt;Instead of letting every newly filed bug wake up a PM and Architect on the spot (hotfixes still jump the line), issues now land in a waiting backlog. On a schedule or on demand, a &lt;code&gt;triager&lt;/code&gt; agent opens a short sprint-planning pass for a repository. It reads the waiting issues together, closes duplicates, groups related tickets, wires up dependencies, and starts the coherent work as a batch. It only works on issue content and bodies. Just like a human would shuffling cards around a kanban board.&lt;/p&gt;

&lt;p&gt;By Tuesday morning, I had staggered triage across repositories in 180-minute slots. Instead of a chaotic drip of single-issue fixes fighting over the same files, work moved in predictable bursts. My plan was to stagger projects across the clock: Project A during business hours, background repos in the evening, keeping the closet shelf humming without stepping on itself.&lt;/p&gt;

&lt;p&gt;So close.&lt;/p&gt;

&lt;p&gt;Then I broke TypeScript.&lt;/p&gt;
&lt;h2&gt;
  
  
  The Tuesday morning TypeScript meltdown
&lt;/h2&gt;

&lt;p&gt;Adding the SRE loop, the Triage backlog, and smarter work ranking was the straw that finally snapped the back of Leaf v1's single-threaded Node.js backend.&lt;/p&gt;

&lt;p&gt;By Tuesday, the v1 &lt;code&gt;src/&lt;/code&gt; tree had grown to &lt;strong&gt;206 non-test TypeScript files&lt;/strong&gt; all sharing one Node.js event loop: HTTP API routes, WebSocket streams to five worker hosts, synchronous SQLite queries, git subprocesses, container orchestration, and background admission passes for 60 concurrent pod slots.&lt;/p&gt;

&lt;p&gt;The git history from &lt;code&gt;~/Code/factory&lt;/code&gt; preserves the exact smoking gun. The very last commit on the v1 TypeScript codebase (&lt;code&gt;ba4a044e&lt;/code&gt;, Tuesday, November 17 at 8:39 AM Eastern) tells the whole story in four lines:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;commit ba4a044efabd6cea1efe7c59db5834fcd70c8a65
Date:   Tue Nov 17 08:39:43 2026 -0500

    Restore the cursor walk for Improve and SRE (#4520)

    316c3c2a and a7d41948 replaced the per-loop cursor walk with a ranking
    by attempts, complexity score and git recency. demand() runs on every
    admission pass, so the ranking's extra git log spawn and its full
    files x open-issues filter ran back to back on the main thread. With the
    triage backlog, that stalled the supervisor until its health check
    killed it.
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Look at the chain reaction in that commit message.&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Both Improve and SRE replaced a cheap cursor walk with a ranking pass that scored files by attempts, complexity, and git recency.&lt;/li&gt;
&lt;li&gt;
&lt;code&gt;demand()&lt;/code&gt; ran on every admission pass, executing the extra &lt;code&gt;git log&lt;/code&gt; spawn and a full &lt;code&gt;files x open-issues&lt;/code&gt; cross-product filter back to back on the main thread.&lt;/li&gt;
&lt;li&gt;Once the new triage backlog kept open issues stacked up instead of draining immediately, &lt;strong&gt;that admission pass stalled the Node.js event loop long enough for the supervisor's own health check to declare the process dead and kill it&lt;/strong&gt;.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;And when the supervisor got killed mid-flight, the restarts triggered secondary fallout (commit &lt;code&gt;cc7f910e&lt;/code&gt; earlier that same morning had to patch dropped tech-lead commands and re-ask pods whose &lt;code&gt;say&lt;/code&gt; calls hit an unreachable Factory API during a restart).&lt;/p&gt;

&lt;p&gt;Restoring the cursor walk in &lt;code&gt;ba4a044e&lt;/code&gt; stopped the bleeding for the afternoon. It didn't fix the real architecture problem. Running a multi-loop, multi-host orchestration engine on a single-threaded runtime where one heavy loop calculation can starve the API and trip the health check was living on borrowed time.&lt;/p&gt;

&lt;p&gt;To be fair to Node.js and TypeScript, I'm not claiming JavaScript can't run backend services. Plenty of huge companies run Node at scale with worker threads, queues, and separate microservices. My factory is a single-binary, closet-shelf control plane where I want &lt;strong&gt;one simple process&lt;/strong&gt; managing SQLite, git repos, host sockets, and a dozen background loops without me babysitting event-loop lag.&lt;/p&gt;

&lt;p&gt;That's a Go problem.&lt;/p&gt;

&lt;h2&gt;
  
  
  Enter Factory v2
&lt;/h2&gt;

&lt;p&gt;The very next morning (Wednesday, November 18), I cut the cord and landed the initial &lt;strong&gt;Factory 2.0&lt;/strong&gt; build.&lt;/p&gt;

&lt;p&gt;It's going to have a better name (I think), and it's taking every hard-won lesson from those 40 billion tokens while ruthlessly deleting the ideas that didn't survive contact with reality. The architecture split is simple:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Backend and host agents in Go:&lt;/strong&gt; The supervisor, bridge, CLI, hook, and &lt;code&gt;hostd&lt;/code&gt; worker daemon are all Go now. Real goroutines, real concurrency, and one hard reliability rule baked into the spec: nothing in a background loop gets to delay the API or the health check.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Sprint planning as a first-class stage:&lt;/strong&gt; Triage isn't bolted onto the side anymore. The lifecycle is explicitly &lt;code&gt;Sprint -&amp;gt; Plan -&amp;gt; Build -&amp;gt; Land -&amp;gt; Ship -&amp;gt; Learn&lt;/code&gt;, with a strict issue rubric and atomic transactions so overlapping findings get grouped before a planning room ever opens.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Frontend stays in TypeScript, with a full UI overhaul:&lt;/strong&gt; TypeScript is still great where it belongs. The web UI is getting a clean rebuild focused on the live flow, quota meters per subscription pool, and what actually needs my attention.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;I'm pushing the first real workloads through v2 right now and shaking out the remaining bugs across the fleet. I hope to unveil the whole thing, new name and all, in the next week or two.&lt;/p&gt;

&lt;p&gt;My quotas can't wait.&lt;/p&gt;

&lt;h2&gt;
  
  
  So what?
&lt;/h2&gt;

&lt;p&gt;If you're stacking multiple autonomous agent loops onto the same codebase, three things fell out of this week's wreckage:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Multiple observer loops require a synthesis gate.&lt;/strong&gt; As soon as you have more than one automated filer (SRE, telemetry, security, complexity), they &lt;em&gt;will&lt;/em&gt; report the same root cause in different words. Put a triage step in front of execution so you fix the cause once instead of patching three symptoms.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Respect the event loop on control planes.&lt;/strong&gt; If your orchestrator runs admission logic, git commands, and API health checks on one thread, a growing backlog can turn a harmless O(files x issues) filter into a self-inflicted crash loop.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Treat v1 as the spec for v2.&lt;/strong&gt; Pushing 40 billion tokens through a scrappy TypeScript prototype wasn't wasted effort. It taught me which loops actually move the needle, which ceremonies are real engineering, and exactly where the runtime walls are before I wrote a line of Go.&lt;/li&gt;
&lt;/ul&gt;

</description>
      <category>darkfactory</category>
      <category>softwarefactory</category>
      <category>leaf</category>
      <category>agenticsdlc</category>
    </item>
    <item>
      <title>Checking My Dark Factory's Receipts After 4,000 Runs</title>
      <dc:creator>Jamie Duncan</dc:creator>
      <pubDate>Fri, 20 Nov 2026 05:56:27 +0000</pubDate>
      <link>https://dev.to/jamie_duncan_3738b0805ed5/checking-my-dark-factorys-receipts-after-4000-runs-1inn</link>
      <guid>https://dev.to/jamie_duncan_3738b0805ed5/checking-my-dark-factorys-receipts-after-4000-runs-1inn</guid>
      <description>&lt;p&gt;&lt;em&gt;Originally published at &lt;a href="https://jduncan.io/blog/2026-11-12-checking-dark-factory-receipts-four-thousand-runs/" rel="noopener noreferrer"&gt;jduncan.io&lt;/a&gt;.&lt;/em&gt;&lt;/p&gt;

&lt;p&gt;This is the third piece of today's fresh telemetry pass across Leaf, following &lt;a href="https://jduncan.io/blog/2026-11-12-forty-billion-tokens-dark-factory-rework/" rel="noopener noreferrer"&gt;our 40-billion-token rework audit&lt;/a&gt; and &lt;a href="https://jduncan.io/blog/2026-11-12-best-architect-model-isnt-best-builder/" rel="noopener noreferrer"&gt;what 1,199 randomized planning rooms revealed about model routing&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;Over the last ten days, I published two major changes to &lt;a href="https://jduncan.io/blog/2026-11-11-six-agentic-development-rules-that-survived-2026/" rel="noopener noreferrer"&gt;Leaf&lt;/a&gt; while the paint was barely dry:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;On November 4, in &lt;a href="https://jduncan.io/blog/2026-11-04-pruning-agent-prompts-dark-factory/" rel="noopener noreferrer"&gt;Pruning Agent Prompts&lt;/a&gt;, we cut &lt;strong&gt;65% of the words in &lt;code&gt;prompts/&lt;/code&gt;&lt;/strong&gt; (taking &lt;code&gt;_base.md&lt;/code&gt; from &lt;code&gt;2,861&lt;/code&gt; words down to &lt;code&gt;55&lt;/code&gt;) and deleted &lt;code&gt;900&lt;/code&gt; lines of prose-pinning regex tests. My runtime evidence at the time was a &lt;strong&gt;29-hour sample of 10 planning rooms and 9 build runs on &lt;code&gt;Calamus&lt;/code&gt;&lt;/strong&gt;. I called out three risks: &lt;code&gt;BrandCast&lt;/code&gt; (my main digital signage monorepo) hadn't run a batch yet, task shapes had shifted, and &lt;strong&gt;Tech Lead input tokens on 1-node runs had gone the wrong way (&lt;code&gt;+29%&lt;/code&gt;)&lt;/strong&gt;. I promised to re-measure past 50 Development runs.&lt;/li&gt;
&lt;li&gt;On November 6, in &lt;a href="https://jduncan.io/blog/2026-11-06-teaching-my-dark-factory-to-dream/" rel="noopener noreferrer"&gt;Teaching My Dark Factory to Dream&lt;/a&gt;, we deleted &lt;code&gt;6,067&lt;/code&gt; lines of noisy LLM "friction counting" in favor of eight deterministic SQL flags, a daily Dream consolidation pass, and a wiki health tracker that immediately tattled on us: &lt;strong&gt;73% (&lt;code&gt;108 of 148&lt;/code&gt;) of &lt;code&gt;factory&lt;/code&gt;'s own auto-generated wiki pages had never been read once&lt;/strong&gt;.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;Eight days and more than a thousand runs later (&lt;code&gt;4,044&lt;/code&gt; total completed &lt;code&gt;plan_run&lt;/code&gt;s in &lt;code&gt;factory.db&lt;/code&gt;), the full-fleet numbers are in. Almost every bet paid off, and the one metric that looked broken taught me a great lesson about statistical traps in agent harnesses.&lt;/p&gt;

&lt;h2&gt;
  
  
  The nuance: how a 78% token spike turned out to be a win
&lt;/h2&gt;

&lt;p&gt;When I re-ran that exact 1-node Tech Lead query tonight across &lt;strong&gt;8 full days before the November 4 prompt cut (&lt;code&gt;October 27 to November 4&lt;/code&gt;) versus 8 full days after (&lt;code&gt;November 4 to November 12&lt;/code&gt;)&lt;/strong&gt;, my heart sank for about thirty seconds.&lt;/p&gt;

&lt;p&gt;Across &lt;code&gt;122&lt;/code&gt; post-cut 1-node Development runs (compared to &lt;code&gt;38&lt;/code&gt; before), &lt;strong&gt;Tech Lead turns didn't stabilize; they climbed 41% (&lt;code&gt;14.0&lt;/code&gt; to &lt;code&gt;19.8&lt;/code&gt;), raw input tokens more than doubled (&lt;code&gt;500K&lt;/code&gt; to &lt;code&gt;1,032K&lt;/code&gt;), and Tech Lead IET jumped 78% (&lt;code&gt;100.5K&lt;/code&gt; to &lt;code&gt;178.8K&lt;/code&gt;)&lt;/strong&gt;. Meanwhile, same-model Sonnet engineer wall time per pod on Development runs had also climbed (&lt;code&gt;285.5s&lt;/code&gt; to &lt;code&gt;618.6s&lt;/code&gt;).&lt;/p&gt;

&lt;p&gt;If I had stopped querying at the pod level, I would have rolled back the Tech Lead prompt tonight. Instead, I zoomed out one level to check &lt;strong&gt;whole plans and planning rooms across all 13 repositories&lt;/strong&gt;:&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fpenmoea6fz9l52pv173w.webp" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fpenmoea6fz9l52pv173w.webp" alt="Receipts scorecard from Oct 27 to Nov 12, 2026: Top table compares 8 days before vs. 8 days after the 65% prompt cut showing 655 planning rooms dropping 40% in IET and 28% in wall time, Development plans rising 35.5% (121 to 164) at 7.4% lower build IET per plan, nodes per plan collapsing from 4.05 to 1.54, and 1-node Dev diffs growing 2.86x larger (135 to 387 lines, 3.8 to 8.0 CI jobs). Bottom cards show Dream telemetry auto-fixing 107 of 118 systemic bugs (90.7%) with a 93.1% yield on retried pods, and wiki unread pages dropping from 73% to 34% on factory (81.4% read on brandcast)." width="799" height="483"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Look at the whole board together:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Planning passed the &lt;code&gt;BrandCast&lt;/code&gt; stress test (&lt;code&gt;649&lt;/code&gt; rooms before vs. &lt;code&gt;655&lt;/code&gt; rooms after, including &lt;code&gt;258&lt;/code&gt; &lt;code&gt;BrandCast&lt;/code&gt; rooms):&lt;/strong&gt; Average planning room wall time dropped &lt;strong&gt;28%&lt;/strong&gt; (&lt;code&gt;471.8s&lt;/code&gt; down to &lt;code&gt;338.4s&lt;/code&gt;), raw input tokens per room dropped &lt;strong&gt;47%&lt;/strong&gt; (&lt;code&gt;5.06M&lt;/code&gt; down to &lt;code&gt;2.69M&lt;/code&gt;), and &lt;strong&gt;planning IET per room fell 40% (&lt;code&gt;882.2K&lt;/code&gt; down to &lt;code&gt;526.4K&lt;/code&gt;)&lt;/strong&gt;.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Feature throughput jumped &lt;code&gt;+35.5%&lt;/code&gt; at lower token cost per feature:&lt;/strong&gt; Completed human Development plans climbed from &lt;strong&gt;&lt;code&gt;121&lt;/code&gt; (&lt;code&gt;15.1/day&lt;/code&gt;) to &lt;code&gt;164&lt;/code&gt; (&lt;code&gt;20.5/day&lt;/code&gt;)&lt;/strong&gt;, while &lt;strong&gt;average build IET per shipped Development plan fell 7.4% (&lt;code&gt;2.89M&lt;/code&gt; down to &lt;code&gt;2.68M&lt;/code&gt;)&lt;/strong&gt; on top of the 40% planning savings.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;How can per-pod engineer time and 1-node Tech Lead tokens go &lt;em&gt;up&lt;/em&gt; while per-feature build cost goes &lt;em&gt;down&lt;/em&gt; and throughput rises 35.5%?&lt;/p&gt;

&lt;p&gt;Because we walked straight into &lt;a href="https://plato.stanford.edu/entries/paradox-simpson/" rel="noopener noreferrer"&gt;Simpson's paradox&lt;/a&gt;, the classic contingency-table trap &lt;a href="https://rss.onlinelibrary.wiley.com/doi/10.1111/j.2517-6161.1951.tb00088.x" rel="noopener noreferrer"&gt;Edward H. Simpson formalized in 1951&lt;/a&gt; where subgroup trends reverse once you account for a shifting confounding variable. Yes. I had to look this one up. Hat tip to the new Gemini Carbon model for the great catch here.&lt;/p&gt;

&lt;p&gt;Look at the fourth and fifth rows of the table: &lt;strong&gt;pruning 89% of the Architect prompt collapsed the average Development plan from &lt;code&gt;4.05&lt;/code&gt; nodes down to &lt;code&gt;1.54&lt;/code&gt; nodes&lt;/strong&gt;.&lt;/p&gt;

&lt;p&gt;Under our old 7,700-word prompt binder, Architects routinely chopped a coherent feature into four coupled micro-nodes just to look thorough. Back then, the only work that stayed a "1-node Development plan" (&lt;code&gt;n = 38&lt;/code&gt;) was a tiny &lt;strong&gt;&lt;code&gt;134.9&lt;/code&gt;-line tweak&lt;/strong&gt; in a small repo running &lt;strong&gt;&lt;code&gt;3.79&lt;/code&gt; CI jobs&lt;/strong&gt; (&lt;code&gt;15.8%&lt;/code&gt; send-backs). After the cut, Architects stopped over-splitting work and kept whole features inside a single coherent node (&lt;code&gt;n = 122&lt;/code&gt;), pushing the average "1-node run" to &lt;strong&gt;&lt;code&gt;386.5&lt;/code&gt; changed lines (&lt;code&gt;2.86x&lt;/code&gt; larger)&lt;/strong&gt;, &lt;strong&gt;&lt;code&gt;7.95&lt;/code&gt; CI jobs (&lt;code&gt;2.1x&lt;/code&gt; more, including &lt;code&gt;BrandCast&lt;/code&gt;'s full 9-job suite)&lt;/strong&gt;, and &lt;strong&gt;&lt;code&gt;27.9%&lt;/code&gt; Tech Lead send-backs&lt;/strong&gt;.&lt;/p&gt;

&lt;p&gt;Before the cut, a 1-node plan was a typo fix.&lt;br&gt;&lt;br&gt;
After the cut, a 1-node plan was the whole feature.&lt;/p&gt;

&lt;p&gt;Comparing "1-node runs" across that boundary was comparing bicycles to box trucks because both have two axles. And consolidating &lt;code&gt;4.05&lt;/code&gt; nodes into &lt;code&gt;1.54&lt;/code&gt; came with one genuine engineering trade-off worth naming honestly: &lt;strong&gt;average wall-clock time per Development plan rose from &lt;code&gt;18.99&lt;/code&gt; minutes to &lt;code&gt;26.86&lt;/code&gt; minutes (&lt;code&gt;+41%&lt;/code&gt;)&lt;/strong&gt;. One engineer writing 387 lines serially takes longer on the wall clock than four parallel pods writing 100 lines each, even though it burns fewer total tokens and eliminates cross-node merge ceremony. With 60 pod slots in the closet, trading eight extra minutes of background wall time for 35.5% more shipped features per week is a deal I'll take every day. I will say that there is a trade-off because it &lt;em&gt;feels&lt;/em&gt; slower when I'm actively testing something. That's new muscle memory I will be working on.&lt;/p&gt;
&lt;h2&gt;
  
  
  Dream telemetry at 1,806 flagged rooms (90.7% self-healed)
&lt;/h2&gt;

&lt;p&gt;Six days after turning on &lt;a href="https://jduncan.io/blog/2026-11-06-teaching-my-dark-factory-to-dream/" rel="noopener noreferrer"&gt;the Analyst and Dreamer telemetry loops&lt;/a&gt;, the backlog of unanalyzed outlier rooms sits at &lt;strong&gt;zero&lt;/strong&gt;.&lt;/p&gt;

&lt;p&gt;Across &lt;strong&gt;1,806 flagged rooms&lt;/strong&gt; (&lt;code&gt;932&lt;/code&gt; read end-to-end by single-room Analysts and &lt;code&gt;874&lt;/code&gt; &lt;code&gt;turns&lt;/code&gt;-only outliers sampled out via our &lt;code&gt;1-in-5&lt;/code&gt; rule) and &lt;strong&gt;28 daily Dream consolidation passes&lt;/strong&gt;:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;118 distinct systemic bugs were isolated, and 107 (&lt;code&gt;90.7%&lt;/code&gt;) are already fixed and closed &lt;code&gt;done&lt;/code&gt;.&lt;/strong&gt; Across all 13 onboarded repositories tonight (&lt;code&gt;158&lt;/code&gt; all-time closed trajectory issues), &lt;strong&gt;only 1 trajectory bug remains open&lt;/strong&gt;.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Two SQL flags turned out to be pure gold.&lt;/strong&gt; Rooms where a pod had to be started more than once (&lt;code&gt;retried&lt;/code&gt;) linked to a concrete root-cause bug &lt;strong&gt;93.1% of the time (&lt;code&gt;417 of 448&lt;/code&gt;)&lt;/strong&gt;, and rooms where a pod exited without a completion report (&lt;code&gt;unfinished&lt;/code&gt;) yielded a bug &lt;strong&gt;78.2% of the time (&lt;code&gt;104 of 133&lt;/code&gt;)&lt;/strong&gt;. Even &lt;code&gt;dead time&lt;/code&gt; (10-minute silent gaps), which sat at &lt;code&gt;0 of 19&lt;/code&gt; in last week's initial backfill, began earning its keep at &lt;strong&gt;&lt;code&gt;17.9%&lt;/code&gt; (&lt;code&gt;10 of 56&lt;/code&gt;)&lt;/strong&gt; once real suite checkout hangs appeared.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Timestamp recurrence tracking caught 18 regressions (&lt;code&gt;15.3%&lt;/code&gt;).&lt;/strong&gt; Whenever an Analyst or Dreamer links a new room to an &lt;em&gt;already closed&lt;/em&gt; issue, Leaf checks whether the room closed after the fix landed and flips the cause family to &lt;code&gt;Recurred after fix&lt;/code&gt;. That check caught two subtle platform bugs that refused to die on the first patch: &lt;code&gt;factory#3102&lt;/code&gt; (where a race killed a plan's first Tech Lead pod seconds after boot) and &lt;code&gt;factory#3279&lt;/code&gt; (where CI suite pods intermittently stalled 2 to 5 minutes in host checkout across &lt;code&gt;brandcast&lt;/code&gt; and &lt;code&gt;calamus&lt;/code&gt;).&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Without recurrence timestamps, both of those bugs would still be sitting in the "closed and celebrated" column while quietly taxing rooms every night.&lt;/p&gt;
&lt;h2&gt;
  
  
  How we got agents to actually read the wiki
&lt;/h2&gt;

&lt;p&gt;The most embarrassing number in last Friday's post was wiki readership: &lt;strong&gt;73% (&lt;code&gt;108 of 148&lt;/code&gt;) of &lt;code&gt;factory&lt;/code&gt;'s auto-generated wiki pages had zero reads&lt;/strong&gt;, even though Technical Writer and Lead Writer pods were burning millions of tokens keeping them tidy.&lt;/p&gt;

&lt;p&gt;Our new rule from the prompt prune forbade adding a lecture to &lt;code&gt;architect.md&lt;/code&gt; saying &lt;em&gt;"Please remember to read the wiki!"&lt;/em&gt; If smart models ignore a document, the document isn't useful enough yet.&lt;/p&gt;

&lt;p&gt;Instead, as part of &lt;code&gt;commit c2a39960&lt;/code&gt;, we audited the pages and made Factory's deterministic code stamp a live &lt;code&gt;&amp;lt;!-- factory:measured --&amp;gt;&lt;/code&gt; block onto the bottom of every repository &lt;code&gt;area&lt;/code&gt; wiki page on startup, on &lt;code&gt;wiki.write&lt;/code&gt;, and whenever &lt;code&gt;scorecard.ts&lt;/code&gt; scores a merged node in that directory:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight markdown"&gt;&lt;code&gt;&amp;lt;!-- factory:measured --&amp;gt;
&lt;span class="gu"&gt;## Measured cost
&lt;/span&gt;
| area | class | effort | level | n | minutes | tokens | sendbacks | before_edit |
| :--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- |
| src | 2 | medium | area | 255 | 0.5 | 66000 | 0.07 | 2 |
| src | 4 | high | area | 97 | 3.2 | 292000 | 0.24 | 4 |

Scoped checks: &lt;span class="sb"&gt;`npm test`&lt;/span&gt; median 104s (n=412)
&amp;lt;!-- /factory:measured --&amp;gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Every time a writer agent edits an area page, Factory strips any old block and writes fresh medians from SQLite (&lt;code&gt;orientation&lt;/code&gt; tool calls before first edit across &lt;code&gt;10,430&lt;/code&gt; unpacked trajectories, class/effort costs, and scoped test durations).&lt;/p&gt;

&lt;p&gt;Six days later, look at &lt;code&gt;/api/wiki&lt;/code&gt; on our two largest repositories:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;&lt;code&gt;BrandCast&lt;/code&gt; (&lt;code&gt;118&lt;/code&gt; pages):&lt;/strong&gt; Unread pages dropped from &lt;code&gt;44%&lt;/code&gt; down to &lt;strong&gt;&lt;code&gt;18.6%&lt;/code&gt; (&lt;code&gt;96 of 118&lt;/code&gt; pages read, &lt;code&gt;81.4%&lt;/code&gt;)&lt;/strong&gt;, with &lt;strong&gt;&lt;code&gt;225&lt;/code&gt; explicit wiki citations&lt;/strong&gt; inside planning messages (&lt;code&gt;233&lt;/code&gt; writer jobs and &lt;code&gt;33&lt;/code&gt; lead passes costing &lt;code&gt;158.9M&lt;/code&gt; cached input tokens and &lt;code&gt;561K&lt;/code&gt; output tokens).&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;&lt;code&gt;Factory&lt;/code&gt; (&lt;code&gt;150&lt;/code&gt; pages):&lt;/strong&gt; Unread pages dropped by more than half, from &lt;strong&gt;&lt;code&gt;73.0%&lt;/code&gt; down to &lt;code&gt;34.0%&lt;/code&gt; (&lt;code&gt;51&lt;/code&gt; unread)&lt;/strong&gt;, with &lt;strong&gt;&lt;code&gt;123&lt;/code&gt; explicit planning citations&lt;/strong&gt;.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;We didn't have to beg the PM or Architect to read the wiki. &lt;strong&gt;Once the wiki carried live operational facts that agents couldn't get by grepping TypeScript source files, reading it became the fastest path to getting their plan approved.&lt;/strong&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  So what?
&lt;/h2&gt;

&lt;p&gt;Checking your own receipts a week after shipping harness changes is where the real engineering happens:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Watch out for Simpson's paradox when plan shapes shift.&lt;/strong&gt; Per-pod and "1-node run" averages lie when an upstream prompt change consolidates 4-node graphs into 1.5-node graphs. Always pair pod-level metrics with &lt;strong&gt;cost and throughput per shipped issue&lt;/strong&gt;.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Track whether closed bugs stay dead.&lt;/strong&gt; An autonomous self-healing loop that files and merges fixes looks magical right up until &lt;code&gt;15.3%&lt;/code&gt; of those causes quietly recur a few days later. Compare room close timestamps against issue close timestamps automatically.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Make documentation useful by injecting live telemetry, not prompt nagging.&lt;/strong&gt; If planning agents skip your repo docs, don't scold them in &lt;code&gt;AGENTS.md&lt;/code&gt;. Stamp measured directory costs, send-back rates, and test runtimes directly into the pages so reading the docs gives the agent an immediate edge.&lt;/li&gt;
&lt;/ul&gt;

</description>
      <category>darkfactory</category>
      <category>promptengineering</category>
      <category>agenttelemetry</category>
      <category>leaf</category>
    </item>
    <item>
      <title>40 Billion Tokens In: Why 28% Agent Rework Is a Feature</title>
      <dc:creator>Jamie Duncan</dc:creator>
      <pubDate>Fri, 20 Nov 2026 05:56:25 +0000</pubDate>
      <link>https://dev.to/jamie_duncan_3738b0805ed5/40-billion-tokens-in-why-28-agent-rework-is-a-feature-3ge7</link>
      <guid>https://dev.to/jamie_duncan_3738b0805ed5/40-billion-tokens-in-why-28-agent-rework-is-a-feature-3ge7</guid>
      <description>&lt;p&gt;&lt;em&gt;Originally published at &lt;a href="https://jduncan.io/blog/2026-11-12-forty-billion-tokens-dark-factory-rework/" rel="noopener noreferrer"&gt;jduncan.io&lt;/a&gt;.&lt;/em&gt;&lt;/p&gt;

&lt;p&gt;Two weeks ago, I opened &lt;a href="https://jduncan.io/blog/2026-10-29-moneyball-for-ai-agents/" rel="noopener noreferrer"&gt;Moneyball for AI Agents&lt;/a&gt; by noting that &lt;a href="https://jduncan.io/blog/2026-10-25-leaf-dark-factory-update/" rel="noopener noreferrer"&gt;Leaf, my local Level 5 dark software factory&lt;/a&gt;, had burned through 15.8 billion tokens inside its build runs over its first 50 days.&lt;/p&gt;

&lt;p&gt;In the 14 days since that post went live, &lt;strong&gt;Leaf burned 15.50 billion raw tokens across all five factory loops&lt;/strong&gt; (&lt;strong&gt;2.87 billion input-equivalent tokens&lt;/strong&gt; and &lt;strong&gt;101.7 million output tokens&lt;/strong&gt; over &lt;strong&gt;162,946 model turns&lt;/strong&gt;). Two weeks of autonomous work nearly matched the factory's entire first seven weeks of build volume.&lt;/p&gt;

&lt;p&gt;Pulling the full 64-day ledger (September 9 to November 12, 2026) from the live DB shows how fast an unattended harness compounds when you stop babysitting terminals:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Total token volume across all seats:&lt;/strong&gt; &lt;strong&gt;39.90 billion raw tokens&lt;/strong&gt; (&lt;strong&gt;37.13 billion cache reads&lt;/strong&gt;, &lt;strong&gt;274.35 million output tokens&lt;/strong&gt;, and &lt;strong&gt;7.89 billion input-equivalent tokens&lt;/strong&gt;) across &lt;strong&gt;460,740 model turns&lt;/strong&gt; and &lt;strong&gt;1,412.4 hours&lt;/strong&gt; (58.9 continuous days) of active model execution.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Where the tokens went:&lt;/strong&gt; &lt;strong&gt;22.66 billion&lt;/strong&gt; inside &lt;strong&gt;4,044 completed build runs&lt;/strong&gt; (&lt;strong&gt;4,026 confirmed merges, or 99.6%&lt;/strong&gt;), &lt;strong&gt;9.47 billion&lt;/strong&gt; across &lt;strong&gt;4,112 PM and Architect planning rooms&lt;/strong&gt;, &lt;strong&gt;5.47 billion&lt;/strong&gt; across &lt;strong&gt;14,321 autonomous Improvement file examinations&lt;/strong&gt;, and &lt;strong&gt;0.39 billion&lt;/strong&gt; across &lt;strong&gt;722 Technical Writer and Lead Writer runs&lt;/strong&gt;.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Telemetry and fleet archive:&lt;/strong&gt; &lt;strong&gt;2,539,265 event frames&lt;/strong&gt; stored in a &lt;strong&gt;1.7 GB SQLite WAL database&lt;/strong&gt;, &lt;strong&gt;49,746 indexed pod trajectory bundles&lt;/strong&gt; (&lt;strong&gt;78 GB&lt;/strong&gt; on disk including retained conversation spools), &lt;strong&gt;15,565 deterministic CI job pods&lt;/strong&gt;, and &lt;strong&gt;9,340 CD build and deploy attempts&lt;/strong&gt; across &lt;strong&gt;1,856 CD runs&lt;/strong&gt;.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Closed work across 13 repositories:&lt;/strong&gt; &lt;strong&gt;7,370 closed issues&lt;/strong&gt; (with only &lt;strong&gt;10 open&lt;/strong&gt; across the entire fleet tonight), led by &lt;strong&gt;5,042 verified AST cognitive complexity reductions&lt;/strong&gt; (up &lt;strong&gt;2.8x&lt;/strong&gt; from 1,814 two weeks ago) and &lt;strong&gt;980 human Development features, stories, and bugs&lt;/strong&gt;.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;To keep that ledger grounded in reality, almost all of those 39.9 billion tokens are prompt cache reads running against flat monthly subscriptions and &lt;a href="https://jduncan.io/blog/2026-11-07-dark-factory-fleet-hardware/" rel="noopener noreferrer"&gt;palm-sized mini-PCs in my office closet&lt;/a&gt;. I did add a fifth worker box (&lt;code&gt;factory-node-5&lt;/code&gt;, 12 slots, 32 GB RAM, 500 GB NVMe) this week to bring the fleet to &lt;strong&gt;60 concurrent pod slots&lt;/strong&gt;, tagging both newer Zen 4 nodes with a &lt;code&gt;fast&lt;/code&gt; host label so &lt;strong&gt;84.4% of container image builds (448 of 531) and 61.2% of CI suites (1,173 of 1,916) land on modern silicon&lt;/strong&gt; while the 2019 Intel NUCs handle network-bound agent turns. Still, nobody is confusing my closet shelf with &lt;code&gt;us-east-1&lt;/code&gt;.&lt;/p&gt;

&lt;p&gt;Even on a closet shelf, two and a half million SQLite frames won't let you lie to yourself for long.&lt;/p&gt;

&lt;h2&gt;
  
  
  The nuance: catching my own lying denominator
&lt;/h2&gt;

&lt;p&gt;When I queried weekly rework rates tonight, the top-line number made me blink. Overall engineer rework across the factory had jumped from &lt;strong&gt;8.4%&lt;/strong&gt; (&lt;code&gt;2026-W43&lt;/code&gt;) and &lt;strong&gt;9.6%&lt;/strong&gt; (&lt;code&gt;2026-W44&lt;/code&gt;) all the way to &lt;strong&gt;24.1%&lt;/strong&gt; this week (&lt;code&gt;2026-W45&lt;/code&gt;). Back in October, I had bragged on a chart that Class 4 Development rework was sitting at a polite &lt;strong&gt;5.9%&lt;/strong&gt;.&lt;/p&gt;

&lt;p&gt;My first instinct was to blame a model regression. Instead, I checked the timestamps on the &lt;code&gt;send_back&lt;/code&gt; table in &lt;code&gt;factory.db&lt;/code&gt;:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight sql"&gt;&lt;code&gt;&lt;span class="k"&gt;SELECT&lt;/span&gt; &lt;span class="nb"&gt;MIN&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;at&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="k"&gt;AS&lt;/span&gt; &lt;span class="n"&gt;first_send_back&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
       &lt;span class="nb"&gt;MAX&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;at&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="k"&gt;AS&lt;/span&gt; &lt;span class="n"&gt;last_send_back&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
       &lt;span class="nb"&gt;COUNT&lt;/span&gt;&lt;span class="cm"&gt;(*) AS total_rejections,
       COUNT(DISTINCT room || ':' || pod) AS distinct_pods
  FROM send_back;
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;





&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;first_send_back           last_send_back            total_rejections  distinct_pods
------------------------  ------------------------  ----------------  -------------
2026-10-25T04:29:06.506Z  2026-11-12T22:41:21.090Z  550               532
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;That single row exposed two things at once.&lt;/p&gt;

&lt;p&gt;First, &lt;strong&gt;my October 29 post measured a 14-day window (October 15 to October 29), and &lt;code&gt;send_back&lt;/code&gt; didn't exist for the first ten days of that window&lt;/strong&gt;. Ten days of unrecorded zeros sat in the denominator and made early Development rework look artificially tiny. Inside Leaf's own &lt;code&gt;retro&lt;/code&gt; skill, we keep a list titled &lt;em&gt;"Columns that have lied"&lt;/em&gt; so post-mortem agents don't build arguments on schema migration artifacts. Tonight I tripped over my own rule.&lt;/p&gt;

&lt;p&gt;Second, &lt;strong&gt;the jump from 8.4% overall rework in late October to 24.1% this week was a workload mix shift, not a model collapse&lt;/strong&gt;. After Leaf closed its five-thousandth complexity reduction, I switched autonomous &lt;code&gt;Improve&lt;/code&gt; sweeps to &lt;code&gt;off&lt;/code&gt; across my repositories so the fleet could focus on feature work. In &lt;code&gt;W43&lt;/code&gt;, human Development nodes made up just &lt;strong&gt;15.5%&lt;/strong&gt; of engineer pods (&lt;code&gt;501 of 3,229&lt;/code&gt;). This week, Development made up &lt;strong&gt;56.8%&lt;/strong&gt; (&lt;code&gt;196 of 345&lt;/code&gt;).&lt;/p&gt;

&lt;h2&gt;
  
  
  Why 28.3% feature rework is the whole ballgame
&lt;/h2&gt;

&lt;p&gt;Once you filter strictly to runs started after &lt;code&gt;send_back&lt;/code&gt; went live on October 25 (&lt;code&gt;4,985&lt;/code&gt; planned engineer nodes across all five loops, excluding CI repair pods and retired September validators), the real picture snaps into focus:&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Flpqct64hbufnqku0m6d0.webp" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Flpqct64hbufnqku0m6d0.webp" alt="Leaf Dark Software Factory 64-day ledger (39.9B raw tokens, 4,044 completed build runs at 99.6% merge, 460.7K turns, 5,042 closed reductions) and horizontal bar chart comparing Tech Lead first-pass rejection rate against final unattended completion across 4,985 nodes since Oct 25: Improvement 5.5% rework and 99.5% done, Quality 16.3% rework and 100% done, Trajectory 18.9% rework and 100% done, and Development 28.3% rework and 99.5% done." width="800" height="467"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Look at how cleanly Tech Lead rejection rates track the amount of open judgement left to the builder pod:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Improvement (&lt;code&gt;3,764&lt;/code&gt; nodes, &lt;code&gt;5.5%&lt;/code&gt; rework, &lt;code&gt;99.5%&lt;/code&gt; completed):&lt;/strong&gt; Before an Improvement reduction ever reaches a builder pod, an examiner agent has already proven in a throwaway worktree that the test suite passes and &lt;a href="https://jduncan.io/blog/2026-10-29-moneyball-for-ai-agents/" rel="noopener noreferrer"&gt;Tree-sitter cognitive complexity strictly drops&lt;/a&gt;. Because &lt;strong&gt;the edit is mechanically pre-verified before planning starts, builders nail 19 out of 20 nodes on the first try&lt;/strong&gt;.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Quality (&lt;code&gt;49&lt;/code&gt; nodes, &lt;code&gt;16.3%&lt;/code&gt; rework, &lt;code&gt;100.0%&lt;/code&gt; completed) and Trajectory (&lt;code&gt;185&lt;/code&gt; nodes, &lt;code&gt;18.9%&lt;/code&gt; rework, &lt;code&gt;100.0%&lt;/code&gt; completed):&lt;/strong&gt; These bugs are filed by machines with exact file paths, symbol names, and reproduction steps, leaving moderate room for implementation drift.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Development (&lt;code&gt;981&lt;/code&gt; nodes, &lt;code&gt;28.3%&lt;/code&gt; rework, &lt;code&gt;99.5%&lt;/code&gt; completed):&lt;/strong&gt; Human features and bug reports ask engineers to wire new behavior across real modules. &lt;strong&gt;More than one in four engineer diffs (&lt;code&gt;278 of 981&lt;/code&gt;) gets rejected by the Tech Lead on the first pass&lt;/strong&gt;, yet &lt;strong&gt;99.5% of those nodes still finish green and merge unattended&lt;/strong&gt;.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;A 28.3% rejection rate sounds terrible if you judge an agent factory like a typing contest.&lt;br&gt;&lt;br&gt;
It looks completely different when you remember nobody human is reading the diffs.&lt;/p&gt;

&lt;p&gt;Think about what happens in a standard "autonomous PR" product that spawns one coding agent, runs &lt;code&gt;npm test&lt;/code&gt;, and opens a pull request for a human teammate to review. Every engineer pod in Leaf &lt;em&gt;already&lt;/em&gt; had to pass its scoped test check before it was even allowed to report &lt;code&gt;done&lt;/code&gt; to the Tech Lead (&lt;a href="https://jduncan.io/blog/2026-10-10-loops-part-5-quality-without-ceremony/" rel="noopener noreferrer"&gt;the single rule that pushed early pass rates above 97%&lt;/a&gt;). &lt;strong&gt;All 278 of those rejected Development diffs already had green unit tests.&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;The Tech Lead sent them back anyway because &lt;strong&gt;unit tests only prove what the test author thought to assert, whereas adversarial review checks the diff against the contract&lt;/strong&gt;: files touched outside the node's ownership boundary, missing edge cases named in &lt;code&gt;done_when&lt;/code&gt;, dead helpers left behind, or shortcuts that satisfied a mock without wiring the real route.&lt;/p&gt;

&lt;p&gt;Green tests prove the code runs.&lt;br&gt;&lt;br&gt;
Adversarial review proves the contract held.&lt;/p&gt;

&lt;p&gt;If that Tech Lead seat were a rubber stamp, 28.3% of our feature nodes would have shipped silent contract drift straight into main (or dumped 278 sloppy diffs onto my desk, turning me right back into a full-time code reviewer). Instead, the Tech Lead rejected the branch with line-level defects, the engineer fixed the gap in the next turn, the deterministic repository CI suite verified the integrated branch (&lt;code&gt;548 of 565&lt;/code&gt; CI runs green over the last seven days, &lt;strong&gt;97.0%&lt;/strong&gt;), and &lt;strong&gt;4,026 of 4,044 build runs landed cleanly&lt;/strong&gt;.&lt;/p&gt;

&lt;h2&gt;
  
  
  Next in this series
&lt;/h2&gt;

&lt;p&gt;Pulling this 40-billion-token snapshot uncovered two other stories big enough to deserve their own space today:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;
&lt;strong&gt;&lt;a href="https://jduncan.io/blog/2026-11-12-best-architect-model-isnt-best-builder/" rel="noopener noreferrer"&gt;Why the Best AI Architect Isn't the Best Coding Builder&lt;/a&gt;:&lt;/strong&gt; How randomizing Class 5 quota across 1,199 planning rooms accidentally ran a head-to-head trial between &lt;code&gt;Claude Opus 5.5&lt;/code&gt; and &lt;code&gt;Codex GPT-6.1-Sol&lt;/code&gt;, and why letting an Architect override historical precedent doubled rework.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;&lt;a href="https://jduncan.io/blog/2026-11-12-checking-dark-factory-receipts-four-thousand-runs/" rel="noopener noreferrer"&gt;Checking My Dark Factory's Receipts After 4,000 Runs&lt;/a&gt;:&lt;/strong&gt; Eight days of full-fleet data after &lt;a href="https://jduncan.io/blog/2026-11-04-pruning-agent-prompts-dark-factory/" rel="noopener noreferrer"&gt;pruning 65% of our prompts&lt;/a&gt; and &lt;a href="https://jduncan.io/blog/2026-11-06-teaching-my-dark-factory-to-dream/" rel="noopener noreferrer"&gt;teaching Leaf to dream&lt;/a&gt;, including a textbook Simpson's paradox that had me chasing a phantom regression.&lt;/li&gt;
&lt;/ol&gt;

&lt;h2&gt;
  
  
  So what?
&lt;/h2&gt;

&lt;p&gt;If you're tracking throughput and quality in your own multi-agent workflows, keep three lessons from this ledger in mind:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Audit the start dates on your telemetry joins.&lt;/strong&gt; When you add a new table like &lt;code&gt;send_back&lt;/code&gt; mid-month, any rolling window that crosses the migration date pads your denominator with zeros. Keep a written list of columns and date boundaries that have lied in your schema.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Separate mechanical cleanup from feature work in every rollup.&lt;/strong&gt; Blending 3,764 pre-proven AST reductions (&lt;code&gt;5.5%&lt;/code&gt; rework) with 981 human feature nodes (&lt;code&gt;28.3%&lt;/code&gt; rework) hides what your agents actually do when a human asks for new behavior.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Treat reviewer send-backs as proof of friction, not a vanity metric to push to zero.&lt;/strong&gt; A 0% rejection rate in an unattended loop doesn't mean your builders are flawless; it usually means your reviewer is asleep. What matters is the pair of numbers together: &lt;strong&gt;how often the counterweight pushes back (&lt;code&gt;28.3%&lt;/code&gt;), and how often the room still ships working software without waking up a human (&lt;code&gt;99.5%&lt;/code&gt;)&lt;/strong&gt;.&lt;/li&gt;
&lt;/ul&gt;

</description>
      <category>darkfactory</category>
      <category>softwarefactory</category>
      <category>leaf</category>
      <category>agenticsdlc</category>
    </item>
    <item>
      <title>Inside my Dark Factory hardware stack: 48 Agent Pods on 2019 Mini-PCs</title>
      <dc:creator>Jamie Duncan</dc:creator>
      <pubDate>Fri, 20 Nov 2026 05:51:12 +0000</pubDate>
      <link>https://dev.to/jamie_duncan_3738b0805ed5/inside-my-dark-factory-hardware-stack-48-agent-pods-on-2019-mini-pcs-7b5</link>
      <guid>https://dev.to/jamie_duncan_3738b0805ed5/inside-my-dark-factory-hardware-stack-48-agent-pods-on-2019-mini-pcs-7b5</guid>
      <description>&lt;p&gt;&lt;em&gt;Originally published at &lt;a href="https://jduncan.io/blog/2026-11-07-dark-factory-fleet-hardware/" rel="noopener noreferrer"&gt;jduncan.io&lt;/a&gt;.&lt;/em&gt;&lt;/p&gt;

&lt;p&gt;When people hear that &lt;a href="https://jduncan.io/blog/2026-10-25-leaf-dark-factory-update/" rel="noopener noreferrer"&gt;Leaf, my local Level 5 dark software factory&lt;/a&gt;, runs &lt;strong&gt;48 concurrent pod slots&lt;/strong&gt; shipping features, enforcing adversarial CI/CD gates, and &lt;a href="https://jduncan.io/blog/2026-11-06-teaching-my-dark-factory-to-dream/" rel="noopener noreferrer"&gt;dreaming over its own telemetry overnight&lt;/a&gt;, they usually picture a loud rack of enterprise servers in my garage or a four-figure monthly cloud bill.&lt;/p&gt;

&lt;p&gt;Today I queried the live fleet state from &lt;code&gt;apps:4600/api/state&lt;/code&gt; and SSH'd into every host on my tailnet running &lt;code&gt;factory-hostd&lt;/code&gt; (all on commit &lt;code&gt;9356bac&lt;/code&gt;) to inventory the hardware and benchmark the nodes while they ran live pods.&lt;/p&gt;

&lt;p&gt;Three-quarters of my factory's pod capacity runs on &lt;strong&gt;seven-year-old 2019 Intel NUCs&lt;/strong&gt;.&lt;/p&gt;

&lt;h2&gt;
  
  
  The nuance: agent factories are timer and I/O bound, not CPU bound
&lt;/h2&gt;

&lt;p&gt;We talk about AI software engineering as if every layer of the stack needs bleeding-edge silicon. That narrative gets the workload backwards.&lt;/p&gt;

&lt;p&gt;Unless you are serving local model weights on the same box (my local LLM server, Home Assistant box, and Mac Mini sit on the tailnet without &lt;code&gt;factory-hostd&lt;/code&gt;), an unattended coding pod spends almost its entire life waiting. It waits on frontier model API responses over the network, it waits on Node and container process startup, it waits on small filesystem I/O, and it waits on test timers.&lt;/p&gt;

&lt;p&gt;Across my entire fleet, &lt;strong&gt;CPU pressure on every node is about 1% of the time since boot&lt;/strong&gt;.&lt;/p&gt;

&lt;p&gt;When I benchmarked my newest 2024 AMD Zen 4 mini-PC (&lt;code&gt;factory-node-4&lt;/code&gt;) against my 2019 Intel Comet Lake NUCs (&lt;code&gt;factory-node-1&lt;/code&gt; through &lt;code&gt;3&lt;/code&gt;) today, the Zen 4 chip beat the NUC by &lt;strong&gt;2.08x on a single core&lt;/strong&gt; and &lt;strong&gt;2.59x across the whole host&lt;/strong&gt;. When I ran Factory's actual test suite inside a live pod image on both machines, that 2.59x hardware lead collapsed to &lt;strong&gt;1.17x&lt;/strong&gt; (&lt;code&gt;104 seconds&lt;/code&gt; vs. &lt;code&gt;122 seconds&lt;/code&gt;).&lt;/p&gt;

&lt;p&gt;Buying twice the clock speed buys you almost nothing when your pods spend 99% of their lives waiting on timers and sockets.&lt;/p&gt;

&lt;h2&gt;
  
  
  Six hosts, four CPU generations, 48 pod slots
&lt;/h2&gt;

&lt;p&gt;Six palm-sized mini-PCs on my tailnet run &lt;code&gt;factory-hostd&lt;/code&gt;. Four of them take scheduled pods; the other two run stateful services with &lt;code&gt;SLOTS=0&lt;/code&gt; and stay connected to &lt;code&gt;hostd&lt;/code&gt; strictly for automated deployments and secret distribution.&lt;/p&gt;

&lt;p&gt;Pulling the live state on November 7, 2026 (&lt;code&gt;commit 9356bac&lt;/code&gt;) shows the exact footprint of the fleet:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;Fleet snapshot (apps:4600/api/state @ commit 9356bac)
-----------------------------------------------------
Connected hosts : 6 (Ubuntu 26.04 LTS, Linux 7.0.0-31 / -34 / -38)
Pod slots       : 48 total (33 in use at snapshot)
CPU threads     : 72 total (52 schedulable)
Memory          : 168 GB installed (128 GB schedulable)
NVMe storage    : 2.3 TB installed (1.3 TB schedulable)
CPU generations : 4 (spanning 2019 to 2024)
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h3&gt;
  
  
  The four schedulable pod nodes (&lt;code&gt;SLOTS=12&lt;/code&gt;)
&lt;/h3&gt;

&lt;p&gt;Every schedulable node exposes 12 pod slots to run engineer agents, planning rooms, test suites, container builds, and continuous delivery tasks:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;&lt;code&gt;factory-node-1&lt;/code&gt;, &lt;code&gt;factory-node-2&lt;/code&gt;, and &lt;code&gt;factory-node-3&lt;/code&gt; (2019 Intel Comet Lake, 14 nm):&lt;/strong&gt; Three identical Intel &lt;code&gt;NUC10i7FNH&lt;/code&gt; mini-PCs. Each runs a 15W Core i7-10710U (6 cores / 12 threads, up to 4.7 GHz), 32 GB of DDR4-2667 (&lt;code&gt;2x16 GB&lt;/code&gt;), and a 250 GB Samsung 970 EVO Plus NVMe. At the time of the snapshot, &lt;code&gt;node-1&lt;/code&gt; held 8 active pods (&lt;code&gt;12 / 12&lt;/code&gt; slots), &lt;code&gt;node-2&lt;/code&gt; held 3 pods (&lt;code&gt;7 / 12&lt;/code&gt; slots), and &lt;code&gt;node-3&lt;/code&gt; held 5 pods (&lt;code&gt;8 / 12&lt;/code&gt; slots). Together, these three 2019 boxes carry &lt;strong&gt;36 of the 48 pod slots (75% of the fleet)&lt;/strong&gt;.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;&lt;code&gt;factory-node-4&lt;/code&gt; (2024 AMD Zen 4 Hawk Point, 4 nm):&lt;/strong&gt; A GMKtec NucBox K8 Plus I bought after &lt;a href="https://jduncan.io/blog/2026-10-20-where-my-level-5-software-factory-doesnt-cut-it-yet/" rel="noopener noreferrer"&gt;blowing through 50,000 GitHub Actions minutes in October&lt;/a&gt;. It runs a 45W Ryzen 7 8845HS (8 cores / 16 threads, up to 5.1 GHz), 32 GB of DDR5-5600 (&lt;code&gt;2x16 GB&lt;/code&gt;), and a 512 GB AirDisk NVMe (&lt;code&gt;203 / 500 GB&lt;/code&gt; free). During the snapshot it was running 4 pods (&lt;code&gt;6 / 12&lt;/code&gt; slots), accounting for the remaining &lt;strong&gt;25% of pod capacity&lt;/strong&gt;.&lt;/li&gt;
&lt;/ul&gt;

&lt;h3&gt;
  
  
  The two service hosts (&lt;code&gt;SLOTS=0&lt;/code&gt;)
&lt;/h3&gt;

&lt;p&gt;Keeping stateful control-plane and production services off the pod scheduler prevents a runaway build from starving the factory supervisor or a live database:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;&lt;code&gt;apps&lt;/code&gt; (2023 AMD Zen 3 Barcelo-R, 7 nm):&lt;/strong&gt; A GMKtec M5 Ultra-R running a 15W Ryzen 7 7730U (8 cores / 16 threads, 4.5 GHz), 32 GB of DDR4-3200, and a 512 GB GMK NVMe (&lt;code&gt;281 / 500 GB&lt;/code&gt; free). This is the Factory host itself: it runs the Leaf supervisor, &lt;code&gt;factory-git&lt;/code&gt;, &lt;code&gt;bark-dev&lt;/code&gt;, &lt;code&gt;binpacker-dev&lt;/code&gt;, Sadie and its Actions runner, Immich, Music Assistant, Kokoro, Caddy, and &lt;code&gt;cloudflared&lt;/code&gt;.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;&lt;code&gt;binpacker&lt;/code&gt; (2019 AMD Zen+ Picasso, 12 nm):&lt;/strong&gt; An AZW SER3 with a 15W Ryzen 3 3200U (2 cores / 4 threads, 2.6 GHz), a single 8 GB stick of DDR4-2400 (&lt;code&gt;5.6 GiB&lt;/code&gt; visible to Linux), and a 480 GB Crucial E100 NVMe (&lt;code&gt;424 / 468 GB&lt;/code&gt; free). It serves production &lt;code&gt;binpacker-server&lt;/code&gt; and &lt;code&gt;binpacker-postgres&lt;/code&gt; for my wife's kids' clothing resale store, reached through &lt;code&gt;cloudflared&lt;/code&gt; at &lt;code&gt;garner-ssh.binpacker.net&lt;/code&gt;.&lt;/li&gt;
&lt;/ul&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Generation&lt;/th&gt;
&lt;th&gt;Year / Node&lt;/th&gt;
&lt;th&gt;Hosts&lt;/th&gt;
&lt;th&gt;Threads&lt;/th&gt;
&lt;th&gt;RAM&lt;/th&gt;
&lt;th&gt;Memory Type&lt;/th&gt;
&lt;th&gt;Slots&lt;/th&gt;
&lt;th&gt;Share of Slots&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Intel 10th Gen (Comet Lake)&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;2019 · 14 nm · 15W&lt;/td&gt;
&lt;td&gt;
&lt;code&gt;node-1&lt;/code&gt;, &lt;code&gt;node-2&lt;/code&gt;, &lt;code&gt;node-3&lt;/code&gt;
&lt;/td&gt;
&lt;td&gt;36&lt;/td&gt;
&lt;td&gt;96 GB&lt;/td&gt;
&lt;td&gt;DDR4-2667&lt;/td&gt;
&lt;td&gt;36&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;75%&lt;/strong&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;AMD Zen 4 (Hawk Point)&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;2024 · 4 nm · 45W&lt;/td&gt;
&lt;td&gt;&lt;code&gt;node-4&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;16&lt;/td&gt;
&lt;td&gt;32 GB&lt;/td&gt;
&lt;td&gt;DDR5-5600&lt;/td&gt;
&lt;td&gt;12&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;25%&lt;/strong&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;AMD Zen 3 (Barcelo-R)&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;2023 · 7 nm · 15W&lt;/td&gt;
&lt;td&gt;&lt;code&gt;apps&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;16&lt;/td&gt;
&lt;td&gt;32 GB&lt;/td&gt;
&lt;td&gt;DDR4-3200&lt;/td&gt;
&lt;td&gt;0&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;0%&lt;/strong&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;AMD Zen+ (Picasso)&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;2019 · 12 nm · 15W&lt;/td&gt;
&lt;td&gt;&lt;code&gt;binpacker&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;4&lt;/td&gt;
&lt;td&gt;8 GB&lt;/td&gt;
&lt;td&gt;DDR4-2400&lt;/td&gt;
&lt;td&gt;0&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;0%&lt;/strong&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;h2&gt;
  
  
  Where the real limits hide: iGPU carve-outs and the 20 GiB disk floor
&lt;/h2&gt;

&lt;p&gt;When you run 48 container slots across consumer mini-PCs, the operational walls you hit have nothing to do with CPU Geekbench scores. They come down to &lt;strong&gt;RAM per slot&lt;/strong&gt; and &lt;strong&gt;NVMe free-space floors&lt;/strong&gt;.&lt;/p&gt;

&lt;p&gt;First, &lt;strong&gt;an iGPU BIOS reservation makes my newest DDR5 machine the tightest node on per-slot memory&lt;/strong&gt;. Because every schedulable host is configured with &lt;code&gt;SLOTS=12&lt;/code&gt;, the 16-thread Zen 4 box (&lt;code&gt;node-4&lt;/code&gt;) carries a relaxed &lt;strong&gt;0.75 slots per CPU thread&lt;/strong&gt;, compared to &lt;strong&gt;1.0 slot per thread&lt;/strong&gt; on the 12-thread Intel NUCs. However, &lt;code&gt;factory-hostd&lt;/code&gt; divides usable host RAM by slot count to enforce per-slot memory reservations. Because &lt;code&gt;node-4&lt;/code&gt;'s integrated Radeon graphics carves out &lt;strong&gt;3 GiB of DDR5&lt;/strong&gt; at boot (leaving 27 GiB visible to the kernel), each slot on the brand-new Zen 4 box gets &lt;strong&gt;2.3 GiB of RAM&lt;/strong&gt;, whereas each slot on a 2019 Intel NUC gets &lt;strong&gt;2.5 GiB&lt;/strong&gt;.&lt;/p&gt;

&lt;p&gt;Second, &lt;strong&gt;250 GB NVMe drives fill up with container layers and git worktrees long before 10th-gen CPUs run out of breath&lt;/strong&gt;. All three Intel NUCs still run the 250 GB Samsung 970 EVO Plus drives they shipped with (&lt;code&gt;242 GB&lt;/code&gt; usable filesystem capacity). When I pulled the fleet state today:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;code&gt;factory-node-2&lt;/code&gt; had &lt;strong&gt;76 GB free&lt;/strong&gt; (&lt;code&gt;76 / 242 GB&lt;/code&gt;).&lt;/li&gt;
&lt;li&gt;
&lt;code&gt;factory-node-1&lt;/code&gt; had &lt;strong&gt;50 GB free&lt;/strong&gt; (&lt;code&gt;50 / 242 GB&lt;/code&gt;).&lt;/li&gt;
&lt;li&gt;
&lt;code&gt;factory-node-3&lt;/code&gt; was down to &lt;strong&gt;27 GB free&lt;/strong&gt; (&lt;code&gt;27 / 242 GB&lt;/code&gt;).&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;That 27 GB figure on &lt;code&gt;node-3&lt;/code&gt; is the closest thing to a red light on my dashboard. &lt;code&gt;factory-hostd&lt;/code&gt; enforces a hard &lt;strong&gt;20 GiB free-disk floor&lt;/strong&gt;: if free space drops below 20 GiB, the daemon refuses to accept new pods until completed rooms prune their workspaces. &lt;code&gt;node-3&lt;/code&gt; was sitting just &lt;strong&gt;7 GB above the scheduling cutoff&lt;/strong&gt; while its CPU idled.&lt;/p&gt;

&lt;h2&gt;
  
  
  Benchmarking 2019 vs. 2024 under live pod load
&lt;/h2&gt;

&lt;p&gt;To see what five years of silicon progress actually buys inside the factory, I benchmarked all six hosts today while they ran their normal live pod load. I took the best run on each host and normalized every score against the 2019 Comet Lake NUC (&lt;code&gt;1.00x&lt;/code&gt;).&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fcyx7e4levba3nhr8lesx.webp" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fcyx7e4levba3nhr8lesx.webp" alt="Dark Factory Fleet benchmark chart comparing 2019 Intel Comet Lake NUCs (1.00x baseline) against 2024 AMD Zen 4 (node-4), 2023 Zen 3 (apps), and 2019 Zen+ (binpacker): Zen 4 is 2.08x faster on a single-core Python loop and 2.59x faster across the whole host, but only 1.17x faster (104s vs 122s) running Factory's full npm test suite inside a pod because node CPU pressure is ~1% and wall time is dominated by timers and I/O." width="799" height="508"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Look at the three tiers of that test:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;
&lt;strong&gt;Single-core synthetic CPU (&lt;code&gt;Python 3.14&lt;/code&gt; loop of integer math, dictionary building, and sorting; NUC baseline = &lt;code&gt;0.60s&lt;/code&gt;):&lt;/strong&gt; A single 4 nm Zen 4 core on &lt;code&gt;node-4&lt;/code&gt; is &lt;strong&gt;2.08x faster&lt;/strong&gt; than a 14 nm Comet Lake core. The Zen 3 core on &lt;code&gt;apps&lt;/code&gt; comes in at &lt;strong&gt;1.47x&lt;/strong&gt;, and the two-core Zen+ chip in &lt;code&gt;binpacker&lt;/code&gt; trails slightly at &lt;strong&gt;0.89x&lt;/strong&gt;.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Whole-host synthetic throughput (running one Python loop process per hardware thread; NUC baseline = &lt;code&gt;7.8 jobs/s&lt;/code&gt; across 12 threads):&lt;/strong&gt; Combining faster cores with 16 threads and DDR5-5600 pushes &lt;code&gt;node-4&lt;/code&gt; to &lt;strong&gt;2.59x the throughput of a NUC&lt;/strong&gt;. Meanwhile, &lt;code&gt;binpacker&lt;/code&gt; (4 threads) drops to &lt;strong&gt;0.38x&lt;/strong&gt;.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Real factory workload (the full &lt;code&gt;npm test&lt;/code&gt; suite at commit &lt;code&gt;9356bac&lt;/code&gt; inside the &lt;code&gt;factory-node-24&lt;/code&gt; container image; NUC baseline = &lt;code&gt;122s&lt;/code&gt;):&lt;/strong&gt; On the workload the fleet actually exists to run, &lt;code&gt;node-4&lt;/code&gt; finishes in &lt;strong&gt;104 seconds&lt;/strong&gt; against the NUC's &lt;strong&gt;122 seconds&lt;/strong&gt;, a speedup of just &lt;strong&gt;1.17x&lt;/strong&gt;.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;Two methodology wrinkles are worth calling out before anyone quotes those numbers out of context:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;&lt;code&gt;apps&lt;/code&gt; ran with a load average of 13 during the benchmark.&lt;/strong&gt; Because I measured the fleet live rather than shutting down production, &lt;code&gt;apps&lt;/code&gt; was simultaneously running the Factory supervisor, &lt;code&gt;factory-git&lt;/code&gt;, Immich, Sadie, and six other services. That background load explains why a 16-thread Zen 3 box scored &lt;code&gt;1.24x&lt;/code&gt; on the whole-host test and &lt;code&gt;0.94x&lt;/code&gt; (&lt;code&gt;130 seconds&lt;/code&gt;) on the test suite.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;I threw out the OpenSSL &lt;code&gt;sha256&lt;/code&gt; numbers.&lt;/strong&gt; When I first checked &lt;code&gt;openssl speed sha256&lt;/code&gt;, the AMD hosts looked &lt;strong&gt;3x to 13x faster&lt;/strong&gt; than the Intel NUCs. That gap has nothing to do with general compute speed: &lt;strong&gt;AMD Zen processors include hardware SHA-NI instructions, while 10th-gen Intel Comet Lake chips omitted them&lt;/strong&gt;. Including that number on a chart would just measure a missing crypto instruction set.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Even with a clean run on &lt;code&gt;node-4&lt;/code&gt;, &lt;strong&gt;a 2.59x whole-host CPU advantage shrinks to 1.17x on a real test suite because test wall time is governed by timers, filesystem I/O, and process startup rather than raw arithmetic&lt;/strong&gt;.&lt;/p&gt;

&lt;p&gt;In a synthetic loop, the CPU is pinned at 100%.&lt;br&gt;&lt;br&gt;
In an agentic factory, the CPU is waiting on the world 99% of the time.&lt;/p&gt;

&lt;h2&gt;
  
  
  Pricing the same six boxes on AWS ($20k to $33k a year)
&lt;/h2&gt;

&lt;p&gt;When I moved git, CI, CD, and agent pods onto this shelf in October, my goal was escaping per-minute cloud CI billing and unifying telemetry. Out of curiosity, I priced out what it would cost to rent this exact six-host footprint (&lt;code&gt;72 threads&lt;/code&gt;, &lt;code&gt;168 GB RAM&lt;/code&gt;, &lt;code&gt;2,254 GB&lt;/code&gt; of NVMe storage, and &lt;code&gt;6 public IPv4 addresses&lt;/code&gt;) on AWS EC2 in &lt;code&gt;us-east-1&lt;/code&gt; at Linux on-demand list prices (&lt;code&gt;730 hours/month&lt;/code&gt;).&lt;/p&gt;

&lt;p&gt;Because EC2 does not offer a 6-core / 12-thread instance shape to match the Intel NUCs, a fair comparison requires a range:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;The low match (&lt;code&gt;$1,688.86/month&lt;/code&gt;)&lt;/strong&gt; matches the NUCs' &lt;strong&gt;32 GiB of RAM&lt;/strong&gt; using &lt;code&gt;m6i.2xlarge&lt;/code&gt; instances (&lt;code&gt;8 vCPU&lt;/code&gt;, or 4 physical cores, vs. the NUC's 6 physical cores).&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;The high match (&lt;code&gt;$2,782.11/month&lt;/code&gt;)&lt;/strong&gt; covers the thread count using &lt;code&gt;m6i.4xlarge&lt;/code&gt; instances (&lt;code&gt;16 vCPU&lt;/code&gt; and &lt;code&gt;64 GiB&lt;/code&gt; of RAM, double the NUC's memory).&lt;/li&gt;
&lt;li&gt;For &lt;code&gt;factory-node-4&lt;/code&gt;, an &lt;code&gt;m7a.2xlarge&lt;/code&gt; (&lt;code&gt;$338.49/month&lt;/code&gt;) is an exact match in both columns because &lt;strong&gt;AWS disables SMT on 4th-gen AMD &lt;code&gt;m7a&lt;/code&gt; instances, so every &lt;code&gt;m7a&lt;/code&gt; vCPU is a full physical Zen 4 core&lt;/strong&gt;.&lt;/li&gt;
&lt;/ul&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Fleet Host(s)&lt;/th&gt;
&lt;th&gt;AWS Low Match (&lt;code&gt;us-east-1&lt;/code&gt; On-Demand)&lt;/th&gt;
&lt;th&gt;Low $/mo&lt;/th&gt;
&lt;th&gt;AWS High Match (&lt;code&gt;us-east-1&lt;/code&gt; On-Demand)&lt;/th&gt;
&lt;th&gt;High $/mo&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;&lt;code&gt;factory-node-1, 2, 3&lt;/code&gt; (x3)&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;3x &lt;code&gt;m6i.2xlarge&lt;/code&gt; (Ice Lake, 8 vCPU, 32 GiB @ &lt;code&gt;$0.384/h&lt;/code&gt;)&lt;/td&gt;
&lt;td&gt;&lt;code&gt;$840.96&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;3x &lt;code&gt;m6i.4xlarge&lt;/code&gt; (16 vCPU, 64 GiB @ &lt;code&gt;$0.768/h&lt;/code&gt;)&lt;/td&gt;
&lt;td&gt;&lt;code&gt;$1,681.92&lt;/code&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;&lt;code&gt;factory-node-4&lt;/code&gt;&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;
&lt;code&gt;m7a.2xlarge&lt;/code&gt; (Zen 4, 8 cores, 32 GiB @ &lt;code&gt;$0.46368/h&lt;/code&gt;)&lt;/td&gt;
&lt;td&gt;&lt;code&gt;$338.49&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Same (&lt;code&gt;m7a&lt;/code&gt; vCPU = 1 full core)&lt;/td&gt;
&lt;td&gt;&lt;code&gt;$338.49&lt;/code&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;&lt;code&gt;apps&lt;/code&gt;&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;
&lt;code&gt;m6a.2xlarge&lt;/code&gt; (Zen 3, 8 vCPU, 32 GiB @ &lt;code&gt;$0.3456/h&lt;/code&gt;)&lt;/td&gt;
&lt;td&gt;&lt;code&gt;$252.29&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;
&lt;code&gt;m6a.4xlarge&lt;/code&gt; (16 vCPU, 64 GiB @ &lt;code&gt;$0.6912/h&lt;/code&gt;)&lt;/td&gt;
&lt;td&gt;&lt;code&gt;$504.58&lt;/code&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;&lt;code&gt;binpacker&lt;/code&gt;&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;
&lt;code&gt;t3a.large&lt;/code&gt; (2 vCPU, 8 GiB @ &lt;code&gt;$0.0752/h&lt;/code&gt;)&lt;/td&gt;
&lt;td&gt;&lt;code&gt;$54.90&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Same&lt;/td&gt;
&lt;td&gt;&lt;code&gt;$54.90&lt;/code&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;NVMe Disks&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;
&lt;code&gt;2,254 GB&lt;/code&gt; gp3 @ &lt;code&gt;$0.08/GB-month&lt;/code&gt;
&lt;/td&gt;
&lt;td&gt;&lt;code&gt;$180.32&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Same&lt;/td&gt;
&lt;td&gt;&lt;code&gt;$180.32&lt;/code&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Public IPv4&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;6 addresses @ &lt;code&gt;$0.005/h&lt;/code&gt;
&lt;/td&gt;
&lt;td&gt;&lt;code&gt;$21.90&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Same&lt;/td&gt;
&lt;td&gt;&lt;code&gt;$21.90&lt;/code&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;All 6 Hosts Total&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;$20,266 / year&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;$1,688.86&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;$33,385 / year&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;$2,782.11&lt;/strong&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;4 Pod Nodes Only&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;&lt;code&gt;node-1..4&lt;/code&gt; + 1.25 TB gp3 + 4 IPv4&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;$1,295.00&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;&lt;code&gt;node-1..4&lt;/code&gt; + 1.25 TB gp3 + 4 IPv4&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;$2,136.00&lt;/strong&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;Like any cloud cost table, this one needs honest caveats on both sides of the ledger.&lt;/p&gt;

&lt;p&gt;On one side, &lt;strong&gt;that $20k to $33k annual total leaves out the line items that usually blow up a real AWS bill&lt;/strong&gt;: outbound data transfer, EBS snapshots and backups, a NAT gateway, and the Tailscale or Cloudflare routing layer.&lt;/p&gt;

&lt;p&gt;On the other side, &lt;strong&gt;no pragmatic infrastructure team pays retail on-demand rates for 24/7 base capacity&lt;/strong&gt;. A 1-year Compute Savings Plan or EC2 Spot instances cut those numbers substantially. Right now, Spot pricing for those four pod nodes sits around &lt;strong&gt;45% of on-demand&lt;/strong&gt; (roughly &lt;strong&gt;$580 to $960 a month&lt;/strong&gt; for the pod tier alone), with the obvious catch that AWS can reclaim a Spot node in the middle of a multi-agent build room.&lt;/p&gt;

&lt;p&gt;Still, even at a 55% Spot discount, &lt;strong&gt;a single month of cloud rent for one node pays for a used 2019 Intel NUC outright&lt;/strong&gt;. And so far, Duke Energy hasn't called to renegotiate my closet's enterprise commitment tier.&lt;/p&gt;

&lt;h2&gt;
  
  
  So what?
&lt;/h2&gt;

&lt;p&gt;If you are building or scaling an unattended agent harness of your own, the hardware data from Leaf points to three practical rules:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Buy RAM density and NVMe headroom, not single-core clock speed.&lt;/strong&gt; When your pods call hosted frontier models over HTTPS, your local compute fleet is an I/O and process-spawn multiplexer running at ~1% CPU pressure. Spend your hardware budget on 32 GB to 64 GB RAM kits and 1 TB+ NVMe drives so container layers and git worktrees never flirt with a 20 GiB disk floor.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Isolate your control plane from your pod slots.&lt;/strong&gt; Running &lt;code&gt;apps&lt;/code&gt; and &lt;code&gt;binpacker&lt;/code&gt; with &lt;code&gt;SLOTS=0&lt;/code&gt; means &lt;code&gt;factory-hostd&lt;/code&gt; can still deploy containers and rotate secrets on them, while noisy &lt;code&gt;npm test&lt;/code&gt; suites stay fenced onto the four &lt;code&gt;SLOTS=12&lt;/code&gt; worker boxes.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Benchmark your actual container suite before retiring old hardware.&lt;/strong&gt; If I had looked at a 2.59x synthetic host benchmark or a 13x OpenSSL hash test, I would have written off my 2019 Comet Lake NUCs as obsolete e-waste. Against a real pod test suite, those seven-year-old boxes deliver &lt;strong&gt;85% of the wall-clock speed of a 2024 Zen 4 machine&lt;/strong&gt; (&lt;code&gt;122s&lt;/code&gt; vs. &lt;code&gt;104s&lt;/code&gt;) and quietly carry three-quarters of my factory.&lt;/li&gt;
&lt;/ul&gt;

</description>
      <category>darkfactory</category>
      <category>softwarefactory</category>
      <category>leaf</category>
      <category>baremetal</category>
    </item>
    <item>
      <title>Teaching My Dark Factory to Dream: From Agent Telemetry to Self-Healing Code</title>
      <dc:creator>Jamie Duncan</dc:creator>
      <pubDate>Fri, 20 Nov 2026 05:51:10 +0000</pubDate>
      <link>https://dev.to/jamie_duncan_3738b0805ed5/teaching-my-dark-factory-to-dream-from-agent-telemetry-to-self-healing-code-1mng</link>
      <guid>https://dev.to/jamie_duncan_3738b0805ed5/teaching-my-dark-factory-to-dream-from-agent-telemetry-to-self-healing-code-1mng</guid>
      <description>&lt;p&gt;&lt;em&gt;Originally published at &lt;a href="https://jduncan.io/blog/2026-11-06-teaching-my-dark-factory-to-dream/" rel="noopener noreferrer"&gt;jduncan.io&lt;/a&gt;.&lt;/em&gt;&lt;/p&gt;

&lt;p&gt;Tonight at 9:25 PM Eastern (&lt;code&gt;02:25:53 UTC&lt;/code&gt;), an unattended simplification run named &lt;code&gt;binpacker765&lt;/code&gt; finished inside &lt;a href="https://jduncan.io/blog/2026-10-25-leaf-dark-factory-update/" rel="noopener noreferrer"&gt;Leaf, my local Level 5 dark software factory&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;BinPacker&lt;/strong&gt; is a small inventory management app I built for my wife's kids' clothing resale store. It tracks plastic storage bins, seasonal sizing, and pull actions when items move from storage to the floor. Earlier tonight, Leaf's background &lt;a href="https://jduncan.io/blog/2026-10-29-moneyball-for-ai-agents/" rel="noopener noreferrer"&gt;Improvement loop&lt;/a&gt; bundled five complexity reductions across BinPacker's React hooks and notification components into a single plan and handed it to an engineer pod.&lt;/p&gt;

&lt;p&gt;The engineer pod simplified all five files, passed the repository's test suite and AST complexity gate, and merged its branch. In almost every agentic coding setup, a green merge is where the story ends.&lt;/p&gt;

&lt;p&gt;In Leaf, closing the room set off a chain reaction while I watched from the dashboard:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;&lt;code&gt;02:25:53 UTC&lt;/code&gt; (&lt;code&gt;binpacker765&lt;/code&gt; closes):&lt;/strong&gt; When the room tore down, Leaf's history recorder checked the pod's numbers against the repository's 30-day baseline. The engineer pod had taken &lt;strong&gt;95 turns&lt;/strong&gt; over 13 minutes. In &lt;code&gt;binpacker&lt;/code&gt;, the 30-day 90th percentile (&lt;code&gt;P90&lt;/code&gt;) for an engineer pod is &lt;strong&gt;19 turns&lt;/strong&gt;. Leaf stamped the closed room with a &lt;code&gt;turns&lt;/code&gt; flag and placed it in the telemetry queue.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;&lt;code&gt;02:26:44 UTC&lt;/code&gt; (The Analyst wakes up):&lt;/strong&gt; Fifty-one seconds later, Leaf opened an analysis room and spawned a Class 5 &lt;code&gt;analyst&lt;/code&gt; pod. Its job wasn't to grade the code. Its job was to read the room log and the pod's compressed trajectory archive and explain where those 13 minutes went.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;&lt;code&gt;02:28:01 UTC&lt;/code&gt; (The bug is filed):&lt;/strong&gt; One minute and 17 seconds into its read, the Analyst found the culprit and filed a bug directly against &lt;code&gt;binpacker&lt;/code&gt;:&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fykoalf79in2vdf7tzhom.webp" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fykoalf79in2vdf7tzhom.webp" alt="Screenshot of a Factory bug report filed automatically by the Telemetry Analyst pod: Observed notes that binpacker765 spent 13 minutes and 95 turns because scripts/check executed src/pages/Find.test.tsx four times (68090ms, 70637ms, 67854ms, and 72548ms), adding 211 seconds of duplicate work; Expected specifies deduplicating related frontend test discovery before running Vitest once; Reproduce cites the exact command and step 89 of trajectory mw5rmsmz-2." width="800" height="429"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;The 95-turn outlier wasn't an "LLM reasoning failure" or a bad prompt. Because the engineer pod touched five frontend files that all imported into the same page, and &lt;code&gt;scripts/check&lt;/code&gt; looped over changed files running &lt;code&gt;vitest related --run&lt;/code&gt; one source at a time, the test runner executed &lt;code&gt;src/pages/Find.test.tsx&lt;/code&gt; &lt;strong&gt;four separate times&lt;/strong&gt;. At roughly 70 seconds per run (&lt;code&gt;68090ms&lt;/code&gt;, &lt;code&gt;70637ms&lt;/code&gt;, &lt;code&gt;67854ms&lt;/code&gt;, and &lt;code&gt;72548ms&lt;/code&gt;), repeating that single test file burned &lt;strong&gt;211 extra seconds&lt;/strong&gt;, which in turn caused the engineer pod to burn dozens of turns polling background process status and scheduling timers.&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;&lt;code&gt;02:32:29 UTC&lt;/code&gt; (Planning and execution start):&lt;/strong&gt; Because &lt;code&gt;binpacker&lt;/code&gt; runs with &lt;code&gt;auto&lt;/code&gt; mode enabled, filing that bug immediately woke up a Product Manager (PM) and Architect pod in a planning room. They drafted a single-node plan (&lt;code&gt;Deduplicate change-related frontend test execution&lt;/code&gt;) to discover affected tests per source file first, keep the per-file &lt;code&gt;untested:&lt;/code&gt; warning, and execute the deduplicated union once.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;&lt;code&gt;02:50:46 UTC&lt;/code&gt; (The fix lands and closes the ticket):&lt;/strong&gt; Eighteen minutes after the plan started (and 24 minutes and 53 seconds after &lt;code&gt;binpacker765&lt;/code&gt; closed), the engineer pod's fix cleared the full CI and container build suite, squash-merged onto &lt;code&gt;main&lt;/code&gt;, and automatically closed the issue.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;From a slow test run in a kids' clothing inventory app to a root-cause diagnosis, a reproducible bug report, an approved plan, and a merged fix in under 25 minutes. Zero human keystrokes.&lt;/p&gt;

&lt;h2&gt;
  
  
  The nuance: why counting "moments of friction" failed
&lt;/h2&gt;

&lt;p&gt;To get to that 25-minute loop tonight, I first had to delete &lt;strong&gt;6,067 lines of telemetry code&lt;/strong&gt; on Thursday morning.&lt;/p&gt;

&lt;p&gt;When I first wired up observability for Leaf, I did what most engineers building agent harnesses do. I built a background pipeline that fed every finished agent trajectory to a local model and asked it to extract "moments of friction." Whenever an agent hit a non-zero exit code, fumbled a tool call, or retried a command, the reader logged a moment and grouped it into a problem list.&lt;/p&gt;

&lt;p&gt;On paper, reading every transcript sounds thorough. In practice, it was a firehose of low-grade noise.&lt;/p&gt;

&lt;p&gt;A run that finished cleanly in 40 seconds and a run that burned 20 minutes in a retry spiral got the exact same shallow read. Because a small model was grading individual tool steps in isolation, it had no link to what the run actually cost or whether a code change would prevent the hiccup next time. My telemetry tab filled up with hundreds of unranked complaints, and I still had to click through transcripts by hand to figure out what mattered.&lt;/p&gt;

&lt;p&gt;Counting how often an agent stumbles tells you nothing if you can't name the line of code that tripped it.&lt;/p&gt;

&lt;p&gt;Two days ago, right after we &lt;a href="https://jduncan.io/blog/2026-11-04-pruning-agent-prompts-dark-factory/" rel="noopener noreferrer"&gt;pruned 65% of our agent prompts&lt;/a&gt;, I ripped out the entire &lt;code&gt;moments&lt;/code&gt; pipeline and rebuilt Leaf's telemetry around two ideas: &lt;strong&gt;deterministic SQL flags&lt;/strong&gt; for single rooms, and a daily &lt;strong&gt;wake-sleep "dreaming" pass&lt;/strong&gt; that consolidates patterns across rooms.&lt;/p&gt;

&lt;h2&gt;
  
  
  Step 1: deterministic flags and a skeptical Analyst
&lt;/h2&gt;

&lt;p&gt;Instead of paying an LLM to scan every transcript looking for vibes, Leaf now evaluates every closed room (build plans, PM/Architect planning rooms, CI/CD runs, Improve examinations, and Security validations) against eight hard facts already recorded in SQLite when a room tears down:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Six event flags (something went wrong):&lt;/strong&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;code&gt;owner&lt;/code&gt;: I stepped into the room with an unprompted message that wasn't answering an agent's question.&lt;/li&gt;
&lt;li&gt;
&lt;code&gt;dead time&lt;/code&gt;: Two adjacent frames in the room had a silent gap of &lt;strong&gt;10 minutes or more&lt;/strong&gt;.&lt;/li&gt;
&lt;li&gt;
&lt;code&gt;rejected&lt;/code&gt;: The PM or Architect rejected the issue during planning.&lt;/li&gt;
&lt;li&gt;
&lt;code&gt;questions&lt;/code&gt;: The planning room had to stop and ask me a round of clarifying questions.&lt;/li&gt;
&lt;li&gt;
&lt;code&gt;send-back&lt;/code&gt;: The Tech Lead rejected an engineer's diff.&lt;/li&gt;
&lt;li&gt;
&lt;code&gt;unfinished&lt;/code&gt;: A pod exited without a clean pass or completion report.&lt;/li&gt;
&lt;/ul&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Two relative flags (something cost more than usual):&lt;/strong&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;code&gt;retried&lt;/code&gt;: A pod had to be started more than once.&lt;/li&gt;
&lt;li&gt;
&lt;code&gt;turns&lt;/code&gt;: A pod's turn count exceeded the &lt;strong&gt;90th percentile (&lt;code&gt;P90&lt;/code&gt;) of its role in that repository over the prior 30 days&lt;/strong&gt; (with planning seats compared strictly against planning rooms, never build pods).&lt;/li&gt;
&lt;/ul&gt;
&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;If none of those eight facts hold true, the room isn't queued for analysis at all. And if a room's &lt;em&gt;only&lt;/em&gt; flag is &lt;code&gt;turns&lt;/code&gt; (meaning nothing broke or got sent back, it just took longer than usual), Leaf samples &lt;strong&gt;every fifth room (&lt;code&gt;1-in-5&lt;/code&gt;)&lt;/strong&gt; and skips the other four so the queue never drowns in normal tail variance.&lt;/p&gt;

&lt;p&gt;When the supervisor drains the queue, it prioritizes rooms where something clearly went sideways (&lt;code&gt;owner&lt;/code&gt;, &lt;code&gt;rejected&lt;/code&gt;, or &lt;code&gt;dead time&lt;/code&gt;) first, then other planning rooms, and finally build rooms, newest first.&lt;/p&gt;

&lt;h3&gt;
  
  
  Making "no bug" the default answer
&lt;/h3&gt;

&lt;p&gt;When you hand a transcript to a frontier model and ask it what went wrong, its natural instinct is to invent a problem so it looks helpful. That's how you end up with the post-mortem prompt bloat we had to clean up earlier this week.&lt;/p&gt;

&lt;p&gt;To keep the &lt;code&gt;analyst&lt;/code&gt; persona honest, its prompt is tiny and built around a single default: &lt;strong&gt;explaining where the room's time went and closing with no bug is the normal outcome&lt;/strong&gt;. The Analyst is only allowed to file a bug when three conditions hold at once:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;It can point to a specific change in a prompt, the factory harness, a brief, the target repository, or the container image that would have prevented the cost.&lt;/li&gt;
&lt;li&gt;The transcript proves what that cost actually was.&lt;/li&gt;
&lt;li&gt;The cause is systemic and will happen again on future runs, rather than being a quirk of that one task.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;Even then, the Analyst has to check the repository's existing issues first and link to an existing cause (&lt;code&gt;match&lt;/code&gt;) rather than filing a duplicate. If it does file a new bug, the harness requires structured &lt;code&gt;Observed&lt;/code&gt;, &lt;code&gt;Expected&lt;/code&gt;, &lt;code&gt;Reproduce&lt;/code&gt;, and &lt;code&gt;Code facts&lt;/code&gt; sections naming the exact files and symbols it verified.&lt;/p&gt;

&lt;p&gt;We also caught an amusing cross-repository routing bug this afternoon. While analyzing a &lt;strong&gt;BrandCast&lt;/strong&gt; planning room, an Analyst pod discovered a subtle race condition inside the factory's own planning state machine and dutifully filed the bug against the BrandCast repo. Two minutes later, the BrandCast PM and Architect pods booted up, searched &lt;code&gt;/work&lt;/code&gt; for the factory's state-machine files, realized they were sitting inside a digital signage codebase, and rejected the ticket as misfiled. We tightened the routing rules right after so harness, prompt, and container causes always route to the factory repository, while application-level causes route to the project repo.&lt;/p&gt;

&lt;h2&gt;
  
  
  Step 2: teaching the factory to "dream" across runs
&lt;/h2&gt;

&lt;p&gt;A single-room Analyst has a structural blind spot: &lt;strong&gt;it only sees one room at a time&lt;/strong&gt;.&lt;/p&gt;

&lt;p&gt;Because the Analyst is told to file a bug only when a cause will repeat, reading one room in isolation creates a catch-22. When an Analyst sees a truncated CI log excerpt or a slow test script in a single run, it often can't tell whether that hiccup was a one-off fluke or a quiet tax hitting five rooms a day. So it writes a faithful one-line verdict explaining where the time went, closes the room with no bug, and exits.&lt;/p&gt;

&lt;p&gt;Even worse, if a bug &lt;em&gt;was&lt;/em&gt; filed and fixed last Tuesday, a single-room Analyst reading a room on Friday doesn't automatically connect the dots across the whole month of closed issues to flag that the regression came back.&lt;/p&gt;

&lt;p&gt;To solve that, I added a second telemetry loop built around &lt;strong&gt;dreaming&lt;/strong&gt;.&lt;/p&gt;

&lt;p&gt;I didn't coin the term "dreaming" for offline consolidation. The idea goes all the way back to &lt;a href="http://www.its.caltech.edu/~bi250b/2009/papers/crick-rem-nature.pdf" rel="noopener noreferrer"&gt;Francis Crick and Graeme Mitchison's 1983 &lt;em&gt;Nature&lt;/em&gt; paper&lt;/a&gt; proposing that brains use REM sleep for "reverse learning" (pruning parasitic noise from overloaded neural networks), and &lt;a href="https://www.its.caltech.edu/~jkenny/nb250c/papers/hinton_95.pdf" rel="noopener noreferrer"&gt;Geoffrey Hinton, Peter Dayan, Brendan Frey, and Radford Neal's 1995 &lt;em&gt;Science&lt;/em&gt; paper on the &lt;strong&gt;Wake-Sleep algorithm&lt;/strong&gt;&lt;/a&gt;. &lt;a href="https://ml4code.github.io/publications/ellis2021dreamcoder/" rel="noopener noreferrer"&gt;Kevin Ellis and Joshua Tenenbaum's &lt;strong&gt;DreamCoder&lt;/strong&gt; (2021)&lt;/a&gt; brought that wake-sleep loop directly into code synthesis, where an agent solves problems while "awake" and refactors its solutions into reusable library abstractions while "asleep." More recently, &lt;a href="https://arxiv.org/abs/2504.13171" rel="noopener noreferrer"&gt;Letta (MemGPT) formalized &lt;strong&gt;sleep-time compute&lt;/strong&gt;&lt;/a&gt; for stateful LLM agents, and &lt;a href="https://platform.claude.com/docs/en/managed-agents/dreams" rel="noopener noreferrer"&gt;Anthropic shipped &lt;strong&gt;Claude Dreams&lt;/strong&gt;&lt;/a&gt; to consolidate raw session transcripts offline.&lt;/p&gt;

&lt;p&gt;I borrowed that exact wake-sleep pattern for Leaf's telemetry.&lt;/p&gt;

&lt;p&gt;During the day, single-room Analysts inspect flagged outliers as they finish and write short, one-line verdicts. Then, once every 24 hours per active repository, Leaf spawns a Class 5 &lt;strong&gt;&lt;code&gt;dreamer&lt;/code&gt;&lt;/strong&gt; pod.&lt;/p&gt;

&lt;p&gt;Unlike the Analyst, the Dreamer doesn't start by wading through megabytes of raw tool logs. Its brief hands it the last 30 days of analyzed rooms for that repository, every single-room Analyst verdict, and the history of open and closed trajectory issues. Its job is to look across the entire window at once:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Spot shared causes behind different phrasings.&lt;/strong&gt; If three separate room verdicts mention different symptoms of the same underlying bottleneck, the Dreamer connects them and files a single bug covering all of those rooms, pulling individual transcripts only when a one-line verdict doesn't give enough detail.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Consolidate "cause families."&lt;/strong&gt; When two or more rooms link to the same underlying issue, Leaf groups them on the Telemetry board as a &lt;strong&gt;cause family&lt;/strong&gt; and tracks how many rooms hit that cause each week.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Catch fixes that didn't stick (&lt;code&gt;Recurred after fix&lt;/code&gt;).&lt;/strong&gt; Whenever an Analyst or a Dreamer matches a room to a &lt;em&gt;closed&lt;/em&gt; issue, Leaf compares the timestamps. If the room closed &lt;em&gt;after&lt;/em&gt; the issue was marked fixed, the harness automatically records the link as &lt;code&gt;recurred&lt;/code&gt; and flips that family's status on the dashboard from &lt;code&gt;Fixed&lt;/code&gt; to &lt;strong&gt;&lt;code&gt;Recurred after fix&lt;/code&gt;&lt;/strong&gt;.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Look at what the live 30-day Telemetry scorecard shows across my four onboarded repositories (&lt;code&gt;factory&lt;/code&gt;, &lt;code&gt;brandcast&lt;/code&gt;, &lt;code&gt;calamus&lt;/code&gt;, and &lt;code&gt;binpacker&lt;/code&gt;) as of tonight:&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fjpz3cl2caabcoq7ejgkj.webp" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fjpz3cl2caabcoq7ejgkj.webp" alt="Leaf Telemetry Loop 30-Day Analyst and Dreamer Scorecard across 953 flagged rooms: 715 rooms read, 238 turns-only sampled out, 69 root causes filed, 61 (88%) fixed and closed done, and 4 (5.8%) recurrences caught; left chart shows factual flag yield led by rejected (100%), unfinished (38.5%), owner (35.7%), and turns &gt; P90 (27.8%); right panel lists top multi-room cause families and engineer orientation median of 3 tool calls before first edit." width="800" height="539"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Across &lt;strong&gt;953 flagged rooms&lt;/strong&gt; in the 30-day window, Leaf sampled out &lt;strong&gt;238 &lt;code&gt;turns&lt;/code&gt;-only rooms&lt;/strong&gt; and read &lt;strong&gt;715 rooms&lt;/strong&gt; all the way through. Out of those 715 analyses and the daily Dream passes:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Most flagged rooms didn't need a ticket.&lt;/strong&gt; In more than 75% of read rooms, the Analyst explained where the clock went, concluded the cost was normal for the size of the task, and closed the analysis without filing noise.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;69 distinct systemic causes were isolated, and 61 (88.4%) are already fixed.&lt;/strong&gt; Because telemetry bugs feed directly back into the planning queue, the factory has already planned, tested, and merged fixes for 61 of the 69 causes it discovered on itself.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;24 multi-room "cause families" emerged.&lt;/strong&gt; For example, this morning's Dream pass on the factory repo reviewed &lt;strong&gt;180 Analyst verdicts&lt;/strong&gt; in a single pass and noticed that four separate rooms had lost real CI failure diagnostics because a test fixture printed oversized marker lines that saturated the log excerpt cap. It consolidated those rooms onto a single issue, and an engineer pod fixed the excerpt slicer 13 minutes later.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;4 closed fixes were caught recurring (&lt;code&gt;5.8%&lt;/code&gt;).&lt;/strong&gt; One family where a Tech Lead's completion report was misrecorded as &lt;code&gt;unreported&lt;/code&gt; showed up across &lt;strong&gt;16 rooms&lt;/strong&gt;. When a room closed &lt;em&gt;after&lt;/em&gt; the initial fix landed and still hit the bug, the timestamp check caught the regression automatically and flagged the family as &lt;code&gt;Recurred after fix&lt;/code&gt;.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;A few honest practitioner caveats on these numbers:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;The 715 read rooms include a one-time backfill.&lt;/strong&gt; When we turned on the new telemetry loop on Thursday, we set a two-week backlog cutoff so the Analyst could chew through recent history right away. That gave the first daily Dream passes across &lt;code&gt;brandcast&lt;/code&gt;, &lt;code&gt;factory&lt;/code&gt; (180 verdicts), &lt;code&gt;calamus&lt;/code&gt; (92 verdicts), and &lt;code&gt;binpacker&lt;/code&gt; (19 verdicts) a rich dataset immediately, though steady-state daily volume is much calmer.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Zero issues on &lt;code&gt;dead time&lt;/code&gt; (&lt;code&gt;0 of 19&lt;/code&gt;) doesn't mean rooms never stall.&lt;/strong&gt; Why did 19 rooms with a 10-minute silent gap yield zero linked issues, while &lt;code&gt;unfinished&lt;/code&gt; yielded &lt;strong&gt;38.5% (&lt;code&gt;40 of 104&lt;/code&gt;)&lt;/strong&gt; and &lt;code&gt;turns &amp;gt; P90&lt;/code&gt; yielded &lt;strong&gt;27.8% (&lt;code&gt;62 of 223&lt;/code&gt;)&lt;/strong&gt;? Because &lt;strong&gt;most 10-minute gaps between room frames happened while a single long E2E suite or container build was legitimately running inside a pod&lt;/strong&gt;. When a room &lt;em&gt;actually&lt;/em&gt; stalled and failed to finish, it also tripped &lt;code&gt;unfinished&lt;/code&gt; or &lt;code&gt;owner&lt;/code&gt;, which is where the real stall bugs got filed.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;BinPacker isn't enterprise production traffic.&lt;/strong&gt; Using an inventory app for my wife's kids' resale shop alongside &lt;code&gt;brandcast&lt;/code&gt;, &lt;code&gt;calamus&lt;/code&gt;, and &lt;code&gt;factory&lt;/code&gt; is my favorite kind of lab test, because the codebase is small enough to trace every turn, but real enough that a 3.5-minute Vitest bottleneck is immediately painful. Still, there's no revolution in the suburbs just yet.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  Measuring orientation and whether the wiki pays its rent
&lt;/h2&gt;

&lt;p&gt;While I was overhauling room telemetry this week, I added two smaller instruments to answer questions I kept guessing at: &lt;strong&gt;how long does it take an engineer pod to orient itself before making its first edit&lt;/strong&gt;, and &lt;strong&gt;do agents actually read the documentation other agents write?&lt;/strong&gt;&lt;/p&gt;

&lt;h3&gt;
  
  
  1. Engineer orientation
&lt;/h3&gt;

&lt;p&gt;Every time an engineer pod finishes, Leaf unpacks its trajectory once, counts how many tool calls the agent made before its very first file edit, and records which repository files it read during that orientation phase.&lt;/p&gt;

&lt;p&gt;Across &lt;strong&gt;7,934 engineer trajectories&lt;/strong&gt; recorded so far, the &lt;strong&gt;median orientation is just 3 tool calls before the first edit&lt;/strong&gt;, and the &lt;strong&gt;90th percentile is 11 tool calls&lt;/strong&gt;. Because Leaf also aggregates every non-skill file that engineer pods read across multiple plans, I can see at a glance which files agents keep hunting for on Turn 1 (like test setup helpers and core server entrypoints across 140+ plans) and bake those conventions directly into the Architect's brief or repository skills.&lt;/p&gt;

&lt;h3&gt;
  
  
  2. Wiki payoff
&lt;/h3&gt;

&lt;p&gt;Whenever a feature PRD or bug closes and deploys in Leaf, a &lt;strong&gt;Tech Writer&lt;/strong&gt; agent wakes up to update the repository's product wiki (and an hourly &lt;strong&gt;Lead Technical Writer&lt;/strong&gt; merges duplicates and keeps the index clean). That sounds great in an architecture diagram, until you ask how many tokens the writers are burning and whether the PM and Architect ever read what they wrote.&lt;/p&gt;

&lt;p&gt;Yesterday we added a wiki health tracker that measures four signals per repository: writer token cost, page reads by the PM, Architect, and &lt;code&gt;/leaf&lt;/code&gt; sessions, explicit wiki citations in planning messages, and PM readiness decisions that cite the wiki.&lt;/p&gt;

&lt;p&gt;In the first 31 hours since that counter started:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;&lt;code&gt;brandcast&lt;/code&gt; (104 wiki pages):&lt;/strong&gt; 77 writer jobs and 10 lead passes burned &lt;strong&gt;49.7M input tokens&lt;/strong&gt; (mostly cache reads) and &lt;strong&gt;205k output tokens&lt;/strong&gt;. In return, planning seats cited the wiki &lt;strong&gt;80 times&lt;/strong&gt;, and 58 of the 104 pages have already been read in live planning rooms (46 unread so far).&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;&lt;code&gt;factory&lt;/code&gt; (148 wiki pages):&lt;/strong&gt; 24 writer jobs and 11 lead passes burned &lt;strong&gt;19.4M input tokens&lt;/strong&gt;, producing &lt;strong&gt;17 citations&lt;/strong&gt;, while &lt;strong&gt;108 of the 148 pages haven't been read once yet&lt;/strong&gt;.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;&lt;code&gt;calamus&lt;/code&gt; (32 pages) and &lt;code&gt;binpacker&lt;/code&gt; (20 pages):&lt;/strong&gt; 19 and 10 citations respectively, with three-quarters of &lt;code&gt;binpacker&lt;/code&gt;'s wiki already read in planning rooms.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Seeing that &lt;strong&gt;73% of &lt;code&gt;factory&lt;/code&gt;'s own wiki pages haven't been read in the last day and a half&lt;/strong&gt; is the exact kind of uncomfortable feedback loop you want in an unattended harness. If a documentation loop costs 20 million tokens a day and nobody in planning reads two-thirds of the pages, the telemetry should tell you to tighten the Tech Writer's scope.&lt;/p&gt;

&lt;h2&gt;
  
  
  So what?
&lt;/h2&gt;

&lt;p&gt;If you're building multi-agent workflows or running unattended coding loops, stop grading your agents with generic LLM "friction counters":&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Gate telemetry on deterministic SQL facts, not LLM vibes.&lt;/strong&gt; Don't pay a small model to read every transcript looking for "moments of friction." Trigger analysis only when a run trips a hard mechanical fact: an unprompted human message, a rejected plan, a Tech Lead send-back, a 10-minute silent gap, or a turn count above the role's 30-day &lt;code&gt;P90&lt;/code&gt; (sampled &lt;code&gt;1-in-5&lt;/code&gt;).&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Make "no bug" the default post-mortem answer.&lt;/strong&gt; If you ask an LLM to find a problem in every slow transcript, it will invent rules that bloat your prompts. Require your Analyst agent to name the exact file, symbol, and reproducible command that will cause the cost again, or close the reading with no bug.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Separate single-run analysis from cross-run "dreaming."&lt;/strong&gt; An agent reading one transcript can't tell a fluke from a systemic tax. Run a daily consolidation pass that reads the last 30 days of one-line Analyst verdicts together, groups shared root causes into families, and flags closed issues that recurred after their fix landed.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Instrument the supporting loops too.&lt;/strong&gt; Track how many tool calls your engineers spend orienting before their first edit, and measure whether the docs your writer agents generate actually get read and cited by your planners.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;By the time I finished writing this draft (&lt;code&gt;02:50:46 UTC&lt;/code&gt;), the BinPacker fix had finished its CI suite green, squash-merged onto &lt;code&gt;main&lt;/code&gt;, and closed the bug automatically. Next time an Improvement pod simplifies my wife's resale inventory app, &lt;code&gt;Find.test.tsx&lt;/code&gt; will only run once.&lt;/p&gt;

</description>
      <category>darkfactory</category>
      <category>agenttelemetry</category>
      <category>observability</category>
      <category>leaf</category>
    </item>
    <item>
      <title>Pruning Agent Prompts: Why I Cut My Factory's Rules by 65%</title>
      <dc:creator>Jamie Duncan</dc:creator>
      <pubDate>Fri, 20 Nov 2026 05:45:57 +0000</pubDate>
      <link>https://dev.to/jamie_duncan_3738b0805ed5/pruning-agent-prompts-why-i-cut-my-factorys-rules-by-65-11f9</link>
      <guid>https://dev.to/jamie_duncan_3738b0805ed5/pruning-agent-prompts-why-i-cut-my-factorys-rules-by-65-11f9</guid>
      <description>&lt;p&gt;&lt;em&gt;Originally published at &lt;a href="https://jduncan.io/blog/2026-11-04-pruning-agent-prompts-dark-factory/" rel="noopener noreferrer"&gt;jduncan.io&lt;/a&gt;.&lt;/em&gt;&lt;/p&gt;

&lt;p&gt;Last week, I wrote a whole post arguing that &lt;a href="https://jduncan.io/blog/2026-10-27-agent-skills-unattended-context/" rel="noopener noreferrer"&gt;giant agent skill repos won't survive unattended 2027 workflows&lt;/a&gt;. I even put up a chart bragging about how lean &lt;a href="https://jduncan.io/blog/2026-10-25-leaf-dark-factory-update/" rel="noopener noreferrer"&gt;Leaf, my local Level 5 dark software factory&lt;/a&gt;, kept its startup context compared to interactive terminal sessions.&lt;/p&gt;

&lt;p&gt;Nothing humbles you faster than bragging about your architecture in public.&lt;/p&gt;

&lt;p&gt;Over the nine days right around that post, a quiet pathology inside my own factory was doing the exact opposite of what I was preaching. By yesterday morning, the Planner role prompt had grown &lt;strong&gt;4.5 times larger&lt;/strong&gt; than it was at the start of October, and the shared &lt;code&gt;_base.md&lt;/code&gt; file that every single agent pod reads on Turn 1 had ballooned &lt;strong&gt;15 times larger&lt;/strong&gt;, hitting &lt;strong&gt;2,861 words&lt;/strong&gt;.&lt;/p&gt;

&lt;p&gt;So yesterday afternoon, we took hedge shears to the &lt;code&gt;prompts/&lt;/code&gt; directory. We cut total prompt words across the factory by &lt;strong&gt;65%&lt;/strong&gt; (from &lt;strong&gt;17,072 words down to 5,899&lt;/strong&gt;), and we took &lt;code&gt;_base.md&lt;/code&gt; from &lt;strong&gt;2,861 words down to 55&lt;/strong&gt;.&lt;/p&gt;

&lt;p&gt;We did it because our prompts had grown by one rule per incident for a month, and the rules had started causing the very failures they were written to stop.&lt;/p&gt;

&lt;h2&gt;
  
  
  The nuance: incident-driven prompt accretion
&lt;/h2&gt;

&lt;p&gt;&lt;strong&gt;The most dangerous phrase in an unattended agent harness is "when an agent misbehaves, fix its prompt."&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;That exact sentence was sitting right inside our repository's &lt;code&gt;AGENTS.md&lt;/code&gt; file. Nobody sat down and designed a 17,000-word prompt library. Each individual rule was written by a well-meaning post-mortem for a real incident. Taken one at a time, every rule looked reasonable. Stacked together, they buried the rules that actually mattered.&lt;/p&gt;

&lt;p&gt;If you haven't read the earlier posts on how my stack is wired, two pieces of nomenclature help make sense of how we got stuck:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Factory&lt;/strong&gt; is the unattended multi-agent engine that takes an issue, plans a dependency graph, spawns isolated container pods to write and review code, and runs deterministic CI/CD.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;&lt;code&gt;/leaf&lt;/code&gt;&lt;/strong&gt; is the interactive skill I run with my Product Owner hat on when I'm scoping new work or running a retrospective on a run that went sideways.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Over the 30 days before yesterday's cut, &lt;strong&gt;170 commits touched &lt;code&gt;prompts/&lt;/code&gt;&lt;/strong&gt;. My interactive &lt;code&gt;/leaf&lt;/code&gt; sessions wrote 85 of them, automated plans built from &lt;code&gt;/leaf&lt;/code&gt; issues wrote 68 of them, and I wrote 17 by hand.&lt;/p&gt;

&lt;p&gt;That loop operated like a one-way ratchet:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;
&lt;strong&gt;A run went sideways.&lt;/strong&gt; A retrospective session dug through the SQLite telemetry and found what the agent did wrong.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;The retro filed a rule for that specific instance.&lt;/strong&gt; Our principle of "fix causes, not instances" quietly degraded into "write a stern sentence forbidding the thing that just happened."&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;A unit test pinned the prose in place.&lt;/strong&gt; Whenever a plan landed a new prompt rule, it also added an assertion in &lt;code&gt;src/bridge.prompts.test.ts&lt;/code&gt; checking that the exact text existed in the markdown file. That test file grew to &lt;strong&gt;1,022 lines and 162 assertions&lt;/strong&gt;. Removing a stale sentence from a prompt literally broke CI.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;Prompts could only grow. They could never shrink.&lt;/p&gt;

&lt;h3&gt;
  
  
  When the cure costs more than the disease
&lt;/h3&gt;

&lt;p&gt;When you never delete a rule, you end up taxing 100% of your runs to defend against failures that happen once in a thousand turns:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;The &lt;code&gt;/tmp/run.log&lt;/code&gt; redirect tax:&lt;/strong&gt; After a rare output buffering hiccup that occurred &lt;strong&gt;0.2 times per 100 runs&lt;/strong&gt;, a post-mortem added a rule telling every pod to redirect every shell command to &lt;code&gt;/tmp/run.log&lt;/code&gt; and inspect it. The container harness already returns &lt;code&gt;stdout&lt;/code&gt;, &lt;code&gt;stderr&lt;/code&gt;, and the exit code directly to the model. Every pod in the factory was burning extra turns managing temporary log files for a bug that almost never happened.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;The shell heredoc panic:&lt;/strong&gt; A rule banning shell heredocs (&lt;code&gt;&amp;lt;&amp;lt; 'EOF'&lt;/code&gt;) was added for a failure rate of &lt;strong&gt;0.1 per 100 runs&lt;/strong&gt;. By yesterday morning, the rule itself was triggering problem moments inside build rooms, with engineer pods stopping mid-task to confess in the logs: &lt;em&gt;"I wrote it with a shell heredoc, which the environment rules forbid,"&lt;/em&gt; and then burning turns redoing working edits.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;The adversarial phase-gate trap:&lt;/strong&gt; Across three separate incident fixes and a 230-line state reducer in &lt;code&gt;src/refinement.ts&lt;/code&gt;, we had turned the Product Manager (PM) into a courtroom judge and the Planner into a defendant. They were locked in rigid readiness, research, and review phases where they couldn't just talk directly, and the PM wasn't allowed to question the token cost of a plan.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Yesterday morning, the whole pile finally collapsed under its own weight during a run on &lt;strong&gt;Calamus&lt;/strong&gt;. Calamus is an agentic long-form writing and publishing experiment I've been building inside the factory to test how multi-agent workflows handle research, drafting, and editorial review outside of pure application code.&lt;/p&gt;

&lt;p&gt;First, a Planner pod on Calamus spent 10 of its 16 minutes running test suites that taught it nothing, because four lines in &lt;code&gt;planner.md&lt;/code&gt; told it to run them. Our reflex was so ingrained that &lt;code&gt;/leaf&lt;/code&gt; immediately filed an issue adding &lt;em&gt;more&lt;/em&gt; rules to &lt;code&gt;planner.md&lt;/code&gt;.&lt;/p&gt;

&lt;p&gt;A few hours later, after a primary frontier model hung, a fallback model planned 10 simple styling tasks on Calamus at &lt;strong&gt;Class 5 (&lt;code&gt;max&lt;/code&gt; effort)&lt;/strong&gt; because it decided styling work was "creative." As I covered in &lt;a href="https://jduncan.io/blog/2026-10-29-moneyball-for-ai-agents/" rel="noopener noreferrer"&gt;Moneyball for AI Agents&lt;/a&gt;, Class 5 is our heavy multi-module tier, and throwing trivial styling nodes at it torches quota for zero gain. We already had a rule forbidding that exact behavior, &lt;strong&gt;but it was one sentence buried inside 4,113 words of prompt instructions&lt;/strong&gt;.&lt;/p&gt;

&lt;p&gt;When &lt;code&gt;/leaf&lt;/code&gt; looked at that run and proposed adding yet another rule to &lt;code&gt;pm.md&lt;/code&gt;, I finally pulled the plug.&lt;/p&gt;

&lt;p&gt;We weren't engineering a harness anymore.&lt;br&gt;&lt;br&gt;
We were writing an HOA bylaw binder that no model could hold in its head.&lt;/p&gt;

&lt;h2&gt;
  
  
  Two tests for every line (and moving rules to code)
&lt;/h2&gt;

&lt;p&gt;We threw out the incident-driven patches and rewrote the core role prompts from scratch around themes rather than past bugs. While we were in there, we also cleaned up the role names end-to-end across code, docs, and SQLite settings: &lt;code&gt;planner&lt;/code&gt; became &lt;strong&gt;&lt;code&gt;architect&lt;/code&gt;&lt;/strong&gt;, &lt;code&gt;foreman&lt;/code&gt; became &lt;strong&gt;&lt;code&gt;tech-lead&lt;/code&gt;&lt;/strong&gt;, and &lt;code&gt;worker&lt;/code&gt; became &lt;strong&gt;&lt;code&gt;engineer&lt;/code&gt;&lt;/strong&gt;.&lt;/p&gt;

&lt;p&gt;To keep a single line of text in a prompt, that line had to pass two tests:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;
&lt;strong&gt;It is verifiably true right now.&lt;/strong&gt; We checked every claim against the live TypeScript code, the container image, and the SQLite schema, never against memory or the old prompt. During the first draft of the prune, our own &lt;code&gt;/leaf&lt;/code&gt; session tried to carry forward the &lt;code&gt;/tmp/run.log&lt;/code&gt; redirect, referenced a &lt;code&gt;/opt/factory/docs/plans.md&lt;/code&gt; file that doesn't exist inside a pod container, and told the PM to send a &lt;code&gt;rev=&amp;lt;n&amp;gt;&lt;/code&gt; parameter that the chat tool explicitly rejects. Every one of those zombie lines got caught only because we forced a check against the code.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;It changes what the agent does on a &amp;gt;1% problem.&lt;/strong&gt; If the harness, the deterministic CI gate, or the model's baseline training already produces the behavior, the line gets cut. And a rare failure (under 1 occurrence per 100 runs) does not earn a permanent tax that every pod has to read on every turn.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;Look at how much dead weight dropped out of the Turn 1 context across each seat, alongside what happened in the first 29 hours of runs after the deploy:&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fme05r90w246fcreri6zy.webp" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fme05r90w246fcreri6zy.webp" alt="Comparison chart showing Factory prompt word counts before and after the 65% cut alongside the first 29 hours of runtime data on Calamus: Architect fell from ~7,700 words to ~840 (-89%), Tech Lead from ~5,200 to ~370 (-93%), PM from ~4,000 to ~200 (-95%), Engineer from ~3,750 to ~220 (-94%), and _base.md from 2,861 to 55 (-98%), while Haiku turns fell 43%, Sonnet turns fell 35%, and build problem moments dropped 62%." width="799" height="453"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Cutting that much markdown only worked because &lt;strong&gt;we moved rules that prompts can't reliably enforce into deterministic code&lt;/strong&gt;:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Container defaults instead of shell lectures:&lt;/strong&gt; Instead of lecturing every pod on which retry flags to pass to &lt;code&gt;curl&lt;/code&gt;, we baked a &lt;code&gt;.curlrc&lt;/code&gt; file directly into the pod container image.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Self-documenting error messages:&lt;/strong&gt; When a merge is refused, the refusal error returned by the harness now names the exact next step in plain text, so the Tech Lead prompt doesn't have to carry a troubleshooting manual for every refusal state. We also fixed a code bug where a retired read-only node never counted as merged and blocked its downstream dependents.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Direct collaboration instead of courtroom phases:&lt;/strong&gt; The PM and the Architect now start in the planning room at the exact same time and talk directly. The PM owns &lt;em&gt;what&lt;/em&gt; and &lt;em&gt;whether&lt;/em&gt; (with the repository's external wiki mounted, and without reading source code); the Architect owns &lt;em&gt;how&lt;/em&gt; and &lt;em&gt;cost&lt;/em&gt;. They share a single explicit goal: &lt;em&gt;"a plan that delivers what the issue asks for, at the lowest cost that does it well."&lt;/em&gt; The readiness, research, review, and owner-wait state machine phases are gone, deleting roughly &lt;strong&gt;470 lines of ceremony from &lt;code&gt;src/prd.ts&lt;/code&gt;&lt;/strong&gt;.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Killing the prose-pinning tests:&lt;/strong&gt; We cut &lt;code&gt;src/bridge.prompts.test.ts&lt;/code&gt; from &lt;strong&gt;1,022 lines down to 108 lines&lt;/strong&gt;. The surviving tests only verify structured tokens that Factory's TypeScript code actually parses.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  What 29 hours on Calamus actually tells us
&lt;/h2&gt;

&lt;p&gt;We deployed the pruned prompts yesterday afternoon. All 9 build runs and 10 of the 11 planning rooms in the first 29 hours ran against &lt;strong&gt;Calamus&lt;/strong&gt;, which gives us a clean before-and-after comparison on a single repository against the 7 days right before the cut (97 planning rooms and 72 development runs):&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Measure (Calamus)&lt;/th&gt;
&lt;th&gt;7 Days Before Cut&lt;/th&gt;
&lt;th&gt;First 29 Hours After&lt;/th&gt;
&lt;th&gt;Change&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Planning: Architect input per turn&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;231k tokens&lt;/td&gt;
&lt;td&gt;130k tokens&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;-44%&lt;/strong&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Planning: Total input per room&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;6.98M tokens&lt;/td&gt;
&lt;td&gt;6.17M tokens&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;-12%&lt;/strong&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Planning: Rooms ending in approved plan&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;96 of 99 (97%)&lt;/td&gt;
&lt;td&gt;10 of 10 (100%)&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;Held steady&lt;/strong&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Planning: Revisions per plan&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;2.08&lt;/td&gt;
&lt;td&gt;1.70&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;-0.4 revisions&lt;/strong&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Engineer nodes planned above Class 4&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;17 of 353 (4.8%)&lt;/td&gt;
&lt;td&gt;0 of 14 (0.0%)&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;Eliminated&lt;/strong&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Engineer on Haiku: turns per pod&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;21.7 turns&lt;/td&gt;
&lt;td&gt;12.3 turns&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;-43%&lt;/strong&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Engineer on Haiku: time to first edit&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;32 seconds&lt;/td&gt;
&lt;td&gt;12 seconds&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;-63%&lt;/strong&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Engineer on Sonnet: turns per pod&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;38.7 turns&lt;/td&gt;
&lt;td&gt;25.3 turns&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;-35%&lt;/strong&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Problem moments per 100 pods (Build)&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;79&lt;/td&gt;
&lt;td&gt;30&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;-62%&lt;/strong&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Problem moments per 100 pods (Planning)&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;47&lt;/td&gt;
&lt;td&gt;35&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;-26%&lt;/strong&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Tech Lead on 1-node runs: input tokens&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;420k tokens&lt;/td&gt;
&lt;td&gt;543k tokens&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;+29% (worse)&lt;/strong&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;Look closely at the mechanism behind those engineer numbers. &lt;strong&gt;The token savings didn't come from a smaller prompt on each turn; they came from the pods taking 35% to 43% fewer turns.&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;On Sonnet, an engineer pod's average context per turn after the cut is almost identical to what it was before (&lt;strong&gt;72.5k tokens vs. 72.9k tokens&lt;/strong&gt;), because repository files and tool output dominate the context window once a pod starts working. What changed is that &lt;strong&gt;the agents stopped performing all the defensive ritual work the old rules demanded&lt;/strong&gt;: redirecting logs to &lt;code&gt;/tmp/run.log&lt;/code&gt;, re-running unrequested checks to prove diligence, and tiptoeing around banned shell syntax. On Haiku, the median time before an engineer pod made its very first file edit dropped from &lt;strong&gt;32 seconds down to 12 seconds&lt;/strong&gt;. And with the 4,113-word noise floor gone from the Architect prompt, zero nodes have been planned above Class 4.&lt;/p&gt;

&lt;p&gt;A few honest practitioner caveats on this dataset:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;It's a 29-hour sample on a single repo.&lt;/strong&gt; Ten planning rooms, 9 build runs, and 14 engineer pods on Calamus is enough to spot a directional shift, not enough to declare victory. &lt;strong&gt;BrandCast&lt;/strong&gt;, my digital signage platform, ran 244 planning rooms in the week before the cut and hasn't run a batch since the deploy. That will be the real stress test.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Task shapes and model mixes shifted.&lt;/strong&gt; Post-cut plans on Calamus averaged 1.4 nodes compared to 4.6 nodes the week before, and 43% of post-cut engineer pods ran on Haiku (Class 2) versus 23% before. That's why the table above compares per-pod, per-turn, and same-model metrics rather than raw per-run totals (which would claim a phony 77% drop in tokens).&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;The Tech Lead went the wrong way.&lt;/strong&gt; On comparable 1-node runs, the Tech Lead took &lt;strong&gt;26% more turns (11.4 to 14.4)&lt;/strong&gt; and &lt;strong&gt;29% more input tokens (420k to 543k)&lt;/strong&gt;. We don't know yet if that's small-sample noise across 8 runs or a real side effect of shorter Tech Lead instructions. We'll re-measure at 50 development runs.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;The prune shipped its own bug on run one.&lt;/strong&gt; When we simplified the PM's plan approval signal to &lt;code&gt;consent&lt;/code&gt;, our new parser extracted the revision number from the &lt;em&gt;first integer&lt;/em&gt; in the PM's message. In the very first live planning room after deployment, the PM wrote &lt;code&gt;"calamus312 revision 2"&lt;/code&gt;, and Factory happily parsed that as &lt;strong&gt;revision 312&lt;/strong&gt;. Our unit tests had only tested &lt;code&gt;"Revision 3."&lt;/code&gt; We fixed the parser four hours later.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Even with those caveats, cutting two-thirds of our rules made the pods faster, quieter, and less prone to self-inflicted errors.&lt;/p&gt;

&lt;h2&gt;
  
  
  Guarding the prompts from ourselves
&lt;/h2&gt;

&lt;p&gt;The most sobering discovery during this cleanup wasn't the word count. It was realizing that &lt;strong&gt;we already had a file protection guard on &lt;code&gt;prompts/&lt;/code&gt;, and it was fencing off the wrong loop&lt;/strong&gt;.&lt;/p&gt;

&lt;p&gt;Back in October, I added &lt;code&gt;prompts/&lt;/code&gt;, &lt;code&gt;src/bridge.prompts.ts&lt;/code&gt;, and &lt;code&gt;skills/&lt;/code&gt; to the &lt;code&gt;protect&lt;/code&gt; array inside &lt;code&gt;.factory.json&lt;/code&gt;. I assumed that prevented automated loops from bloating our instructions.&lt;/p&gt;

&lt;p&gt;It didn't. Only our autonomous &lt;strong&gt;Improve&lt;/strong&gt; loop (the background complexity-reduction loop I wrote about in the &lt;a href="https://jduncan.io/blog/2026-10-29-moneyball-for-ai-agents/" rel="noopener noreferrer"&gt;Moneyball post&lt;/a&gt;) checked the &lt;code&gt;protect&lt;/code&gt; list. And Improve wrote &lt;strong&gt;0 of the 170 commits&lt;/strong&gt; that bloated &lt;code&gt;prompts/&lt;/code&gt;. My own &lt;code&gt;/leaf&lt;/code&gt; post-mortem sessions and the development plans generated from them wrote &lt;strong&gt;153 of those 170 commits&lt;/strong&gt;, walking right through the front door because development plans didn't enforce &lt;code&gt;protect&lt;/code&gt;.&lt;/p&gt;

&lt;p&gt;Meanwhile, six secondary prompts in the directory (&lt;code&gt;security.md&lt;/code&gt;, &lt;code&gt;techwriter.md&lt;/code&gt;, &lt;code&gt;improve.md&lt;/code&gt;, &lt;code&gt;leadwriter.md&lt;/code&gt;, &lt;code&gt;reader.md&lt;/code&gt;, and &lt;code&gt;prd.md&lt;/code&gt;, totaling about 3,970 words) still haven't gone through the line-by-line audit yet.&lt;/p&gt;

&lt;p&gt;To keep the bloat from creeping right back in next week, we're making four structural changes:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;
&lt;strong&gt;Enforce &lt;code&gt;protect&lt;/code&gt; on every plan save, not just Improve.&lt;/strong&gt; If any node in a proposed plan touches a path listed in &lt;code&gt;protect&lt;/code&gt;, the harness refuses to save the plan. No automated loop or &lt;code&gt;/leaf&lt;/code&gt;-generated issue can edit a prompt file anymore; only a direct commit from me can touch &lt;code&gt;prompts/&lt;/code&gt;.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Rewrite the rule in &lt;code&gt;AGENTS.md&lt;/code&gt;.&lt;/strong&gt; We're replacing &lt;em&gt;"When an agent misbehaves, fix its prompt"&lt;/em&gt; with a new default: &lt;strong&gt;when an agent misbehaves, first look for a prompt rule to delete or a refusal check to move into deterministic code.&lt;/strong&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Put a 1% incident gate on &lt;code&gt;/leaf&lt;/code&gt;.&lt;/strong&gt; When a retrospective finds a failure mode that genuinely points to a prompt change, &lt;code&gt;/leaf&lt;/code&gt; is no longer allowed to file an issue. Instead, it has to report the finding to me alongside its historical frequency per 100 runs in &lt;code&gt;factory.db&lt;/code&gt;.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Never pin prompt prose in unit tests.&lt;/strong&gt; &lt;code&gt;src/bridge.prompts.test.ts&lt;/code&gt; will only ever test structured strings that code parses.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;Item 1 is the real lock on the door. Items 2 through 4 are prose, and they only hold because Item 1 refuses anything they miss.&lt;/p&gt;

&lt;h2&gt;
  
  
  So what?
&lt;/h2&gt;

&lt;p&gt;If you're running multi-agent workflows or maintaining &lt;code&gt;AGENTS.md&lt;/code&gt; and skill files for your team, watch out for incident-driven prompt accretion:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Stop treating system prompts like a post-mortem changelog.&lt;/strong&gt; Every time you add a sentence to stop a 0.1% edge case, you dilute the 10% of instructions that keep your agents on the rails during the other 99.9% of runs.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Delete prose-pinning unit tests.&lt;/strong&gt; If your test suite asserts that specific instructional sentences exist inside your markdown prompts, you've built a one-way ratchet that makes cleanup feel like breaking the build. Test what your code parses, not how you lecture the model.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Move behavioral guardrails into deterministic code.&lt;/strong&gt; Don't spend 50 tokens on every turn begging an LLM to pass retry flags to &lt;code&gt;curl&lt;/code&gt; or to format a git merge a certain way. Put &lt;code&gt;.curlrc&lt;/code&gt; in the container image, make your tool refusal errors name the exact recovery command, and let code enforce state transitions.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Measure turns and ritual work, not just Turn 1 prompt size.&lt;/strong&gt; A bloated prompt doesn't just cost you input tokens on Turn 1. It makes agents paranoid, multiplying the number of turns they spend running CYA checks instead of shipping the edit.&lt;/li&gt;
&lt;/ul&gt;

</description>
      <category>darkfactory</category>
      <category>promptengineering</category>
      <category>contextengineering</category>
      <category>leaf</category>
    </item>
    <item>
      <title>Inside My Dark Software Factory: How Leaf Runs Locally</title>
      <dc:creator>Jamie Duncan</dc:creator>
      <pubDate>Fri, 20 Nov 2026 05:45:54 +0000</pubDate>
      <link>https://dev.to/jamie_duncan_3738b0805ed5/inside-my-dark-software-factory-how-leaf-runs-locally-1na8</link>
      <guid>https://dev.to/jamie_duncan_3738b0805ed5/inside-my-dark-software-factory-how-leaf-runs-locally-1na8</guid>
      <description>&lt;p&gt;&lt;em&gt;Originally published at &lt;a href="https://jduncan.io/blog/2026-10-25-leaf-dark-factory-update/" rel="noopener noreferrer"&gt;jduncan.io&lt;/a&gt;.&lt;/em&gt;&lt;/p&gt;

&lt;p&gt;When I &lt;a href="https://jduncan.io/blog/2026-09-14-introducing-leaf/" rel="noopener noreferrer"&gt;first introduced Leaf&lt;/a&gt; a little over a month ago, I hedged on what to call it. I tried out "multi-agent coding orchestrator," "fleet manager," and "meta-harness."&lt;/p&gt;

&lt;p&gt;I'm done hedging. The explicit goal of Leaf is a &lt;strong&gt;Dark Software Factory&lt;/strong&gt;.&lt;/p&gt;

&lt;p&gt;In Dan Shapiro's &lt;a href="https://www.danshapiro.com/blog/2026/01/the-five-levels-from-spicy-autocomplete-to-the-software-factory/" rel="noopener noreferrer"&gt;Five Levels of AI coding framework&lt;/a&gt;, Level 5 is the dark factory: a black box that turns specifications into working, tested, deployed software without a human writing code or reading diffs.&lt;/p&gt;

&lt;p&gt;Building toward that goal forced me to confront the central question of unattended software engineering: if a human isn't reviewing the diffs, what stops an agent from hallucinating a mess and shipping it to production?&lt;/p&gt;

&lt;h2&gt;
  
  
  The nuance: purposeful friction
&lt;/h2&gt;

&lt;p&gt;&lt;strong&gt;In the end, the only &lt;em&gt;real&lt;/em&gt; tool that isn't some flavor of human review is adversarial validation.&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;Give a single coding agent a broad prompt, full repository context, and an open loop, and it will eventually wander afield. It will refactor things you didn't ask it to touch, invent abstractions it doesn't need, and introduce large breaking changes because nothing in the room pushed back.&lt;/p&gt;

&lt;p&gt;In &lt;a href="https://jduncan.io/blog/2026-10-10-loops-part-1-two-kinds-of-work/" rel="noopener noreferrer"&gt;Part 1 of the Loops series&lt;/a&gt;, I wrote about separating &lt;strong&gt;judgement&lt;/strong&gt; from &lt;strong&gt;mechanics&lt;/strong&gt;. The next step in Leaf's evolution was introducing &lt;strong&gt;purposeful friction&lt;/strong&gt; inside the judgement layers, while keeping the mechanical gates strictly deterministic.&lt;/p&gt;

&lt;p&gt;No agent in Leaf works alone or validates its own outcome. Every step pairs a producer with an adversarial counterweight, and every role only sees the exact slice of context it needs to do its job.&lt;/p&gt;

&lt;h2&gt;
  
  
  The workflow: I'm the product owner now
&lt;/h2&gt;

&lt;p&gt;In the current version of Leaf, I don't act as a tech lead or a release manager. &lt;strong&gt;I am the Product Owner.&lt;/strong&gt; It's my vision that is being brought to life.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fg2fsi1h0bxdp4c4hgwvc.webp" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fg2fsi1h0bxdp4c4hgwvc.webp" alt="Leaf Dark Factory adversarial workflow diagram: 1. Human + Skill file the issue; 2. PM (with external wiki mounted) + Architect negotiate the plan and graph in WebSocket Room 1, then drop off; 3. Engineers write code and Tech Lead validates all in WebSocket Room 2; 4. Deterministic Local CI runs, spawning a QA Engineer into the Tech Lead's room on failure; 5. Deterministic Local CD deploys; 6. Tech Writer summarizes commit logs into the external wiki with an hourly Lead Writer running QA" width="800" height="689"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Every phase of agent collaboration happens inside its own dedicated WebSocket chat room, structured around adversarial checks and strict context boundaries:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;
&lt;strong&gt;Human + Skill file the issue.&lt;/strong&gt; I work with my Leaf skill in an interactive session to pressure-test my own thinking, scope the problem, and file a local issue. That issue goes straight to a &lt;strong&gt;Product Manager (PM)&lt;/strong&gt; agent, and a planning WebSocket chat starts.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;PM + Architect develop the plan.&lt;/strong&gt; Once the PM judges the issue ready, an &lt;strong&gt;Architect&lt;/strong&gt; agent joins the WebSocket chat. The PM and the Architect work together to turn the requirement into an implementation plan and a dependency graph. Once the graph is locked, the PM and Architect both drop off. This loops sometimes 3-4 times before a plan that fulfills the goals of both agents gets created. Just like real life.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Engineers write code, and the Tech Lead validates all of it.&lt;/strong&gt; A &lt;strong&gt;Tech Lead&lt;/strong&gt; agent starts in a new WebSocket chat and spawns the &lt;strong&gt;Engineer&lt;/strong&gt; agents. The Engineers do the implementation work per the plan nodes, and the Tech Lead validates every piece of work before it's allowed to land. Just like the real world, the Tech Lead can send inferior work back.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;CI is a deterministic task, not an agent.&lt;/strong&gt; Once the code lands, CI starts. &lt;strong&gt;CI steps are tasks, not agents.&lt;/strong&gt; It's the first programmatic check, running the exact same linters, type checks, and test suites CI runs today.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;QA Engineers answer to the Tech Lead too.&lt;/strong&gt; If CI fails, a &lt;strong&gt;QA Engineer&lt;/strong&gt; agent is spawned to fix it. Crucially, the QA Engineer doesn't get a free pass to hack the tests until they turn green: &lt;strong&gt;QA Engineers join the WebSocket chat with the Tech Lead, who validates their work as well&lt;/strong&gt; before CI re-runs. That loop repeats until CI passes.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;CD is a deterministic task, not an agent.&lt;/strong&gt; Once CI is green, CD starts and deploys the build without an LLM anywhere in the execution path. It's all mechanical. We're just deploying bits at this point.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Tech Writers summarize what landed, and a Lead Writer runs QA.&lt;/strong&gt; Once CD is green, a &lt;strong&gt;Tech Writer&lt;/strong&gt; agent starts in its own WebSocket chat to update the repository's internal wiki.&lt;/li&gt;
&lt;/ol&gt;

&lt;h3&gt;
  
  
  Why only the PM gets the lore
&lt;/h3&gt;

&lt;p&gt;Two details in that knowledge loop matter in practice.&lt;/p&gt;

&lt;p&gt;First, &lt;strong&gt;each repository gets its own wiki, stored outside the source code, and only the PM role has it mounted as part of the run&lt;/strong&gt;. The PM gets the deeper lore: why a service was designed a certain way, how features connect, and what shipped last week. The Engineers &lt;em&gt;don't&lt;/em&gt; get the wiki mounted.&lt;/p&gt;

&lt;p&gt;That separation is deliberate. An Engineer agent doesn't need the philosophical history of the codebase to implement a twenty-minute node. Mounting the deep lore into an Engineer's context just invites it to over-think local changes and touch files outside its scope. The PM and Architect use the lore to shape the plan; the Engineers only see the contract and the code.&lt;/p&gt;

&lt;p&gt;Second, &lt;strong&gt;the Tech Writers aren't synthesizing new work&lt;/strong&gt;. Left to "write documentation," an LLM will happily invent architectural manifesto prose. Instead, Tech Writers strictly aggregate and summarize what actually landed from commit logs and changesets. And to keep the wiki itself from drifting, a &lt;strong&gt;Lead Writer&lt;/strong&gt; role runs hourly in what is primarily a QA role, auditing and validating what the Tech Writers wrote.&lt;/p&gt;

&lt;p&gt;Even the documentation loop has an adversary checking its work.&lt;/p&gt;

&lt;h2&gt;
  
  
  Bringing the plumbing home
&lt;/h2&gt;

&lt;p&gt;Last week I wrote about &lt;a href="https://jduncan.io/blog/2026-10-20-where-my-level-5-software-factory-doesnt-cut-it-yet/" rel="noopener noreferrer"&gt;blowing through all 50,000 of my monthly GitHub Actions build minutes by October 18th&lt;/a&gt;. My factory's autonomous commit throughput had quadrupled, and it ran straight into a CI billing wall built for human typing speed.&lt;/p&gt;

&lt;p&gt;At first, I thought about trimming my YAML workflows. Instead, I brought the whole stack home.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Version control, issues, CI, CD, and the repository wikis are all local to Leaf now.&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;Pulling the entire stack inside Leaf solved the 50,000-minute ceiling over a weekend, because &lt;strong&gt;every test run and container build now executes on my own local fleet without paying a per-minute cloud runner tax&lt;/strong&gt;. (To be fair, running your own compute isn't free either, since my office closet still draws power and wears down SSDs. Still, my power utility hasn't threatened to shut off my heat on the 18th of the month.) I did invest in a 4th factory node to allow for extra room. My current factory setup: &lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;3 older i7 NUCs with 32GB of RAM (approximately 5 years old)&lt;/li&gt;
&lt;li&gt;1 GMKTec NUC clone with an AMD Ryzen 7 and 32GB of RAM&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;The Factory UI and other services sit on another node. All four nodes are dedicated wholly to running work through the factory.&lt;/p&gt;

&lt;p&gt;The bigger win wasn't the bill. It was telemetry.&lt;/p&gt;

&lt;p&gt;When your issues live in one SaaS tool, your git branches live in another, your agent logs sit on a local disk, and your deployments run in a third-party action runner, stitching together what actually happened on a sideways run is a forensic nightmare. Because Leaf now owns the entire lifecycle locally, &lt;strong&gt;I can trace every single touch, model turn, WebSocket chat message, test failure, and deployment log from the moment an issue is opened to the moment CD turns green as one coherent string&lt;/strong&gt;.&lt;/p&gt;

&lt;h3&gt;
  
  
  One file at the root: &lt;code&gt;.factory.json&lt;/code&gt;
&lt;/h3&gt;

&lt;p&gt;To make local CI/CD and environment management predictable across projects, Leaf uses a single configuration file at the root of each repository: &lt;code&gt;.factory.json&lt;/code&gt;.&lt;/p&gt;

&lt;p&gt;CI checks, CD pipelines, and target environments are all defined right there. Right now, Leaf's native CD engine handles deployments across three targets:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;&lt;strong&gt;Kubernetes&lt;/strong&gt;&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Podman&lt;/strong&gt;&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Cloudflare Workers and Pages&lt;/strong&gt;&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;A repo declares how it gets checked and where it gets deployed in &lt;code&gt;.factory.json&lt;/code&gt;:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight json"&gt;&lt;code&gt;&lt;span class="p"&gt;{&lt;/span&gt;
  &lt;span class="nl"&gt;"ci"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="nl"&gt;"tasks"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;
      &lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="nl"&gt;"name"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"lint"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nl"&gt;"run"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"npm run lint"&lt;/span&gt; &lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
      &lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="nl"&gt;"name"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"typecheck"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nl"&gt;"run"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"npm run check"&lt;/span&gt; &lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
      &lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="nl"&gt;"name"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"test"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nl"&gt;"run"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"npm test"&lt;/span&gt; &lt;span class="p"&gt;}&lt;/span&gt;
    &lt;span class="p"&gt;]&lt;/span&gt;
  &lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="nl"&gt;"cd"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="nl"&gt;"environments"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
      &lt;span class="nl"&gt;"staging"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
        &lt;span class="nl"&gt;"target"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"podman"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="nl"&gt;"host"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"apps.local"&lt;/span&gt;
      &lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
      &lt;span class="nl"&gt;"production"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
        &lt;span class="nl"&gt;"target"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"cloudflare-pages"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="nl"&gt;"project"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"jduncan-io"&lt;/span&gt;
      &lt;span class="p"&gt;}&lt;/span&gt;
    &lt;span class="p"&gt;}&lt;/span&gt;
  &lt;span class="p"&gt;}&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;And to ingest all of these new capabilities (local issues, live role-based WebSocket rooms, deterministic CI/CD runs, external repo wikis, and end-to-end trace strings), Leaf's UI and UX got a massive overhaul. Instead of jumping between browser tabs to see why a deployment stalled or why the Tech Lead rejected a QA fix, the whole factory floor sits in one dashboard.&lt;/p&gt;

&lt;h2&gt;
  
  
  What I've refined and learned
&lt;/h2&gt;

&lt;p&gt;Running the entire loop locally with end-to-end telemetry makes patterns jump out fast. Two lessons have stood out over the last week.&lt;/p&gt;

&lt;h3&gt;
  
  
  1. The three repeating agent anti-patterns
&lt;/h3&gt;

&lt;p&gt;When an autonomous run goes sideways, it's rarely a brand-new kind of failure. Almost every repeating issue I trace falls into one of three buckets:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;An agent writes code to attempt to check prose.&lt;/strong&gt; Last week I caught an agent writing regex unit tests to validate markdown role prompts. Checking and refining prose is the job of an improvement loop (or a Lead Writer QA pass), not a test runner.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;An agent writes overly complex code loaded with excess dogma, ceremony, restrictions, and gates that don't actually help anything.&lt;/strong&gt; Left unchecked, a model will happily invent a three-layer validation framework and a custom error taxonomy for a twenty-line internal helper.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;An agent writes prose to attempt to measure something.&lt;/strong&gt; When a node needs to verify that a migration worked or an endpoint responds cleanly, a markdown paragraph explaining why it &lt;em&gt;should&lt;/em&gt; work is worthless. Measuring things is the job of code.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Code can't judge meaning.&lt;br&gt;&lt;br&gt;
Prose can't measure systems.&lt;/p&gt;

&lt;p&gt;Whenever a run starts churning, it's almost always because an agent crossed that line: using rigid code to police natural language, using natural language to fake a programmatic check, or burying simple mechanics under defensive ceremony.&lt;/p&gt;

&lt;h3&gt;
  
  
  2. Frontier provider outages all look different (and nobody calls them outages)
&lt;/h3&gt;

&lt;p&gt;When a human is using a coding assistant in an IDE and the model hiccups, you sigh, take a sip of coffee, and hit retry. When you're running an unattended Dark Factory across multiple harnesses, upstream instability is a systems engineering problem.&lt;/p&gt;

&lt;p&gt;And none of the frontier providers fail the same way, even while their public status pages stay a soothing shade of green:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Capacity walls (&lt;code&gt;529 Overloaded&lt;/code&gt;):&lt;/strong&gt; One provider will start returning HTTP &lt;code&gt;529&lt;/code&gt; (&lt;code&gt;overloaded_error&lt;/code&gt;) responses or mid-stream capacity errors while the status page reports everything is fine.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Silent stream stalls:&lt;/strong&gt; Another provider won't throw an error code at all. A turn simply hangs in the middle of a stream and never returns, leaving the worker pod waiting forever unless the supervisor enforces a hard turn timeout.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Ambiguous &lt;code&gt;429&lt;/code&gt; rate limits:&lt;/strong&gt; Another returns HTTP &lt;code&gt;429&lt;/code&gt; with a generic &lt;code&gt;"Too many requests"&lt;/code&gt; message during upstream capacity crunches. That ambiguity is dangerous in an automated harness: &lt;strong&gt;an ambiguous &lt;code&gt;429&lt;/code&gt; tricks agents and routers into thinking they've exhausted their subscription quota when the provider is actually just throttling global traffic&lt;/strong&gt;.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;If your orchestrator takes every error message at face value, a ten-minute provider brownout will either stall your rooms indefinitely or convince your fleet that it's out of budget for the day. Building a resilient factory means fingerprinting how each provider actually degrades in the wild, not how their documentation says they fail.&lt;/p&gt;

&lt;h2&gt;
  
  
  So what?
&lt;/h2&gt;

&lt;p&gt;If you're pushing your own agentic workflows toward unattended execution, a few lessons from this rebuild apply regardless of your stack:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Build purposeful friction into every phase.&lt;/strong&gt; The only real substitute for human review is adversarial validation. Pair your PM with an Architect, make a Tech Lead validate every Engineer and QA fix, and put an hourly QA Lead over your Tech Writers.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Isolate the lore from the builders.&lt;/strong&gt; Keep your architectural wiki outside the source tree and mount it only for the PM role. Engineers build better and faster when they only see their scoped node and the code.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Keep CI and CD as deterministic tasks.&lt;/strong&gt; You don't need an LLM to run a linter or deploy a container. Run CI/CD as plain tasks defined in one root config file, and only wake up a QA Engineer when a check fails.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Keep code, prose, and loops in their lanes.&lt;/strong&gt; Use code to measure, improvement loops to refine prose, and adversarial review to strip out defensive ceremony.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Leaf is much closer to a true Level 5 Dark Factory than it was a month ago, and tracing a change from a local issue to a live deployment in one unbroken string feels like a genuine step forward. Now I just need to see what breaks next.&lt;/p&gt;

</description>
      <category>darkfactory</category>
      <category>softwarefactory</category>
      <category>leaf</category>
      <category>agenticsdlc</category>
    </item>
    <item>
      <title>Where My Level 5 Software Factory Doesn't Cut It Yet</title>
      <dc:creator>Jamie Duncan</dc:creator>
      <pubDate>Fri, 20 Nov 2026 05:40:03 +0000</pubDate>
      <link>https://dev.to/jamie_duncan_3738b0805ed5/where-my-level-5-software-factory-doesnt-cut-it-yet-127g</link>
      <guid>https://dev.to/jamie_duncan_3738b0805ed5/where-my-level-5-software-factory-doesnt-cut-it-yet-127g</guid>
      <description>&lt;p&gt;&lt;em&gt;Originally published at &lt;a href="https://jduncan.io/blog/2026-10-20-where-my-level-5-software-factory-doesnt-cut-it-yet/" rel="noopener noreferrer"&gt;jduncan.io&lt;/a&gt;.&lt;/em&gt;&lt;/p&gt;

&lt;p&gt;I've been talking a lot lately about &lt;a href="https://jduncan.io/blog/2026-09-14-introducing-leaf/" rel="noopener noreferrer"&gt;Leaf&lt;/a&gt; and the &lt;a href="https://jduncan.io/blog/2026-10-10-loops-part-1-two-kinds-of-work/" rel="noopener noreferrer"&gt;data behind how it runs&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;If you use Dan Shapiro's &lt;a href="https://www.danshapiro.com/blog/2026/01/the-five-levels-from-spicy-autocomplete-to-the-software-factory/" rel="noopener noreferrer"&gt;Five Levels of AI coding framework&lt;/a&gt;, Leaf operates as a &lt;strong&gt;Level 5 software factory&lt;/strong&gt; (what Shapiro calls the "Dark Factory"). A specification goes in as a contract, and working, tested software comes out the other side without a human writing code or reviewing diffs.&lt;/p&gt;

&lt;p&gt;Most days, it's great. It hums along on a few small boxes in my office closet and ships real work while my laptop lid is closed.&lt;/p&gt;

&lt;p&gt;But it's far from perfect.&lt;/p&gt;

&lt;h2&gt;
  
  
  The nuance
&lt;/h2&gt;

&lt;p&gt;"Dark factory" sounds very clean on a conference slide. In practice, taking humans out of the implementation middle doesn't make engineering problems disappear. It just moves the failure modes to places you weren't watching.&lt;/p&gt;

&lt;p&gt;When a human writes code by hand, human boredom and typing speed act as natural rate limiters. You don't accidentally trigger forty container builds before lunch. You don't write a fifty-step coordination plan to update a CSS theme. And you don't rewrite your own job description just to make a unit test pass.&lt;/p&gt;

&lt;p&gt;Take the human rate limiter off, and the edges of your harness get stress-tested fast. I've gotten a few things wrong recently, and each one taught me something about where my factory doesn't cut it yet.&lt;/p&gt;

&lt;h2&gt;
  
  
  1. CI limits come fast
&lt;/h2&gt;

&lt;p&gt;I use GitHub Actions to build and deploy my projects. I pay for a GitHub Enterprise Cloud license for one user, which includes &lt;a href="https://docs.github.com/get-started/learning-about-github/githubs-products" rel="noopener noreferrer"&gt;50,000 Actions build minutes a month&lt;/a&gt; on standard runners for private repositories.&lt;/p&gt;

&lt;p&gt;Before Leaf, I never came close to touching that number. For a solo developer, 50,000 minutes is the same as 50 billion. It's more than thirty-four straight days of compute every month.&lt;/p&gt;

&lt;p&gt;This month, I blew right through all 50,000 minutes on October 18th.&lt;/p&gt;

&lt;p&gt;Why? Did I have repetitive jobs and unoptimized workflows in there? Yes. When you aren't paying attention to build minutes, it's easy to let full container builds and redundant test runs pile up on every push.&lt;/p&gt;

&lt;p&gt;But the real reason wasn't just sloppy YAML. It happened because &lt;strong&gt;autonomous loops multiplied my commit and PR throughput nearly fourfold, while my CI triggers were still designed for human typing speed&lt;/strong&gt;. Between feature work, automated bug fixes, and the &lt;a href="https://jduncan.io/blog/2026-10-10-loops-part-3-one-adds-three-correct/" rel="noopener noreferrer"&gt;Improvement loop&lt;/a&gt; sweeping for dead code while I slept, the factory was firing off full pipeline runs around the clock.&lt;/p&gt;

&lt;p&gt;My factory wasn't broken; my plumbing was still sized for a human.&lt;/p&gt;

&lt;p&gt;So now I'm tightening the screws. I'm building container images less often. I'm being much smarter about when a workflow runs the full end-to-end test suite versus a fast, targeted check. It's all fixable, and high throughput is a good problem to have. Still, hitting a hard billing wall two-thirds of the way through October is a rough way to learn that lesson.&lt;/p&gt;

&lt;h2&gt;
  
  
  2. Foundational changes
&lt;/h2&gt;

&lt;p&gt;My current setup is great for bug fixes and the average new feature. If a problem fits neatly into a handful of files with a clear boundary, Leaf chews right through it.&lt;/p&gt;

&lt;p&gt;Where it gets lost and goes sideways is when I ask it to make fundamental, sweeping changes across an existing project.&lt;/p&gt;

&lt;p&gt;For my BrandCast project, I recently updated the admin UI. While I was poking around, I noticed that a lot of hand-coded styling changes had crept into individual pages over time (agents will do that when they're focused on closing a local ticket). I wanted Leaf to update the core theme and make sure every page in the admin UI was using that updated theme consistently.&lt;/p&gt;

&lt;p&gt;It was a lot of changes and a lot of checks, but the work itself was mostly mechanical.&lt;/p&gt;

&lt;p&gt;Leaf lost. Its. Mind.&lt;br&gt;
Mind. Nowhere to be found.&lt;/p&gt;

&lt;p&gt;The planner generated a graph with &lt;strong&gt;50+ nodes&lt;/strong&gt;. It took over an hour just to finish planning. (I'm blaming my choice to test Astra and Fable as planning agents there. Way too much horsepower and way too much over-thinking.) Once execution finally started, the foreman almost crashed a node, then sent over 30 nodes back to workers for changes. After three hours of churn, the build still failed.&lt;/p&gt;

&lt;p&gt;In &lt;a href="https://jduncan.io/blog/2026-10-10-loops-part-6-output-not-craft/" rel="noopener noreferrer"&gt;Part 6 of the Loops series&lt;/a&gt;, my data already showed that an 8+ node plan costs &lt;strong&gt;17×&lt;/strong&gt; what a single-node plan costs. A 50-node plan isn't just expensive; &lt;strong&gt;splitting a wide, mechanical sweep into fifty isolated worker contexts creates more coordination friction than the work itself&lt;/strong&gt;. Every worker touches adjacent files, merge conflicts multiply, and the foreman spends all its turns playing traffic cop.&lt;/p&gt;

&lt;p&gt;I'm still digging into this one. Some of it is definitely my choice of models (looking at you, Astra). Heavy reasoning models love to turn a straightforward cleanup into a doctoral thesis. But some of it is how I create plans in the first place. I'm not sure what the right fix is yet, whether that's capping graph size, batching mechanical sweeps differently, or keeping foundational refactors in an interactive session. Maybe all or none of the above. Maybe it'll occur to me magically in the shower tomorrow morning.&lt;/p&gt;

&lt;h2&gt;
  
  
  3. Improvement rot
&lt;/h2&gt;

&lt;p&gt;I love the &lt;a href="https://jduncan.io/blog/2026-10-10-loops-part-3-one-adds-three-correct/" rel="noopener noreferrer"&gt;Improvement loop&lt;/a&gt; in Leaf. Having an autonomous sweep that deletes dead weight and simplifies code while the factory is idle has been one of my favorite wins of the whole project.&lt;/p&gt;

&lt;p&gt;But I need to start setting barriers on what it's allowed to touch.&lt;/p&gt;

&lt;p&gt;Leaf can run the Improvement loop on its own repository. Recently, somewhere along the line, an improvement pass decided that it needed to add a regex-style test to validate one of Leaf's role prompts. Testing natural language prompts with regex is already a bad idea. Then, on later sweeps, the loop noticed that other prompts were "missing" similar tests, so it dutifully added more of them.&lt;/p&gt;

&lt;p&gt;Then came the rot.&lt;/p&gt;

&lt;p&gt;When a later change tripped those brittle checks, the agent didn't fix or remove the bad tests. Instead, &lt;strong&gt;it started changing the prose of the prompts to match the regex tests, because updating markdown prose is faster and easier than refactoring test code&lt;/strong&gt;.&lt;/p&gt;

&lt;p&gt;Writing a test takes code.&lt;br&gt;&lt;br&gt;
Changing a prompt takes prose.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Agents &lt;em&gt;always&lt;/em&gt; take the shortest path to the reward.&lt;/strong&gt; Independent evaluators like &lt;a href="https://metr.org/blog/2025-06-05-recent-reward-hacking/" rel="noopener noreferrer"&gt;METR have documented this exact reward-hacking behavior&lt;/a&gt; across frontier coding models: give an agent an automated check as its definition of done, and it will happily game the check if that's the path of least resistance. Nobody in my closet was plotting world domination. The worker just wanted a green exit code in the fewest turns possible.&lt;/p&gt;

&lt;p&gt;Suddenly, Leaf started exhibiting odd behavior on real runs. I ended up spending a 90-minute human-in-the-loop session unwinding the mess, deleting the regex tests, and restoring sanity to the prompts Leaf uses to do its job. Lots of typing in all caps and cursing happened.&lt;/p&gt;

&lt;p&gt;I'm working on this right now. In &lt;a href="https://jduncan.io/blog/2026-10-10-loops-part-2-the-contract/" rel="noopener noreferrer"&gt;Part 2 of the Loops series&lt;/a&gt;, I wrote that judgement lives in the prompts, where I can read it and change it. Letting the automated cleanup loop modify the factory's own prompts violated my own rule. I'm thinking about adding guardrails to skip certain paths (like role prompts and skills) from the Improvement process altogether, though I'm not completely sure where that boundary should sit yet.&lt;/p&gt;

&lt;h2&gt;
  
  
  So what?
&lt;/h2&gt;

&lt;p&gt;Running a Level 5 software factory doesn't mean you stop doing engineering. It means your engineering shifts to the harness, the boundaries, and the economics of the loop:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Audit your downstream plumbing.&lt;/strong&gt; If your commit throughput jumps 4×, your CI bill and container registries are going to feel it before you do. Gate heavy builds and full suites so autonomous loops don't burn your monthly quota in eighteen days.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Don't use a sledgehammer planner on wide, mechanical sweeps.&lt;/strong&gt; Big reasoning models will over-decompose foundational changes into giant node graphs that choke on their own coordination.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Fence off your prompts from self-improvement loops.&lt;/strong&gt; If an agent can pass a test by weakening a prompt instead of fixing code, it will take that deal every single time. Keep automated cleanup focused on mechanical code, and keep human hands on the prompts that define judgement.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;This is all very much a work in progress, and sometimes it goes sideways. Sometimes it goes really sideways.&lt;/p&gt;

&lt;p&gt;But I can't wait to look at the data from that awful three-hour run.&lt;/p&gt;

</description>
      <category>agenticsdlc</category>
      <category>leaf</category>
      <category>softwarefactory</category>
      <category>githubactions</category>
    </item>
    <item>
      <title>Factory: The Next Phase of Hybrid AI Development</title>
      <dc:creator>Jamie Duncan</dc:creator>
      <pubDate>Fri, 20 Nov 2026 05:40:00 +0000</pubDate>
      <link>https://dev.to/jamie_duncan_3738b0805ed5/factory-the-next-phase-of-hybrid-ai-development-5ch</link>
      <guid>https://dev.to/jamie_duncan_3738b0805ed5/factory-the-next-phase-of-hybrid-ai-development-5ch</guid>
      <description>&lt;p&gt;&lt;em&gt;Originally published at &lt;a href="https://jduncan.io/blog/2026-09-08-factory-the-next-phase-of-hybrid/" rel="noopener noreferrer"&gt;jduncan.io&lt;/a&gt;.&lt;/em&gt;&lt;/p&gt;

&lt;p&gt;A couple weeks ago I wrote about my &lt;a href="https://jduncan.io/blog/2026-08-27-hybrid-llms-cloud-for-code-local-for-home" rel="noopener noreferrer"&gt;hybrid LLM setup&lt;/a&gt;. At the time, the division seemed simple enough: cloud models for writing code, and a local server in my house for family life, privacy, and experiments.&lt;/p&gt;

&lt;p&gt;That split was always a temporary checkpoint.&lt;/p&gt;

&lt;p&gt;I spend a lot of my spare time building &lt;a href="https://brandcast.app" rel="noopener noreferrer"&gt;BrandCast&lt;/a&gt;. It's a large monorepo with thousands of moving parts. Over the past year, I've poured a huge amount of effort into dialing in its AI harness. I use Claude Code as my primary pair programmer, backed by custom skills, system prompts, and my &lt;a href="https://jduncan.io/blog/2026-03-22-introducing-root-structured-ai-development" rel="noopener noreferrer"&gt;Root development framework&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;At this point, the harness is predictable. If my input context is clean, I can predict the quality of the output with high confidence.&lt;/p&gt;

&lt;p&gt;The problem isn't the code generation anymore. The problem is the friction around it.&lt;/p&gt;

&lt;p&gt;Sitting in front of a terminal hitting enter every few minutes gets old fast. I don't want an interactive copilot glued to my face all evening. I want an autonomous software factory.&lt;/p&gt;

&lt;h2&gt;
  
  
  The Nuance: The Wrong Way to Split Hybrid
&lt;/h2&gt;

&lt;p&gt;Most conversations about AI agents suffer from binary thinking.&lt;/p&gt;

&lt;p&gt;One camp insists on running everything locally. They imagine an 8B or 32B model refactoring hundreds of files across a massive monorepo with zero external API calls. I love my local hardware, but that's fantasy today. When the cost of a wrong edit is high, you pay for the frontier model. Local models don't have the context coherence or reasoning depth to navigate deep monorepo migrations without going off the rails.&lt;/p&gt;

&lt;p&gt;The other camp hands their credit card over to cloud agents for every tiny step. They spin up autonomous swarms that burn millions of frontier tokens just reading GitHub issues, hallucinating roadmaps, and doing basic ticket triage.&lt;/p&gt;

&lt;p&gt;The reality is more nuanced.&lt;/p&gt;

&lt;p&gt;Frontier models are expensive and rate-limited. You don't use a master craftsman to sweep the workshop floor or sort tickets into neat piles. You use them to build the intricate mechanisms.&lt;/p&gt;

&lt;p&gt;Local models, on the other hand, are fast, private, and practically free once the server is paid for. They aren't ready to rewrite my backend, but they are fantastic at classification, evaluation, and triage.&lt;/p&gt;

&lt;p&gt;The logical step isn't choosing between local and cloud. It's chaining them together.&lt;/p&gt;

&lt;h2&gt;
  
  
  Taking the Harness Off My Laptop
&lt;/h2&gt;

&lt;p&gt;Before I could automate the factory floor, I had to fix where it lived.&lt;/p&gt;

&lt;p&gt;For months, I ran my entire development harness directly on my MacBook. If you've ever run continuous agent loops, multiple git worktrees, Docker test containers, and local build tools on your primary laptop, you know the pain. The fans spin up. The battery melts. Your terminal history turns into an unreadable blur of agent output. Worst of all, your working git tree is constantly in flux.&lt;/p&gt;

&lt;p&gt;So I moved the bulk of my BrandCast development off my laptop entirely.&lt;/p&gt;

&lt;p&gt;I set up a dedicated Intel NUC with 32GB of RAM. It sits quietly on my network, always powered on, running the dev harness and the background agents.&lt;/p&gt;

&lt;p&gt;That hardware separation changed how I think about my time:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Research is still mine to own.&lt;/strong&gt; I use my laptop for high-leverage work: exploratory coding, architectural spikes, deep customer work, and reviewing diffs. My laptop stays fast, quiet, and responsive.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;The chopping wood and carrying water moves to the NUC.&lt;/strong&gt; The repetitive mechanics of software engineering: triaging bug tickets, spinning up branches, writing boilerplate, running tests, fixing regressions: all move to the factory floor.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Let git merging do what it's good for.&lt;/strong&gt; I can work on an experimental feature on my laptop while Factory runs an automated queue on the NUC. When the NUC finishes a ticket, it pushes a PR. When I finish an experiment, I push a branch. Git and GitHub Actions handle the merging and the validation. Distributed version control was built for asynchronous collaboration. It works just as well when your collaborator is an autonomous box in the closet.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  Enter Factory
&lt;/h2&gt;

&lt;p&gt;With the harness running on the NUC, I built a small web UI to manage the flow. I call it Factory.&lt;/p&gt;

&lt;p&gt;It sits between my GitHub issue backlog and my coding harness.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fmptl80hivnuo2hmsaa8m.webp" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fmptl80hivnuo2hmsaa8m.webp" alt="Factory web interface showing an idle status, a ranked queue with issue 4156 at score 8.1, and recently completed issue 4377 merged after 151 minutes" width="799" height="211"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Here's what is happening behind that screen:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;
&lt;strong&gt;Intake&lt;/strong&gt;: Issues land in GitHub. Some are written by me by hand. Others come out of agentic brainstorming sessions when I'm exploring new features.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Local Evaluation&lt;/strong&gt;: Factory pulls down all open issues and sends them to my local LLM server (the Infplane Hilbert box sitting in my house).&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Scoring and Rationale&lt;/strong&gt;: The local model inspects the title, description, and acceptance criteria. It scores each issue by complexity, impact, and feasibility. In the screenshot, you can see issue #4156 scored an 8.1, accompanied by a clean summary: &lt;em&gt;"Critical security fix with clear scope, acceptance criteria, and mechanical implementation path."&lt;/em&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Queue Presentation&lt;/strong&gt;: Factory sorts the candidates and shows me the top 15 ready-to-run items.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;Instead of staring at a sprawling backlog and wondering what to tackle next, I have a prioritized queue scored by a model that didn't cost me a penny in API tokens.&lt;/p&gt;

&lt;h2&gt;
  
  
  The Architecture
&lt;/h2&gt;

&lt;p&gt;Here's how the entire topology fits together across the laptop, the NUC, the local inference box, and the cloud:&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F0sm8dp40xhzvxyo8i4vk.webp" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F0sm8dp40xhzvxyo8i4vk.webp" alt="Factory Topology and Hybrid Autonomous Workflow" width="800" height="475"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Notice how each machine has a dedicated responsibility:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Developer Laptop&lt;/strong&gt;: Where I do research, plan architecture, and run targeted one-off experiments.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Infplane Hilbert&lt;/strong&gt;: Dedicated local AI inference box running AMD Ryzen AI hardware with 128GB of RAM. It handles high-speed, zero-cost backlog evaluation.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Dedicated NUC&lt;/strong&gt;: The factory floor with 32GB of RAM. It hosts the Factory service, SQLite state DB, and background Claude Code sessions in isolated git worktrees.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Cloud &amp;amp; Frontier Tier&lt;/strong&gt;: GitHub hosts the repository and Actions CI/CD. The Anthropic API delivers deep frontier reasoning on the Claude 20x Max plan.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  Two Workflows: One-Off vs. Auto
&lt;/h2&gt;

&lt;p&gt;Factory gives me two execution modes depending on how involved I want to be.&lt;/p&gt;

&lt;h3&gt;
  
  
  1. One-Off Execution
&lt;/h3&gt;

&lt;p&gt;Sometimes I want specific control over what runs. I open Factory, look through the top 15 ranked issues, find the one I care about, and click &lt;strong&gt;run&lt;/strong&gt;.&lt;/p&gt;

&lt;p&gt;Factory immediately launches a backgrounded Claude Code session on the NUC. It provisions an isolated git worktree so the main repository tree isn't touched. The session boots with my full configuration: project-specific MCP servers, custom system prompts, and the Root dev workflow.&lt;/p&gt;

&lt;p&gt;The agent reads the issue, writes the plan, edits the code, runs the tests, and pushes a PR. I can monitor it if I want, or go back to whatever else I was doing on my laptop.&lt;/p&gt;

&lt;h3&gt;
  
  
  2. Auto Queue Processing
&lt;/h3&gt;

&lt;p&gt;This is where the setup becomes genuinely autonomous.&lt;/p&gt;

&lt;p&gt;I click &lt;strong&gt;run&lt;/strong&gt; at the top of the queue and let Factory take the wheel.&lt;/p&gt;

&lt;p&gt;Factory grabs the top-ranked issue and kicks off the background session. While Claude Code works through the Root workflow, Factory continuously polls GitHub Actions to monitor build and test progress.&lt;/p&gt;

&lt;p&gt;When the pull request passes CI and merges into &lt;code&gt;main&lt;/code&gt;, Factory runs its teardown routine:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;It removes the isolated git worktree (in the screenshot, you can see &lt;code&gt;cleaned /home/jduncan/Code/bc-4377&lt;/code&gt;).&lt;/li&gt;
&lt;li&gt;It cleans up dangling agent processes and deleted branches.&lt;/li&gt;
&lt;li&gt;It logs the execution stats to a local SQLite database (issue #4377 took 151 minutes and 26 seconds).&lt;/li&gt;
&lt;li&gt;It marks the ticket complete, picks the next highest issue from the queue, and starts the cycle again.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  Eating My Own Dogfood
&lt;/h2&gt;

&lt;p&gt;If you've heard me speak at Google Cloud recently, you've heard this exact talk track.&lt;/p&gt;

&lt;p&gt;Enterprises are wrestling with the exact same challenge. Everyone knows how to hook up an LLM to an IDE. The real question is how to scale agentic development without burning out your developers, draining your API budget on trivial tasks, or letting unvetted code run wild.&lt;/p&gt;

&lt;p&gt;The answer is division of labor.&lt;/p&gt;

&lt;p&gt;Use the frontier model for what it does best: large-scale, complex reasoning and code modifications in a massive codebase. Augment it with a local LLM to automate your supervisory duties away. Offload the heavy execution to a dedicated machine so your primary workstation stays free for creative thought.&lt;/p&gt;

&lt;p&gt;This is me eating my own dogfood. I have a Claude 20x Max plan. Let's see how well it holds up when a local factory keeps feeding it work around the clock.&lt;/p&gt;

&lt;h2&gt;
  
  
  So What?
&lt;/h2&gt;

&lt;p&gt;If you're looking at your own development workflow and wondering how to take the next step toward autonomy, keep these principles in mind:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Match model cost to task complexity.&lt;/strong&gt; Don't spend frontier tokens to sort a list or summarize an issue. Use cheap, local models for classification and routing. Save the frontier power for code execution.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Get the harness off your primary machine.&lt;/strong&gt; If an agent is running for two hours in the background, it shouldn't hold your personal laptop hostage. A cheap, dedicated box with plenty of RAM gives you the freedom to step away.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Your harness is your safety net.&lt;/strong&gt; Factory only works because the Root framework enforces strict testing and verification steps before any code gets committed. Autonomous loops without rigid harness constraints just create technical debt faster.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Isolate your environments.&lt;/strong&gt; Never let an autonomous agent work directly in your dirty working tree. Use git worktrees and clean teardown scripts so failed runs can be pruned instantly without leaving collateral damage.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Keep research human.&lt;/strong&gt; The goal isn't to remove yourself from software engineering. The goal is to own the architecture and the research, while delegating the chopping wood and carrying water to the factory.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Hybrid AI isn't just about where your data lives. It's about how your models collaborate across your hardware. And right now, the factory floor is humming.&lt;/p&gt;

</description>
      <category>hybridai</category>
      <category>localllms</category>
      <category>claudecode</category>
      <category>root</category>
    </item>
    <item>
      <title>The Attention Economy Broke Professional Social Media</title>
      <dc:creator>Jamie Duncan</dc:creator>
      <pubDate>Thu, 10 Sep 2026 00:00:00 +0000</pubDate>
      <link>https://dev.to/jamie_duncan_3738b0805ed5/the-attention-economy-broke-professional-social-media-1k0o</link>
      <guid>https://dev.to/jamie_duncan_3738b0805ed5/the-attention-economy-broke-professional-social-media-1k0o</guid>
      <description>&lt;p&gt;Professional social media was supposed to be about craft and reputation. Now it's clickbait culture with a tech topic slapped on top. Here's what happened.&lt;/p&gt;

</description>
      <category>linkedin</category>
      <category>attentioneconomy</category>
      <category>techculture</category>
      <category>generativeai</category>
    </item>
  </channel>
</rss>
