<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:dc="http://purl.org/dc/elements/1.1/">
  <channel>
    <title>DEV Community: Christian Anderson</title>
    <description>The latest articles on DEV Community by Christian Anderson (@iam-tech).</description>
    <link>https://dev.to/iam-tech</link>
    <image>
      <url>https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F4110779%2Ff7ed549e-875c-4c07-8cb7-c1d3ff6a355e.jpg</url>
      <title>DEV Community: Christian Anderson</title>
      <link>https://dev.to/iam-tech</link>
    </image>
    <atom:link rel="self" type="application/rss+xml" href="https://dev.to/feed/iam-tech"/>
    <language>en</language>
    <item>
      <title>Discord bot offline a week while systemd said active: a plugin timeout</title>
      <dc:creator>Christian Anderson</dc:creator>
      <pubDate>Thu, 19 Nov 2026 09:00:17 +0000</pubDate>
      <link>https://dev.to/iam-tech/discord-bot-offline-a-week-while-systemd-said-active-a-plugin-timeout-5bkp</link>
      <guid>https://dev.to/iam-tech/discord-bot-offline-a-week-while-systemd-said-active-a-plugin-timeout-5bkp</guid>
      <description>&lt;p&gt;Discord is how I talk to my homelab's AI agents. It's basically the front door. From 30 September to 7 October 2026 that front door was shut, and everything I had watching it said it was fine.&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;code&gt;hermes-gateway&lt;/code&gt; (the agent's gateway, Hermes Agent) was &lt;code&gt;active (running)&lt;/code&gt;
&lt;/li&gt;
&lt;li&gt;my gateway watchdog was green&lt;/li&gt;
&lt;li&gt;the gateway's own state file said Discord was &lt;code&gt;connected&lt;/code&gt;
&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;None of that was true for Discord. This is how a 10 second timeout did a week of damage.&lt;/p&gt;

&lt;h2&gt;
  
  
  The reboot
&lt;/h2&gt;

&lt;p&gt;Around 02:02 on 30 September the Proxmox node that runs the agent rebooted. That node keeps all its guests on one 5400rpm hard disk, so every boot is an IO storm for a good 15 minutes. I wrote about that separately. The short version is that everything starting up is fighting over one slow spindle, so everything is slow.&lt;/p&gt;

&lt;p&gt;The gateway loads its chat platforms as plugins. Each plugin gets a time limit to import and register itself: &lt;code&gt;plugins.load_timeout_seconds&lt;/code&gt;, which defaults to &lt;strong&gt;10 seconds&lt;/strong&gt;. On a calm boot that's loads. In the middle of the storm, the Discord plugin's import didn't make it.&lt;/p&gt;

&lt;h2&gt;
  
  
  What the logs said
&lt;/h2&gt;

&lt;p&gt;When I dug into it on 7 October, the logs were clear enough:&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F2fjhshb1fculu3erqmss.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F2fjhshb1fculu3erqmss.png" alt="Real log lines: the discord-platform plugin load timed out after 10s, then the gateway started with no adapter for discord, connected webhook and email, and reported running with 2 platforms" width="800" height="443"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;The plugin timed out after 10 seconds. Its &lt;code&gt;register_platform()&lt;/code&gt; call did come back, about 0.7 seconds later, and got ignored because the loader had already given up. A few minutes later the gateway started, logged &lt;code&gt;No adapter available for discord&lt;/code&gt;, connected the two platforms it did have (webhook and email), and said &lt;code&gt;Gateway running with 2 platform(s)&lt;/code&gt;.&lt;/p&gt;

&lt;p&gt;So the gateway was up and working. It just wasn't on Discord. And nothing treats "running with 2 platforms instead of 3" as an error.&lt;/p&gt;

&lt;h2&gt;
  
  
  Why every check stayed green
&lt;/h2&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Ft07xm8azwmgmlycl8r0z.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Ft07xm8azwmgmlycl8r0z.png" alt="A table: systemctl said active, the watchdog said green, the state file said connected, and nothing asked whether Discord was actually connected right now" width="800" height="414"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;systemd&lt;/strong&gt; was right. The process was up. It only knows about the process.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;My watchdog&lt;/strong&gt; was right too, about the gateway. It wasn't Discord-aware, so as long as the gateway was alive, it was happy.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;The state file&lt;/strong&gt; was the nasty one. The gateway keeps a &lt;code&gt;gateway_state.json&lt;/code&gt; with a row per platform. The Discord row still said &lt;code&gt;connected&lt;/code&gt;. But that row had a &lt;code&gt;writer_pid&lt;/code&gt; of 15603, and process 15603 was dead. The running gateway never loaded the Discord adapter, so nothing ever replaced that row, and it just sat there saying "connected" for a week.&lt;/p&gt;

&lt;p&gt;That's the general trap: a status row is only as true as whoever last wrote it. If the writer is dead, the row is history, not status.&lt;/p&gt;

&lt;h2&gt;
  
  
  The fix
&lt;/h2&gt;

&lt;p&gt;Two parts.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;1. Give the plugin more time.&lt;/strong&gt; In the gateway's &lt;code&gt;config.yaml&lt;/code&gt;:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight yaml"&gt;&lt;code&gt;&lt;span class="na"&gt;plugins&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
  &lt;span class="na"&gt;load_timeout_seconds&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mi"&gt;90&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Then a gateway restart, and the log said &lt;code&gt;[Discord] Connected as&lt;/code&gt; with the bot's name. A week late, but there.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;2. A health check that asks the actual question.&lt;/strong&gt; I wrote a small script, &lt;code&gt;discord-health.py&lt;/code&gt;, that exits 0 if healthy and 1 with a reason if not. It only trusts the state file if the live gateway wrote it:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;the state file's pid matches the gateway service's current &lt;code&gt;MainPID&lt;/code&gt;
&lt;/li&gt;
&lt;li&gt;the state file's &lt;code&gt;updated_at&lt;/code&gt; is less than 10 minutes old&lt;/li&gt;
&lt;li&gt;the Discord row's &lt;code&gt;writer_pid&lt;/code&gt; matches that same &lt;code&gt;MainPID&lt;/code&gt;
&lt;/li&gt;
&lt;li&gt;the Discord row's state is &lt;code&gt;connected&lt;/code&gt;
&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;The core of it is only a few lines. This is check 3, straight from the script:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="kt"&gt;str&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;d&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;get&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;"writer_pid"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;!=&lt;/span&gt; &lt;span class="n"&gt;pid&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="k"&gt;print&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;f&lt;/span&gt;&lt;span class="s2"&gt;"discord row written by pid {d.get('writer_pid')}, not live gateway {pid} (adapter never loaded)"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt; &lt;span class="n"&gt;sys&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;exit&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mi"&gt;1&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Check 3 is the one that would have caught this on day one. A Discord row written by any process other than the running gateway means the adapter never loaded this time round.&lt;/p&gt;

&lt;p&gt;One gotcha: it shells out to &lt;code&gt;systemctl --user&lt;/code&gt; to get the &lt;code&gt;MainPID&lt;/code&gt;, and when you run that from a timer as root it needs &lt;code&gt;XDG_RUNTIME_DIR=/run/user/0&lt;/code&gt; set, or it can't find the user bus.&lt;/p&gt;

&lt;h2&gt;
  
  
  The watchdog now knows about Discord
&lt;/h2&gt;

&lt;p&gt;I hooked that check into the gateway watchdog, which runs every 3 minutes on a systemd timer:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;5 minutes of grace after the gateway starts, so it doesn't fight a slow boot&lt;/li&gt;
&lt;li&gt;restart only after &lt;strong&gt;2 failures in a row&lt;/strong&gt;
&lt;/li&gt;
&lt;li&gt;at most &lt;strong&gt;3 restarts an hour&lt;/strong&gt;, then it holds off instead of looping&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;When it restarts the gateway, or hits that hold-off, it posts to my network alerts channel. It does that through Discord's REST API, not through the bot's own connection, so the alert doesn't depend on the bot being connected.&lt;/p&gt;

&lt;h2&gt;
  
  
  What I'd do differently
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Health checks should ask about the thing you use, not the process.&lt;/strong&gt; "Is the service active" and "can I talk to it on Discord" are different questions.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Treat stale state as unknown, not as good.&lt;/strong&gt; Any status row should carry who wrote it and when, and the reader should check both.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Default timeouts are tuned for calm boots.&lt;/strong&gt; If your host has a known slow boot, every startup timeout on it is a bet. Ten seconds lost that bet.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;"Running with 2 platforms" should be loud&lt;/strong&gt; when you configured 3. I'd like the gateway itself to fail health when a configured platform has no adapter.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Check your edits survive upgrades.&lt;/strong&gt; That timeout key lives in &lt;code&gt;config.yaml&lt;/code&gt;. If an upgrade regenerates the config, it could quietly go back to 10 seconds. It's on my list to re-check after every update.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;How do you check your bots are actually connected, rather than just running? I'd genuinely like to know what other people use.&lt;/p&gt;




&lt;blockquote&gt;
&lt;p&gt;🤖 &lt;em&gt;Drafted with AI assistance from my own homelab notes, logs and repos, and fact-checked against them before publishing.&lt;/em&gt;&lt;/p&gt;
&lt;/blockquote&gt;

</description>
      <category>ai</category>
      <category>agents</category>
      <category>python</category>
      <category>node</category>
    </item>
    <item>
      <title>Proxmox on a 5400rpm HDD: why every reboot gave me 15 minutes of 502s</title>
      <dc:creator>Christian Anderson</dc:creator>
      <pubDate>Wed, 18 Nov 2026 09:00:18 +0000</pubDate>
      <link>https://dev.to/iam-tech/proxmox-on-a-5400rpm-hdd-why-every-reboot-gave-me-15-minutes-of-502s-3j5p</link>
      <guid>https://dev.to/iam-tech/proxmox-on-a-5400rpm-hdd-why-every-reboot-gave-me-15-minutes-of-502s-3j5p</guid>
      <description>&lt;p&gt;On 28 August 2026 I couldn't log into my AI agent's dashboard. While I was looking into it, the Proxmox node it runs on patched itself and rebooted. Which turned out to be handy, because it showed me the actual problem, and it wasn't the dashboard.&lt;/p&gt;

&lt;h2&gt;
  
  
  What happened
&lt;/h2&gt;

&lt;p&gt;At 09:05:37 the node ran &lt;code&gt;apt-get dist-upgrade&lt;/code&gt;. That's my deliberate patching path, not an accident. It took Proxmox VE from 9.2.2 to 9.2.11 and pulled in a new kernel. At 09:09 the node rebooted. Nothing had scheduled that reboot in a timer, it came out of the patch run.&lt;/p&gt;

&lt;p&gt;Then the agent's dashboard took about 15 minutes to serve a single request. For the whole of that, Caddy in front of it answered &lt;strong&gt;502&lt;/strong&gt;.&lt;/p&gt;

&lt;h2&gt;
  
  
  The disk
&lt;/h2&gt;

&lt;p&gt;Here's where every guest on that node lives:&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fivwzzgbbgwhbo1e28j7f.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fivwzzgbbgwhbo1e28j7f.png" alt="lsblk output showing the guests on a 931.5G 5400rpm HDD and the Proxmox root on a 13.4G Optane, plus io pressure full avg60=81, load 23, 87% iowait" width="800" height="364"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;One 931.5G spinning drive, 5400rpm. The fast Optane is only 13G, so it just holds the Proxmox root.&lt;/p&gt;

&lt;p&gt;After the reboot, five containers cold-booted at once on that one spindle. The heavy ones were:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;ClickHouse, inside my Langfuse container&lt;/li&gt;
&lt;li&gt;MariaDB plus a live Django &lt;code&gt;python manage.py migrate&lt;/code&gt;, inside a 3D-printer monitoring container&lt;/li&gt;
&lt;li&gt;Nextcloud&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;That drove &lt;code&gt;/proc/pressure/io&lt;/code&gt; to &lt;code&gt;full avg60=81&lt;/code&gt;, load average 23, and 87% iowait.&lt;/p&gt;

&lt;p&gt;I checked whether the storage itself was unwell. It wasn't. The thin pool was 7.8% used, no IO errors, no device-mapper or ext4 faults. It was just seek contention: lots of things wanting different bits of one slow disk at the same time. There's no tuning knob for that. A spinning disk can only be in one place.&lt;/p&gt;

&lt;h2&gt;
  
  
  The signature to recognise
&lt;/h2&gt;

&lt;p&gt;This is the bit I actually wanted to write down:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;code&gt;systemctl is-active&lt;/code&gt; for the agent's web service says &lt;strong&gt;active&lt;/strong&gt;
&lt;/li&gt;
&lt;li&gt;nothing is listening on its port&lt;/li&gt;
&lt;li&gt;the process is in &lt;strong&gt;&lt;code&gt;D&lt;/code&gt; state&lt;/strong&gt; (uninterruptible sleep), with its wchan showing &lt;code&gt;wait_on_buffer&lt;/code&gt;, &lt;code&gt;d_alloc_parallel&lt;/code&gt; or &lt;code&gt;dm_thin_find_block&lt;/code&gt;
&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;"Active" plus "not listening" looks like a hung service. The obvious move is to restart it. Don't. &lt;code&gt;D&lt;/code&gt; state with those wait channels means it's waiting on the disk. It hasn't crashed. Restarting it only queues the same IO again, behind everyone else's.&lt;/p&gt;

&lt;h2&gt;
  
  
  What actually shortened it
&lt;/h2&gt;

&lt;p&gt;I stopped the Langfuse container. It's pure tracing, nothing depends on it being up, and ClickHouse is by far the heaviest thing starting on that box.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;The dashboard came up 21 seconds later.&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;I started Langfuse again once &lt;code&gt;full avg60&lt;/code&gt; in &lt;code&gt;/proc/pressure/io&lt;/code&gt; had dropped under 25. That took 260 seconds.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F3xj07z2srqjhq5eb9mf8.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F3xj07z2srqjhq5eb9mf8.png" alt="Timeline: 09:05:37 dist-upgrade, 09:09 reboot, about 15 minutes of 502s, stop Langfuse and the dashboard is up 21 seconds later, restart Langfuse after 260 seconds" width="800" height="364"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;The one I deliberately didn't stop is the printer-monitoring container. It runs a Django database migration at boot, and killing a migration halfway through is a real way to break something. Pick the guest that's heavy and doesn't matter, not just the heaviest one.&lt;/p&gt;

&lt;p&gt;Also worth knowing: every guest has &lt;code&gt;onboot: 1&lt;/code&gt; and they do all come back on their own. A couple of them just lag. So "stopped" a minute after boot isn't a fault. Look again before you start poking them.&lt;/p&gt;

&lt;h2&gt;
  
  
  Two things that broke because the boot was slow
&lt;/h2&gt;

&lt;p&gt;A slow boot shows up the stuff that only worked because boots used to be quick.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;1. The gateway watchdog was dead exactly when I needed it.&lt;/strong&gt; I have a systemd timer that restarts the agent's gateway if it falls over. It fires 2 minutes after boot (&lt;code&gt;OnBootSec=2min&lt;/code&gt;). On this boot, 2 minutes wasn't enough for root's user manager to have created &lt;code&gt;/run/user/0&lt;/code&gt;, so the watchdog's &lt;code&gt;systemctl --user&lt;/code&gt; call died with "Failed to connect to bus". So the thing that revives the gateway failed on the one boot where it mattered.&lt;/p&gt;

&lt;p&gt;The fix: the watchdog script now waits up to 60 seconds for the user bus (and starts &lt;code&gt;user@0.service&lt;/code&gt; itself if it has to), and exits 0 instead of failing. Plus a drop-in with &lt;code&gt;After=&lt;/code&gt; and &lt;code&gt;Wants=user@0.service&lt;/code&gt;.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;2. &lt;code&gt;systemd-journal-flush.service&lt;/code&gt; timed out&lt;/strong&gt; inside the container after 90 seconds and left it &lt;code&gt;degraded&lt;/code&gt;. That one was just collateral. It had worked on every boot before. &lt;code&gt;systemctl restart systemd-journal-flush&lt;/code&gt; once the IO had calmed down cleared it.&lt;/p&gt;

&lt;h2&gt;
  
  
  A side lesson about writing that fix
&lt;/h2&gt;

&lt;p&gt;I first wrote the new watchdog script with a heredoc through &lt;code&gt;ssh&lt;/code&gt; into the node, then &lt;code&gt;pct exec&lt;/code&gt; into the container, then &lt;code&gt;bash -lc&lt;/code&gt;. Three layers of quoting. The file ended up with a literal &lt;code&gt;\$(seq ...)&lt;/code&gt; in it, a syntax error, and the unit failed.&lt;/p&gt;

&lt;p&gt;What works: write the file locally, &lt;code&gt;base64 -w0&lt;/code&gt; it, pipe that in and decode it on the other side. Then run &lt;code&gt;bash -n&lt;/code&gt; on it before you start the unit.&lt;/p&gt;

&lt;h2&gt;
  
  
  What I'd do differently
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Don't put all your guests on one spinning disk.&lt;/strong&gt; I know. The real fix is an SSD, and nothing in this post replaces that.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Know which guest is your release valve before the storm.&lt;/strong&gt; For me it's the tracing box. Heavy, and nobody misses it for five minutes.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Never restart a &lt;code&gt;D&lt;/code&gt; state process to "unstick" it.&lt;/strong&gt; Check &lt;code&gt;/proc/&amp;lt;pid&amp;gt;/wchan&lt;/code&gt; first.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Test your watchdogs on a slow boot.&lt;/strong&gt; A watchdog that only works when everything else is fine isn't much of a watchdog.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Patch reboots are boots too.&lt;/strong&gt; Whatever happens on a cold start happens every time you patch the host.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;There's a follow-up to this. A month later the same node rebooted again, and a slow plugin load during the storm took my Discord bot offline for a week without anything noticing. I've written that one up separately.&lt;/p&gt;

&lt;p&gt;What's the slowest disk you're still running guests on? Mine's still that 5400rpm one, for now.&lt;/p&gt;




&lt;blockquote&gt;
&lt;p&gt;🤖 &lt;em&gt;Drafted with AI assistance from my own homelab notes, logs and repos, and fact-checked against them before publishing.&lt;/em&gt;&lt;/p&gt;
&lt;/blockquote&gt;

</description>
      <category>linux</category>
      <category>node</category>
      <category>performance</category>
      <category>ai</category>
    </item>
    <item>
      <title>ESP32 CYD keeps restarting: my £10 homelab screen and the 5 second watchdog</title>
      <dc:creator>Christian Anderson</dc:creator>
      <pubDate>Tue, 17 Nov 2026 09:00:16 +0000</pubDate>
      <link>https://dev.to/iam-tech/esp32-cyd-keeps-restarting-my-ps10-homelab-screen-and-the-5-second-watchdog-1m1a</link>
      <guid>https://dev.to/iam-tech/esp32-cyd-keeps-restarting-my-ps10-homelab-screen-and-the-5-second-watchdog-1m1a</guid>
      <description>&lt;p&gt;I wanted a little screen on my desk that tells me if anything in the homelab needs me, without opening a laptop. Nothing fancy. A number for "things waiting for you", a number for known-exploited CVEs, whether the servers are up, and when the next bus is.&lt;/p&gt;

&lt;p&gt;So I bought an ESP32-2432S028R, the one everyone calls the Cheap Yellow Display or CYD. About a tenner. 2.8 inch, 320x240, resistive touch, ESP32 on the back. I flashed it with ESPHome and built the whole thing over about a day, around 12 October 2026, and nearly every hour of it was something going wrong. Here's the lot.&lt;/p&gt;

&lt;h2&gt;
  
  
  What it shows
&lt;/h2&gt;

&lt;p&gt;Four pages, you tap the screen to flip between them:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Warden&lt;/strong&gt;: things waiting for me to approve, known-exploited CVEs (goes red if it's above 0), open findings, pending patches and reboots. Warden is my homelab security tool.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Buses&lt;/strong&gt;: next two buses each way at my stop, counting down, from TfL's free API.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Homelab&lt;/strong&gt;: nodes up, services up in Uptime Kuma, backups fresh or not, the busiest CPU, RAM and disk.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Network&lt;/strong&gt;: internet up and ping, how many devices are on the LAN, bans, warden events in the last 24 hours.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;I styled it like e-ink. Paper colour background, black ink, one red ink for anything bad, Libre Baskerville for the big numbers and IBM Plex Mono for the labels. Backlight at 40% so it's not a torch on the desk, and I run the backlight PWM at 20 kHz because lower frequencies flicker when you dim it.&lt;/p&gt;

&lt;h2&gt;
  
  
  Problem 1: it wouldn't even build on my server
&lt;/h2&gt;

&lt;p&gt;First go, I tried compiling on the box that runs my agents. It had 3.2 GB free and was 91% full. ESPHome on the arduino framework still pulls in ESP-IDF 5.5.5, and that wants several GB. Build failed. I cleaned up and built on my Mac instead. A full build later came to about 3 GB, so don't try this on a squeezed VM.&lt;/p&gt;

&lt;h2&gt;
  
  
  Problem 2: ESPHome 2026.9.1 had moved things about
&lt;/h2&gt;

&lt;p&gt;I was on ESPHome 2026.9.1 and a few things had changed under me:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;code&gt;http_request&lt;/code&gt; &lt;code&gt;headers:&lt;/code&gt; is now &lt;code&gt;request_headers:&lt;/code&gt;
&lt;/li&gt;
&lt;li&gt;the &lt;code&gt;ili9xxx&lt;/code&gt; display now &lt;strong&gt;requires&lt;/strong&gt; &lt;code&gt;invert_colors:&lt;/code&gt;, it won't guess&lt;/li&gt;
&lt;li&gt;
&lt;code&gt;id: clock&lt;/code&gt; is reserved, so I called my time source &lt;code&gt;sntp_time&lt;/code&gt;
&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Then two more that &lt;code&gt;esphome config&lt;/code&gt; happily passed, and only a full compile caught:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;I'd named a script &lt;code&gt;poll&lt;/code&gt;. That collides with the C library's &lt;code&gt;poll()&lt;/code&gt;. Rename it.&lt;/li&gt;
&lt;li&gt;
&lt;code&gt;request_headers&lt;/code&gt; lambdas have to return a &lt;code&gt;const char*&lt;/code&gt;, not a string. I ended up with a global buffer.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Lesson there: &lt;code&gt;esphome config&lt;/code&gt; passing means the YAML is valid. It doesn't mean it'll compile. Do the full compile before you get excited.&lt;/p&gt;

&lt;h2&gt;
  
  
  Problem 3: I leaked my own token four times
&lt;/h2&gt;

&lt;p&gt;The first version talked straight to warden's API with a bearer token. Turns out &lt;code&gt;esphome config&lt;/code&gt; and &lt;code&gt;compile&lt;/code&gt; print the token in their output if it's written inline. I did that four times during setup before it clicked, and revoked all four.&lt;/p&gt;

&lt;p&gt;The fix is boring: always use &lt;code&gt;!secret&lt;/code&gt;, because ESPHome redacts those in its output. And later on I got rid of the token on the board completely, more on that below.&lt;/p&gt;

&lt;h2&gt;
  
  
  Problem 4: fuzzy static
&lt;/h2&gt;

&lt;p&gt;First flash, the screen showed fuzzy static. My board is the two-USB version (micro-USB and USB-C), and those use an ST7789 controller, not the ILI9341 the older one-USB version has. So I made the display chip a setting you can override. That was part of it, but not the real cause.&lt;/p&gt;

&lt;p&gt;The real clue was this line in the log:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;Could not allocate buffer for display!
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;A full 320x240 frame in 16-bit colour is 320 × 240 × 2 bytes, about 150 KB. This ESP32 has no PSRAM. Later, with a debug build, the board reported 216 KB of free heap in total but the biggest single free block was 110 KB. A frame buffer needs one unbroken lump, so it was never going to fit.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fz1o1nmq8qumjm6bq9hgy.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fz1o1nmq8qumjm6bq9hgy.png" alt="Bar chart: a 150 KB frame buffer against a 110 KB largest free block, 216 KB total free heap, and a 20 KB strip buffer" width="800" height="364"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;The fix was switching from &lt;code&gt;ili9xxx&lt;/code&gt; to ESPHome's newer &lt;code&gt;mipi_spi&lt;/code&gt; driver. It has presets for all three CYD versions built in (&lt;code&gt;ESP32-2432S028-7789&lt;/code&gt; for the two-USB one, &lt;code&gt;ESP32-2432S028-9342&lt;/code&gt;, and &lt;code&gt;ESP32-2432S028&lt;/code&gt; for the one-USB one) and it draws the screen in strips with a buffer of about 20 KB. Full 16-bit colour still, and the screen came up properly after that.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight yaml"&gt;&lt;code&gt;&lt;span class="na"&gt;display&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
  &lt;span class="p"&gt;-&lt;/span&gt; &lt;span class="na"&gt;platform&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s"&gt;mipi_spi&lt;/span&gt;
    &lt;span class="na"&gt;model&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s"&gt;${display_model}&lt;/span&gt;     &lt;span class="c1"&gt;# ESP32-2432S028-7789 on my two-USB board
&lt;/span&gt;    &lt;span class="na"&gt;invert_colors&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s"&gt;${invert_colors}&lt;/span&gt;
    &lt;span class="na"&gt;rotation&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mi"&gt;270&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;If your CYD shows fuzzy static, it's either the wrong variant or this. Check the log for that buffer line before you start swapping drivers about.&lt;/p&gt;

&lt;h2&gt;
  
  
  Problem 5: it kept restarting and blinking
&lt;/h2&gt;

&lt;p&gt;Now it worked, but every so often it rebooted and the screen blinked. The log had lines like this:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;interval took … 5414 ms
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The board was doing HTTPS calls to warden and TfL itself. TLS on an ESP32 is slow, and those calls blocked the main loop for 5 to 6 seconds a minute. The ESP's watchdog is 5 seconds. So it kept tripping it.&lt;/p&gt;

&lt;p&gt;I could have fiddled with timeouts, but the real answer was to stop making a £10 microcontroller do the slow bit. I moved all of it to a small script on my agent box:&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Ffwbezo6nzddrh9fc1g4g.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Ffwbezo6nzddrh9fc1g4g.png" alt="Before: the board making TLS calls itself and blocking for over 5 seconds. After: a feed script gathers everything every 60 seconds and the board fetches one small JSON over plain HTTP every 30 seconds" width="800" height="391"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;A Python script runs every 60 seconds on a systemd timer. It pulls from warden, TfL, Beszel, Uptime Kuma, my backup check and a LAN scan, and writes one small JSON file. The slow ones are cached, Kuma and the LAN scan every 5 minutes, backups every 30.&lt;/li&gt;
&lt;li&gt;A tiny read-only web server shares that file on the LAN.&lt;/li&gt;
&lt;li&gt;The board does one plain HTTP GET every 30 seconds. The file was 462 bytes when I checked it on 18 October 2026.&lt;/li&gt;
&lt;li&gt;The screen only redraws when the JSON actually changes, plus once a minute for the bus countdown. That got rid of the blink.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Bonus: no token lives on the board any more. If someone nicks it off my desk they get a screen and my WiFi, that's it. If a source fails it just drops off its page instead of breaking the lot.&lt;/p&gt;

&lt;h2&gt;
  
  
  Things I left off on purpose
&lt;/h2&gt;

&lt;p&gt;Two things I tried and dropped because the numbers would lie:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;AdGuard stats.&lt;/strong&gt; My router's AdGuard API returned 401, it's tied to the router's web login session. Not worth hacking round.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;NetBird peers.&lt;/strong&gt; It said &lt;code&gt;Peers 1/9&lt;/code&gt;, but it only counts peers with a live tunnel right now. On a glance screen "1/9" looks like an outage when it isn't.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;A number on a screen you glance at needs to mean what it looks like it means, or it's worse than nothing.&lt;/p&gt;

&lt;h2&gt;
  
  
  Did the restarts stop?
&lt;/h2&gt;

&lt;p&gt;Mostly yeah, and I'll be honest about the bit I can't explain. After moving to the feed, it had 3 restarts between 09:41 and 09:53 that morning. I added ESPHome's &lt;code&gt;debug&lt;/code&gt; component so the board reports its reset reason, free heap and loop time on its own web page. Then it ran 4 hours with zero restarts, uptime 14,387 seconds, and the restarts didn't come back.&lt;/p&gt;

&lt;p&gt;I never caught the cause. My best guess is me, streaming the board's live logs over and over and doing repeated OTA updates that morning. If it comes back, the reset reason is right there on its web page and I don't need a USB cable to read it.&lt;/p&gt;

&lt;h2&gt;
  
  
  If you're building one
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;Work out which CYD you've got first. Two USB ports means ST7789, one means ILI9341.&lt;/li&gt;
&lt;li&gt;Use &lt;code&gt;mipi_spi&lt;/code&gt;, not &lt;code&gt;ili9xxx&lt;/code&gt;. It's deprecated anyway and the full-frame buffer won't fit.&lt;/li&gt;
&lt;li&gt;Don't do TLS on the board if you can help it. Let a proper computer do the slow work and give the ESP32 something small and dull to fetch.&lt;/li&gt;
&lt;li&gt;Keep secrets in &lt;code&gt;!secret&lt;/code&gt;. ESPHome will print anything else.&lt;/li&gt;
&lt;li&gt;Full compile, not just &lt;code&gt;esphome config&lt;/code&gt;.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;What have you got on your desk telling you the homelab's alright? Mine's a tenner and it took a day, but I look at it more than any dashboard I've got.&lt;/p&gt;




&lt;blockquote&gt;
&lt;p&gt;🤖 &lt;em&gt;Drafted with AI assistance from my own homelab notes, logs and repos, and fact-checked against them before publishing.&lt;/em&gt;&lt;/p&gt;
&lt;/blockquote&gt;

</description>
      <category>security</category>
      <category>agents</category>
      <category>python</category>
      <category>ai</category>
    </item>
    <item>
      <title>I let my AI agent look after my NAS, but it can't reboot it: zima-mcp</title>
      <dc:creator>Christian Anderson</dc:creator>
      <pubDate>Mon, 16 Nov 2026 09:00:17 +0000</pubDate>
      <link>https://dev.to/iam-tech/i-let-my-ai-agent-look-after-my-nas-but-it-cant-reboot-it-zima-mcp-1j26</link>
      <guid>https://dev.to/iam-tech/i-let-my-ai-agent-look-after-my-nas-but-it-cant-reboot-it-zima-mcp-1j26</guid>
      <description>&lt;p&gt;My ZimaBlade runs 16 containers: Immich, a dashboard, Uptime Kuma, a reverse proxy and a few others. When something feels slow, the question is always the same one: is the box busy, and which container is it? I got fed up SSHing in to run &lt;code&gt;docker stats&lt;/code&gt;, so my self-hosted agent has had a set of "Zima ops" tools for a while. Today I cleaned them up and published them as &lt;strong&gt;zima-mcp&lt;/strong&gt;.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fikk70o7nyr7colze5uuh.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fikk70o7nyr7colze5uuh.png" alt="zima-mcp running against my real ZimaBlade: tool list, load average, per-container stats and a refused restart" width="800" height="605"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  What it is
&lt;/h2&gt;

&lt;p&gt;An MCP server. MCP is the plug that lets Claude Desktop, Claude Code, Cursor and other AI clients call tools. zima-mcp gives them seven:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;code&gt;zima_load&lt;/code&gt;: uptime and load average&lt;/li&gt;
&lt;li&gt;
&lt;code&gt;zima_top&lt;/code&gt;: top processes by CPU&lt;/li&gt;
&lt;li&gt;
&lt;code&gt;zima_docker_ps&lt;/code&gt; and &lt;code&gt;zima_docker_stats&lt;/code&gt;: containers, and CPU/memory per container&lt;/li&gt;
&lt;li&gt;
&lt;code&gt;zima_disk&lt;/code&gt;: &lt;code&gt;df -h&lt;/code&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;code&gt;zima_container_logs&lt;/code&gt;: the last N lines for one container&lt;/li&gt;
&lt;li&gt;
&lt;code&gt;zima_restart_container&lt;/code&gt;: restart one container, &lt;strong&gt;only if it's on your allowlist&lt;/strong&gt;
&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Every tool is one fixed command over plain SSH. Nothing gets installed on the Zima.&lt;/p&gt;

&lt;h2&gt;
  
  
  The bit I actually care about: what it can't do
&lt;/h2&gt;

&lt;p&gt;ZimaOS is a bit odd. The normal user can run &lt;code&gt;docker&lt;/code&gt; without sudo, because it's in the docker group, but sudo itself wants a password. I leaned into that rather than fighting it. The agent gets no sudo, no reboot, and no killing root processes. There are no tools for those, and there's a test that checks none of the commands contain them.&lt;/p&gt;

&lt;p&gt;Restarts are opt-in per container. I let it bounce the Immich containers, because those are what actually fall over. I don't let it touch the reverse proxy, the login service or the tunnel. Restarting those at the wrong moment locks &lt;em&gt;me&lt;/em&gt; out, and an agent being helpful at 2am is exactly when that happens.&lt;/p&gt;

&lt;h2&gt;
  
  
  What I found tidying it up
&lt;/h2&gt;

&lt;p&gt;The old version quoted the container name before sending it over SSH, so &lt;code&gt;a;reboot&lt;/code&gt; was already harmless. But quoting doesn't stop a name like &lt;code&gt;--help&lt;/code&gt; or &lt;code&gt;-f&lt;/code&gt;. Docker reads that as a flag, not a container. Nothing bad came of it, but it's the kind of thing you only spot when you write tests for someone else's box. Container names are now checked against Docker's own naming rules (letters, digits, &lt;code&gt;_ . -&lt;/code&gt;, no leading dash) before they go anywhere near a shell.&lt;/p&gt;

&lt;h2&gt;
  
  
  Proving the setup guide works
&lt;/h2&gt;

&lt;p&gt;Every one of my repos is getting a setup guide, and I'm sick of guides that were right once. This one is tested. On every change, a GitHub runner plays both parts: it's the computer running the AI client, and it's the "Zima" (SSH to localhost, real Docker, a real nginx container). It installs zima-mcp the way the guide says, calls every tool, restarts an allowlisted container, checks a non-allowlisted one is refused, and talks to the server over real MCP stdio. If the guide goes stale, the check goes red.&lt;/p&gt;

&lt;p&gt;What CI can't prove is the ZimaOS-specific bit and your client's config file. The guide says so rather than pretending.&lt;/p&gt;

&lt;h2&gt;
  
  
  Try it
&lt;/h2&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;pipx install git+https://github.com/casareanderson/zima-mcp
&lt;span class="nv"&gt;ZIMA_HOST&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;your-zima &lt;span class="nv"&gt;ZIMA_USER&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;you zima-mcp --call zima_load
claude mcp add zima -e &lt;span class="nv"&gt;ZIMA_HOST&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;your-zima -e &lt;span class="nv"&gt;ZIMA_USER&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;you -- zima-mcp
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;It works on any Linux box where your user can run Docker, not just a Zima. It's free and MIT: &lt;a href="https://github.com/casareanderson/zima-mcp" rel="noopener noreferrer"&gt;github.com/casareanderson/zima-mcp&lt;/a&gt;. The full walkthrough, with Claude Desktop, Cursor and the common errors, is in &lt;a href="https://github.com/casareanderson/zima-mcp/blob/main/docs/SETUP.md" rel="noopener noreferrer"&gt;docs/SETUP.md&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;Would you let an agent restart containers on your server? And which ones would you never put on the list?&lt;/p&gt;




&lt;blockquote&gt;
&lt;p&gt;🤖 &lt;em&gt;Drafted with AI assistance from my own homelab notes, logs and repos, then reviewed and edited before publishing.&lt;/em&gt;&lt;/p&gt;
&lt;/blockquote&gt;

</description>
      <category>ai</category>
      <category>opensource</category>
      <category>github</category>
      <category>linux</category>
    </item>
    <item>
      <title>My security tool missed my reverse proxy going down. So warden v3 watches the boxes too</title>
      <dc:creator>Christian Anderson</dc:creator>
      <pubDate>Sun, 15 Nov 2026 09:00:05 +0000</pubDate>
      <link>https://dev.to/iam-tech/my-security-tool-missed-my-reverse-proxy-going-down-so-warden-v3-watches-the-boxes-too-gp7</link>
      <guid>https://dev.to/iam-tech/my-security-tool-missed-my-reverse-proxy-going-down-so-warden-v3-watches-the-boxes-too-gp7</guid>
      <description>&lt;p&gt;This morning I updated the OS on my NAS. It rebooted, came back, everything looked fine. It wasn't. Three containers didn't come back up, and one of them was the reverse proxy that sits in front of everything public. My photos site was down for about 35 minutes before I noticed.&lt;/p&gt;

&lt;p&gt;The annoying bit is I already run a security tool on this network. I wrote it. It's called warden, and it was sat there reading logs, scanning for CVEs and checking files for changes the whole time, with no idea anything was wrong. It never asked "is the box up, and is the stuff on it actually running?"&lt;/p&gt;

&lt;p&gt;Turns out the update had run &lt;code&gt;docker stop&lt;/code&gt; on its way down, and Docker treats that as you stopping them on purpose. So &lt;code&gt;restart: unless-stopped&lt;/code&gt; did exactly what it says and left them off. Fair enough, but I'd have liked to know.&lt;/p&gt;

&lt;p&gt;So that's most of warden v3. Here's what's new.&lt;/p&gt;

&lt;h2&gt;
  
  
  A heartbeat for every box
&lt;/h2&gt;

&lt;p&gt;Every 5 minutes warden checks every server and container: up, down, stopped, or unknown, and how long it's been up.&lt;/p&gt;

&lt;p&gt;The bit I'm pleased with: on a Proxmox node it's &lt;strong&gt;one SSH call for the node and every container on it&lt;/strong&gt;. &lt;code&gt;pvesh&lt;/code&gt; already knows the status and uptime of every container, so I just ask it. My 2 nodes, 9 containers and 2 other servers take 4 SSH calls and about 6 seconds.&lt;/p&gt;

&lt;p&gt;If a box misses three beats you get one message. When it's back you get another, and if the uptime is shorter than the outage it tells you it rebooted. If a whole node stops answering, its containers show as "unknown", not "down", so one dead node is one message, not twenty.&lt;/p&gt;

&lt;h2&gt;
  
  
  Who last signed in
&lt;/h2&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fo8mzfnj5iaq51jds5lrz.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fo8mzfnj5iaq51jds5lrz.png" alt="Endpoints on demo data: heartbeat and last sign-in for every box, failed sign-ins, and the sign-in list" width="800" height="681"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Every box now shows who last signed in over SSH or the console, from where, and when, plus failed attempts in the last 24 hours.&lt;/p&gt;

&lt;p&gt;First time I ran it, every single box said the last login was warden itself, a few minutes ago. Obviously. warden signs in to check things, and so does my agent setup on the same box. When I counted, &lt;strong&gt;about three sign-ins in four were my own automation&lt;/strong&gt;. So sign-ins made with a key that lives on the warden box get tagged and left out, and you can name your other automation keys so they show as jobs, not people.&lt;/p&gt;

&lt;p&gt;After that, the first read of each box is a quiet baseline. From then on, a user, address and key a box has never seen sends you a message.&lt;/p&gt;

&lt;p&gt;It also turned up something I didn't know about: two failed sign-ins this morning on an old container I'd parked. Still working out what that was, which is sort of the point.&lt;/p&gt;

&lt;h2&gt;
  
  
  A network map that draws itself
&lt;/h2&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fnukms9v2yv4twta97vl1.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fnukms9v2yv4twta97vl1.png" alt="The network map on demo data: internet, router, Cloudflare and the tunnel, servers with their containers, devices by kind on a LAN and an IoT VLAN" width="800" height="419"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;I wanted a network diagram but I didn't want to draw one, because a hand-drawn diagram is wrong the week after you draw it.&lt;/p&gt;

&lt;p&gt;So the Network tab draws it from stuff warden already knows: internet and line health, the router, Cloudflare and which box runs the tunnel, each subnet with your servers (containers inside their node, coloured by heartbeat) and every other device counted by kind. Click a server and you go to its details, click "bulb 38" and it lists your bulbs. It works on a phone as well, which took two goes.&lt;/p&gt;

&lt;h2&gt;
  
  
  The rest since v2
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;No scanner binary any more.&lt;/strong&gt; I used trivy, then its own releases got compromised (CVE-2026-33634). Now warden reads package lists off each box and checks them straight against OSV.dev, ranked by known-exploited first.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;What the internet can reach:&lt;/strong&gt; vulnerable containers that are actually exposed go to the top.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Every device in the house:&lt;/strong&gt; one inventory from the LAN sweep, router leases and Home Assistant. It learns what each bulb or TV normally looks up and only asks me about odd ones.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Internet health every minute&lt;/strong&gt;, with who was using the line when it went bad.&lt;/li&gt;
&lt;li&gt;An attack map by country, more log formats (Caddy, Traefik, Authentik), and an optional Beszel card.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  What it still can't do
&lt;/h2&gt;

&lt;p&gt;Being straight about it: warden still doesn't check individual Docker containers, so this morning's outage would only show up today as "box is up". That's the next fix. Sign-ins through &lt;code&gt;pct exec&lt;/code&gt; or the Proxmox web console don't get logged anywhere warden can read, so they don't show. Container heartbeats need Proxmox. The full list is in the repo's KNOWN-ISSUES.md, and I've pointed my coding agent at that file so the fixes come in as PRs I review.&lt;/p&gt;

&lt;p&gt;It's free, AGPL, runs on 512 MB, and it's detect-only by default, so nothing gets changed without you saying yes.&lt;/p&gt;

&lt;p&gt;Repo: &lt;a href="https://github.com/casareanderson/warden" rel="noopener noreferrer"&gt;https://github.com/casareanderson/warden&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;If you run it, let me know what it finds. And if you've got a better idea for spotting stopped containers than polling &lt;code&gt;docker ps&lt;/code&gt;, I'm all ears.&lt;/p&gt;




&lt;blockquote&gt;
&lt;p&gt;🤖 &lt;em&gt;Drafted with AI assistance from my own homelab notes, logs and repos, then reviewed and edited before publishing.&lt;/em&gt;&lt;/p&gt;
&lt;/blockquote&gt;

</description>
      <category>security</category>
      <category>opensource</category>
      <category>ai</category>
      <category>discuss</category>
    </item>
    <item>
      <title>Uptime Kuma had never added a monitor on its own, so I wrote kuma-discover</title>
      <dc:creator>Christian Anderson</dc:creator>
      <pubDate>Sat, 14 Nov 2026 09:00:16 +0000</pubDate>
      <link>https://dev.to/iam-tech/uptime-kuma-had-never-added-a-monitor-on-its-own-so-i-wrote-kuma-discover-2af8</link>
      <guid>https://dev.to/iam-tech/uptime-kuma-had-never-added-a-monitor-on-its-own-so-i-wrote-kuma-discover-2af8</guid>
      <description>&lt;p&gt;On 7 October I moved Uptime Kuma onto my ZimaBlade. While I was repointing everything at it, I noticed something a bit embarrassing: nothing had ever added a monitor automatically. Kuma watched exactly what I'd remembered to click into it, and nothing else. Every new container was unwatched until I happened to think of it.&lt;/p&gt;

&lt;p&gt;So I wrote a small script that compares what's actually running with what Kuma is watching. Its first run found &lt;strong&gt;18 services and 5 gaps&lt;/strong&gt;. I approved all 5. Today I've cleaned it up and published it as &lt;strong&gt;kuma-discover&lt;/strong&gt;.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fnz3mbcn06y57tzwrgiv9.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fnz3mbcn06y57tzwrgiv9.png" alt="kuma-discover finding two unwatched containers, adding one after approval and rejecting the other" width="800" height="440"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  What it does
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;Looks at your Docker hosts over SSH for containers with a published port. Loopback-only ports don't count, because Kuma can't reach them anyway.&lt;/li&gt;
&lt;li&gt;Optionally looks at Proxmox LXC containers and your Cloudflare Tunnel hostnames.&lt;/li&gt;
&lt;li&gt;Works out what's already covered: same IP, same host and port, same hostname, a watched tunnel hostname in front of it, or a monitor with the same name.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Asks before adding anything.&lt;/strong&gt; That's either &lt;code&gt;kuma-discover review&lt;/code&gt; at the terminal, or ✅ / ❌ in a Discord channel, where only my reaction counts.&lt;/li&gt;
&lt;li&gt;Remembers every answer. Say no once and it never asks again.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;There's no AI model in it. Every decision is a rule you can read.&lt;/p&gt;

&lt;h2&gt;
  
  
  The three bugs
&lt;/h2&gt;

&lt;p&gt;&lt;strong&gt;1. Kuma 2 broke the library's add call.&lt;/strong&gt; &lt;code&gt;uptime-kuma-api&lt;/code&gt; was written for Kuma 1. On Kuma 2 its &lt;code&gt;add_monitor()&lt;/code&gt; fails, because a new &lt;code&gt;conditions&lt;/code&gt; column can't be empty. The fix is to build the same payload and send it with &lt;code&gt;conditions&lt;/code&gt; set. kuma-discover tries the normal way first and only falls back when it sees that error.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;2. The notification list that added notification "0".&lt;/strong&gt; My first live add failed with a foreign-key error. Kuma loops over the &lt;em&gt;keys&lt;/em&gt; of &lt;code&gt;notificationIDList&lt;/code&gt;, so sending &lt;code&gt;[1]&lt;/code&gt; (a list) meant it tried to attach notification id &lt;code&gt;0&lt;/code&gt;, which doesn't exist. It has to be &lt;code&gt;{"1": true}&lt;/code&gt;. Found and fixed the same night.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;3. It tried to monitor itself.&lt;/strong&gt; I didn't find this one, my own test did. The setup check runs a real Kuma container on GitHub, and I'd named it &lt;code&gt;kuma&lt;/code&gt; instead of &lt;code&gt;uptime-kuma&lt;/code&gt;. kuma-discover only skipped Kuma by name, so it happily proposed "Svc: kuma". It now skips the &lt;code&gt;louislam/uptime-kuma&lt;/code&gt; image too, whatever the container is called.&lt;/p&gt;

&lt;p&gt;There was a fourth, softer one. Logging in sometimes timed out against my Kuma on the Zima, 2 runs out of 7 one morning. It retries 3 times now.&lt;/p&gt;

&lt;h2&gt;
  
  
  The setup guide is tested
&lt;/h2&gt;

&lt;p&gt;Same as my other repos now: &lt;a href="https://github.com/casareanderson/kuma-discover/blob/main/docs/SETUP.md" rel="noopener noreferrer"&gt;docs/SETUP.md&lt;/a&gt; gets followed by CI on every change. It runs against real Uptime Kuma &lt;strong&gt;1.23.16&lt;/strong&gt; and &lt;strong&gt;2.5.6&lt;/strong&gt; containers, and a real Docker host over SSH, from install to an approved monitor actually showing up in Kuma. The screenshot above is from that run. Proxmox, Cloudflare and Discord need real accounts, so they're marked "not CI-tested" instead of quietly assumed.&lt;/p&gt;

&lt;h2&gt;
  
  
  Try it
&lt;/h2&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;pipx install git+https://github.com/casareanderson/kuma-discover
curl -fsSLO https://raw.githubusercontent.com/casareanderson/kuma-discover/main/config.example.toml
mv config.example.toml kuma-discover.toml   &lt;span class="c"&gt;# set your Kuma URL and Docker hosts&lt;/span&gt;
&lt;span class="kd"&gt;export&lt;/span&gt; &lt;span class="nv"&gt;KUMA_USERNAME&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;admin &lt;span class="nv"&gt;KUMA_PASSWORD&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s1"&gt;'...'&lt;/span&gt;
kuma-discover scan --dry-run
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Free and MIT: &lt;a href="https://github.com/casareanderson/kuma-discover" rel="noopener noreferrer"&gt;github.com/casareanderson/kuma-discover&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;How many of your containers does your monitoring actually know about? I thought I was close to all of them. I wasn't.&lt;/p&gt;




&lt;blockquote&gt;
&lt;p&gt;🤖 &lt;em&gt;Drafted with AI assistance from my own homelab notes, logs and repos, then reviewed and edited before publishing.&lt;/em&gt;&lt;/p&gt;
&lt;/blockquote&gt;

</description>
      <category>opensource</category>
      <category>github</category>
      <category>ai</category>
      <category>security</category>
    </item>
    <item>
      <title>One profile, two systems: how I stopped my agent and my open-source tool drifting apart</title>
      <dc:creator>Christian Anderson</dc:creator>
      <pubDate>Fri, 13 Nov 2026 11:33:46 +0000</pubDate>
      <link>https://dev.to/iam-tech/one-profile-two-systems-how-i-stopped-my-agent-and-my-open-source-tool-drifting-apart-5flh</link>
      <guid>https://dev.to/iam-tech/one-profile-two-systems-how-i-stopped-my-agent-and-my-open-source-tool-drifting-apart-5flh</guid>
      <description>&lt;p&gt;I run a private agent at home. It has a profile file with my sizes, my fit, my colours, my brands, and it fetches things from shops I actually buy from.&lt;/p&gt;

&lt;p&gt;Then I open-sourced the fetching engine as a separate tool. It had its own profile file. Its own shop list. Its own idea of what "my size" means.&lt;/p&gt;

&lt;p&gt;For about three weeks they disagreed. Not loudly. The private agent said I was a size XL in trousers. The public tool said XL &lt;strong&gt;or&lt;/strong&gt; XXL. One of them was right and I had no idea which, because the tool's output looked exactly as authoritative as the agent's.&lt;/p&gt;

&lt;h2&gt;
  
  
  The duplication I hadn't counted
&lt;/h2&gt;

&lt;p&gt;I wrote the public tool first, then went back to look at my private agent. Same thing. Same twenty Shopify shops. Same &lt;code&gt;/products.json&lt;/code&gt; endpoint. Same "per-variant stock means in stock in your size" logic.&lt;/p&gt;

&lt;p&gt;Every shop was being hit twice, once by each system, on different schedules, with different parsing. Every week the two pools drifted a little further apart, because the private one had a fix for a size label the public one didn't, and vice versa.&lt;/p&gt;

&lt;p&gt;The private agent was, in fairness, much older and much cleverer: it had six more adapters built on a headless renderer, photo matching, price-error detection, order history. But all of that sat on top of the same twenty shops I'd duplicated.&lt;/p&gt;

&lt;p&gt;I had written the same fetcher twice and called it two features.&lt;/p&gt;

&lt;h2&gt;
  
  
  The obvious fix, and the bit that wasn't
&lt;/h2&gt;

&lt;p&gt;The obvious fix is "have one call the other". Which one? That turned out to be the wrong question to ask first.&lt;/p&gt;

&lt;p&gt;The right question was: &lt;strong&gt;which one owns the facts?&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;My agent's &lt;code&gt;profile.yaml&lt;/code&gt; was the file I actually edited. When my size changed, I changed it there. So that file was the source of truth — and the public tool's profile was a &lt;em&gt;copy&lt;/em&gt; of a copy, written by hand, already stale.&lt;/p&gt;

&lt;p&gt;So the shape of the fix was:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;  agent/profile.yaml            &amp;lt;- the one file I edit (source of truth)
        |
        |  bridge script: projects it into the tool's schema
        v
  tool/profile.yaml             &amp;lt;- generated, never edited by hand
  tool/config.yaml              &amp;lt;- generated: the shop list, from the agent's config
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The bridge is about a hundred lines and does the unglamorous work: copy the sizes, map the fit rules, translate "never Zara" into a brand filter, and drop the fields the tool must not have.&lt;/p&gt;

&lt;p&gt;It drops two fields on purpose. The tool's schema refuses them — and so does my agent's own rules. It stores my height and weight for fit reasoning, and it is not allowed to repeat those numbers to anyone, ever. So the bridge does not copy them, and the tool's validator rejects them if they appear. Two independent rules agreeing on one boundary is exactly the kind of thing you want, because it means the boundary can't be crossed by accident later.&lt;/p&gt;

&lt;h2&gt;
  
  
  The part I did not expect
&lt;/h2&gt;

&lt;p&gt;Then I changed the fetch path. The twenty shared shops were now fetched by the tool — the tested, packaged one — and the agent read the tool's pool instead of fetching those shops itself. The agent kept its six extra adapters and its headless-rendered footwear retailers; it just stopped re-fetching the twenty shops it had in common with the tool.&lt;/p&gt;

&lt;p&gt;The surprising bit is how little changed afterwards. The agent's pool format looked nothing like the tool's — different field names, different shape, one of them stores sizes as a JSON string. About forty lines of mapping sat between them, and then... it worked. The agent's daily digest started including the tool's items and there was no visible seam.&lt;/p&gt;

&lt;p&gt;That forty lines is the whole trick. The two systems never had different &lt;em&gt;facts&lt;/em&gt;. They had different &lt;em&gt;shapes&lt;/em&gt;, and shapes are cheap to translate.&lt;/p&gt;

&lt;h2&gt;
  
  
  The rule I keep relearning
&lt;/h2&gt;

&lt;p&gt;&lt;strong&gt;If two systems need the same facts, one of them must not gather them.&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;Not "should share them" — share them. Gather them once, in the place that owns them, and let everyone else read. The temptation with agent systems is that every soul feels entitled to its own scraper, because a soul that fetches its own data feels self-contained. But self-contained and duplicated are different things, and duplication doesn't announce itself: both copies look right, and the drift shows up months later as "why did the tool think my size was different".&lt;/p&gt;

&lt;p&gt;Two smaller things I learned doing it:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;A generated file needs a generator that runs, not a note that says it should.&lt;/strong&gt; My first attempt was a script I ran by hand whenever I remembered. It was wrong within a fortnight. The version that works is three lines in the weekly job: bridge first, then fetch. Now it cannot drift, because it is regenerated on the same schedule that consumes it.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Keep the estate files out of the public repo.&lt;/strong&gt; The moment the bridge and the merge plan were in the public repo, I had to take them back out again — the wiring belongs on the machine, not in the project. Small thing, but it's the difference between a reusable library and someone's private setup with a public URL attached.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  What I'd tell past me
&lt;/h2&gt;

&lt;p&gt;If you're about to open-source the engine that powers one of your own agents, decide up front which side owns the profile — and make the other side &lt;em&gt;generated&lt;/em&gt;, with the generator wired into the schedule that uses it.&lt;/p&gt;

&lt;p&gt;The tool is better for being public: it has tests it wouldn't have had as a private script, and it has a packaging that forced me to fix a fetcher that had never worked. But the private agent is better for being the source of truth, because it's the one where I actually edit the file.&lt;/p&gt;

&lt;p&gt;Two systems, one profile, one fetch path. The code that glues them together is the boring part, and the boring part is where all of the duplication used to live.&lt;/p&gt;




&lt;blockquote&gt;
&lt;p&gt;🤖 &lt;em&gt;Drafted with AI assistance from my own homelab notes, logs and repos, then reviewed and edited before publishing.&lt;/em&gt;&lt;/p&gt;

&lt;p&gt;🌐 &lt;em&gt;More field notes: &lt;a href="https://iamtech.cn1-lab.uk" rel="noopener noreferrer"&gt;https://iamtech.cn1-lab.uk&lt;/a&gt;&lt;/em&gt;&lt;/p&gt;
&lt;/blockquote&gt;

</description>
      <category>ai</category>
      <category>opensource</category>
      <category>showdev</category>
      <category>agents</category>
    </item>
    <item>
      <title>Shopify blocked my scraper and it wasn't my User-Agent</title>
      <dc:creator>Christian Anderson</dc:creator>
      <pubDate>Fri, 13 Nov 2026 11:33:27 +0000</pubDate>
      <link>https://dev.to/iam-tech/shopify-blocked-my-scraper-and-it-wasnt-my-user-agent-363b</link>
      <guid>https://dev.to/iam-tech/shopify-blocked-my-scraper-and-it-wasnt-my-user-agent-363b</guid>
      <description>&lt;p&gt;I open-sourced a little shopping tool. It had a polite fetcher, respected &lt;code&gt;robots.txt&lt;/code&gt;, sent a real User-Agent, and checking that the address was public before it dialled out.&lt;/p&gt;

&lt;p&gt;It had never once fetched a product.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;Artifacts Apparel            FAILED HTTP 429 for https://artifactsapparel.co.uk/products.json?limit=250&amp;amp;page=1
You Know Who's               FAILED HTTP 429 for https://youknowwhos.co.uk/products.json?limit=250&amp;amp;page=1
Goodhood                     FAILED HTTP 429 for https://goodhoodstore.com/collections/sale/products.json?limit=250&amp;amp;page=1
Represent                    FAILED HTTP 429 for https://representclo.com/products.json?limit=250&amp;amp;page=1
...
0 products checked, 0 in stock in your size
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Twenty shops, twenty-nines, zero products. And here is the part that made it interesting: &lt;strong&gt;the tool never failed loudly.&lt;/strong&gt; It printed a table. It exited 0. It wrote an empty digest and posted it to Discord like a perfectly normal Monday.&lt;/p&gt;

&lt;h2&gt;
  
  
  The test that took two minutes
&lt;/h2&gt;

&lt;p&gt;My first guess was the User-Agent. I'd written &lt;code&gt;ClothingLab/0.1 (+https://github.com/casareanderson/clothinglab)&lt;/code&gt; — identifying myself with a real URL, the polite thing to do. Maybe Shopify didn't like the URL in it. Maybe they wanted a browser.&lt;/p&gt;

&lt;p&gt;So I ran the same URL with the same User-Agent through two different clients:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="k"&gt;import&lt;/span&gt; &lt;span class="n"&gt;urllib&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;request&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;requests&lt;/span&gt;

&lt;span class="no"&gt;URL&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="s2"&gt;"https://artifactsapparel.co.uk/products.json?limit=5&amp;amp;page=1"&lt;/span&gt;
&lt;span class="no"&gt;UA&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="s2"&gt;"ClothingLab/0.1 (+https://github.com/casareanderson/clothinglab)"&lt;/span&gt;

&lt;span class="n"&gt;urllib&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;request&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;urlopen&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;urllib&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;request&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nc"&gt;Request&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="no"&gt;URL&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;headers&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="s2"&gt;"User-Agent"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="no"&gt;UA&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="c1"&gt;# HTTPError: HTTP Error 429&lt;/span&gt;

&lt;span class="n"&gt;requests&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;get&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="no"&gt;URL&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;headers&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="s2"&gt;"User-Agent"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="no"&gt;UA&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="c1"&gt;# &amp;lt;Response [200]&amp;gt;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;client&lt;/th&gt;
&lt;th&gt;User-Agent&lt;/th&gt;
&lt;th&gt;result&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;urllib&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;ClothingLab/0.1 (+github…)&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;429&lt;/strong&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;requests&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;ClothingLab/0.1 (+github…)&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;200&lt;/strong&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;urllib&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;&lt;code&gt;Mozilla/5.0 … Chrome/128.0&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;429&lt;/strong&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;requests&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;&lt;code&gt;Mozilla/5.0 … Chrome/128.0&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;200&lt;/strong&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;Same URL, same User-Agent, same second. The User-Agent was never the variable. &lt;strong&gt;The HTTP client was.&lt;/strong&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  What Shopify was actually looking at
&lt;/h2&gt;

&lt;p&gt;&lt;code&gt;requests&lt;/code&gt; and &lt;code&gt;urllib&lt;/code&gt; look like the same request on paper. They are not:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;&lt;/th&gt;
&lt;th&gt;&lt;code&gt;urllib.request&lt;/code&gt;&lt;/th&gt;
&lt;th&gt;&lt;code&gt;requests&lt;/code&gt;&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;Accept-Encoding&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;&lt;code&gt;identity&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;&lt;code&gt;gzip, deflate&lt;/code&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;Connection&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;&lt;code&gt;close&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;keep-alive&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;
&lt;code&gt;Accept-Encoding&lt;/code&gt; offered&lt;/td&gt;
&lt;td&gt;no compression at all&lt;/td&gt;
&lt;td&gt;compressed&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;cookies / session&lt;/td&gt;
&lt;td&gt;none&lt;/td&gt;
&lt;td&gt;a session that persists&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;Shopify's edge sits behind a WAF with bot rules that key on the shape of the request, not on what it claims to be. A &lt;code&gt;urllib&lt;/code&gt; request announces itself with &lt;code&gt;Connection: close&lt;/code&gt; and refuses compression. A WAF rule written to spot "a script that isn't a browser" catches it perfectly — and &lt;code&gt;429 Too Many Requests&lt;/code&gt; is a much softer answer than &lt;code&gt;403 Forbidden&lt;/code&gt;, which is what you'd get if it thought you were up to no good.&lt;/p&gt;

&lt;p&gt;The thing that had me staring at it for a while: the tool &lt;em&gt;did&lt;/em&gt; have a browser-ish User-Agent option. I had tried it. It still returned 429. Because I had been testing the wrong variable.&lt;/p&gt;

&lt;h2&gt;
  
  
  The fix
&lt;/h2&gt;

&lt;p&gt;One import, one line:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="o"&gt;-&lt;/span&gt;&lt;span class="k"&gt;import&lt;/span&gt; &lt;span class="n"&gt;urllib&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;request&lt;/span&gt;
&lt;span class="o"&gt;+&lt;/span&gt;&lt;span class="k"&gt;import&lt;/span&gt; &lt;span class="n"&gt;requests&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;and the fetch became:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="n"&gt;r&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;requests&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;get&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;url&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;timeout&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="nb"&gt;self&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;timeout&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;headers&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="s2"&gt;"User-Agent"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nb"&gt;self&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;ua&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"Accept"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"application/json, text/html;q=0.9"&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;r&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;status_code&lt;/span&gt; &lt;span class="o"&gt;!=&lt;/span&gt; &lt;span class="mi"&gt;200&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="k"&gt;raise&lt;/span&gt; &lt;span class="nc"&gt;FetchError&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;f&lt;/span&gt;&lt;span class="s2"&gt;"HTTP {r.status_code} for {url}"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;I kept the &lt;code&gt;robots.txt&lt;/code&gt; check, the per-host delay, the public-address check and the User-Agent you configure — all of that was right. I just swapped the thing that was silently wrong.&lt;/p&gt;

&lt;p&gt;Then I wrote the regression test, because "it works now" is not the same as "it works for a reason":&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;test_fetcher_uses_requests_not_urllib&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;monkeypatch&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="s2"&gt;"""Shopify answers urllib with 429 and requests with 200 — the client is the difference, not the UA."""&lt;/span&gt;
    &lt;span class="n"&gt;seen&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;

    &lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;fake_get&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;url&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;timeout&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="kc"&gt;None&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;headers&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="kc"&gt;None&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
        &lt;span class="n"&gt;seen&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"headers"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;headers&lt;/span&gt;
        &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="nf"&gt;_Resp&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mi"&gt;200&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;b&lt;/span&gt;&lt;span class="s2"&gt;"hello"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;

    &lt;span class="n"&gt;monkeypatch&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nb"&gt;setattr&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;fetch&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;requests&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"get"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;fake_get&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="n"&gt;f&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nc"&gt;Fetcher&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;user_agent&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;"ClothingLab/test"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="k"&gt;assert&lt;/span&gt; &lt;span class="n"&gt;f&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;_raw&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;"https://example.com/products.json"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;==&lt;/span&gt; &lt;span class="n"&gt;b&lt;/span&gt;&lt;span class="s2"&gt;"hello"&lt;/span&gt;
    &lt;span class="k"&gt;assert&lt;/span&gt; &lt;span class="n"&gt;seen&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"headers"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"User-Agent"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt; &lt;span class="o"&gt;==&lt;/span&gt; &lt;span class="s2"&gt;"ClothingLab/test"&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The comment is the important part. Six months from now, someone will decide &lt;code&gt;urllib&lt;/code&gt; is tidier and "it's stdlib, why not" — and the comment says exactly why not, in the place they'll be looking.&lt;/p&gt;

&lt;h2&gt;
  
  
  What it actually taught me
&lt;/h2&gt;

&lt;p&gt;The bug wasn't clever. It was the most ordinary kind of bug there is: &lt;strong&gt;a program that runs fine and produces nothing.&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;I had a health table, a status line, a summary line, a digest, a Discord post and a schedule. Every one of them worked. The one number that mattered — products checked — was zero, and it was zero &lt;em&gt;quietly&lt;/em&gt;. A green checkmark at the end of a successful run is the most dangerous thing a tool can produce, because it tells you the run happened, not that the work did.&lt;/p&gt;

&lt;p&gt;Two habits I'd keep from this:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;
&lt;strong&gt;Test the wire, not just the logic.&lt;/strong&gt; Every unit test I'd written passed, because they all stubbed the network. The bug lived in the eleven lines between "I have a URL" and "I have a response", and no unit test could ever have caught it. The two-minute test that fixed it was the first thing in the project that touched a real socket.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Make "zero" impossible to mistake for "fine".&lt;/strong&gt; It costs nothing to have the tool refuse to post a digest that checked zero products from twenty sources, and say &lt;em&gt;why&lt;/em&gt;. Silence and zero are not the same outcome, and a tool that cannot tell you the difference will take your data quietly.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;The wider version of that lesson: when something is wrong in a way that produces a &lt;em&gt;plausible&lt;/em&gt; result — zero products, no findings, no alerts, no changes — the first thing to check is the plumbing, not the content. A scraper that fetches nothing looks exactly like a scraper that found nothing.&lt;/p&gt;

&lt;p&gt;And the specific version, which cost me an afternoon: &lt;strong&gt;if every single source fails the same way, that's a signal about your client, not about the sources.&lt;/strong&gt;&lt;/p&gt;




&lt;blockquote&gt;
&lt;p&gt;🤖 &lt;em&gt;Drafted with AI assistance from my own homelab notes, logs and repos, then reviewed and edited before publishing.&lt;/em&gt;&lt;/p&gt;

&lt;p&gt;🌐 &lt;em&gt;More field notes: &lt;a href="https://iamtech.cn1-lab.uk" rel="noopener noreferrer"&gt;https://iamtech.cn1-lab.uk&lt;/a&gt;&lt;/em&gt;&lt;/p&gt;
&lt;/blockquote&gt;

</description>
      <category>python</category>
      <category>showdev</category>
      <category>opensource</category>
      <category>github</category>
    </item>
    <item>
      <title>My deal bot showed me 148 picks in two weeks. It was 61 products: ClothingLab</title>
      <dc:creator>Christian Anderson</dc:creator>
      <pubDate>Fri, 13 Nov 2026 09:00:20 +0000</pubDate>
      <link>https://dev.to/iam-tech/my-deal-bot-showed-me-148-picks-in-two-weeks-it-was-61-products-clothinglab-3n71</link>
      <guid>https://dev.to/iam-tech/my-deal-bot-showed-me-148-picks-in-two-weeks-it-was-61-products-clothinglab-3n71</guid>
      <description>&lt;p&gt;I built myself a deal bot for clothes. Every morning it posted sale picks to Discord: things from shops I actually buy from, in my size, in stock. It worked. Then I counted what it had posted.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;148 picks in 14 days. 61 different products.&lt;/strong&gt; One brown zip hoodie turned up 23 times over four weeks. A pair of trainers ran 13 days straight. The bot wasn't finding deals any more, it was nagging me.&lt;/p&gt;

&lt;p&gt;That's the first of three failures that ended up as &lt;strong&gt;ClothingLab&lt;/strong&gt;, a free tool that does the boring part properly: sale stuff that's in stock &lt;strong&gt;in your size&lt;/strong&gt;, from shops you pick, without showing you the same thing twice.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fkigmpjyotcedgeu3xwlw.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fkigmpjyotcedgeu3xwlw.png" alt="A ClothingLab digest for a made-up person: picks with the size that's in stock, budget tier and why it was picked, then one look per tier" width="800" height="789"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  Failure 1: it kept showing me the same hoodie
&lt;/h2&gt;

&lt;p&gt;The daily post had sections: top picks, "on trend", looks, new in your size. Each one picked from the same pool. Top picks only remembered the last three days, and when everything had been seen it fell back to showing the whole list again. The "on trend" section didn't check what I'd seen at all.&lt;/p&gt;

&lt;p&gt;The fix is one ledger that every section shares. Anything you've been shown stays out for 14 days. It only comes back if it's worth telling you again: it's at least 5% cheaper than when you saw it, or it went out of your size and came back. Colourways count as the same product, so "Hoodie - Black" and "Hoodie - Navy" don't take turns. And nothing appears twice in one post.&lt;/p&gt;

&lt;h2&gt;
  
  
  Failure 2: EU shoe sizes were silently thrown away
&lt;/h2&gt;

&lt;p&gt;Shoe deals from European shops just never showed up. No error. The shops list sizes as bare numbers, &lt;code&gt;45&lt;/code&gt; or &lt;code&gt;45.5&lt;/code&gt;, and sometimes glue the colour on: &lt;code&gt;46 2/3 / Cream White / Blue&lt;/code&gt;. My size check only understood "EU 46" with the letters in front, so a bare &lt;code&gt;45.5&lt;/code&gt; got read as a UK size, which obviously didn't match, and the item was dropped as "wrong size". It was the biggest drop reason in every run.&lt;/p&gt;

&lt;p&gt;The trick that fixed it: &lt;strong&gt;you can prove a shop is listing EU sizes from the numbers alone&lt;/strong&gt;, because no adult UK or US shoe size is 39 or more. You can't tell UK from US that way, so those stay a per-shop setting. ClothingLab works your EU sizes out from your UK size and stamps them on the result, so you see "EU 44" and know what to order.&lt;/p&gt;

&lt;h2&gt;
  
  
  Failure 3: the stock was there, I was reading the wrong bit
&lt;/h2&gt;

&lt;p&gt;Some shops showed "no size information" for every trainer. I'd assumed they were hiding it. They weren't. The page's visible size picker had no stock markers, but the exact per-size stock was sitting in data embedded in the page. One shop went from 0 usable trainer deals to 9 the day I read the data instead of the buttons.&lt;/p&gt;

&lt;p&gt;The flip side was harder to accept. Some big high-street shops don't put per-size stock in the page at all; it loads later with JavaScript. For those my bot found &lt;strong&gt;zero&lt;/strong&gt; items in my size. ClothingLab's rule is that an item with no per-size stock is never shown as "in your size". Fewer results, but none of them lie.&lt;/p&gt;

&lt;h2&gt;
  
  
  What ClothingLab is
&lt;/h2&gt;

&lt;p&gt;The engine from my bot, rebuilt so it isn't hard-wired to me:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;A short interview&lt;/strong&gt; writes your profile: sizes (tops, waist and leg, UK shoe), fits you like and never wear, colours, brands, budget. It never asks for height or weight.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Two generic shop readers&lt;/strong&gt;: any Shopify shop (its public &lt;code&gt;products.json&lt;/code&gt; has every variant's size and an &lt;code&gt;available&lt;/code&gt; flag) and any shop with a sitemap and proper product data on its pages. robots.txt is checked, there's a delay between requests, and you have to set a user agent that says who you are before it'll run.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Budget tiers&lt;/strong&gt; 💸 Low / 👌 Mid / 💎 High, as plain £ lines per garment you can move, and &lt;strong&gt;one look per tier&lt;/strong&gt; from a top, bottom and shoes in your size, using things you already own if you list them. If a tier has no full look today, it says so.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;A watchlist&lt;/strong&gt; for price drops, a daily digest (file, email or Discord), and an &lt;strong&gt;MCP server&lt;/strong&gt;, so you can onboard and ask "any navy hoodies under £50?" from Claude, Cursor or any MCP client.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F5kypqjiq7xw5ac2mb3wj.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F5kypqjiq7xw5ac2mb3wj.png" alt="refresh, looks and search in a terminal" width="800" height="358"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Both images are real output, from the made-up shop the tests use and a made-up person called Sam. My own sizes aren't in the repo.&lt;/p&gt;

&lt;h2&gt;
  
  
  What's not in it
&lt;/h2&gt;

&lt;p&gt;My bot also reads a list of UK shops with their quirks worked out: the hidden stock data, the odd size formats, which ones block bots. That list stays mine. The free version ships with four example Shopify shops and makes adding your own a three-line job. It also doesn't read your email; if you want looks to use what you own, list it.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;pipx install git+https://github.com/casareanderson/clothinglab
clothinglab init &lt;span class="o"&gt;&amp;amp;&amp;amp;&lt;/span&gt; clothinglab onboard
clothinglab refresh &lt;span class="o"&gt;&amp;amp;&amp;amp;&lt;/span&gt; clothinglab digest
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;It's free and AGPL: &lt;a href="https://github.com/casareanderson/clothinglab" rel="noopener noreferrer"&gt;github.com/casareanderson/clothinglab&lt;/a&gt;. The setup guide gets run on a fresh machine by GitHub every week, so if it says a step works, it does.&lt;/p&gt;

&lt;p&gt;What size are you, and how many "deals" do you scroll past before you find one that's actually in it?&lt;/p&gt;




&lt;blockquote&gt;
&lt;p&gt;🤖 &lt;em&gt;Drafted with AI assistance from my own homelab notes, logs and repos, then reviewed and edited before publishing.&lt;/em&gt;&lt;/p&gt;
&lt;/blockquote&gt;

</description>
      <category>ai</category>
      <category>opensource</category>
      <category>showdev</category>
      <category>github</category>
    </item>
    <item>
      <title>Set up F1 print-status lights on a Klipper printer in 30 minutes</title>
      <dc:creator>Christian Anderson</dc:creator>
      <pubDate>Thu, 12 Nov 2026 09:00:16 +0000</pubDate>
      <link>https://dev.to/iam-tech/set-up-f1-print-status-lights-on-a-klipper-printer-in-30-minutes-mpm</link>
      <guid>https://dev.to/iam-tech/set-up-f1-print-status-lights-on-a-klipper-printer-in-30-minutes-mpm</guid>
      <description>&lt;p&gt;My Voron's LED strip changes colour with the print: blue when idle, orange heating, white printing, green chequered flag when it's done, red for errors. It's &lt;a href="https://github.com/casareanderson/WLED_Voron--F1-Lights" rel="noopener noreferrer"&gt;WLED_Voron--F1-Lights&lt;/a&gt;, and it now has a setup guide that GitHub checks every week.&lt;/p&gt;

&lt;h2&gt;
  
  
  What a printer repo can test with no printer
&lt;/h2&gt;

&lt;p&gt;Not Klipper itself. But it can check the bits people actually get wrong:&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fnmqq86epj9vl3a5dtr54.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fnmqq86epj9vl3a5dtr54.png" alt="The weekly check: config consistent, the sed edits nine addresses, every macro's JSON reaches a fake WLED" width="800" height="575"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;all eight &lt;code&gt;LED_*&lt;/code&gt; macros exist and every value is in range&lt;/li&gt;
&lt;li&gt;the guide's one-line &lt;code&gt;sed&lt;/code&gt; really does change all &lt;strong&gt;nine&lt;/strong&gt; IP addresses (there's no single variable for it, which is the most annoying part of the setup)&lt;/li&gt;
&lt;li&gt;the optional print macros only call macros that exist&lt;/li&gt;
&lt;li&gt;each macro's numbers arrive at a fake WLED as valid JSON with the right colour, brightness and effect&lt;/li&gt;
&lt;li&gt;a strip that's switched off can't hang a print: the script gives up in about 3 seconds&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  The colour trap
&lt;/h2&gt;

&lt;p&gt;Most WS2812B and COB strips are GRB, so red and green arrive swapped. The defaults are written for that: heating sends &lt;code&gt;80 255 0&lt;/code&gt; to get orange. If your WLED colour order is already set to match the strip, heating comes out green and errors come out... also green. The guide says exactly which three lines to swap.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Ff5ojodvrxdahxkp351mg.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Ff5ojodvrxdahxkp351mg.png" alt="What each macro sends and what a swapped strip shows" width="800" height="382"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  Short version
&lt;/h2&gt;

&lt;p&gt;Flash WLED, install &lt;code&gt;gcode_shell_command&lt;/code&gt;, copy the script and &lt;code&gt;wled.cfg&lt;/code&gt;, change the nine addresses, &lt;code&gt;[include wled.cfg]&lt;/code&gt;, restart, and type &lt;code&gt;LED_IDLE&lt;/code&gt; in the console. Full steps, with the Moonraker bit and every error I've hit: &lt;a href="https://github.com/casareanderson/WLED_Voron--F1-Lights/blob/main/docs/SETUP.md" rel="noopener noreferrer"&gt;docs/SETUP.md&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;Based on &lt;a href="https://github.com/Gliptopolis/WLED_Klipper" rel="noopener noreferrer"&gt;Gliptopolis/WLED_Klipper&lt;/a&gt;, reworked on my Voron 2. What colours would you use for a failed print, red or something more dramatic?&lt;/p&gt;




&lt;blockquote&gt;
&lt;p&gt;🤖 &lt;em&gt;Drafted with AI assistance from my own homelab notes, logs and repos, then reviewed and edited before publishing.&lt;/em&gt;&lt;/p&gt;
&lt;/blockquote&gt;

</description>
      <category>opensource</category>
      <category>tutorial</category>
      <category>github</category>
      <category>ai</category>
    </item>
    <item>
      <title>Set up Jamf scripts safely: check them before they touch a Mac</title>
      <dc:creator>Christian Anderson</dc:creator>
      <pubDate>Wed, 11 Nov 2026 09:00:28 +0000</pubDate>
      <link>https://dev.to/iam-tech/set-up-jamf-scripts-safely-check-them-before-they-touch-a-mac-59jk</link>
      <guid>https://dev.to/iam-tech/set-up-jamf-scripts-safely-check-them-before-they-touch-a-mac-59jk</guid>
      <description>&lt;p&gt;&lt;a href="https://github.com/casareanderson/JSS-Scripts" rel="noopener noreferrer"&gt;JSS-Scripts&lt;/a&gt; is my fork of &lt;a href="https://github.com/bp88/JSS-Scripts" rel="noopener noreferrer"&gt;Balmes Pavlov's Jamf Pro scripts&lt;/a&gt;: deferrable macOS updates, upgrade nags, TCC resets, printer resets. Almost all of it is his work, and his repo has newer versions. Check there first.&lt;/p&gt;

&lt;p&gt;I've added a &lt;a href="https://github.com/casareanderson/JSS-Scripts/blob/main/docs/SETUP.md" rel="noopener noreferrer"&gt;setup guide&lt;/a&gt; anyway, because the process matters more than the scripts: nothing like this should reach a fleet without passing one test Mac first.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fsa1t039kxe1a8vqnye0k.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fsa1t039kxe1a8vqnye0k.png" alt="The syntax check on a fresh GitHub runner: each script parsed with the shell on its first line, one here-document warning from OS_Upgrade.sh, and the LaunchDaemon plist validated" width="800" height="286"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  Check they parse, with the right shell
&lt;/h2&gt;

&lt;p&gt;The README used to say "&lt;code&gt;bash -n&lt;/code&gt; passes for every script". True, and a bit misleading: three of them are zsh. The check now reads each script's first line and uses that shell, and validates the LaunchDaemon as a real plist. It runs every week on a fresh Ubuntu machine.&lt;/p&gt;

&lt;p&gt;It turned up two things worth knowing:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;code&gt;OS_Upgrade.sh&lt;/code&gt; gets a here-document warning from bash 5 at line 786. It still parses, and macOS's own bash is 3.2, but I haven't run it on a Mac, so check the "quit all apps" step on yours.&lt;/li&gt;
&lt;li&gt;The Homebrew script starts with a UTF-8 byte-order mark before &lt;code&gt;#!/bin/bash&lt;/code&gt;, which can stop macOS reading the shebang. Strip it before uploading.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  Read the header, then upload
&lt;/h2&gt;

&lt;p&gt;Every script lists its Jamf parameters, defaults and exit codes at the top. &lt;code&gt;AppleSoftwareUpdate.sh&lt;/code&gt;, for example: &lt;code&gt;$4&lt;/code&gt; postponements (default 3), &lt;code&gt;$5&lt;/code&gt; dialog seconds (900), &lt;code&gt;$6&lt;/code&gt; contact, &lt;code&gt;$7&lt;/code&gt; icon. Exit 11 means no power, 13 FileVault still encrypting.&lt;/p&gt;

&lt;p&gt;In Jamf Pro: Settings → Computer Management → Scripts → New, paste it, copy the parameter labels from the header. Then a policy scoped to one test Mac, run it with a custom trigger, and read the policy log. Try it with someone logged in and with nobody logged in.&lt;/p&gt;

&lt;h2&gt;
  
  
  What CI can't prove
&lt;/h2&gt;

&lt;p&gt;Nearly everything. CI proves the files parse and the plist is valid. Nothing runs on a Mac or against Jamf, and these target macOS 10.12 to 10.15. Treat every one as untested until it's done the right thing on your test Mac.&lt;/p&gt;

&lt;p&gt;Full guide: &lt;a href="https://github.com/casareanderson/JSS-Scripts/blob/main/docs/SETUP.md" rel="noopener noreferrer"&gt;github.com/casareanderson/JSS-Scripts/blob/main/docs/SETUP.md&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;What's your process for testing a Jamf script before it hits the fleet?&lt;/p&gt;




&lt;blockquote&gt;
&lt;p&gt;🤖 &lt;em&gt;Drafted with AI assistance from my own homelab notes, logs and repos, then reviewed and edited before publishing.&lt;/em&gt;&lt;/p&gt;
&lt;/blockquote&gt;

</description>
      <category>devops</category>
      <category>tutorial</category>
      <category>github</category>
      <category>ai</category>
    </item>
    <item>
      <title>Set up t212-autopilot on the demo account, tested against a fake broker</title>
      <dc:creator>Christian Anderson</dc:creator>
      <pubDate>Tue, 10 Nov 2026 09:00:16 +0000</pubDate>
      <link>https://dev.to/iam-tech/set-up-t212-autopilot-on-the-demo-account-tested-against-a-fake-broker-3jc1</link>
      <guid>https://dev.to/iam-tech/set-up-t212-autopilot-on-the-demo-account-tested-against-a-fake-broker-3jc1</guid>
      <description>&lt;p&gt;First thing: &lt;a href="https://github.com/casareanderson/t212-autopilot" rel="noopener noreferrer"&gt;t212-autopilot&lt;/a&gt; doesn't make money. Its own README says the signals it was built around averaged &lt;strong&gt;−0.187R over 213 picks&lt;/strong&gt;, and the one pattern that looked positive got all of it from a single trade. That's why it defaults to the demo account, a £100 ceiling and only 10% on picks. It's a tool for running a plan you've already proved, not a way to make one work. Not financial advice.&lt;/p&gt;

&lt;p&gt;With that said, I've written a proper setup guide, and it's tested every week.&lt;/p&gt;

&lt;h2&gt;
  
  
  A fake broker
&lt;/h2&gt;

&lt;p&gt;You can't let CI near a real brokerage account, demo or not. So the check runs the real CLI against a fake Trading 212 API that answers the reads the CLI makes and records orders instead of placing them. That lets it prove the safety rules, not just that the code imports:&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fdt1w4q0m4cg3sw1wm064.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fdt1w4q0m4cg3sw1wm064.png" alt="The weekly check against a fake broker" width="800" height="1110"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;code&gt;plan&lt;/code&gt; prints the orders and the broker receives &lt;strong&gt;zero&lt;/strong&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;code&gt;T212_MODE=live&lt;/code&gt; without &lt;code&gt;--yes-live&lt;/code&gt; prints the plan, says REFUSED and exits 2, and the broker still receives zero&lt;/li&gt;
&lt;li&gt;demo &lt;code&gt;apply&lt;/code&gt; sends exactly the planned orders&lt;/li&gt;
&lt;li&gt;a plain-http picks feed is refused&lt;/li&gt;
&lt;li&gt;the MCP server answers discovery, but every tool call without the right token gets "not authorised"&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  Short version
&lt;/h2&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;git clone https://github.com/casareanderson/t212-autopilot &lt;span class="o"&gt;&amp;amp;&amp;amp;&lt;/span&gt; &lt;span class="nb"&gt;cd&lt;/span&gt; t212-autopilot
python3 -m pip install -r requirements.txt
cp .env.example .env      &lt;span class="c"&gt;# a DEMO key and secret&lt;/span&gt;
&lt;span class="nb"&gt;set&lt;/span&gt; -a&lt;span class="o"&gt;;&lt;/span&gt; . ./.env&lt;span class="o"&gt;;&lt;/span&gt; &lt;span class="nb"&gt;set&lt;/span&gt; +a
python3 -m autopilot check
python3 -m autopilot plan --picks picks.example.json
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Generate the key on the demo account. A live key gets 401 from the demo host, and that 401 nearly always means the wrong account, not a bad key.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fftrs6qkysk9ra8zdsd7p.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fftrs6qkysk9ra8zdsd7p.png" alt="autopilot plan against the example feed" width="800" height="348"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;The guide explains the limit you'll hit first: Trading 212's public API has no price endpoint, so ETFs you don't already hold get skipped. Full guide: &lt;a href="https://github.com/casareanderson/t212-autopilot/blob/main/docs/SETUP.md" rel="noopener noreferrer"&gt;docs/SETUP.md&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;Has anyone actually found an edge they'd automate with real money? Genuine question.&lt;/p&gt;




&lt;blockquote&gt;
&lt;p&gt;🤖 &lt;em&gt;Drafted with AI assistance from my own homelab notes, logs and repos, then reviewed and edited before publishing.&lt;/em&gt;&lt;/p&gt;
&lt;/blockquote&gt;

</description>
      <category>opensource</category>
      <category>python</category>
      <category>tutorial</category>
      <category>github</category>
    </item>
  </channel>
</rss>
