DEV Community: Christian Anderson

Set up ZimaBlade backups in 20 minutes, and practise first on any box

Christian Anderson — Fri, 30 Oct 2026 09:00:18 +0000

My ZimaBlade recipes are the stuff I learned running a ZimaBlade like a server instead of an appliance. The one most people need is backups, because the obvious way is broken in a way you don't notice.

On ZimaOS your SSH user isn't root and can't sudo. Parts of /DATA/AppData are root-only, mode 0700: your reverse proxy's certificates, tunnel tokens, auth databases. A plain tar can't read them. It complains, exits 2, and still writes the archive. You get a backup of a believable size with exactly the files you can't recreate missing from it.

The new setup guide lets you watch that happen, and the fix, on any Linux machine with Docker before you touch your real data. That practice run is tested on a fresh Ubuntu machine every week.

Practise on a throwaway folder

The guide builds a fake AppData in /tmp with one ordinary app and one root-only tls folder, then points the scripts at it:

export SRC_DIR=/tmp/zima-practice/AppData DEST_DIR=/tmp/zima-practice/Backups CONFIG_DIRS=/tmp/zima-practice/apps
tar -czf /tmp/plain.tar.gz -C /tmp/zima-practice AppData   # Permission denied, exit 2, archive written anyway
bash scripts/backup-appdata.sh --dry-run                     # lists the folder a plain tar would miss
bash scripts/backup-appdata.sh                               # root via a privileged container: it gets in
bash scripts/verify-backup.sh                                # proves it

The line that matters is OK — all 1 root-only director(ies) are in the archive. An ordinary tar couldn't have read that folder, so seeing it in the archive is the proof the backup worked as root.

Then on the Zima

Copy the two scripts over, dry run, back up, verify. The backup runs docker run --privileged -v /:/host, which is root on the machine, so read the scripts first. Recipe 00 explains that one-liner bit by bit.

ZimaOS doesn't run cron, so the schedule lives on another always-on box that SSHes in with remote-trigger.sh. Nothing gets installed on the Zima.

Three new recipes this week

03 Apps CasaOS can't eat. My Homepage dashboard vanished from CasaOS twice. The second time the compose file I'd restored from was gone too. It runs as a plain docker run now.
04 Uptime Kuma on the Zima. Upgrading to the newer full image didn't remove the actively exploited Chromium bugs inside it. The -slim image did, and it's 540 MB instead of 1.71 GB.
05 Disk pressure and a coldstore. I nearly moved data to the Zima to free space elsewhere. The space was Docker images, which can't live on NFS anyway.

What CI can't prove

Anything on a real Zima. The weekly run proves the scripts' root trick and the verify checks on Ubuntu with Docker. ZimaOS differences (BusyBox find, read-only root, no cron) are written up from my own box, not tested in CI.

Full guide: github.com/casareanderson/zimablade-recipes/blob/main/docs/SETUP.md

When did you last actually restore something from your homelab backups?

I rebuilt warden's console: 16 tabs down to 6 (and found my own DB world-readable)

Christian Anderson — Thu, 29 Oct 2026 13:34:49 +0000

warden is my homelab security console — it reads the logs my WAF can't see, scans the boxes, watches my AI agents, and proposes bans and patches I approve with a ✅ in Discord. It never does anything on its own. I wrote up what it does before; this is about the cleanup I just did, because it had got messy.

It had grown 16 tabs

Every time I added something — incidents, agent tracking, vulnerabilities, endpoint integrity — it got its own tab. Dashboards grow like that. One day I counted and it was 16 tabs across four groups, and I realised I was hunting for things I used every day.

So I cut it down to 6:

Overview — the dashboard
Review — the one place for everything waiting on my ✅ / ❌
Threats — detections, edge, network, incidents, geo
Agents — what my AI agents are reaching out to
Posture — attack surface, vulnerabilities, endpoints, health
Settings

The related views sit under a little sub-nav inside each tab, so nothing's gone — it's just one click deeper. And instead of a scary number on every tab, a parent tab shows a small dot when something inside actually needs me. The real counts live where they make sense, so I don't get a meaningless "512" on a tab that's really just counting known CVEs.

The small things that were quietly annoying

The settings page listed raw config keys like patch.expire_hours and cloudflare.zone_id. Now each field has a plain label ("Approval expires after (hours)", "Cloudflare Zone ID") with the raw key dimmed underneath, so I still know what maps to the config file.
API tokens and the "how do I connect a tool" bits were off on their own. They live under Account now, where I'd actually look for them.
The approvals queue used to sit on top of the Settings page. It's its own Review tab now.

The bit that mattered: my database was world-readable

While I was in there I checked how warden stores anything sensitive. Most of it was already fine — API tokens are only kept as a SHA-256 hash, passwords are scrypt, session cookies are random and only their hash is stored, and actual secrets (bot tokens, API keys) never touch the database at all.

Then I looked at the file permissions on the database itself. 644. World-readable.

On a shared box that's a real problem: the database holds logs, session hashes, and — if you use warden's own 2FA — the TOTP seed. Anything else running on that machine as another user could just read it. In my case that includes the sandboxed coding agent I let loose on my repos, which is exactly the kind of thing warden exists to keep honest.

Fix was simple but I made it stick: warden now forces the database (and its write-ahead files) to 0600 on every start, and each process sets a umask so anything it creates is owner-only from birth. I tested it by deliberately setting it back to 644 and restarting — it put itself straight back to 600.

The lesson I keep relearning: the dangerous gaps aren't the exotic ones. It was a default file permission sitting there the whole time.

Get it

It's on GitHub, AGPL-3.0, dependency-free Python — it'll run on a Pi. The v3.2 release has the full changelog, and there's a 5-minute demo setup if you want to click around before wiring it to anything.

Still detect-only and read-only by default. It tells me what it'd do; I decide.

Set up a GPU watch on a ZimaBlade: get told when the card falls off

Christian Anderson — Wed, 28 Oct 2026 09:00:19 +0000

My ZimaBlade has a Quadro P600 on a powered riser for Immich. It works, for a while, then logs Xid 79, GPU has fallen off the bus and stays gone until a reboot. The nasty part is Immich doesn't break. It quietly falls back to the CPU and carries on, slowly. Mine went six days before I noticed.

So the GPU repo has a new script: zima-gpu-watch.sh. It checks every few minutes, logs how long each boot's GPU lasts, and alerts once when it goes and once when it's back. Not every five minutes.

That first line is my actual Zima this morning: dead, 408 hours into the boot. The rest is the check the setup guide runs every week on a fresh machine, using a stand-in for nvidia-smi so it can fake alive and dead.

Diagnose first

git clone https://github.com/casareanderson/zimablade-gpu-immich && cd zimablade-gpu-immich
bash diagnostics/zima-gpu-check.sh

Nine read-only sections. As a normal user, five are accurate and the rest tell you they need root (on ZimaOS that's the privileged-container route in the guide). Read section 04 first: width 0 is a physical fault no driver will fix.

Then watch it

From another always-on box with an SSH key to the Zima:

ZIMA=you@zima GPU_MATCH=P600 WEBHOOK=https://discord.com/api/webhooks/... bash diagnostics/zima-gpu-watch.sh

Put that in cron every 15 minutes. Two rules it follows:

Unreachable isn't dead. If SSH can't reach the Zima (rebooting, network blip) it logs unreachable and doesn't touch the state.
One alert per change. First run just records the starting state.

What CI can't prove

The real hardware. The weekly run proves both scripts run cleanly on a machine with no GPU and that the alive/dead logic alerts exactly once per change. Whether your card stays on the bus is between you, your riser cable and your grounds.

Full guide: github.com/casareanderson/zimablade-gpu-immich/blob/main/docs/SETUP.md

Anyone got a riser setup in a small box that's actually stable? What fixed it?

Set up warden in 5 minutes: try my homelab security console on demo data

Christian Anderson — Tue, 27 Oct 2026 09:00:21 +0000

warden is my in-house security console: it reads the logs, scans the machines, and proposes bans and patches that I approve with a ✅ in Discord. It never does anything on its own.

The usual problem with a tool like this is you have to wire it to everything before you can see if you even like it. So the setup guide starts with demo data. Five minutes, nothing to connect.

The guide is tested too. A GitHub job pulls the commands out of docs/SETUP.md and runs them on a fresh Ubuntu machine every week.

Get it

Linux with systemd and Python 3. Tested on Ubuntu 24.04 and Debian 13.

sudo apt install python3-yaml python3-requests nmap
sudo git clone https://github.com/casareanderson/warden.git /opt/warden
sudo chown -R "$USER" /opt/warden && cd /opt/warden

Look before you wire anything

export WARDEN_DATA=/tmp/warden-demo
python3 tools/demo.py
./warden-setup user add demo --role admin
python3 warden-ui.py

Open http://127.0.0.1:8792 and sign in as demo with the one-time password it printed. You'll be asked to pick your own.

It also has a REST API and an MCP endpoint, so an AI agent can ask it "anything dodgy last night?". Mint a token and try it:

TOKEN=$(./warden-setup token try-it | head -1)
curl -H "Authorization: Bearer $TOKEN" http://127.0.0.1:8792/api/v1/summary

You get JSON with "mode": "detect-only" and the headline counts. Every MCP tool is read-only, and a test enforces that: an agent can ask, it can't ban or patch anything.

Then the real install

./warden-setup init                  # asks about your estate, alerts and machines
./warden-setup check --hosts         # ✓ / ✗ / ○ for every feature
sudo ./warden-setup units --install  # only turns on the timers you need

check is the bit I like. It asks the running system what actually works, and tells you what to do about each ✗. It exits 1 while anything required is missing, so it works in a script too.

What CI can't prove

Everything after the demo. init asks you questions, and the real install talks to your own machines, logs and Cloudflare account, so the weekly run stops at "the example config loads and the checks run". The guide says that next to each step.

Full guide: github.com/casareanderson/warden/blob/main/docs/SETUP.md

What would you want a homelab security console to tell you first thing in the morning?

Set up sieve in 5 minutes: find the CVEs that matter on a Linux box

Christian Anderson — Mon, 26 Oct 2026 09:00:20 +0000

I wrote sieve last week to replace trivy in my homelab. A few people asked how to actually get it going, so this is the short version. The long version is docs/SETUP.md in the repo.

One thing I've done differently this time: the guide is tested. A GitHub job pulls the commands out of SETUP.md and runs them on a brand new Ubuntu machine every week. If a step breaks, the job goes red before you find out the hard way.

That's a real run from today. Funny thing is, a fresh GitHub runner isn't clean: 32,445 findings, 9 of them actively exploited (all kernel tool packages with no fix yet), 31 fixable.

Install

You need Linux and Python 3.10 or newer. No database, no server, no API key.

pipx install git+https://github.com/casareanderson/sieve
sieve --version

If it says command not found, run pipx ensurepath and open a new terminal.

Scan the box you're on

sieve host --limit 10

First run is the slow one, it downloads the CISA known-exploited list and the EPSS scores and asks OSV.dev about every package. After that it's cached.

Read it top down. A ⚠ in the KEV column means someone's exploiting it right now. A massive number with "0 fixable" is normal on a patched box, most of it is "affected, no fix released yet".

The exit code is the useful bit for cron: 0 nothing known-exploited, 1 at least one KEV finding, 2 part of the scan failed.

Scan your containers

This needs root, because container filesystems live under /proc/<pid>/root:

sudo "$(command -v sieve)" containers

Or run the collector on another Docker host with nothing installed there except python3:

sieve collector | ssh root@dockerhost python3 - > packages.tgz
sieve containers --from-tar packages.tgz

Put it on a timer

The line I'd actually use, only shouts about things you can fix:

0 7 * * 1  sieve all --fixable-only --min-severity high

There's also a Docker image and a GitHub Action, both in the guide.

What CI can't prove

Not much for this one. The weekly run covers install, host scan, JSON output, a real container scan and the Docker image. It can't tell you how your own distro feeds behave: Fedora, RHEL and Arch get their Python and container language packages checked, not the OS ones.

Full guide: github.com/casareanderson/sieve/blob/main/docs/SETUP.md

Anyone else running a scanner on a timer at home? What do you do with the "no fix yet" pile?

Everything I've open-sourced from my homelab, in one list

Christian Anderson — Sun, 25 Oct 2026 09:00:17 +0000

People keep finding one of my repos and not the others, so here's the lot in one place. Everything below is free and on GitHub, and every one now has a setup guide in docs/SETUP.md that a GitHub job runs on a fresh machine every week. If a guide stops working, the job goes red, so I find out before you do.

Each guide also says what CI can't prove (anything that needs your hardware or accounts). That bit's in there on purpose.

Security

warden: a homelab security console. Reads the logs your WAF can't see, scans your machines, proposes bans and patches, and waits for your ✅. Detect-only by default. For anyone self-hosting behind a Cloudflare tunnel. setup
sieve: finds the CVEs that matter on a Linux box and its running containers, using OSV.dev, CISA's known-exploited list and EPSS. No scanner binary, no database, no server. setup
farrier: owner-approved patching as a standalone CLI. Plan, snapshot, apply (apt, npm, pip), and it always hands you the rollback. setup

Homelab

zimablade-recipes: running a ZimaBlade like a server. Root through Docker, AppData backups that actually include your certificates, offsite photos, apps CasaOS can't remove, a slimmer Uptime Kuma. setup
zimablade-gpu-immich: a GPU in a ZimaBlade for Immich. The slot's real limits, the driver trap for older cards, and a watcher that tells you when the card falls off the bus. setup
garden-room-lights: time-of-day lighting for a room of WLED strips, with brightness that respects each strip's power limit. setup
WLED_Voron--F1-Lights: Klipper macros that turn a Voron 2's LED strip an F1 colour for each print state. setup

AI agents

model-radar: finds free models that really make tool calls, and fails your agents over when the paid model or the credit runs out. setup
toolsmith: a Hermes agent that researches a replacement for one of your tools each week, tests it in a sandbox, and withholds anything it can't back with a measured fact. Needs Hermes and Proxmox. setup
ephor: passive, detect-only monitor of what your AI agents connect to, download and can access. Reads /proc and the socket table, so nothing can route around it. setup
proctor: an AI code reviewer you run from a terminal. Point it at a diff, it reports what's wrong and fact-checks every finding against the patch. setup

Music

setlisted: a headless Spotify CLI for non-repeating discovery mixes, hour-by-hour event sets and playlist flow. setup
spin-that-dice: a jukebox dice for parties. Tap the die on a tablet, get a random track from one of 43 categories of music, and it plays. setup

Shopping and eBay

sold-shadow: build your own eBay sold-price data from the official API, and measure whether it can replace the paid data you use. setup
cex-uk-prices: CeX UK sell, cash and voucher prices and store stock, with the traps measured (£0 means they won't buy it, voucher isn't cash). setup

Writing

blog-kit: what I publish these posts with. A dev.to publisher with a leak check that refuses private IPs and keys, plus diagrams and covers. setup

Just for fun

cryptonym-desk: name the films and characters you like and it issues you a spy-style cryptonym. One HTML file. setup
cryptonym-desk-sanity: the same desk rebuilt on Astro and a public Sanity dataset. setup

If one of these is useful to you, a star on GitHub genuinely helps people find it. Which one should get a proper write-up next?

16 containers on 2 cores: what my ZimaBlade home server actually uses

Christian Anderson — Sat, 24 Oct 2026 08:00:17 +0000

People ask if a little box like a ZimaBlade can really run a homelab. Mine runs 16 containers on a dual-core Celeron N3350 from 2016. So I measured what each one actually uses.

The box: 2 cores (1.1 GHz, boosts to 2.4), 16 GB of RAM, two 6 TB IronWolf drives on SATA. It had been up 17 days when I took these, and Docker hadn't restarted a single container.

RAM: Immich is most of it

Added up, the 16 containers use just under 3 GB. The Immich stack (server, Postgres, machine learning, Redis) is about 2.2 GB of that, roughly three quarters:

immich-server: 1.34 GB
immich-postgres: 602 MB
immich-machine-learning: 239 MB
uptime-kuma: 183 MB
npmplus (reverse proxy): 137 MB
homepage: 112 MB

Everything else is tiny. Authelia, the thing every login goes through, is 32 MB. The Cloudflare tunnel is 29 MB. CrowdSec is 57 MB.

The OS says 5.4 GB "used" overall, but 10.5 GB is still available because the rest is file cache it'll hand back. It's also got about 1 GB sitting in swap.

CPU: idle, mostly

Two snapshots 20 seconds apart and almost everything was under 1% CPU. The busiest were Uptime Kuma (4% then 1%) and immich-server (under 1%, then nearly 3%). Earlier in the day the load average sat around 0.9 on 2 cores, so about half busy, and it sat at 41-44 °C.

So for always-on stuff that mostly waits for requests, 2 old cores are fine.

Where 2 cores hurt

Here's the bit I didn't expect. While I was taking these measurements, the 1-minute load jumped from about 0.9 to 3.4 and the kernel's CPU pressure figure said something was stuck waiting for CPU about 75% of the last 10 seconds:

docker stats itself isn't free, and I can't promise nothing else kicked off at the same time. But it shows the shape: with only 2 cores, any burst queues up behind everything else. Idle services are cheap. Bursty work, like ML indexing, video transcoding or someone uploading 300 photos, is where this CPU runs out. And my Immich machine learning runs on the CPU (the P600 is too old for Immich's GPU build), so that work lands on these same 2 cores.

Disk, not CPU, is the other limit

The fast storage is a 17 GB eMMC partition, now 72% full, mostly a 5.2 GB swap file and 3 GB of OS update files. Docker's images (12.5 GB, 5.4 GB of it old ones I could prune) live on the big array, which is only 36% used. So the thing I watch isn't CPU or RAM, it's that little eMMC.

What it means if you're picking hardware

From my numbers, not a benchmark:

RAM was the right thing to have plenty of. 16 GB means Immich can be greedy and nothing else notices.
The CPU is fine for services that idle. It's not fine for bursts, so keep heavy jobs off it or schedule them.
Real SATA mattered more to me than CPU speed. Photos live on proper drives, not USB.
On paper I'd expect a Raspberry Pi 5 to beat this CPU. I haven't benchmarked that yet. That's the next post, with prices and power once I've got real numbers.

Pi or mini PC for your homelab? And what's the biggest RAM hog on yours?

What a ZimaBlade can't do: the limits I hit running it as my home server

Christian Anderson — Fri, 23 Oct 2026 08:00:19 +0000

I've run a ZimaBlade as my main home server for a few months now. It does photos (Immich), single sign-on, the reverse proxy, monitoring, CrowdSec and a Cloudflare tunnel. I like it. But it's a small board and it has proper limits, and most of them I found the hard way.

I already wrote up how I set it up as a server rather than an appliance and the saga of the Quadro P600 in it. This one is the list I wish I'd had before buying.

1. Power comes in over USB-C, and there isn't much of it

The stock power brick is 12 V / 3 A, so 36 W, over USB-C. Kits since late 2025 come with a 60 W one. The board, RAM and two SATA drives already sit around 25-30 W, so there is no headroom for anything else.

I got this wrong in my own repo at first. I'd written instructions about a barrel jack. There is no barrel jack, it's USB-C. Fixed now, but if you're planning to power a riser or a GPU off the same supply, don't. Anything hungry needs its own PSU.

2. The PCIe slot is small in every direction

What the slot actually reports:

PCIe 2.0 x4, so about 2 GB/s at best
a 25 W slot power limit, so a 40 W card can't run off it directly
no hot-plug: a card only gets found at a cold boot, rescanning does nothing
no Above 4G decoding: the biggest PCI window is about 1.25 GB, fine for a 2 GB card, probably a wall for anything with big VRAM

So "stick a GPU in it" really means a riser, an external PSU and a small, old card.

3. Two power supplies, two grounds, one very confused GPU

My riser setup ran the Zima off its own unearthed brick and the GPU off an earthed ATX PSU. Two ground domains, joined only by the thin grounds in the riser's cable. The fan spun, so I assumed power was fine. The PCIe link never trained at all.

After a lot of physical faff in the loft it did enumerate. Then it ran for about 8.5 hours and dropped off:

The link had also dropped itself to PCIe 1.0 x1. These days a small watcher script records how long the card survives each boot and only shouts when it changes state, rather than every 15 minutes. The full story's in the P600 post.

4. The OS is read-only and you don't get sudo

ZimaOS is immutable. / is a read-only squashfs that always shows 100% full. That's by design, it's not a full disk, but it scared me the first time.

The one SSH user can run Docker without sudo, but sudo itself wants a password I don't have. So editing anything root-owned (a systemd unit, /etc/exports) means borrowing root through a privileged container:

docker run --rm --privileged --pid=host -v /:/host alpine \
  chroot /host systemctl status some-unit

If that looks dodgy, it is a bit. But being in the docker group already means being root in all but name, so it's not adding a hole, it's just honest about the one that's there.

5. The app store deleted my dashboard. Twice.

I ran Homepage as a CasaOS app. In July it vanished: container gone, config still on disk. I reinstalled it from the leftover compose file. In September it vanished again, and this time the compose file had gone too.

Now it's a plain docker run with --restart unless-stopped, and CasaOS has no say in whether it lives. Same for Uptime Kuma when I moved it onto the Zima. Anything I care about stays out of the app store.

Related trap: back in July five of my containers had restart=no. CasaOS starts them at boot, so it looked fine, but a container that crashed mid-life just stayed dead. One sat dead for six days before I noticed. I switched them all to unless-stopped then. When I checked again for this post, 8 of the 16 running were on restart=no again, so it's a check worth repeating.

6. 17 GB of eMMC fills up fast

The fast storage is a 29 GB eMMC, with a 17 GB /DATA partition. The two big drives are a separate 5.5 TB array.

12 GB used and 4.7 GB free. The biggest things on it are a 5.2 GB swap file and 3 GB of OS update files. Docker's 12.5 GB of images (5.4 GB of that is old images I could prune) actually sit on the array. Rule I follow now: anything big or cold goes on the array, never /DATA. I also share a "coldstore" folder off the array to my Proxmox boxes over NFS, mounted with nofail and automount so a Zima outage can never stop them booting.

7. It can't watch itself

Uptime Kuma runs on the Zima. Which means when the Zima dies, the thing that's meant to tell me is dead too. So a script on a different box checks Kuma every 5 minutes and restarts the container over SSH if it's gone. I tested it by stopping Kuma, it was back in seconds.

So would I buy one again?

Yeah, for what it is. Two real SATA ports, 16 GB of RAM and an x86 CPU that runs every Docker image without thinking about ARM builds is a lot in a box that size. But it's a 2016-era dual-core Celeron with a weedy PCIe slot and a USB-C power brick, and you have to work around that rather than pretend it's a NAS appliance. Next post is what 16 containers actually cost it.

What's the limit that bit you on your homelab box?

FontForge brand fonts: why changeWeight failed and offset outlines worked

Christian Anderson — Thu, 22 Oct 2026 08:00:37 +0000

I run two side projects that look nothing alike on purpose. The tech one (homelab notes, tips, the Server reels) is black and acid green with IBM Plex Sans Condensed Bold for headlines. The poster one is gold on black with Anton. Both fonts are free and brilliant, which is the problem. Half the internet uses them.

On 29 September I wanted something "a little off what we use, sharper lines, more impact". Not a new typeface, I'm not a type designer. Just my existing fonts, a bit heavier with a signature detail, so a headline looks like mine before you read it.

Here's what broke, the licence rules, and the colours, which were less tidy than I thought.

The tools

FontForge has Python bindings (apt install python3-fontforge on Debian/Ubuntu). The whole build is one 82-line script: open the source font, loop over every glyph, change the outline, rename the font, write .otf and .ttf, then render a specimen with Pillow so I can actually see it.

Two fonts come out of it:

iC Volt Condensed, from IBM Plex Sans Condensed Bold, for the tech brand
C1 Cut, from Anton Regular, for the poster brand

v1: changeWeight made a mess

FontForge has a built-in for this, glyph.changeWeight(). One call per glyph, give it a stroke amount, job done. So v1 did exactly that: 3% of the em for the Anton one, 3.4% for Plex, then tightened the letter spacing by 1.2% and 1.8%.

It never errored. It was wrapped in a try/except: pass (not proud of that), so I went back and counted: 596 glyphs on Anton, 330 on Plex, zero exceptions.

It just looked rubbish. The heavy Anton one clogged up, the holes in R, G and 8 nearly closed. The R leg grew a weird curled foot. The S on the Plex one got little nicks at its ends. And the tighter spacing made the clogging worse, letters started kissing.

That left-hand side is a rebuild of v1 from the old script today, so it's real output, not a mock up.

v2: offset the outline yourself

What I actually wanted was "the same letter, a bit fatter, corners still sharp". So v2 does that directly:

removeOverlap() so the glyph is one clean outline
take a copy of the outline and stroke it with a circular pen, butt caps and mitre joins, keeping only the outer band
add that band back onto the original body and merge it with removeOverlap() again

The mitre join is the bit that matters. A round join would soften every corner, which is the opposite of "sharper". A mitre keeps the point.

body = g.foreground.dup()
band = g.foreground.dup()
band.stroke("circular", 2 * weight * em, "butt", "miter", "removeinternal")
body += band
g.foreground = body
g.removeOverlap()

v2 also uses way less weight: 1% of the em on C1 Cut, 2% on iC Volt. A little goes a long way when the shapes aren't getting mangled.

The cut corners

The signature detail is a chamfer, like a machined edge. For every corner point the script checks:

the point and both neighbours are on-curve, so it's a straight line meeting a straight line
both edges are longer than 3x the chamfer size, so tiny details don't get eaten
the angle is between 60° and 120°

If all three pass, the corner point gets swapped for two points a set distance back along each edge. That's 3% of the em on C1 Cut and 2.8% on iC Volt. Curves are never touched, so S, G and 8 keep their shape and only the hard corners on things like M, T and K get cut.

After that it's housekeeping: correctDirection(), addExtrema(), round(), then spacing. C1 Cut went 0.4% looser, iC Volt 0.6% tighter, split evenly across both side bearings.

The licence bit (do read this)

Both source fonts are under the SIL Open Font License 1.1. It's a generous licence, but it has rules for modified versions:

Reserved Font Names. IBM's licence for Plex says "Reserved Font Name 'Plex'". A modified version can't use that name. So no "Plex Volt" or anything like it, hence iC Volt Condensed. Anton doesn't reserve a name, but I renamed it to C1 Cut anyway so nobody mixes it up with the real thing.
Rename it properly. The script strips the old Family, Full name, PostScript name, Unique ID, Preferred Family, WWS and Trademark entries from the name table and writes new ones. It appends "Modified 2026 ... (SIL Open Font License 1.1)" to the copyright, it doesn't replace it.
Ship the licence with the font. And the modified font stays under the OFL.
Don't sell the font files on their own. You can use them in stuff you sell, but the font itself isn't a product.

Where I got it wrong: the first licence file the script wrote for iC Volt was a three-line notice, credit and a link, without the actual licence text. When the font went on my site I shipped the proper file instead: the notice plus the full OFL 1.1 text, served next to the font with a "Font licence" link in the footer.

Where they're used right now

iC Volt went live on 8 October on my tech site, headlines only. It's a Latin-only subset made with fontTools, 12,232 bytes as WOFF2. Body text stays IBM Plex Sans and Plex Mono. The Friday newsletter template uses it too, with Impact as the fallback because loads of email clients won't load web fonts.

C1 Cut isn't used anywhere yet. The poster renderers still use Anton, and the tech Instagram renderer still uses Plex Condensed Bold. Swapping is mostly changing a font constant in each renderer, but I'd rather the owner of each brand (me, wearing a different hat) looks at it on real posts first.

And the colours

I'd been calling these "colour packs" in my head. Being honest, there's no colour pack system. What exists:

The tech brand has one palette of eight tokens: ink #0B0C0E, panel #15181C, grid #262A30, paper #F1EFE8, mute #8B9099, volt #D7FF3A, red #FF5A4E, green #39D98A. Volt is the headline and accent colour, red and green are for the do/don't slides and result labels. It lives as a BRAND dict in the Instagram renderer, as RGB tuples in the reel renderer, and as CSS variables on the site.

Three copies, no shared file. And it's already drifted. The newer "Server explains" reel script defines its own red and green, (255, 92, 92) and (92, 220, 120), which are close to the brand ones but not the same, plus an amber the palette doesn't have. Nobody would spot it on a phone, but that's how a brand goes muddy. Next job is one palette file that everything imports.

The poster brand is simpler at the top: gold #D4AF37 on #0B0B0B with #F5F3EE text in the video renderer. The posters themselves use named colourways, which is the closest thing I've got to actual packs. Each one is a dict with a background, a gradient or solid fill for the glyphs, an accent, title, subtitle and fine-print colours, plus a flag for light or dark. There are five base ones (noir-gold, noir-silver, paper, neon, pop), and limited runs add their own with COLOURWAYS.update() so they never reuse a base one.

What I'd tell you

No errors isn't the same as working. changeWeight ran clean on 926 glyphs and still made rubbish. Render a specimen every build.
Mitre joins if you want weight without losing sharp corners.
Check the licence for a Reserved Font Name before picking a name.
Colours in one file from day one.

Has anyone made a proper derivative font for their own stuff? I'd like to know how you checked yours at small sizes, because I've only really looked at mine big.

Why I built warden: fail2ban can't see attackers behind a Cloudflare tunnel

Christian Anderson — Wed, 21 Oct 2026 08:06:03 +0000

I already wrote up what warden does and the bugs I hit building it in Homelab security on 512 MB. This one is about why it exists at all. I didn't set out to write a security tool. I kept finding gaps in the setup I already had, and warden is what was left after I filled them.

The address you want to ban isn't on the wire

My public services sit behind a Cloudflare tunnel. That's good for lots of reasons, but it breaks the usual homelab reflex of "see a bad IP, drop it in iptables".

Behind the tunnel, the source address at the network layer is cloudflared or the Cloudflare edge. The attacker's real address only exists in the HTTP layer, in a header the proxy passes along. So a local firewall ban keyed on the attacker never matches a single packet. And if you ban what does show up at L3, you've banned Cloudflare and cut off your own estate.

I wrote that down before writing any code, and it's why warden has no iptables code in it. Any ban has to happen at the Cloudflare edge, before traffic reaches the tunnel. Cloudflare's own edge ranges sit on warden's allowlist from the start, next to my own addresses.

CrowdSec was running. It just couldn't see everything.

I'd had CrowdSec in place since June, sending its local bans to Cloudflare. On paper that was covered. When I looked properly in August, it wasn't.

The web tier was basically unwatched. My reverse proxy doesn't write the standard nginx log format, so when I mounted its log as nginx, every line landed in "unparsed". That's coverage that looks real and detects nothing. I wrote a custom parser for it, but the bigger hole was the tunnel itself. Some hostnames go straight from the tunnel to a service and skip the proxy completely, and nothing was reading cloudflared's log. That's where I found someone probing one of those direct routes for a Vite file-read bug, against a service that wasn't even running any more.

So warden started on 30 August as a small, dependency-free second opinion to CrowdSec. It reads the logs CrowdSec can't see or doesn't parse (the proxy log, my SSO's log, and the tunnel log) and scores what it finds. CrowdSec keeps running alongside it.

Why it asks before it acts

CrowdSec bans automatically. My sync pushed its local bans to the edge every 60 seconds. That's fine until you look at who is getting flagged.

The only local CrowdSec alert in one September week was my own WAN IP: the Immich app on my phone asking for 11 thumbnails I'd deleted, all within four seconds. Warden agreed with it: 27 of the 29 app-layer events that week came from that same address. On 7 October, 281 of the 295 edge events I pulled were me as well. On 10 October it was 1,526 of 1,577 events in 24 hours, until I filtered my own rows out of the counts.

The loudest "attacker" on my network is me. A detector you don't fully trust, pointed at traffic that's mostly yours, shouldn't be holding the ban button.

I'd also seen what automation does with bad readings. My security daemon once restarted CrowdSec every ten minutes for a day. CrowdSec was never down. An SSH timeout came back empty and got read as "service dead". Warden's own first ingest put weeks of July history inside the live ban window, which in enforcing mode would have banned people for July (the details are in the other post).

So warden ships detect-only. It records what it would ban and changes nothing. When I want something blocked at the edge, it proposes it in Discord and I tap ✅ or ❌. Approved bans go into one WAF rule and lapse after 30 days. A ❌ is remembered for 90 days so it doesn't keep asking. It refuses to propose my own addresses, private ranges or Cloudflare's. Patching works the same way: it shows me the upgrade plan first, takes a snapshot, and never reboots a Proxmox node on its own.

One place, on a small box, with nothing installed

The other reason was sprawl. Logs in one tool, CVEs nowhere, patches done by hand, and no real list of what was on my network. When I needed a SIEM layer in September I chose to extend warden instead of deploying Loki or Wazuh.

Everything it does is agentless. Vulnerability scanning copies the package list off each box and checks it against OSV.dev, the open database that Debian, Ubuntu, Alpine, GitHub and the Go team publish their advisories into. For Docker it reads what's actually in each running container, not just the image it came from. Integrity checks are one read-only shell collector per box. Nothing gets installed on the things being watched. I measured the core at 1 vCPU, 512 MB of RAM and 300 MB of disk.

It used to run Trivy for the vulnerability side, and that was the weak point: a 161 MB binary, a server process, a 1.4 GB database, and a project whose own release got backdoored this year (CVE-2026-33634). So I swapped it out on 10 October. On my Proxmox node the new scanner found 680 of the 691 CVEs Trivy found, both agreed nothing was fixable, and the odd ones out were the two databases updating at different times. It also binned two "actively exploited" Trivy alerts for a package one container doesn't even run any more.

None of the pieces are new. Wazuh, CrowdSec, Vuls, Security Onion and others all exist, and warden leans on OSV.dev's data, Lynis and Suricata for the hard parts. What I couldn't find was that combination: agentless, small, and asking in chat before it does anything. It's AGPL-3.0 since 8 October.

Agents read it, they don't decide

I run Hermes agents at home, and the obvious temptation is to let one of them do the security work. I don't. Warden has a read-only MCP server, and my sec agent connects to it as a reader: summaries, detections, devices, CVEs. My IT admin agent includes warden's facts in its daily report, and its rules say it reports and never approves. Scripts gather the facts and I make the call. The agent is there to explain things, not to decide them.

The newest bit: the house itself

The newest gap was the house. Warden knew about servers, but not the bulbs, plugs and TVs. On 10 October it gained a device inventory that merges ARP sweeps with the router's DHCP leases: 107 real devices from 133 addresses, with 5 left as honest unknowns rather than guesses.

Then I gave each device a DNS baseline from the router's query log. Replaying a week of history gave 1.17 million lookups across 85 devices. Some devices are very predictable. One WiZ bulb has talked only to wiz.world for nine days. A smart lock bridge talks only to its maker.

That made scored decisions possible instead of "count over threshold". When a fixed-function device that's past its 7-day learning period hits a domain it has never used, warden starts at 40 points and adjusts. Plus 25 if no other device uses that domain, minus 25 if two or more do, minus 30 if the same kind of device all changed together, minus 20 if the maker's name is in the domain, plus 15 for 00:00 to 05:00, plus 15 for a risky TLD. Under 25 it just learns it, 25 to 54 gets logged, and 55 or more asks me.

Replaying the real week produced zero reviews. A simulated WiZ bulb calling update-check.xyz at 3am scored 95 and showed up for review.

It's still limited. Device scoring only sees DNS, my IoT VLAN has no devices on it yet, and the edge ban path is brand new: three bans approved so far, all on 10 October. But each part exists because something I already ran was missing it, not because a feature list said so.

If you run services behind a Cloudflare tunnel, how do you deal with bans when the address you'd block only exists in a header?

Agent-run social media for a one-person Etsy shop: what 205 views taught me

Christian Anderson — Tue, 20 Oct 2026 08:06:04 +0000

I run two tiny things on my own. THEC1LAB is a print shop on Etsy: posters drawn in code, gold on black. IAMC/TECH is my tech page: homelab field notes, tips, AI tools. So over the last few weeks I've had agents on my homelab run the social side, and I just approve or veto.

Two brands, kept apart on purpose

Early on I saw one combined intro covering all my pages and it looked wrong. They're different brands with different people buying. So the rule is simple: no Etsy CTAs on @iamc_tech, no homelab tips on the shop pages. Each brand has its own intro, its own colours, its own sign-off.

It still bit me twice. A planets post on the shop account got tech hashtags. And the tech page's "what's inside" reels wanted to end on a THEC1LAB poster card, which had to be stripped.

Each channel and how it's fed

Channel	How it's fed	What needs a human	Result so far
TikTok @thec1lab	Daily video rendered from the fact bank, sent as an inbox draft at 17:30	Me: add a sound, press post	200 to 740 views per video
Instagram @thec1lab	Reels, carousels, stories; preview in Discord about an hour before	❌ to cancel; pins, bio	Carousels blocked since 7 Oct; reels still go out
Instagram @iamc_tech	"Server" reels: jokes, tips, explainers	❌ to cancel	Started from 1 follower
YouTube Shorts	Uploader takes the day's approved video at 19:45	Nothing after approval	First Short: 1,019 views, 15 likes
Pinterest	Weekly bulk CSV of fresh pins	Me: upload the CSV	First shop pin scheduled for 11 Oct
Threads @iamc_tech	One tech news story per slot, plus my own dev.to posts	❌ to cancel	Too early to say

TikTok. My TikTok is a personal account. Switching to business wanted Companies House details and I'm a sole trader. Posting through the API from an unaudited app gets forced to SELF_ONLY, which means only I can see it. Cookie bots were off the table because of shadowban risk. So the agent sends each video as an inbox draft, drops it in Discord at 17:30, and I add a sound and post it myself. Sound matters: early on a video with a commercial sound and a question caption got 389 views, a silent shop drop got 0.

Instagram. Posting to my own account through the API is allowed here, so it's fully automated. Every post gets previewed in Discord about an hour before it goes out and I can ❌ it. The agent also re-weights the mix every week from insights.

YouTube Shorts. I set the channel up on 8 Oct, uploaded one Short, and then nothing. Turns out the daily uploader was never actually built. That first Short got 1,019 views and 15 likes, better than any single TikTok I've done. The uploader went in on 10 Oct: it takes the day's approved video, fixes the loudness, uploads it and checks it really came out public.

Pinterest. This one's about links. On 10 Oct my 138 active Etsy listings had 205 views ever and 1 favourite, while TikTok videos do 200 to 740 views each. The problem is the bridge: a personal TikTok gets no clickable bio link until 1,000 followers. Pins are links. So every week a script makes fresh pins (2 a day, one per listing, words only from the listing itself) and writes a bulk CSV. I upload it by hand. Pinterest's API trial access only makes pins visible to you, and I turned down using my own cookie to automate it.

Threads. The tech page's Threads is for quick tech and AI news: one Hacker News story, why it matters, a clickable source link, under 500 characters. Since 8 Oct one slot a day shares my own dev.to posts, because I don't self-promote enough.

Scripts gather facts, models never do

This is the rule that holds the whole thing together. The model can write a caption or a joke. It does not get to invent a number.

So the moon distances, binary puzzles and "which part is this" answers are all computed by code from a fact bank. Tech facts carry a source and a verbatim quote. Then validators check the output: numbers and names only from the cited facts, no hype words, no "closest" or "only" unless the fact text says so. In testing it caught a made-up 360,000 km, a stray London and a model typing "Manches... nah let me try again" into a caption.

There's also a small decision model that scores captions. Genuine questions came back as supported at 0.81 to 0.95. A made-up claim scored 0.05. "Comment YES and tag 3 friends" scored 0.97 for bait.

The jokes on the tech page work the same way. The comic writer gives up to six fresh jokes a day, filters throw out anything with digits or a reused punchline, and then the judge asks "would most people smile". I added the judge because the first fresh jokes passed every check and still weren't funny. One example: a foreign-key joke scored 0.81, a flat one 0.13. Anything under 0.6 falls back to a fixed list. Since 10 Oct the shop's talking-Moon banter gets a fresh judged joke every day too, so it doesn't run out.

And on top of all that, my ✅ and ❌ in Discord. The agents do the day-to-day running. I keep the veto.

The honest numbers

As of 10 Oct:

Etsy: 138 active listings, 205 views ever, 1 favourite.
TikTok: 200 to 740 views a video. The anatomy smartphone one did 716.
YouTube: one Short, 1,019 views, 15 likes. Uploader built 10 Oct.
Instagram shop account: carousels have failed 6 of 7 times since 7 Oct 16:00 with "Action is blocked". Quota use was 1 out of 100 and reels went straight through, so it's an account-level spam flag, not volume. Reels and stories still publish daily.

Also, the comment agent has never seen a single comment on either Instagram account. The Meta app is still in Development mode, which only shows comments from people with a role on the app. Strangers are invisible until I switch it to Live.

The bug that hid in plain sight

The worst one was audio. The reel mixer was built for silent videos. So it only kept the music, loudness-normalised it to -14 LUFS, and silently dropped the talking-Moon voices. Every banter reel went to Instagram as full-volume music with the characters miming. I only caught it because the music sounded too loud.

For the tech page it's now voice at -16 LUFS with music at -30 underneath. The shop account kept the old mix for now. On the TikTok side the banter originals still come out at about -36 LUFS, which is very quiet, and that's not fixed yet.

What didn't work

Cookie bots. I tried a headless browser with my own Etsy cookies to send buyer messages. The first page loaded, the second got a 403: "Access is temporarily restricted … automated (bot) activity on your network". That's my home connection, so it could hit my normal browsing too. Stopped there, nothing sent. Same answer for TikTok and Pinterest cookies: not worth the ban.

Engagement bots. I asked for something to follow and like for traction on Instagram. The answer was no: it breaks Meta's terms, and the API can't do it anyway. Better content and fast replies instead.

£1 drops. Cheap limited drops on TikTok didn't work, so they've stopped.

What's next

The talking-Moon banter shorts are the content model now, and from 16 Oct they start promoting other products in the shop, not just one.

The real test is Pinterest. The first shop pin was scheduled for 11 Oct, and around 22 Oct I'll check Etsy's traffic sources to see if pins actually send anyone. The evidence for Pinterest driving Etsy printable traffic is anecdotal, so I'm not counting on it.

If you've run a small shop off a personal TikTok, what actually got people from a video to your listing before you hit 1,000 followers?

Toolsmith: building a homelab research agent that says 'I don't know'

Christian Anderson — Mon, 19 Oct 2026 08:02:00 +0000

The two-year itch

Every six months or so I stare at my homelab and wonder if the thing doing DNS should be doing SSO, and the thing doing SSO should be doing MFA, and the thing doing backups should be doing... you get the picture. Tool fatigue is real when you're running fifteen services on a low-power box and each one made sense when you installed it but the landscape has shifted twice since then.

I wanted an agent that would do the research for me. Not a chatbot that vomits a comparison table, but something that comes back every week with a measured brief: here are the three alternatives to your current tool, here's why you'd switch, and here's why you wouldn't.

I built it. I called it toolsmith. And the most important piece of it is the component that refuses to answer the question.

What toolsmith actually is

Toolsmith is a weekly research agent that lives in a Docker sandbox and has one job: given a homelab tool you're running, find credible replacements and write up what you'd gain and lose by switching. The repository lives on GitHub if you want the full picture.

The architecture hits three constraints that shaped everything else.

Measured fact packs. Every recommendation must cite where the information comes from. Not a link dropped at the bottom — the claim and the source are paired in the output. If the source says one thing and the marketing page says another, the agent is told to surface the conflict, not pick a side.

Capped Docker sandbox with no egress by default. The agent runs in a container that can't reach the internet unless a specific research phase explicitly opens an outbound channel. That means no "let me just check the latest blog post mid-run" drift. The research window is defined upfront, the sandbox opens, the data comes in, the sandbox closes. It keeps the agent honest about working with what it gathered rather than live-navigating and getting distracted.

A validator that withholds unsupported recommendations. This is the one I want to talk about.

The validator

Most agent projects I've built or studied optimise for giving an answer. The agent is embarrassed if it says "I don't know" — so it finds something, anything, to recommend. Maybe it conflates two tools, maybe it extrapolates from a forum post with three upvotes, maybe it just makes something plausible.

Toolsmith's validator does the opposite. It looks at the proposed recommendation and checks: is there a direct source for every factual claim? If the answer is no for any claim that the recommendation depends on, the validator strips that recommendation from the output. Not marks it as low confidence — withholds it entirely.

The first time I saw this in action I thought I'd broken something. The agent ran, did its research, produced a perfectly reasonable-looking analysis, and the validator gutted half of it. Two of three tool recommendations vanished. What remained was thin. I nearly patched the validator to be more lenient.

I'm glad I didn't.

What remained was correct. The surviving recommendation cited actual documented feature matrices, a real changelog entry about the breaking change I was trying to avoid, and a comparison that accounted for the specific hardware constraints of my homelab box. The two that got cut were built on forum speculation, a six-month-old Reddit thread that the agent had treated as authoritative, and one that was just plain hallucinated version numbers.

The commit history shows a few rounds of "validator identifier fixes" — the nitpicky work of getting the validator to recognise source material reliably. It is not a solved problem. But the direction is the right one.

What broke

A few things didn't survive contact with reality.

The knowledge base was only as good as its last sync. I integrated a Sanity Context MCP knowledge base to give the agent a curated starting point. Great in theory. In practice, the knowledge base needed prose updates, identifier corrections, and new demo images constantly. If I wasn't tending it weekly, the agent was working from stale ground truth. The knowledge base became a second project to maintain, which defeats the purpose of an agent that's supposed to save you time.

The sandbox isolation added friction to iteration. Every time the research scope changed — "oh, also check if this tool supports LDAP" — I had to redeclare the sandbox permissions. The no-egress-by-default design is principled, but when you're iterating on prompts it feels like the system is working against you. I don't have a clean solution for this. I suspect the answer is better tooling around ephemeral sandbox configuration rather than softening the isolation.

The Buy Me a Coffee button went up before the thing was useful. I added it early, before the validator was tuned, before the knowledge base was populated. Nobody bought me a coffee, which is exactly what should happen when you ask for support on a project that still eats its own homework. The commit that added that link sits there as a reminder to ship value before asking for gratitude.

The lesson that transfers

The takeaway isn't about toolsmith specifically. It's about where to put the defensive boundary in an AI-assisted workflow.

Every agent project I see optimises for throughput — more completions, faster answers, lower latency, broader knowledge. The implicit assumption is that more information is better and the human will filter out the bad parts. But that's exactly wrong for homelab work. A bad recommendation in production infrastructure doesn't waste an hour, it wastes a weekend of rebuilding something that was working fine.

The right defensive boundary is the one that says "I have an answer but I cannot vouch for it, so I will not give it to you." Not a confidence score, not a disclaimer paragraph, not a footnote. Withhold.

That design choice — a validator with teeth — changes the optimisation target of the whole system. The agent stops trying to sound smart and starts trying to be verifiable. The research phase becomes about collecting sources that can survive inspection rather than assembling a plausible narrative. The prompt engineering shifts from "be thorough and helpful" to "be correct or be silent."

I don't think this applies only to agents. Any automation that makes decisions about infrastructure should have the same bias. A backup script that half-works is worse than one that fails loudly. A migration tool that mostly migrates data is worse than one that refuses to proceed on schema mismatch. Sometimes the most helpful thing a tool can do is tell you it can't help.

Where it stands now

Toolsmith runs weekly in my homelab. Some weeks it produces useful research. Some weeks the validator cuts everything and I get an empty report. Those weeks I curse it, then I go manually look up what I needed, and usually I find that the validator was right — there wasn't a clear enough answer to automate.

That's fine. An empty report from an honest system is worth more than a confident recommendation from one that doesn't want to disappoint you.

The repo is at casareanderson/toolsmith if you want to see the current state. It's early, it's rough, and the validator eats its own output regularly. But the direction — measured facts, isolated research, and a hard no when the evidence isn't there — is one I think more homelab tools should take.

Built for the shelf, not for the shop window.