DEV Community: Prateek Rai

Building a Vitals Monitor for Claude Code

Prateek Rai — Sat, 18 Jul 2026 23:23:02 +0000

A Slack thread about our climbing AI bill, plus word that per-user usage limits were landing soon, sent me to check what I was actually spending on Claude Code. The answer was that I had no idea.

The cost only showed up after the fact, in a dashboard I never opened. So I built a gauge instead. My terminal footer now reads out the session's vitals on every render, and the whole thing is one shell script.

In a hurry? Skip to what the line shows.

What the status line gets to see

Claude Code pipes a JSON blob to whatever you register as its statusLine command, once on every render. My script reads that blob with a single jq pass and prints one line back. Claude Code renders each stdout line as its own status row, so I get to decide exactly what sits under the prompt.

Registering it is two lines in ~/.claude/settings.json:

{
  "statusLine": {
    "type": "command",
    "command": "bash ~/.claude/statusline-command.sh"
  }
}

Here's the field extraction, trimmed to what matters:

# One jq pass over the JSON Claude Code pipes in on every render.
echo "$input" | jq -r '[
  (.model.display_name                       // ""),
  (.context_window.used_percentage           // "" | tostring),
  (.context_window.total_input_tokens        // "" | tostring),
  (.context_window.context_window_size       // "" | tostring),
  (.cost.total_cost_usd                      // "" | tostring),
  (.rate_limits.five_hour.used_percentage    // "" | tostring),
  (.rate_limits.seven_day.used_percentage    // "" | tostring),
  (.exceeds_200k_tokens                      // false | tostring),
  (.effort.level                             // "" | tostring)
] | join("\u001f")'

That \u001f is a unit separator, not a tab. Tab is IFS whitespace, so read would collapse the empty fields and shift every value one slot to the left. The rate-limit and usage fields are absent early in a session, so empty fields are the normal case, not the edge case. Learned that one the hard way.

What I watch, and why each segment earns its place

Every segment has to justify the pixels. Here is the full line and what each part answers.

Segment	Reads from	Why it's on screen
`Opus 4.8 (1M context)`	`model.display_name`	Which model and window I'm on, without guessing
`xhigh`	`effort.level`	High effort burns output tokens fast; seeing it nudges me to dial down for cheap tasks
`▓▓░░░ 34% 343k/1M`	`context_window.*`	How full the window is. Near the top means auto-compaction is close, and per-turn cost is climbing
`⚠>200K`	`exceeds_200k_tokens`	This request's input crossed 200K tokens (see below)
`⚡99%`	`context_window.current_usage.*`	Share of input served from cheap cache reads instead of full-price fresh input
`$18.20`	`cost.total_cost_usd`	Dollars this session
`5h:25%↻2h33m 7d:10%↻3d13h`	`rate_limits.*`	Rolling rate-limit usage and time until each window resets

The meter is a five-slot bar that goes green under 50%, yellow through 79%, red at 80% and up. The cache indicator is computed, not read: cache_read / (input + cache_creation + cache_read), colored the same way. A low number there means I'm paying full input price every turn, which is real money on a long session.

In a narrow tmux split the line wraps to two rows and starts shedding segments in a fixed order: effort, then limits, then cache, then the token fraction, then cost. The ⚠>200K warning is never dropped. It's the one thing I never want to lose sight of, for reasons in the next section.

The two costs that surprise people

There are two different cost mechanisms in play, and the line treats them as separate things because they are.

The first is the long-context tier. The ⚠>200K flag comes straight from the exceeds_200k_tokens boolean, and it lights up when a single request's input crosses 200K tokens. On the models that carry a long-context pricing tier, that's the point where input past 200K starts billing at a higher per-token rate. The honest caveat: at the time of writing, the current Opus (Opus 4.8, 1M context) bills the whole window at standard rates with no long-context premium. So treat the flag as "you're in long-context territory, go check what your model charges here," not a guaranteed price jump. Anthropic's pricing is the source of truth, and these tiers move.

The second is plan-level "extra usage," and it's what the 5h/7d segments are really about. Those track rolling usage against the plan's rate limits. On Team plans with spillover enabled, usage past the included allotment bills at API rates, roughly 4–5× the effective included rate at the time of writing. The status line can't show a dollar figure for it, because there is no field for it in the JSON. The percentages and reset countdowns are the proxy: 5h:25%↻2h33m means I'm a quarter into the five-hour window with 2h33m until it resets.

Hot Take
Cost is the number everyone watches and it's the wrong one to watch alone. Crossing the 200K threshold and burning down a rate-limit window are leading indicators; the dollar figure is the lagging one that arrives after the decision that caused it. The reset countdown turned rate limits from an invisible ceiling into something I can pace against.

Reading the line at a glance

Here it is in the wild: a minimal WezTerm window, tmux doing the multiplexing, and the footer carrying the whole story.

Bottom row, left to right: identity and branch, then Opus 4.8 (1M context), xhigh, the context meter at 34% 343k/1M, ⚠>200K lit (343k is well past the threshold), ⚡99% cache hits, $18.20 spent, and the two reset clocks. One glance and I know where the session stands before I fire off the next expensive turn.

Why I kept my own script

Before committing to this, I compared it against ccstatusline, the popular off-the-shelf option. The deciding factor was where the data lives.

The statusLine JSON hands you almost everything worth showing for free: model, effort, context, cost, cache, rate limits. The one thing it has no field for is an extra-usage or overage dollar figure — that lives behind a separate Usage API, which is exactly what ccstatusline calls to fill the gap. I didn't want the dollar figure badly enough to take on a second data source and its auth, so I kept the script and just added the fields it was missing.

rtk, the token saver that briefly lied to me

Separate from the status line, but part of the same "spend the context window on thinking, not scrollback" goal, I run rtk (Rust Token Killer). It's an output-compression proxy wired in as a Claude Code PreToolUse Bash hook (rtk hook claude). It squeezes the command output that flows back into the model's context — long git log, directory listings, build dumps — so scrollback doesn't eat the window.

It works. It also bit me this session. rtk was aggressive enough to collapse my own diagnostic output: a wc -l came back 0, an ls -la came back (empty), and for a minute I was convinced real commands were failing. They weren't. The hook had compressed the output to nothing before I ever saw it. The fix was to redirect to a file and read the file directly, bypassing the hook entirely.

You can catch it happening in the screenshot above, where a cd dotfiles reads Bash completed with no output. A token-saver that hides real output is a trade-off, and I'd rather say that plainly than pretend it's free.

How the pieces fit

Claude Code renders, pipes the JSON, the script parses it once and prints, tmux shows it. No daemon, no polling, no state. It recomputes from scratch on every render, so what you see is always the current session, with no cached numbers to go stale.

What it changed

The line changed how I work more than I expected. I match effort to the task now instead of leaving xhigh on for a one-line edit. I keep the prompt prefix stable so the cache-hit rate stays high. I pace long runs against the reset clock instead of running into a wall I couldn't see. None of that is new information — it was all available before, in a dashboard, after the fact. Putting it under the cursor is what made me act on it.

The script and its install live in my dotfiles, symlinked and set up by the bootstrap only when the claude CLI is present.

Chopper is the Straw Hats' doctor. He doesn't wait for a crewmate to drop mid-fight; he watches their vitals and calls it early. That's what this footer is — a vitals monitor for the session, taking its pulse on every render. And when the window gets tight there's a bit of Sanji in it too: rationing a limited pantry so nothing goes to waste, which is the entire job of rtk and, fittingly, the name over the door of this blog.

Chopper only became that doctor because Dr. Hiriluk, the quack everyone laughed at, believed in him first.

Intro and closing images are from the One Piece anime via the One Piece Wiki: the intro scene of Luffy inviting Chopper to join and the closing scene of Dr. Hiriluk and Chopper. Attribution and credits — please contact if anything needs updating.

How Python's rglob Silently Loses Files, and Why macOS Makes It Worse

Prateek Rai — Sat, 04 Jul 2026 12:31:26 +0000

A Python service I work on did something that shouldn't be possible. It wrote a directory full of files, turned around to upload them, and found nothing. The files were right there — ls saw them from another terminal.

The upload logged success. Zero files. No exception. Digging into why took me through three separate layers of the stack — none of which would tell me what actually happened — and a paper trail going back to 2015 that almost nobody has read.

The record of every one of these problems has been sitting in public bug trackers and mailing lists for years — some of it for two decades. This post is the story of the bug, the archaeology of who reported what and when, and the fix we shipped while the deeper layers stay unfixed.

If this corner of the stack is new to you, here's the short version of each layer, in the order the post walks them:

Python's rglob hides errors. It walks a folder tree and returns every file it finds — and if the operating system refuses or fails anywhere along the walk, it silently skips instead of complaining. A folder it couldn't read looks exactly like a folder with nothing in it, and there is no way to find out which one you got.
The operating system never promised your listing was fresh. A listing that races concurrent writes is allowed to miss the newest files — on every system — and on macOS the filesystem adds behaviour Apple has never documented in either direction.
The obvious repair tool doesn't do what its name says. fsync, the call that's supposed to force pending writes all the way down to disk, only goes halfway on macOS — and it was never designed to freshen directory listings in the first place.

Each section below expands one layer, with receipts. When the jargon gets thick, look for the collapsible "in plain English" boxes — they're there so you can keep up without leaving the page. And if you're here because you have this bug right now, jump straight to the fix.

A Program That Succeeds at Doing Nothing

The service writes a batch of parquet files (the columnar data files pipelines pass around) into a directory, then hands that directory to an uploader. The uploader does what every Python tutorial would tell you to do:

discovered = [p for p in src.rglob("*") if p.is_file()]
results = [upload(p) for p in discovered]
reason = "uploaded" if discovered else "skipped:nothing_to_upload"

pathlib is the modern, blessed way to walk a directory. On a quiet machine this always works. In production — Linux containers on ARM, like almost everyone's production — it intermittently returned an empty iterator for a directory that had files in it. An os.listdir in the same process, microseconds later, saw them fine.

The insidious part is what happens next: nothing. The caller sees an empty list and proceeds. Logs say the upload completed. The next pipeline stage treats the empty result as a legitimate zero-row dataset. A program that crashes gets fixed the same day; a program that silently does nothing can lie to you for months.

So I did the obvious thing and tried to reproduce it locally — on an Apple Silicon MacBook, because that's what dev laptops are now. It reproduced. Then it kept reproducing in ways production had never shown: wider windows, weirder timings, listings that disagreed with each other from one call to the next. The repro environment was lying in more ways than the production one.

It turned out three independently-true facts were involved. The first fires on every platform and is the heart of the production bug. The second is allowed everywhere but gets dramatically worse on a Mac. The third is purely a macOS problem — and it matters because a Mac is where you'll be standing when you try to debug this. None of the three is a bug in the villain sense. All of them are documented — if you know where to dig.

Layer One — pathlib Swallows Errors, and Has Since 2015

rglob works by walking into a folder, asking the operating system what's inside, and repeating that for every subfolder it finds. At any step of that walk, the OS can say no: you don't have permission, the thing it was reading vanished a moment ago, a read transiently failed. In Python, those refusals arrive as an OSError exception.

Here's the design decision at the heart of this post: when that happens inside glob() or rglob(), pathlib catches the error and just keeps going. No exception, no warning, no hook to find out it happened. Whatever it was scanning at that moment quietly drops out of the results. So the output of rglob cannot distinguish "this directory is empty" from "this directory has a thousand files I wasn't allowed to see."

In plain English — what 'swallowing an error' looks like

Deep inside pathlib's directory walk sits the moral equivalent of this:

try:
    entries = os.scandir(directory)   # ask the OS what's inside
except OSError:
    pass                              # couldn't read it? pretend it was empty

If the read fails, pretend it succeeded and found nothing. There's no log, no flag, no exception for the caller to catch — a failed read and a genuinely empty folder produce the identical return value. That's the property this whole post hangs on: not just that errors are dropped, but that their absence is unobservable.

The paper trail on this is remarkable, and nobody has ever written it up as a story. It starts in May 2015, when a bug tracker user named Gregorio filed cpython#68308 — asking, reasonably, that rglob stop crashing on unreadable directories and skip them the way os.walk does. Core developer Serhiy Storchaka was the lone dissent, proposing a bash-failglob-style option to surface errors instead. That idea was deflected to "a different issue" which was never filed.

The suppression patch (by Ulrich Petri) sat unreviewed for months, until Guido van Rossum personally hit the bug in January 2016: "I'm just going to commit this." Two days later he noticed his own fix made results depend on directory scan order — "dependent on the ordering of the names returned by listdir(), which is just wrong" — re-patched it, and closed with a line that reads differently a decade later: "Should be fixed for real now."

It wasn't. In 2019, Thierry Parmentelat filed cpython#83075 with strace output (a tool that records every call a program makes to the OS) showing that a single symlink into an unreadable directory made glob() silently drop other, perfectly readable files. Matt Wozniski reduced it to a five-command repro, and Pablo Galindo root-caused it as a regression from a performance rewrite. Nobody had noticed — because the failure mode is silence, exactly as Guido's warning predicted.

The endgame is the telling part. In 2021, cpython#90208 asked for an opt-in flag to surface the errors. It was resolved in 2023 by pathlib's maintainer Barney Gale going the other way: suppressing all OSError below the top-level path. That shipped in Python 3.13, whose docs now state that errors are "suppressed in many cases, but not all." And in March 2026, Jakub Kuczys demonstrated that list(Path('/root').glob('*')) returns [] with zero indication anything went wrong — and filed cpython#146646 as a docs-only issue, writing: "Based on #68308 (and just the fact that it has stayed unchanged for years), I assume this is intentional." Eleven years in, the behaviour's age had become the argument for keeping it.

What makes this sting is that CPython knows how to do better — it just didn't do it here. os.walk has taken an onerror callback since forever. Path.walk, added in 3.12, takes on_error. POSIX find prints to stderr and exits nonzero when it can't read a directory. glob and rglob give you nothing. "Errors should never pass silently. Unless explicitly silenced" is in the Zen of Python; there is no way to explicitly un-silence these.

The most rigorous Python codebases already know. Black recurses with iterdir and handles errors itself. pytest's internal visit() helper walks with os.scandir. mypy's file cache catches OSError and re-raises it. Ruff walks the tree in Rust.

Trivia
Ruff has a lint rule, PTH207, that tells you to replace glob.glob with Path.glob because pathlib is more idiomatic. The migration it recommends moves you from an API that suppresses most errors to one that, since 3.13, suppresses all of them below the root. The linter whose own authors bypass pathlib for traversal nudges everyone else toward it.

Hot Take
Documenting a footgun is not fixing a footgun. The 3.13 docs note and the 2026 glob-module note upgraded a decade-old surprise from "undocumented behaviour" to "intended behaviour" without adding a single way to opt out. That's the cheapest possible resolution each time, and it's now load-bearing: any future fix would be a documented-behaviour break.

Now the honest question, because you should ask it: did suppression actually cause our production incident? I can't prove it — and that is precisely the indictment. If an error fired during those listings, rglob ate it. If the kernel simply returned a stale view, rglob passed it along. The return value is identical either way: []. The API doesn't just fail — it destroys the evidence of how it failed, which means the first question of any incident review ("what actually happened?") is unanswerable by design.

What I can offer is the one measurement we did get. While reproducing locally, rglob would come back empty while os.scandir — the lower-level directory-scanning call that pathlib itself is built on — returned the full set of files when called at essentially the same instant. Two readers of the same kernel state disagreeing is what turns pathlib from bystander into suspect. Though even here, honesty: the two calls are microseconds apart, and we could never fully separate "pathlib dropped entries" from "the view changed between the calls."

Switching to os.scandir closes the silence class entirely — errors propagate, so at minimum you learn which failure you have. But it can't conjure files the operating system didn't report. That's the next layer down.

Layer Two — The Filesystem Never Promised You a Fresh Listing

Here's the part I would have called a kernel bug, before I read the spec. Whenever any program lists a folder — ls, Finder, Python's os.listdir, pathlib's walk — it ultimately goes through one low-level operation: readdir, the operating system's "tell me what's in this directory" call. POSIX, the standard that defines how that call must behave on Unix-like systems, says something remarkable in the readdir specification: "If a file is removed from or added to the directory after the most recent call to opendir() or rewinddir(), whether a subsequent call to readdir() returns an entry for that file is unspecified."

Unspecified. A listing that races concurrent writes is allowed to miss files, by design, on every POSIX system. But read the clause carefully, because it's easy to apply too broadly — I did at first: it covers files added while the directory is being read. Our production pipeline was, as far as we could reconstruct, sequential — write, close, then list. POSIX has no carve-out for that. A filesystem that returns a stale view after a completed write-and-close isn't exercising the spec's freedom; something else is going on.

So here is the honest statement about production, on Linux: we never definitively identified the mechanism. The candidates: the container filesystem (production runs in containers, and overlay filesystems have a documented history of readdir caching quirks); writes that weren't as complete as the code believed; or an error that fired and was eaten — and Layer One explains why that last one can never be ruled out. The listing was wrong, the API was silent about why, and the crime scene was already clean by the time anyone looked.

The local repro on my Mac is where things got measurably stranger — and that's an APFS question. First, the mental model that makes any of this thinkable: creating a file is really two writes — the file's bytes, and the folder's record that the name exists — and nothing guarantees they become visible at the same instant. APFS is the copy-on-write filesystem on every modern Mac, and the honest answer to what it does to directory listings under pressure is that nobody outside Apple knows.

In plain English — a new file is two writes, not one

Creating a file changes two separate things: the file's contents (the bytes you wrote) and the folder's own records saying "a file with this name now exists here" — the metadata. They are not committed at the same instant. APFS batches its metadata updates for speed using a copy-on-write structure: it builds a new version of the folder's records off to the side and atomically swaps a pointer when it commits. Until that swap happens, both of these can be true at once: the file's bytes are fully written and you can open() it by name, and a directory listing still reads the old records and doesn't show it. "The file exists" and "the folder doesn't list it" are not contradictions — they're two different questions asked of two different data structures.

I want to be careful here, because this is the least-documented layer of the three. Apple's APFS Reference describes the copy-on-write B-tree design across 181 pages and does not mention readdir once. There is no published contract for when a freshly created file becomes visible to directory enumeration. What I saw while reproducing locally — files that stat (the "does this exact path exist?" check) and open could see by name while a concurrent recursive listing came back without them, under burst writes on an Apple Silicon laptop — is consistent with metadata commits lagging data writes, but I can't point you at an Apple document that says so, because there is no Apple document that says anything about it at all. The silence is the finding.

What is on the record is enough to stop giving APFS the benefit of the doubt — with precision about what it shows. In 2020, Giovanni Pizzi of the AiiDA project filed bpo-41291: a delete-versus-open race, where a file being replaced while another process opens it yields empty reads and st_ino == 0 (an inode number of zero — the file's on-disk identity, reading back as "nothing"). That is not a listing bug, and I'm not claiming it as one — but it is the closest thing on the public record to an Apple-acknowledged APFS race of any kind. CPython's macOS maintainer Ronald Oussoren reproduced it in pure C and bisected it by filesystem: fires on APFS, not on HFS+. His conclusion: "This appears to be a bug in the APFS filesystem implementation." Reported to Apple as FB8009608. No public resolution.

Two years earlier, Gregory Szorc had profiled Firefox builds on macOS and found readdir on APFS serializing on a global kernel lock — the machine spending ~75% of its CPU time in the kernel just listing directories. A performance finding, not a correctness one — but the same neglected code path, and the same ending: a radar filed, a partial improvement shipped without comment, no acknowledgement.

One precision note, because it matters for anyone debugging their own version of this: several distinct mechanisms produce the "listing lies" symptom, and they are not the same bug. A writer in another process racing your listing is POSIX-unspecified territory. Files changing while you're mid-walk is a TOCTOU problem — time-of-check to time-of-use, meaning the world changed between when you looked and when you acted. bpo-41291 is an apparent APFS defect. iCloud and Dropbox "dataless" files break globbing in their own special way. And a file you wrote but never flushed is a you problem. The fix depends on which one you have — and this is where I have to practice what I'm preaching about evidence: production, on Linux, remains formally unsolved; the mechanism candidates above are indistinguishable after the fact. The local Mac repro showed listing inconsistency we could measure but never fully attribute. When your listing API is silent, every one of these mechanisms produces the identical observable: [].

So the instinctive fix is: force the pending metadata down before listing — sync the directory. Hold that instinct, because it's about to get complicated twice over: the syncing tool doesn't mean what its name says on a Mac, and it was never a listing-freshness tool anywhere.

Layer Three — fsync Is a Polite Request on macOS

On Linux, fsync(fd) — the fd is a file descriptor, the numeric handle the OS gives you for anything you've opened, folders included — means: this data is on durable storage before the call returns. On macOS, fsync(fd) means: this data has been handed to the drive, which may keep it in a volatile cache and write it whenever it feels like it. Apple's own fsync man page has said so for over twenty years, and it's blunt about the consequences: "This is not a theoretical edge case."

In plain English — the three stops between your program and the disk

When you write a file, the bytes make three stops: your operating system's memory (fast, lost if the OS crashes), the drive's own internal cache (fast, lost if power cuts out), and finally the durable flash storage (survives anything short of hardware failure). fsync is the call that's supposed to mean "push my data all the way down, then tell me." On Linux it pushes through all three stops. On macOS it stops at the drive's cache — so data that fsync just claimed was safe can still be eaten by a power cut. The macOS call that pushes all the way is fcntl(fd, F_FULLFSYNC), and it's dramatically slower — which is exactly why it isn't the default, and exactly why almost nothing uses it.

The picture is worth having in your head — three stops, and the two systems disagree about which stop "safe" means:

fsync on Linux pushes your data to stop 3. fsync on macOS stops at 2 and reports success anyway. The real flush is a separate call, fcntl(fd, F_FULLFSYNC), which pushes to stop 3 and has existed since Mac OS X 10.3 in 2003. The canonical explanation came from Apple filesystem engineer Dominic Giampaolo on the PostgreSQL mailing list in 2005 — he'd verified drive-cache behaviour with a logic analyzer on the ATA cable, and described the full flush as "a heavy handed operation... But in an app like a database, it is essential." The database world quietly listened: SQLite added PRAGMA fullfsync in 2004, PostgreSQL grew fsync_writethrough. Everyone who truly needed durability opted in twenty years ago, which is precisely why there was never enough pressure to change the default.

The rest of us found out in February 2022, when Asahi Linux's Hector Martin measured an M1's SSD doing ~46 write operations per second with real flushes versus ~40,000 without and called it benchmark cheating. Michael Tsai's roundup of the ensuing fight is the fairest record — including the counter-voices: most non-Apple consumer drives simply lie about flushes, and when Russ Bishop power-loss-tested four NVMe SSDs, half of them lost data they had claimed to flush. Alex Miller's "Darwin's Deceptive Durability" distilled it for database people into three deadpan headings: fsync does not fsync, O_DSYNC does not O_DSYNC, SQLite is not durable.

This genre has a famous Linux chapter too — fsyncgate, 2018: PostgreSQL's Craig Ringer discovered that when Linux fsync fails, the kernel silently discards the data and the next call reports success. His report is dry understatement at its finest — "The write never made it to disk, but we completed the checkpoint, and merrily carried on our way. Whoops, data loss." Jonathan Corbet's LWN writeup and Dan Luu's verbatim archive tell it properly, and a later USENIX study found no major application handles fsync failure correctly even now. The takeaway for this post: durability primitives whose names promise more than they deliver are a genre, not a macOS quirk.

Now the caveat this section has been building to — and it cuts against our own fix, so it stays in. fsync is a durability tool: it controls what survives a power cut. It is not, on any system I can find documentation for, a visibility tool — nothing in POSIX or Apple's docs says that syncing a directory's file descriptor refreshes what the next readdir returns on a running machine. On Linux it has no defined effect on our problem at all. On macOS, the theory that F_FULLFSYNC forces APFS to commit the metadata transaction that feeds enumeration is exactly that — a theory, resting on the same undocumented behaviour as Layer Two. We put the barrier in our fix anyway: it costs 5–20ms on NVMe, and if the theory is right, it tightens the window. But I won't pretend we proved it does anything. Keep that skepticism handy for the next section.

How Three Half-Truths Compound

Any one of these alone is trivia. Stacked, they turn the friendliest file API in Python into silent data loss. Read the diagram top to bottom — this is the order it happens in at runtime:

| Layer | What it's supposed to guarantee | What it actually guarantees |
|---|---|---|
| Path.rglob | Returns every file in the tree | Returns whatever readdir delivered, silently dropping anything that errored — and never telling you whether anything did |
| readdir | The current contents of the directory | Under concurrent writes: unspecified by POSIX. After sequential writes: should be current — but APFS's behaviour is undocumented and container filesystems have their own quirks |
| fsync(dir_fd) | Pending metadata committed to durable storage | Kernel buffers only on macOS (F_FULLFSYNC for the drive) — and on no system is it a documented listing-freshness barrier |

In production on Linux we can name the symptom but not the mechanism — the listing was wrong and the API was silent about why. On a Mac dev box, two more undocumented layers join in, which is why the bug was easier to reproduce at my desk than it ever was to catch in the logs. The window is tens of milliseconds. The probability per call is tiny. The probability across a multi-hour pipeline run writing thousands of files is not — and every miss looks exactly like a legitimately empty directory.

The reframe that finally made the fix obvious: a directory listing is not a query. It's a measurement. It has noise, it has a sampling window, and if you need it to be right, you have to engineer for that the way you would for any other measurement.

What We Shipped

First: do you even need this? Only if your code lists a directory in the same breath as writing into it — write-then-immediately-list, or listing while writers may still be active. A static folder read at startup is fine with plain rglob. If that's you, close this tab guilt-free.

For everyone else, four moves, each honest about what it does and doesn't buy:

Replace rglob with an os.scandir recursion that raises on error. The one provable fix: silence becomes a loud failure, so you always know which kind of empty you got.
Retry briefly with backoff. The unglamorous move that, in truth, probably does most of the work.
Nudge the directory metadata before each attempt — F_FULLFSYNC on macOS, os.fsync elsewhere. This is the theory-not-proof part flagged above: cheap, harmless, unproven.
A tripwire and a loud log. Two same-instant top-level listings that disagree mean the directory is visibly in flux; if it never settles, operators hear about it instead of nobody.

The block below is complete — imports, exception, logging — paste it and it runs:

import fcntl
import logging
import os
import sys
import time
from pathlib import Path

logger = logging.getLogger(__name__)

# In the stdlib on macOS builds of Python; the raw value elsewhere is inert.
F_FULLFSYNC = getattr(fcntl, "F_FULLFSYNC", 51)


class StorageListingError(Exception):
    """A directory listing failed — surfaced instead of swallowed."""


def _flush_directory_metadata(path: Path) -> None:
    """Nudge the kernel to commit pending directory metadata.

    fsync is a durability primitive; treating it as a listing-freshness
    barrier is a theory, not a documented contract. Kept because it is
    cheap and cannot make things worse."""
    fd = os.open(path, os.O_DIRECTORY)
    try:
        if sys.platform == "darwin":
            try:
                fcntl.fcntl(fd, F_FULLFSYNC)
            except OSError:
                os.fsync(fd)  # e.g. network/virtual filesystems
        else:
            os.fsync(fd)
    finally:
        os.close(fd)


def _scandir_recursive(path: Path) -> list[Path]:
    """Like Path.rglob('*') for files, except errors raise instead of
    silently shrinking the result."""
    out: list[Path] = []
    try:
        with os.scandir(path) as it:
            for entry in it:
                if entry.is_dir(follow_symlinks=False):
                    out.extend(_scandir_recursive(Path(entry.path)))
                elif entry.is_file(follow_symlinks=False):
                    out.append(Path(entry.path))
    except OSError as exc:
        raise StorageListingError(f"could not list {path}") from exc
    return out


def safe_list_directory(path: Path, max_attempts: int = 3) -> list[Path]:
    """Recursive file listing, hardened for directories written moments ago."""
    backoffs = (0.0, 0.05, 0.10)
    last: list[Path] = []
    for attempt in range(max_attempts):
        if attempt:
            time.sleep(backoffs[min(attempt, len(backoffs) - 1)])
        _flush_directory_metadata(path)
        listing = _scandir_recursive(path)
        # Tripwire, not proof: two top-level listings taken back-to-back
        # that DISAGREE mean the directory is visibly in flux. Agreement
        # does not prove the recursive listing is complete.
        if set(os.listdir(path)) == {p.name for p in path.iterdir()}:
            return listing
        last = listing
    logger.warning(
        "directory listing never converged: path=%s attempts=%d",
        path, max_attempts,
    )
    return last

It drops in at every rglob call site with one deliberate behavioural change: where rglob silently returned partial results, this raises. That is the point — if your caller can't handle a StorageListingError, it was already mishandling the invisible version of the same event.

Which parts can I defend, and how hard? The scandir-and-raise part: fully — it provably closes the silence class. The retry: empirically — it's the standard answer to transient inconsistency, whatever the mechanism. The flush: it's the belt-and-braces theory flagged in Layer Three; we never ran the ablation that would separate its contribution from the retry's, and I'd genuinely love to see someone do it. The tripwire: it detects churn between two instants, not stable staleness — a steadily wrong view sails through it, which is why the warning log exists. The whole thing is probabilistic, not deterministic. We tightened the window; we did not close it. The Darwin branch exists because dev machines live on Macs even though production is Linux; the cost on a Mac is 5–20ms of F_FULLFSYNC per attempt, far cheaper on Linux, and negligible against upload latencies either way.

The Fixes That Don't Trust Listings

Closing the window for real means restructuring so the race can't exist, and every mature system that has faced this converged on one of three shapes — none of which trust a directory listing:

Sentinel markers. The writer touches a marker file last, after everything is flushed; readers refuse to list until the marker exists. Hadoop's _SUCCESS file, carried forward by Spark, Hive, and Iceberg.
Atomic rename. Write into a staging directory, flush, then rename it into place — POSIX guarantees observers see the old state or the new state, never a partial one. Maildir has done this since 1995; Git's index.lock is the same idea.
Kernel events instead of listings. Subscribe to FSEvents/inotify before writing and build the file set from events. Watchman, Buck2, and Mercurial's fsmonitor live here.

There's an optimistic precedent for the "or the platform just fixes it" ending, too. S3's LIST calls were eventually consistent for fourteen years, and it bit production hard enough that Netflix built s3mper — Daniel Weeks's 2014 writeup could be this post's abstract: "there is no indication that a problem occurred... if only a small portion of input is missing the results will appear convincing." Hadoop built S3Guard for the same reason. Then in December 2020, AWS made S3 strongly consistent for everyone, retroactively, for free — Werner Vogels's engineering post-mortem notes the formal verification took more work than the implementation. The S3Guard docs now just say: delete the DynamoDB table.

Sit with the irony for a second: an object store on the other side of the planet now gives you stronger listing guarantees than the filesystem on the laptop you're reading this on.

The People Who Dug First

Almost nothing in this post is my discovery. The record was public the whole time — it just took a production incident to make me read it. Credit where it belongs:

The pathlib thread: Gregorio (the 2015 report), Serhiy Storchaka (the road not taken), Ulrich Petri (the patch), Guido van Rossum (the warning that came true), Thierry Parmentelat and Matt Wozniski (the strace proof), Pablo Galindo (the regression fix), Barney Gale (3.13's resolution), and Jakub Kuczys (the 2026 issue that finally got it documented).
The Darwin durability thread: Dominic Giampaolo (the 2005 explanation that still holds), Hector Martin (the numbers), Michael Tsai (the fairest record of the fight), Alex Miller (the essay to read first), and Howard Oakley (the calm explainer).
The fsyncgate thread: Craig Ringer (the discovery), Jonathan Corbet (the no-villains account), Dan Luu (the archive, and the whole "files are hard" genre), Thomas Munro and Andres Freund (the fix), and Ted Ts'o (the kernel's side of the story, which deserves to be heard).
The APFS thread: Giovanni Pizzi and Ronald Oussoren (bpo-41291 — the one Apple-acknowledged APFS race, a delete-vs-open one rather than a listing one), and Gregory Szorc (the readdir profiling). This thread is the shortest because almost nobody has dug here. That should probably change.

If you take one action from this post, make it structural: treat directory listings as measurements, surface your errors, and when correctness matters, publish atomically or use a marker. The standard library teaches you to trust the listing. Production says don't.

The scholars of Ohara, with the Buster Call bearing down on them, threw their books into the lake so the knowledge would outlive the island. Twenty years later it was all still there, waiting for someone willing to dive.

The knowledge about this bug was never hidden either. It's been sitting in open issue trackers, mailing list archives, and man pages since before some of the people hitting it learned Python — eleven years of it in one CPython thread alone. Nobody burned the library. We just stopped reading the poneglyphs.

Intro and closing images are from the One Piece anime via the One Piece Wiki. Attribution and credits — please contact if anything needs updating.

Your Blog Deserves a Second Life: Syndicating to dev.to with Canonical URLs

Prateek Rai — Sun, 28 Jun 2026 10:54:50 +0000

I spent months writing posts for this blog. Each one went live on the site, got maybe three readers (one of them me checking the RSS feed), and sat there in peace.

Then I started copy-pasting the same content to dev.to manually — fixing broken image paths, stripping out MkDocs-specific markdown, re-pasting. Every. Single. Post.

The third time I did it, I wrote a Python script instead. That script became a CI pipeline. That pipeline became the subject of this post.

The Canonical URL Problem

Here is the thing that kept me up at night: dev.to has a domain authority of 89. Your personal blog has a domain authority of about 4 (if you are lucky). If you post the same content on both without telling Google which one is original, guess which one ranks first?

Hint: not yours.

This is where canonical_url comes in. It is an HTML <link> tag that tells search engines: "This article originally lives at this URL. Index that one, not me." dev.to supports it natively — if you include canonical_url in your API payload, every dev.to article page renders a <link rel="canonical" href="..."> in the head. Google sees it and credits the original source.

Without canonical URLs, dev.to syndication actively harms your SEO. With them, it becomes a pure distribution channel — your site keeps the ranking, dev.to gets the traffic.

Hot Take
If you syndicate to dev.to without setting a canonical URL, you are actively cannibalizing your own SEO. dev.to's domain authority means Google will rank their copy above yours every time. Canonical URLs are not a nice-to-have — they are the difference between syndication helping you and hurting you.

The canonical URL for this very post? https://prateek11rai.github.io/sanji/blog/2026/06/28/devto-syndication/. The script builds it from the front matter date and the slug.

The dev.to API

The dev.to API is refreshingly simple. No OAuth flow, no Bearer tokens, no scopes to negotiate. You generate an API key from dev.to → Settings → Account → API Keys and send it as a header.

Authentication

api-key: YOUR_DEVTO_API_KEY
Content-Type: application/json

That is it. One header. No Authorization: Bearer, no refresh tokens.

The Payload

The entire payload nests under a single article key:

{
  "article": {
    "title": "Your Blog Deserves a Second Life",
    "body_markdown": "# The full markdown content...",
    "published": true,
    "canonical_url": "https://prateek11rai.github.io/sanji/blog/2026/06/28/devto-syndication/",
    "description": "Short excerpt shown in previews",
    "tags": ["tooling", "python", "meta"],
    "main_image": "https://raw.githubusercontent.com/prateek11rai/sanji/main/docs/assets/images/blog/devto-syndication/devto-intro.png"
  }
}

Endpoints

Endpoint	Method	Purpose
`/api/articles`	POST	Create new article
`/api/articles/{id}`	PUT	Update existing article
`/api/articles/me/all?per_page=100`	GET	List all articles (including drafts)
`/api/articles/me/published?per_page=100`	GET	List published articles only

Rate Limits

10 POST requests per 30 seconds
30 PUT requests per 30 seconds

We hit the POST limit during the first bulk syndication. The API does not return rate-limit headers — you only learn you hit the wall when a 429 comes back. The fix is a retry with backoff, which I added after the first batch failed halfway through.

Trivia
The dev.to API has no DELETE /api/articles/{id} endpoint. If you need to remove a syndicated article, you must delete it manually through the web dashboard. This is surprisingly common among community-focused platforms — they want human oversight on deletions.

Tag Restrictions

dev.to tags must be:

Purely alphanumeric (no hyphens, no spaces, no special characters)
Max 4 tags per article

This means self-hosting (MkDocs category) becomes selfhosting (dev.to tag). Our script lowercases categories and strips hyphens automatically.

Draft Quirk

Setting published: false creates an unpublished article — but GET /api/articles/{id} returns a 404 for drafts. You must use /me/all to find them. This caught us during development when we tried to verify a draft by fetching it back.

The Transformation Pipeline

MkDocs Material produces beautiful markdown — for MkDocs Material. dev.to uses its own renderer (based on Redcarpet/github-linguist), and the two speak completely different dialects.

Here is every transform our syndicate.py script applies, in order:

Strip `` — MkDocs excerpt marker, irrelevant on dev.to
Strip <style> blocks — Material inline styles that break dev.to rendering
Strip `` grid cards — Material's HTML grid markup is stripped by dev.to anyway
Render Mermaid diagrams as images — `mermaid blocks become ![Architecture diagram](https://mermaid.ink/img/pako:{compressed})
Strip `` — MkDocs image attribute, dev.to does not support it
Strip attr_list attributes — `` annotations from the attr_list plugin
Replace Material icons — ✅ becomes ✅, unknown icons are removed
Convert admonitions to blockquotes — !!! note "Title"\n content becomes > **Title**\n> content
Rewrite image URLs — ../../assets/images/foo.jpg → https://raw.githubusercontent.com/...
Unwrap footnote definitions — : text moved to bottom under a --- separator
Strip footnote references — `` inline refs removed
Strip H1 title — The # Title is removed (dev.to uses its own title field from the payload)

The most complex transform is the Mermaid rendering, which deserves its own section.

Mermaid Rendering

MkDocs Material renders Mermaid diagrams natively in the browser using JavaScript. dev.to does not. The solution: pre-render every Mermaid block as an image using the free mermaid.ink service.

The trick is in how mermaid.ink accepts diagrams. It expects a URL-safe, zlib-compressed, base64-encoded JSON payload in the path. The payload structure:

{
  "code": "graph TB\n    A[Post] --> B(syndicate.py)",
  "mermaid": "{\"theme\":\"neutral\"}",
  "updateEditor": false,
  "autoSync": false,
  "updateDiagram": true
}

Our script compresses this, encodes it, and produces a URL:

def _render_mermaid(body: str) -> str:
    def _repl(m: re.Match) -> str:
        code = m.group(1).strip()
        payload = json.dumps({
            "code": code,
            "mermaid": json.dumps({"theme": "neutral"}),
            "updateEditor": False,
            "autoSync": False,
            "updateDiagram": True,
        }, separators=(",", ":"))
        compressed = zlib.compress(payload.encode(), level=9)
        encoded = base64.urlsafe_b64encode(compressed).decode().rstrip("=")
        url = f"https://mermaid.ink/img/pako:{encoded}"
        return f"![Architecture diagram]({url})\n"
    return re.sub(
        r"```

mermaid\n(.*?)

```\s*\n*", _repl, body, flags=re.DOTALL,
    )

The result is an inline image that renders the diagram exactly as it would on the site. The level=9 compression is crucial — Mermaid payloads can be large, and URLs have length limits.

Here is the full flow visualized:

The CI Workflow

The workflow lives in .github/workflows/syndicate.yml and runs on every push to main. It detects which blog posts changed, runs the syndication script on each one, and publishes (or updates) them on dev.to.

Key excerpt:

name: Syndicate blog to dev.to
on:
  push:
    branches: [main]

jobs:
  syndicate:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
        with:
          fetch-depth: 0

      - name: Detect changed blog posts
        id: changed
        run: |
          BEFORE="${{ github.event.before }}"
          if [ "$BEFORE" = "0000000000000000000000000000000000000000" ]; then
            FILES=$(git diff-tree --no-commit-id -r "${{ github.sha }}" \
              --name-only -- 'docs/blog/posts/*.md')
          else
            FILES=$(git diff --name-only "$BEFORE" "${{ github.sha }}" \
              -- 'docs/blog/posts/*.md')
          fi
          FILES=$(echo "$FILES" | grep -v 'sanji\.md$' || true)
          echo "changed<<DELIM" >> $GITHUB_OUTPUT
          echo "$FILES" >> $GITHUB_OUTPUT
          echo "DELIM" >> $GITHUB_OUTPUT

      - name: Syndicate to dev.to
        if: steps.changed.outputs.changed != ''
        run: |
          echo "$CHANGED_FILES" | while IFS= read -r file; do
            [ -z "$file" ] && continue
            uv run python scripts/syndicate.py "$file" --publish
          done
        env:
          CHANGED_FILES: ${{ steps.changed.outputs.changed }}
          DEVTO_API_KEY: ${{ secrets.DEVTO_API_KEY }}

A few design decisions worth noting:

fetch-depth: 0 is required because the workflow needs the full git history for git diff to compare against the previous commit. A shallow clone would fail here.
The zero-SHA check handles the first push to a branch, where github.event.before is all zeros. In that case, git diff-tree lists every file in the initial commit.
sanji.md is filtered out — that is the about page, not a blog post.
uv run is used instead of raw python because the project uses uv for Python package management. Much faster than pip, lockfile for reproducibility.

The Canonical URL Builder

The most important function in the script. Every post gets a deterministic URL based on its front matter date and title:

def build_payload(post_file: Path, slug: str | None = None, publish: bool = False) -> dict:
    content = post_file.read_text(encoding="utf-8")
    fm, body = parse_front_matter(content)
    title = extract_title(body) or slug or post_file.stem.replace("-", " ").title()
    description = extract_description(body)
    slug = slug or make_slug(title)

    tags = [t.lower().replace("-", "") for t in fm.get("categories", [])][:MAX_TAGS]
    if not tags:
        tags = ["meta"]

    date_str = fm.get("date", "")
    date_parts = date_str.split("-")
    if len(date_parts) == 3:
        canonical_url = f"{SITE_BASE}/blog/{date_parts[0]}/{date_parts[1]}/{date_parts[2]}/{slug}/"
    else:
        canonical_url = f"{SITE_BASE}/blog/{slug}/"

    # ... payload assembly continues ...

The slug is generated using pymdownx.slugs.uslugify(title, "-") — the same function MkDocs Material uses internally. This ensures the dev.to canonical URL matches the exact URL the blog generates at build time.

Idempotency & Dedup

The worst thing a syndication script can do is create duplicates. Our script prevents this by checking the existing articles before creating:

def find_existing(api_key: str, canonical_url: str) -> int | None:
    headers = {"api-key": api_key}
    for url in [
        f"{DEVTO_API}/me/all?per_page=100",
        f"{DEVTO_API}/me/published?per_page=100",
    ]:
        resp = requests.get(url, headers=headers)
        if resp.status_code != 200:
            continue
        for article in resp.json():
            if article.get("canonical_url") == canonical_url:
                return article["id"]
    return None

If an article with matching canonical_url exists, it calls PUT instead of POST. The same post can be re-syndicated a hundred times — it creates one article, updated in place.

The script also preserves the existing published state on updates unless --publish is explicitly passed. This means a draft post that gets edited stays a draft on dev.to.

Why This Matters

Syndication infrastructure is invisible when it works and very visible when it does not. The script runs on every push, transforms every block, and posts to dev.to without anyone thinking about it. That is the point.

The Going Merry was not the biggest or strongest ship. It was a caravel — an obsolete design by Grand Line standards. But it carried the Straw Hats through East Blue, across Reverse Mountain, into the sky, and all the way to Enies Lobby. It was the sending-ship that connected their private world to the public one.

This syndication pipeline is the same thing. A small, purpose-built tool that carries each post from my personal dock to the public port. The canonical URL is the flag in the mast — everyone knows where it came from.

Image credit and attribution — One Piece artwork is the property of Eiichiro Oda and Shueisha. Please contact if anything needs updating.

Crafting Your Terminal Like a Chef Sharpens His Knives

Prateek Rai — Sat, 27 Jun 2026 19:27:21 +0000

I used the default macOS Terminal for years and thought that was just how terminals look.

Then I saw someone else's — proper font, syntax highlighting, a prompt that actually tells you something — and realized mine looked like a hospital monitor from the 90s.

Why Not the Defaults

macOS ships with Terminal.app and bash (well, zsh now, but the point stands). They are designed to be adequate, not good. Here is what I wanted instead:

Default	Why It Falls Short
Terminal.app	No GPU acceleration, no Lua config, no cross-platform
bash	No autosuggestions, weaker completion than zsh
Bare prompt	No git status, no Python version, no command timing
No multiplexer	Lose your session when the window closes
Default font	Menlo at 12px — designed for legibility, not for staring at 8 hours a day

I wanted a terminal that felt like my space — configured exactly how I like it, persistent across sessions, and beautiful enough that I do not mind spending hours in it. The result is a stack of four tools that each do one thing well: WezTerm for rendering, Tmux for session management, Zsh for the shell, and Starship for the prompt. All tied together with a single dotfiles repo and a bootstrap script that sets up a new Mac in one command.

Hot Take
iTerm2 is the default recommendation for anyone switching from Terminal.app, and I think it is overrated. It is heavy, its tab management has always felt bolted on, and its Python scripting API is powerful but unnecessary for 95% of users. WezTerm does everything iTerm2 does but with a config file you can version-control and a GPU-accelerated renderer that actually matters on high-DPI displays.

WezTerm — The Terminal Itself

WezTerm is a GPU-accelerated terminal emulator configured entirely through Lua. No GUI settings panel, no hidden preferences — just a wezterm.lua file you check into git.

The Config

The main file lives at ~/.config/wezterm/wezterm.lua, split into modules loaded with require():

~/.config/wezterm/
├── wezterm.lua           # Main config
├── startup/
│   └── init.lua          # Auto-attach tmux + maximize on launch
└── keybindings/
    └── init.lua          # Custom keybindings (C-S-f to fit screen)

Key settings:

Setting	Value	Why
`enable_tab_bar = false`	Off	Tmux handles tabs — WezTerm just renders
`window_decorations = "RESIZE"`	Minimal	No title bar, no close/minimize buttons
`font = JetBrains Mono Bold`	Bold	Clean, developer-friendly, bold for readability
`font_size = 15`	15px	Comfortable without wasting space
`color_scheme = "Dracula (Official)"`	Dracula	Consistent theme across everything
`window_background_opacity = 0.99`	Near-solid	Slight transparency for a modern feel

The Startup Module

This is the glue that connects WezTerm to Tmux:

-- ~/.config/wezterm/startup/init.lua
return {
    wezterm.on("gui-startup", function()
        local tmux_path = "/opt/homebrew/bin/tmux"
        wezterm.mux.spawn_window({
            args = { tmux_path, "new-session", "-A" }
        })
    end)
}

When WezTerm launches, it runs tmux new-session -A — attach to the most recent Tmux session if one exists, or create a new one. The full path to tmux is required because WezTerm's spawn process does not inherit the user shell PATH.

The Keybindings Module

-- ~/.config/wezterm/keybindings/init.lua
return {
    keys = {
        {
            key = "F",
            mods = "CTRL|SHIFT",
            action = wezterm.action({ FitToDisplay = {} }),
        },
    }
}

One binding: Ctrl+Shift+F fits the window to the active display. This exists because disconnecting an external monitor leaves the terminal window sized for a larger screen. One keystroke fixes it.

Tmux — The Session Manager

WezTerm renders the pixels. Tmux keeps your work alive. The two together are better than either alone.

Config at ~/.config/tmux/tmux.conf:

set -g prefix C-a                # Changed from C-b — easier reach
set -g mouse on                   # Click panes, scroll, resize with mouse
bind r source-file ~/.config/tmux/tmux.conf  # Quick reload
bind i run ~/.config/tmux/plugins/tpm/bin/install_plugins

Plugins

Managed by TPM (Tmux Plugin Manager):

Plugin	Purpose
`tmux-plugins/tpm`	Plugin manager itself
`tmux-plugins/tmux-sensible`	Saner defaults (better escape time, UTF-8, improved pane behavior)
`tmux-plugins/tmux-yank`	Copy to system clipboard from inside tmux

Why Tmux Over WezTerm Tabs

WezTerm can do tabs natively. I chose Tmux because:

Persistence — close WezTerm, reopen it, and you are back in the same Tmux session. Everything is where you left it.
SSH — Tmux works over SSH exactly as it works locally. Detach from a remote session, reconnect later from a different machine.
Muscle memory — Tmux has been doing this since 2007. Every terminal multiplexer since then has been catching up.

Zsh — The Shell

The shell config at ~/.zshrc sets up the environment that WezTerm and Tmux run inside:

# Prompt
eval "$(starship init zsh)"

# Python
export PYENV_ROOT="$HOME/.pyenv"
command -v pyenv >/dev/null && eval "$(pyenv init -)"

# Editor
export EDITOR="code --wait"

# Autosuggestions
source "$(brew --prefix)/share/zsh-autosuggestions/zsh-autosuggestions.zsh"

# Java
export JAVA_HOME="/Library/Java/JavaVirtualMachines/jdk-22.jdk/Contents/Home"

# Corporate SSL (Netskope — only on work machines)
if [ -f "/Library/Application Support/Netskope/STAgent/stagent.pem" ]; then
    export AWS_CA_BUNDLE="/Library/Application Support/Netskope/STAgent/stagent.pem"
    export CURL_CA_BUNDLE="/Library/Application Support/Netskope/STAgent/stagent.pem"
    export SSL_CERT_FILE="/Library/Application Support/Netskope/STAgent/stagent.pem"
    export NODE_EXTRA_CA_CERTS="/Library/Application Support/Netskope/STAgent/stagent.pem"
fi

# Aliases
alias vc="vcluster platform connect vcluster"
alias ap="argopm install . -n default -f -c ."

The Netskope block is only active on corporate machines where the cert file exists. On personal machines, those env vars are never set, and nothing breaks.

Starship — The Prompt

Starship is a cross-shell prompt written in Rust. It is fast, minimal, and shows contextual information without clutter.

# ~/.config/starship.toml
# Each module gets its own format string
[git_branch]
format = "on [$symbol$branch]($style) "

[python]
format = "via [${symbol}${version}]($style) "

[cmd_duration]
show_milliseconds = true

What it shows on every prompt:

Current directory — truncated to fit
Git branch and status — dirty/clean, ahead/behind
Python version (if in a venv or project)
Command duration — how long the last command took
Exit code — red indicator on failure

The Dotfiles Repository

Everything above lives in a single repo: github.com/prateek11rai/dotfiles

The structure mirrors ~/.config/ directly:

dotfiles/
├── .zshrc
├── .config/
│   ├── starship.toml
│   ├── tmux/tmux.conf
│   ├── neofetch/
│   │   ├── config.conf
│   │   └── custom-ascii.txt
│   └── wezterm/
│       ├── wezterm.lua
│       ├── startup/init.lua
│       └── keybindings/init.lua
├── wallpapers/macos.png
├── scripts/mac-bootstrap.sh
└── README.md

The Bootstrap Script

The script at scripts/mac-bootstrap.sh automates setting up a new Mac:

Install Homebrew (if missing)
Install packages: tmux, starship, gh, zsh-autosuggestions, pyenv, wezterm, font-jetbrains-mono, neofetch
Back up any existing config files that are NOT already symlinks (renames to .backup.YYYYMMDD-HHMMSS)
Symlink every config file from the repo into the correct ~/ or ~/.config/ location
Install TPM and Tmux plugins

Running it on a fresh Mac:

git clone https://github.com/prateek11rai/dotfiles.git ~/github/prateek11rai/dotfiles
~/github/prateek11rai/dotfiles/scripts/mac-bootstrap.sh

The backup step makes this safe to run on an already-configured machine — nothing gets lost.

The Architecture

Neofetch is Archived
Neofetch has been unmaintained since 2023. The bootstrap script falls back to downloading the raw script from GitHub if brew install fails. Consider migrating to fastfetch as a maintained replacement.

What It Looks Like

WezTerm opens, attaches to the last Tmux session, and I see my Zsh prompt with the Dracula theme, Starship showing git status, and Neofetch greeting me with system info. Everything is the same as I left it — even if I closed the terminal an hour ago, on a different display, in a different city.

The Philosophy

Five principles that shaped this setup:

Tmux-first — Tmux manages sessions, not the terminal. WezTerm is just a display layer. This separation means I can close the window, switch machines, or lose power without losing my place.
One config repo — All dotfiles in one place, one bootstrap script, one git clone to set up a new machine.
Minimal bootstrap — Only Homebrew and git needed to go from zero to fully configured.
Consistent theme — Dracula everywhere. Same colors in the terminal, editor, and browser. Reduces visual friction.
Safe to re-run — The bootstrap script backs up existing files before symlinking. I can run it on an already-configured machine without breaking anything.

Every chef has their own knife kit, sharpened and arranged exactly how they like it. This is mine. WezTerm is the blade, Tmux is the workbench, Zsh is the cutting board, and Starship is the scale that tells you exactly what you are working with. The dotfiles repo is the roll that keeps it all together.

Image credits and attribution — please contact if anything needs updating.
WezTerm has its own multiplexer, but Tmux is preferred because sessions persist when the window closes, it works identically over SSH, and it has decades of community tooling around it.

Freeing the Screensaver: 4K Anime Art on Google TV

Prateek Rai — Sat, 27 Jun 2026 19:26:34 +0000

My TV spends more time idle than playing content. And every idle minute, Google TV shows me the same curated AI art or Google Photos memories — things I did not choose, do not control, and cannot change.

I wanted my own screensaver. 4K anime art from Wallhaven, video loops from MoeWalls — playing on my TV, not streamed from a cloud service I never asked for.

The Problem

Since Android TV 12, Google TV devices have locked down the screensaver selection. The Settings UI no longer lists third-party screensaver apps. You get Ambient Mode — Google Photos, AI art, or a black screen. That is it.

The change affected every Google TV device: Chromecast with Google TV, Google TV Streamer, and built-in Google TV on Sony, TCL, and Philips TVs.

The Solution

Aerial Views is an open-source screensaver app that supports 4K HDR videos, local photos, and media from USB/Samba/WebDAV. It is on the Play Store, but Google TV hides it from the launcher. You install it anyway via ADB and force the system to use it.

The full pipeline:

1. Enable Developer Options on the TV

Settings → Device Preferences → About → Build → click 7 times
Settings → System → Developer options → USB debugging → ON

Note down the TV's IP address: Settings → Network → Wi-Fi → IP address.

2. Install ADB

brew install android-platform-tools
adb version

3. Connect to the TV

adb connect <TV_IP>:5555

If the connection is refused (common on Android 14+ Chromecast with Google TV):

Go to Developer options → Wireless debugging on the TV
Select Pair device with pairing code
Run the pair command shown on TV, then connect:

adb pair <IP>:<PORT>   # enter the 6-digit code
adb connect <IP>:<PORT>

4. Download and Install Aerial Views

curl -LO https://github.com/theothernt/AerialViews/releases/download/1.8.2/aerial-views-1.8.2.apk
adb install aerial-views-1.8.2.apk

5. Set It as the System Screensaver

This is the critical step — it bypasses Google TV's UI lock:

adb shell settings put secure screensaver_components \
  com.neilturner.aerialviews/.ui.screensaver.DreamActivity

# Verify
adb shell settings get secure screensaver_components
# Expected: com.neilturner.aerialviews/.ui.screensaver.DreamActivity

6. Push Your Own Media

# Create a folder on the TV
adb shell mkdir -p /sdcard/AerialViews

# Push 4K images and videos
adb push ~/Downloads/wallhaven/*.png /sdcard/AerialViews/
adb push ~/Downloads/moewall/*.mp4 /sdcard/AerialViews/

# Verify
adb shell find /sdcard/AerialViews -type f

7. Configure Aerial Views on the TV

Open the Aerial Views app from the app drawer and configure:

Sources → Local storage
- Mode: Folder access
- Path: /AerialViews
- Ensure only one volume is selected
Playlist — disable ALL online sources (Apple, Amazon, Jetson Creative, Robin Fourcade). Enable only Local storage
- Set Media type: Mixed or Photos only
- Adjust Photo duration to your preference
Overlays (optional) — clock, date, location
Back → Test screensaver to preview

Where to Find 4K Anime Media

Google TV is 16:9 4K, so aim for 3840×2160 content:

Site	Type	Notes
Wallhaven	Images	4K PNG, best for anime art, bans AI art
DesktopHut	Video loops	Free 4K MP4, anime/gaming/nature
MoeWalls	Video loops	Anime-focused 4K MP4/WEBM, free
Pixabay	Videos	15K+ free backgrounds, 4K, royalty-free

The Current Setup

My TV now cycles through a mix of 4K Wallhaven images and a video loop from MoeWalls — about 151 MB of local content that plays regardless of internet connectivity.

/sdcard/AerialViews/
├── wallhaven-je86dy_3840x2160.png
├── wallhaven-d86o9g_3840x2160.png
├── wallhaven-6llg7q_3840x2160.png
├── ...                # 8 more images
└── moewalls/
    └── cat-in-the-swamp-moewalls-com.mp4

Quick Reference

Action	Command
Connect	`adb connect <IP>:5555`
Pair (if needed)	`adb pair <IP>:<PORT>`
Install APK	`adb install app.apk`
Push files	`adb push <local> <remote>`
Set screensaver	`adb shell settings put secure screensaver_components com.neilturner.aerialviews/.ui.screensaver.DreamActivity`
Restore default	`adb shell settings put secure screensaver_components com.google.android.apps.tv.dreamx/.service.Backdrop`
Change idle timeout	`adb shell settings put system screen_off_timeout 300000`

The fix took 15 minutes. Most of that was downloading wallpapers. The TV sits idle, and now it shows exactly what I want — no cloud, no AI, no algorithm.

Image credits and attribution — please contact if anything needs updating.

Ohara Learns to Talk: Self-Hosting an AI Agent with Hermes

Prateek Rai — Sat, 27 Jun 2026 19:26:24 +0000

Ohara had been running quietly under my desk for weeks. Jellyfin streamed movies. Suwayomi downloaded manga chapters overnight. The server did its job without complaint, invisible and silent.

But I wanted more. I wanted it to talk back.

The idea was simple: an always-on AI agent I could message from my phone, anywhere in the world, running on my own hardware. No subscriptions, no cloud lock-in, no monthly fees for toys I half-use. Just a Telegram bot that could think, remember, and eventually do things.

Why Hermes

Hermes Agent by Nous Research is an open-source AI agent framework designed to run on your own infrastructure. It connects to LLM providers (OpenRouter, OpenAI, Anthropic, local models) and messaging platforms (Telegram, WhatsApp, Discord) to become a persistent AI assistant.

What sold me:

Requirement	Why Hermes
Runs on a Core 2 Duo	No local GPU needed — all inference is API-based
Self-hosted	My data, my config, my keys
Multi-platform	One agent, many messaging backends
Extensible	Skills, cron jobs, terminal access, memory

The server's 15-year-old hardware does not matter here. Hermes does not run a model locally — it calls APIs. Ohara just needs to run a Docker container and have internet access. That is it.

The Docker Setup

Hermes ships as an official Docker image, so it fits right into Ohara's existing infrastructure. I keep all app data under /srv/appdata, so Hermes lives at /srv/appdata/hermes/:

/srv/appdata/hermes/
├── docker-compose.yml
├── .env
└── data/
    ├── .env
    ├── config.yaml
    ├── platforms/
    ├── sessions/
    ├── memories/
    └── skills/

docker-compose.yml

services:
  hermes:
    image: nousresearch/hermes-agent:latest
    container_name: hermes
    restart: unless-stopped
    command: gateway run
    ports:
      - "8642:8642"
    volumes:
      - /srv/appdata/hermes/data:/opt/data
    environment:
      - OPENROUTER_API_KEY=${OPENROUTER_API_KEY}

The .env file beside the compose file holds just the OpenRouter key:

OPENROUTER_API_KEY=sk-or-...

Start it:

cd /srv/appdata/hermes
docker compose up -d

The Permission Trap

On first boot, the data/ directory gets created as root. Hermes then drops privileges to UID 10000 (the hermes user) inside the container and can no longer read its own files. You will know this happened because the container enters a restart loop with PermissionError in the logs.

The fix is simple — one-time, then it stays fixed:

sudo chown -R 1000:1000 /srv/appdata/hermes/data
docker compose down && docker compose up -d

This is the single most important thing to know about running Hermes in Docker. Every permission issue traces back to this. If you write files with sudo tee or docker cp, run that chown command afterwards.

OpenRouter: Free LLM Access

Hermes needs an LLM to think. Running one locally on a Core 2 Duo with 4 GB RAM is not realistic. Instead, I route everything through OpenRouter — a single API key that gives access to 200+ models from every provider.

# Inside the container
docker exec -it hermes bash
source /opt/hermes/.venv/bin/activate
hermes model

This launches an interactive wizard. Two choices:

Select OpenRouter (the key is auto-picked from the OPENROUTER_API_KEY env var)
Pick a model

The model I chose: nvidia/llama-3.1-nemotron-70b-instruct:free

This is a NVIDIA-hosted Llama 3.1 variant available at no cost through OpenRouter's free tier.

The key insight: the API route means your server's hardware does not matter. A Raspberry Pi would work just as well. The server just needs to make HTTP requests and relay responses. No GPU, no RAM for model weights, no quantization — just a network connection.

The Rate Limit Reality

Here is where the free tier fantasy meets reality. A brand-new OpenRouter key gets 50 free requests per day. That sounds like a lot until you realize each Hermes conversation turn calls the model 3-5 times — one for thinking, one for tool calls, one for the final response. Those 50 requests become 10-15 usable conversations before the key hits the wall.

You know you have hit the limit when Hermes stops responding and OpenRouter returns a 429 or a vague gateway error:

The fix is unsurprising:

Add a minimum of $10 in credits to your OpenRouter account. This unlocks 1000 requests/day — more than enough for a personal bot.

$10 on OpenRouter lasts months for a single-user bot. The free tier is a generous trial — the paid tier is where it becomes actually useful. And $10 is still cheaper than a month of any subscription service.

Telegram Integration

The messaging layer is what makes this useful. A web UI is fine for configuration, but the real power is messaging your server like a person.

Step 1: Create the Bot

Open Telegram, search for @BotFather, and send /newbot:

Pick a name (I used Ohara Hermes)
Pick a username ending in _bot (e.g. ohara_hermes_bot)
BotFather replies with a token: YOUR_BOT_TOKEN

Step 2: Get Your User ID

Telegram bots need to know who is allowed to talk to them. Find your numeric user ID by messaging @userinfobot.

Step 3: Configure Hermes

echo 'TELEGRAM_ENABLED=true' | sudo tee -a /srv/appdata/hermes/data/.env
echo "TELEGRAM_BOT_TOKEN=YOUR_BOT_TOKEN" | sudo tee -a /srv/appdata/hermes/data/.env
echo 'TELEGRAM_ALLOWED_USERS=8523398033' | sudo tee -a /srv/appdata/hermes/data/.env
sudo chown -R 1000:1000 /srv/appdata/hermes/data
docker restart hermes

Environment variable name
The correct variable is TELEGRAM_ALLOWED_USERS. Using TELEGRAM_ALLOWED_USER_IDS (which some older docs reference) will cause "Unauthorized user" errors in the logs and the bot will reject every message.

Step 4: Test It

Open Telegram, find your bot by username, send a message. The agent responds. That is the whole setup — four commands and you have an AI assistant on your phone.

The WhatsApp Detour

Hermes also supports WhatsApp through a Baileys-based bridge (emulates WhatsApp Web, no Business API needed). I tried it. I removed it.

The self-chat mode works in theory — you message yourself, Hermes replies in the same thread. In practice:

The bridge rejected messages with self_chat_mode_rejects_non_self even though the sender was the paired number
Permission conflicts between the bridge log (owned by root) and the gateway
The user experience is worse than Telegram — no voice messages, limited formatting, and the self-chat thread gets confusing

Removal is clean:

# Inside container
sed -i '/^WHATSAPP/d' /opt/data/.env
rm -rf /opt/data/platforms/whatsapp
docker restart hermes

Telegram is the first-class messaging platform for Hermes. WhatsApp works but has rough edges I did not want to maintain.

The Permission Pattern (It Keeps Happening)

Every Hermes issue I ran into traced back to the same root cause: file ownership. Here is the complete list of symptoms and the universal fix:

Symptom	Cause
Container restart loop	`data/` owned by root, container UID 10000 can't write
`PermissionError: .bundled_manifest`	Skills file owned by root
`I have no name!` shell prompt	Cosmetic — UID 10000 not in /etc/passwd
Telegram unauthorized	Wrong env var name (`_USER_IDS` vs `_USERS`)
`.env` changes not picked up	File written with sudo, owned by root

The fix every time:

sudo chown -R 1000:1000 /srv/appdata/hermes/data
docker restart hermes

A DNS Detour: When Tailscale Broke GitHub Pages

This one was unexpected. After setting up Hermes, I tried to visit my GitHub Pages site (https://prateek11rai.github.io/sanji/) to check the previous blog post. The browser showed nothing. DNS lookup failed. But everything else on the internet worked fine.

The culprit was hiding in plain sight:

$ dig prateek11rai.github.io

;; SERVER: 100.100.100.100#53  # ← Tailscale DNS
;; status: SERVFAIL

Tailscale's MagicDNS was intercepting the .github.io lookup and failing to resolve it. The server at 100.100.100.100 — Tailscale's built-in DNS resolver — did not know how to handle GitHub Pages domains.

The fix
Temporarily disable Tailscale — the site loads immediately through the normal DNS resolver. Alternatively, add a secondary DNS server (like 8.8.8.8) in your system network settings while Tailscale is connected.

This is a quirk of how Tailscale handles DNS on macOS. It overrides your system DNS with its own resolver, and that resolver does not forward non-tailnet domains correctly in all configurations. The .github.io namespace is a GitHub-controlled TLD that Tailscale's DNS does not recognize.

The Complete Architecture

Here is what Ohara looks like now:

What Comes Next

Hermes has a built-in cron scheduler. I have started experimenting with daily prompts:

hermes cron add --schedule "0 6 * * *" --prompt "Check my Jellyfin library and recommend something to watch tonight" --platform telegram

The longer-term plan is a coding agent — Hermes that can clone a repo, run Claude Code through OpenRouter, make changes, commit, and create a PR, all triggered from a Telegram message. The infrastructure is ready: Hermes has terminal access, file system access, and network connectivity. It just needs the right skills configured.

But that is a post for another day.

Things I Learned

Permission issues are the #1 cause of Docker container crashes — any file written as root inside a volume will eventually break something. chown 1000:1000 after every file write and move on.
API-based AI does not need a GPU — a 15-year-old Core 2 Duo with 4 GB RAM is enough to run a full AI agent. The only requirement is internet access.
OpenRouter free tier is a trial, not a plan — 50 requests/day sounds fine until Hermes burns 3-5 per turn. Budget $10 to unlock 1000/day. It lasts months for a personal bot.
Tailscale's DNS can interfere with non-tailnet domains — .github.io, and potentially other TLDs, may fail to resolve through Tailscale's MagicDNS. Know how to bypass it.
Telegram is the better platform — WhatsApp integration exists but Telegram's bot API is cleaner, more reliable, and better documented. Start with Telegram.

The server under my desk still runs Jellyfin and Suwayomi. But now it also talks back. When I send a message to @ohara_hermes_bot, a container on that dusty old OptiPlex wakes up, calls a free NVIDIA-hosted model through OpenRouter, thinks about what I asked, and replies. No cloud subscription. No exposed ports. No GPU required.

That is the magic of self-hosting in 2026. The hardware you already own is enough.

Image attribution and credits — please contact if anything needs updating.

Breathing Life Into an Old PC: Building a Self-Hosted Media & Manga Server

Prateek Rai — Sat, 27 Jun 2026 19:01:44 +0000

I had an old PC lying around — one of those machines you keep meaning to throw out but never do. It was slow, loud, and completely useless for modern computing. No one was going to browse the web on it, let alone run any real software.

Then it hit me: I could turn it into a server.

This is the story of how that dusty old tower became the backbone of my home media setup. I call it Ohara — after the island of scholars in One Piece — because it preserves and serves knowledge (and manga, and movies, and music) to every device in my house.

The Hardware: What I Was Working With

Before you go out and buy a Raspberry Pi or a NAS, look under your desk. Chances are you already own a server.

🖥️ The Specs
- Old Dell OptiPlex office PC
- Intel Core 2 Duo (~15 years old)
- 4 GB DDR3 RAM
- NVIDIA GeForce 8400 GS (2007-era, useless for AI)
- 500 GB HDD + external USB drives
- Gigabit Ethernet port
- No monitor, no keyboard, no mouse
✅ What it can do
- Stream 1080p media to multiple devices ✔️
- Self-host a manga server with auto-downloads ✔️
- Run 24/7 on ~40W power ✔️
- Serve files over a private encrypted network ✔️
- Run LLMs or AI workloads ❌

The GPU is so old it predates CUDA entirely. But here is the thing: media serving is not GPU-bound. Jellyfin can hardware-transcode if you have a modern GPU, but for direct-play or software transcoding of 1080p content, even this Core 2 Duo handles it fine. The real bottleneck is always the network, not the CPU.

The OS Choice: Ubuntu Server

I went with Ubuntu Server 24.04 LTS — minimal install, no desktop environment. The alternatives were:

Debian — lighter, but hardware detection is worse and you lose out on broader community support
Alpine — very light, but musl libc can cause Docker compatibility surprises
Proxmox — overkill for a single machine, eats RAM for the hypervisor layer

Ubuntu Server wins because when something breaks, someone has already asked and answered the question on the internet.

Installation Steps

Download the Ubuntu Server ISO via torrent — browser downloads are slow, magnets are faster
Create a bootable USB with Rufus (Windows) or dd (Mac/Linux)
Boot the old PC from the USB, install Ubuntu Server — do not install a GUI, select "Install Ubuntu Server" and keep it minimal
During install, enable SSH and create your user account
After first boot:

sudo apt update && sudo apt upgrade -y
sudo apt install -y docker.io docker-compose-v2 ufw
sudo snap remove lxd  # remove unnecessary bloat

Running It Headless

The PC sits under my desk with nothing plugged in except power and ethernet. No monitor, no keyboard. Everything is done over SSH.

# Find the server on your network
nmap -sn 192.168.1.0/24

# SSH in (use the IP you found)
ssh prateek@192.168.1.50

Making Sure It Never Sleeps

Servers should run 24/7. The cleanest way to disable sleep on Ubuntu Server is through logind:

sudo mkdir -p /etc/systemd/logind.conf.d
sudo tee /etc/systemd/logind.conf.d/nosleep.conf << 'EOF'
[Login]
HandleSuspendKey=ignore
HandleHibernateKey=ignore
HandleLidSwitch=ignore
HandleLidSwitchExternalPower=ignore
IdleAction=ignore
EOF

sudo systemctl restart systemd-logind

Storage Layout

All data lives under /srv, following the Linux Filesystem Hierarchy Standard:

/srv/
├── media/
│   ├── movies/
│   ├── tv/
│   ├── music/
│   ├── music-videos/
│   └── manga/
└── appdata/
    ├── jellyfin/
    └── suwayomi/

The separation is deliberate: /srv/media holds the actual content (large, slow-changing), while /srv/appdata holds Docker configs and databases (small, needs backup). You can back up configs independently of media this way.

Tailscale: The Networking Layer

I never expose any port to the public internet. Instead, everything is accessed through Tailscale — a WireGuard-based mesh VPN that makes all my devices think they are on the same local network.

# On the server
curl -fsSL https://tailscale.com/install.sh | sh
sudo tailscale up
# Opens a URL — authenticate once, done forever

# On Mac
brew install tailscale && open /Applications/Tailscale.app

# On iPhone / Android TV
# Install from App Store / Play Store, log in

Once Tailscale is running, every device gets a 100.x.x.x IP. I reach my server by its hostname from anywhere in the world — no DDNS, no port forwarding, no Cloudflare tunnels. It survives reboots automatically. If the server restarts, Tailscale reconnects without manual intervention.

Verification Checklist

Test	Command	Expected
Device visibility	`tailscale status`	All 4-5 devices listed
Network connectivity	`ping 100.x.x.x`	Replies from server
SSH over mesh	`ssh ohara`	Login prompt from anywhere
File transfer	`rsync -avz test.txt ohara:/tmp/`	File arrives

Docker Compose: The Infrastructure as Code

Every service runs in a container defined by a docker-compose.yml. The entire server can be rebuilt from these files in minutes.

The Main Compose File

# /srv/appdata/docker-compose.yml
services:
  jellyfin:
    image: jellyfin/jellyfin:latest
    container_name: jellyfin
    ports:
      - "8096:8096"
    volumes:
      - /srv/appdata/jellyfin:/config
      - /srv/media:/media:ro
    devices:
      - /dev/dri:/dev/dri  # hardware transcoding (if available)
    restart: unless-stopped

Suwayomi: The Manga Server

Suwayomi is a self-hosted manga reader server. It is a rewrite of Tachiyomi (the popular Android manga reader) for the server — it downloads chapters, stores them on disk, and serves them to clients over the web. The workflow is simple:

Browse and add manga from your phone via the WebUI
Suwayomi downloads the chapters to /srv/media/manga/
Read from any device — progress syncs automatically

The Docker Compose for Suwayomi lives alongside Jellyfin in my setup:

# Suwayomi service — add to /srv/appdata/docker-compose.yml
services:
  suwayomi:
    image: ghcr.io/suwayomi/suwayomi-server:latest
    container_name: suwayomi
    ports:
      - "4567:4567"
    environment:
      - SUWAYOMI_USER=prateek
      - SUWAYOMI_PASSWORD=your_secure_password_here
    volumes:
      - /srv/appdata/suwayomi:/home/suwayomi/.local/share/Suwayomi
      - /srv/media/manga:/manga
    restart: unless-stopped

Start it:

cd /srv/appdata
docker compose up -d suwayomi

# Check it's running
docker compose ps

# Watch logs
docker compose logs -f suwayomi

Adding Manga Sources (Extensions)

Suwayomi is compatible with the Tachiyomi/Mihon extension ecosystem, but the way you add sources is different from the Android app. Extensions are added from inside the Suwayomi WebUI, not from the Docker config. This separation is intentional — infrastructure stays clean, and application config stays flexible.

Here is the important backstory. The original Tachiyomi project was discontinued. The community forked it into Mihon (the Android reader), and the extension ecosystem moved to Keiyoushi — a community-maintained repository of manga sources. Suwayomi can use this same repository.

I have both repos starred on GitHub:

Repo	What it is
`Suwayomi/Suwayomi-Server`	The server itself — a rewrite of Tachiyomi for desktop/server
`keiyoushi/extensions`	Extension repository for Mihon and Suwayomi — sources like MangaDex, MangaSee, etc.

To add sources to Suwayomi:

# Step 1: Open the WebUI
# In your browser (over Tailscale):
# http://ohara:4567
# or http://100.x.x.x:4567

Log in with the credentials you set in docker-compose.yml
Go to Extensions → Repositories
Add the Keiyoushi extensions repository URL:

   https://keiyoushi.github.io/extensions/index.json

Click Save — the repository is now registered
Go to Extensions → All, refresh the list
You should now see sources like MangaDex, MangaSee, Comick available for install
Click Install on each source you want

Once installed, search for manga, add to your library, and enable auto-download for new chapters. The server becomes largely autonomous — new chapters are downloaded in the background, and your library updates on a schedule.

Jellyfin: Movies, TV, and Music

Jellyfin is my unified media server. From the single docker-compose above, I get:

Movies & TV — organized by metadata scrapers, streamed to any device
Music — with album art, artist info, and gapless playback
Live TV (optional) — if you plug in a tuner

The library structure maps directly to /srv/media/:

Library Type	Path in Container	Path on Disk
Movies	`/media/movies`	`/srv/media/movies`
TV Shows	`/media/tv`	`/srv/media/tv`
Music	`/media/music`	`/srv/media/music`

# Inside Jellyfin WebUI, add each library:
# Dashboard → Libraries → Add Media Library
# Type: Movies → Folder: /media/movies
# Type: TV Shows → Folder: /media/tv
# Type: Music → Folder: /media/music

Configure the "Prefer ARTISTS tag if available" setting under the Music library for correct artist metadata.

How I Consume Everything Across Devices

The best part of this setup is that content follows me everywhere. Here is exactly how each device connects:

iPhone

Jellyfin: Install the Jellyfin Mobile app from the App Store, sign in with http://ohara:8096 (the Tailscale MagicDNS hostname resolves over the mesh)
Manga (Suwayomi): Open Safari → http://ohara:4567 → log in → tap the Share button → Add to Home Screen. It behaves like a native app: full-screen, dedicated icon, persistent session
Tailscale: Always-on in the background. Without it, none of the above works outside my home WiFi

Android TV

My TV runs Android TV, which does not come with Tailscale or Jellyfin pre-installed. You can sideload both via ADB:

# 1. Enable Developer Options on TV
# Settings → Device Preferences → About → Build → tap 7 times

# 2. Enable USB Debugging
# Settings → Developer Options → USB Debugging → ON

# 3. Find TV IP from network settings
# Example: 192.168.1.100

# 4. On your Mac/Laptop, connect and install
adb connect 192.168.1.100
adb install jellyfin-androidtv.apk
adb install tailscale.apk

# 5. On the TV, open Tailscale → log in
# 6. Open Jellyfin → sign in with http://ohara:8096

Once set up, the TV works outside home too — take it to a different city, connect to WiFi, Tailscale reconnects automatically and Jellyfin works.

MacBook

The MacBook is my management machine. I use it for:

# SSH into the server from anywhere
ssh ohara

# Rsync backups
rsync -avz --progress ohara:/srv/media/manga/ ~/ohara-backup/manga/

# Edit compose files remotely
ssh ohara "vim /srv/appdata/docker-compose.yml"

# Rebuild containers after config changes
ssh ohara "cd /srv/appdata && docker compose up -d"

For media consumption, I use the Jellyfin WebUI in a browser. There are desktop clients (Jellyfin Media Player, Swiftfin) but the WebUI is good enough for occasional viewing.

Windows Laptop

The Windows laptop serves as a secondary backup target. Over Tailscale:

# Pull backup from server
rsync -avz prateek@ohara:/srv/media/manga/ D:\ohara-backup\manga\

# Push files to server
rsync -avz D:\documents\ prateek@ohara:/srv/media/documents\

No SMB shares, no mapped drives, no exposed ports — just SSH and rsync over the encrypted mesh.

The Complete Architecture

Backup Strategy

# From Mac — pull a full media backup
rsync -avz --progress ohara:/srv/media/ ~/ohara-backup/

# From Windows — mirror to external drive
rsync -avz prateek@ohara:/srv/media/manga/ D:\ohara-backup\manga\

# From server itself — backup to USB drive
rsync -avz /srv/appdata/ /mnt/usb-backup/appdata/

Three golden rules:

Always use --dry-run first — rsync is silent when it works and destructive when it doesn't
Trailing slashes matter — ohara:/srv/media/ copies contents, ohara:/srv/media copies the folder itself
No SMB needed — SSH + rsync is cleaner, scriptable, and works over Tailscale from anywhere

What Running This Feels Like Day to Day

The server is invisible. It sits under my desk, silent, doing its job. Here is what a typical day looks like:

Morning: New manga chapters were downloaded overnight (Suwayomi auto-updates at 3 AM)
Afternoon: I start a movie on the TV via Jellyfin, pause it, continue on my phone in another room
Evening: I discover a new manga on my phone, add it to my library via Suwayomi WebUI
Night: An automated rsync cron job backs up appdata to an external drive

The only time I SSH into the server is when I add a new service or update a container. Everything else runs itself.

Why "Ohara"?

In One Piece, Ohara was the island of scholars — a place where knowledge was pursued, preserved, and archived. The World Government destroyed it for holding too much knowledge, but the legacy survived through the survivors who carried books in their memory.

A home server is a much smaller act of preservation. But it is the same impulse: keep your own things on your own hardware, under your own control. Your media library, your manga collection, your files — they should outlive any streaming service's licensing decisions or any cloud provider's terms of service changes.

You do not need a data hoarding disorder to want that. You just need an old PC, a weekend, and the willingness to ssh into a box that lives under your desk.

What You Will Have at the End

✅ A 24/7 headless server that needs no monitor, keyboard, or mouse

✅ Remote access from anywhere via Tailscale — no ports open to the internet

✅ A full media server (Jellyfin) for movies, TV, and music

✅ A self-hosted manga server (Suwayomi) with auto-downloads and cross-device sync

✅ Clients working on iPhone, Android TV, Mac, and Windows

✅ An automated backup pipeline over SSH

✅ Infrastructure-as-code with Docker Compose — rebuild the entire server from a single YAML file

The barrier to entry for homelabbing has never been lower. Ubuntu Server is free. Docker Compose turns deployment into a single YAML file. Tailscale makes networking trivially secure. All you need is the hardware you already have lying around, gathering dust.

Go build your Ohara.

If I have used any images that need attribution, please contact me.