DEV Community: Wes

When AI Writes Your Firewall, Check the Math

Wes — Sun, 12 Apr 2026 11:52:30 +0000

A Python developer with "AI Solutions Architect" in their GitHub bio pushes 8,500 lines of eBPF Rust in a single commit. The commit author is "Blackwall AI." The .gitignore lists .claude/. The README reads like marketing copy, and the author later confirms the AI handled the "marketing glaze." Four days later, the repo has 119 stars.

This is the new normal. People are shipping real projects in languages they don't primarily work in, at speeds that weren't possible two years ago. The question isn't whether AI-assisted code can be good. It's whether you can tell the difference between code that's good and code that looks good.

I went to find out.

What Is Blackwall?

Blackwall is an eBPF/XDP firewall with an LLM-powered honeypot, written in Rust. Named after the AI containment barrier in Cyberpunk 2077. It runs packet filtering at kernel level via aya (pure Rust eBPF, no C, no libbpf), tracks per-IP behavioral profiles through a state machine that escalates from New to Normal to Probing to EstablishedC2, does JA4 TLS fingerprinting and deep packet inspection, and includes a tarpit that uses a local LLM through Ollama to simulate a compromised Ubuntu server. The idea is that when the behavioral engine flags an IP as hostile, traffic gets redirected to the honeypot, which pretends to be a real machine while logging everything the attacker does.

One maintainer. Two commits. 119 stars in four days. MIT license.

The Snapshot


Project	blackwall
Stars	~119 at time of writing
Maintainer	Solo (xzcrpw), shipped everything in one commit
Code health	8,544 lines, 123 tests, zero unwrap() in production, real bugs in the glue code
Docs	AI-generated README that oversells, no architecture docs
Contributor UX	No CONTRIBUTING.md, no CI, no PR history to judge from
Worth using	Not yet, but worth reading

Under the Hood

Six crates in a Cargo workspace. common holds #[repr(C)] shared types between kernel and userspace with explicit padding fields. blackwall-ebpf has the XDP programs. blackwall is the userspace daemon. tarpit is the honeypot. blackwall-controller coordinates distributed nodes. xtask handles eBPF cross-compilation.

The eBPF code is the strongest part. Every pointer dereference has a bounds check before access, which is not just good practice but a hard requirement: the BPF verifier will reject your program otherwise, and getting this right in Rust via aya is harder than it sounds. The entropy estimation uses a 256-bit bitmap instead of a histogram, which is a clever adaptation for eBPF's 512-byte stack limit. The TLS ClientHello parser handles session IDs, cipher suites, compression methods, extensions, SNI extraction, ALPN, and GREASE filtering. If you want to see how to write correct eBPF in Rust, this is a solid reference.

The behavioral engine is clean too. Seven phases from New through EstablishedC2, monotonically increasing in suspicion with an explicit demotion path to Trusted. The state machine uses deterministic thresholds for fast-path decisions, with optional LLM classification as a slow path. Constants are well-named and consistent: SUSPICION_INCREMENT at 0.15, SUSPICION_MAX at 1.0, SUSPICION_DECAY at 0.02. Trusted promotion requires the score to stay below 0.1 across 300 seconds and 100+ packets.

The tarpit is where things get creative. It runs four protocol handlers: SSH (backed by the LLM pretending to be a bash shell), HTTP (fake WordPress), MySQL (wire protocol responses), and DNS (canary records). The SSH honeypot's system prompt is detailed enough to be convincing. It specifies hostname, kernel version, filesystem layout, installed services, and includes example command/output pairs so the LLM knows to respond with raw terminal output instead of "Sure, here's the output of ls."

Now for the bugs. The project ships with "think": false in its Ollama requests, but some models emit <think> blocks anyway. The stripping code handles exactly one block:

if let Some(start) = content.find("<think>") {
    if let Some(end) = content.find("</think>") {
        let after = &content[end + 8..];
        after.trim_start().to_string()

If the model emits two <think> blocks, the second one passes through to the attacker, leaking the LLM's reasoning about how to respond. A security tool with a security bug.

The DPI path matching is aggressive. It flags any HTTP path starting with /wp-, /adm, /cm, or /cg as suspicious, which would catch /admiral-insurance or /cmarket. The random_window_size() function in the tarpit's anti-fingerprinting module computes a value and discards it (let _window = random_window_size()). TCP window randomization is advertised in the README but doesn't actually happen. The iptables DNAT rules that redirect traffic to the honeypot persist after a SIGKILL because Drop cleanup gets skipped. Several modules (distributed, antifingerprint, canary) carry #[allow(dead_code)] annotations, meaning they're defined but not wired into anything.

No CI. No GitHub Actions. The README claims "Clippy: zero warnings (-D warnings)" but nothing enforces it. The entire development history is two commits: "release: blackwall v1" and "images."

The Contribution

The suspicion score scale mismatch was the clearest bug to fix. The behavioral engine operates on a 0.0 to 1.0 scale. Increments are 0.15, max is 1.0, decay is 0.02 per evaluation, and trusted promotion requires < 0.1. But the DPI event handler in process_events was adding 15.0 and capping at 100.0. A completely different scale. Any IP that triggered a single DPI detection got a suspicion score of 15.0+, making trusted promotion effectively unreachable. At a decay rate of 0.02 per tick, it would take roughly 750 ticks to come back under 0.1.

The fix was small: export SUSPICION_INCREMENT and SUSPICION_MAX from the behavior module and use them in the DPI handler, matching the pattern already used by apply_escalation in the transition logic. Three files, 11 insertions, 5 deletions. All 19 behavior tests pass.

Getting into the codebase was easy despite the size. The workspace structure keeps things separated, the naming is consistent, and the behavioral engine is self-contained enough that you can understand the state machine without reading the eBPF code. Finding the bug was a matter of grepping for suspicion_score and noticing that one callsite looked nothing like the others.

PR #3 got a response the same day. The maintainer had independently found the same bug during a major architectural rewrite and closed the PR to avoid merge conflicts with the new core. "Nice catch on the scale mismatch, man." He shipped v2.0.0 hours later, overhauling the behavioral engine and moving to native eBPF DNAT. Someone else's detailed bug report got a similar response: all findings would be addressed in the rewrite. Both are now closed.

The Verdict

Blackwall is interesting as both a codebase and a case study. The eBPF work is genuinely competent. The behavioral engine is well-designed. The honeypot concept is creative. But it shipped in one commit with no CI, no development history, and bugs in the code that connects the components together. The strongest parts (kernel programs, type contracts, state machine) look like they were built carefully. The weakest parts (DPI integration, think-tag stripping, dead code) look like they were assembled and not tested end-to-end.

That's not a criticism of using AI to write code. It's a criticism of shipping without integration testing, regardless of who or what wrote it. The eBPF verifier forces you to be correct in the kernel programs. Nothing forces you to be correct in the glue.

If you work with eBPF, the XDP programs and the aya patterns are worth studying. If you're interested in honeypot design, the tarpit's multi-protocol approach and LLM integration are genuinely novel. The maintainer shipped a v2.0.0 rewrite the same week, which suggests the integration bugs are being addressed. If you want to run this in production, give the new version a serious read first. The architecture has changed significantly, and it hasn't had time to accumulate the kind of real-world testing you want from something sitting between attackers and your network.

Go Look At This

Blackwall on GitHub. Read the eBPF code even if you skip the rest.

If you want to contribute, the v2.0.0 rewrite is a fresh starting point. Our suspicion score fix and the bug report are both closed, addressed in the rewrite. The new codebase still has no CI, and the <think> tag stripping likely still needs work. The 35 MB of PNG assets are still there.

This is Review Bomb #14, a series where I find under-the-radar projects on GitHub, read the code, contribute something, and write it up. If you know a project that deserves more eyeballs, drop it in the comments.

This post was originally published at wshoffner.dev/blog. If you liked it, the Review Bomb series lives there too.

A Rust TUI for Your UniFi Network That Actually Takes Code Review Seriously

Wes — Thu, 09 Apr 2026 12:44:50 +0000

If you run UniFi gear, you manage it through a web UI. That's fine until you need to script something, automate a deployment, or check your firewall rules from an SSH session on a box that doesn't have a browser. Ubiquiti doesn't ship a CLI. The API exists but it's split across two incompatible surfaces with different auth mechanisms, different response formats, and different ideas about what an entity ID should look like. Most people who try to automate UniFi end up writing bespoke Python scripts against whichever API endpoint they needed that week. The scripts work until a firmware update moves the endpoint.

Someone decided to build the CLI that Ubiquiti didn't. And then kept going until it had a full TUI dashboard.

What Is Unifly?

Unifly is a Rust CLI and terminal dashboard for managing UniFi network infrastructure. Built by Stefanie Jane, it ships as a single binary with 27 CLI commands covering devices, clients, networks, firewall rules, NAT policies, DNS, VPN, Wi-Fi observability, and more. There's also a 10-screen ratatui TUI for real-time monitoring: device health, client connections, traffic charts, firewall policies, network topology. The whole thing speaks both UniFi API dialects (the modern Integration API and the older Session API) and reconciles data between them.

123 stars at time of writing. Eight weeks old. About 15,000 lines of Rust across two crates. The pace of development is aggressive: 71 commits in the first eight days of April alone.

The Snapshot


Project	Unifly
Stars	123 at time of writing
Maintainer	Solo (hyperb1iss), AI-assisted
Code health	250+ tests, pedantic clippy, forbid(unsafe), e2e suite against simulated controller
Docs	Thorough CONTRIBUTING.md, detailed architecture guide, explicit ROADMAP with named gaps
Contributor UX	Same-day responses, content always lands, merge mechanism varies
Worth using	Yes if you have UniFi gear. The CLI alone replaces a lot of web UI clicking.

Under the Hood

The architecture is a two-crate Cargo workspace. unifly-api is the library: HTTP and WebSocket transport, a Controller facade behind an Arc, and a reactive DataStore built on DashMap and tokio::watch channels. unifly is the binary: clap-based CLI commands and ratatui TUI screens. The library is published to crates.io independently, so you could build your own tooling on top of it without pulling in the CLI.

The dual-API problem is the interesting engineering challenge. UniFi controllers expose a modern Integration API (REST, API key auth, UUIDs) and an older Session API (cookie + CSRF, hex string IDs, envelope-wrapped responses). Some data only exists on one side. Client Wi-Fi experience metrics? Session API. Firewall policy CRUD? Integration API. NAT rules? Session v2 API, a third dialect. Unifly handles all of this behind a single Controller type. You call controller.execute(Command::CreateNatPolicy(...)) and the command router figures out which API to hit, which auth to use, and which ID format to resolve.

The lint configuration tells you a lot about a project's standards. Unifly runs clippy::pedantic at deny level, clippy::unwrap_used at deny, and unsafe_code at forbid. The workspace Cargo.toml has 30+ individual clippy rule overrides, each with a clear rationale. This is not someone who pasted a default config. The test suite uses wiremock for API mocking, assert_cmd for CLI testing, and insta for snapshot tests. The e2e suite spins up a simulated UniFi controller and runs full command flows. cargo-deny enforces a license allowlist and vulnerability advisories.

One thing you notice immediately: nearly every commit has Co-Authored-By: Claude Opus 4.6 in the trailer. This is an AI-assisted codebase, and the maintainer isn't hiding it. The code quality is high regardless of how it got there. The architecture is coherent, the tests are real, the error handling uses thiserror and miette properly. AI-assisted doesn't mean AI-dumped. The commit history shows iterative problem-solving, not a single prompt that produced 15,000 lines.

What's rough? The TUI has ten screens and zero test coverage. The controller reconnect lifecycle is broken (the CancellationToken goes permanent after the first disconnect). And the ROADMAP had a few stale entries that claimed features were missing when they'd already been implemented. These are the kinds of gaps you'd expect in a project that's moving this fast with a solo maintainer.

The Contribution

The ROADMAP listed "NAT policies have no update subcommand" as a known gap. The workaround was to delete and recreate, which means re-specifying every field just to change one. NAT rules are the kind of thing you adjust frequently (new port forward, toggling a rule on and off), so this was a real workflow friction.

The fix touched 11 files across both crates. On the library side: an UpdateNatPolicyRequest struct, a new Command variant, and an apply_nat_update function that fetches the existing rule via the Session v2 API, merges only the changed fields, and PUTs it back. On the CLI side: an Update variant in the clap args with all NAT fields as optional flags plus --from-file support, validation that at least one field is provided, and conflicts_with between --name and --description (both map to the same v2 API field, which I caught during my pre-submission review). I also promoted a private ensure_session_access function from the events handler to a shared utility, and wired it into the NAT handler so users get a clean error message when their auth mode is insufficient instead of a deep transport failure.

The codebase was easy to navigate. The project's AGENTS.md (symlinked as CLAUDE.md) doubles as a comprehensive architecture guide. It documents every API quirk, every CLI pattern, every module's responsibility. When I needed to understand how the firewall update command worked (to replicate the pattern for NAT), I read the architecture doc first, then confirmed in code. The existing firewall implementation was an almost perfect blueprint. The CONTRIBUTING.md lays out the full checklist: define args, implement handler, wire dispatch, update skill docs, add tests. I followed it step by step.

PR #10 is open and waiting for review. The maintainer's track record with external PRs is good. Five previous PRs from two contributors all had their content land in main. Clean PRs against current main get proper GitHub merges. PRs that conflict with ongoing refactors get reworked by the maintainer with co-author credit. Either way, the work ships.

The Verdict

Unifly is for anyone who manages UniFi networks and wants to do it from the terminal. If you run a homelab with Ubiquiti gear, or you're an MSP managing multiple sites, or you just want to script your firewall rules instead of clicking through a web UI, this is the tool. The CLI covers enough surface area to be genuinely useful today. The TUI is a bonus for monitoring.

The trajectory is steep. Eight weeks from first commit to 27 commands, a full TUI, dual-API support, and a published crate on crates.io. The codebase is clean enough that an external contributor (me) could implement a missing feature by following the existing patterns. The maintainer responds quickly and ships contributed work. Those are the signals that matter for whether a project has legs.

What would push it further? Test coverage on the TUI screens. Fixing the reconnect lifecycle so long-running TUI sessions survive network blips. And eventually, Cloud/Site Manager API support so you can manage controllers through Ubiquiti's cloud without direct network access. The AuthCredentials::Cloud variant already exists in the code. The transport just isn't wired yet.

Go Look At This

If you have UniFi gear, try unifly. cargo install unifly or grab a binary from the releases page. Run unifly config init to connect to your controller, then unifly devices list and see what happens.

The ROADMAP has more gaps worth picking up. TUI test coverage is explicitly welcomed. The radio data parsing has untested code paths. The Cloud API transport is waiting for someone to implement it.

This is Review Bomb #13, a series where I find under-the-radar projects on GitHub, read the code, contribute something, and write it up. If you know a project that deserves more eyeballs, drop it in the comments.

This post was originally published at wshoffner.dev/blog. If you liked it, the Review Bomb series lives there too.

Anatomy of a GitHub Actions Supply Chain Attack Targeting MCP Repos

Wes — Wed, 08 Apr 2026 11:41:55 +0000

On April 7th, someone submitted a pull request to my project Charlotte. 28 lines. One new file. A GitHub Actions workflow that "validates skill metadata in CI." The PR body quoted my own README back to me and offered to adjust the filename if I preferred something different.

I said I'd review it tomorrow. Then I actually looked at it, and spent the next day tracing an operation that spans 250+ repositories, at least 64 sockpuppet accounts, and five distinct phases of escalating access -- all controlled by a single organization.

This is what I found.

The PR

Charlotte is a browser automation MCP server. The PR came from an account called internet-dot and added .github/workflows/hol-skill-validate.yml:

name: HOL Skill Validate

on:
  push:
    branches: [main, master]
  pull_request:
    branches: [main, master]
  workflow_dispatch:

permissions:
  contents: read
  id-token: write

jobs:
  validate:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@34e114876b...
      - uses: hashgraph-online/skill-publish@1c30734416d9b05948ccd7f4b3cf60baada87e9e
        with:
          mode: validate

If you want to check whether this workflow exists in your own repo, that hash is the string to search for.

Two problems. First, id-token: write grants the workflow permission to mint an OpenID Connect token from GitHub's OIDC provider. That's a signed JWT proving "this request comes from a workflow running in repo X, branch Y, triggered by event Z." It exists for deploying to cloud providers. A local metadata validation step has no reason to request one.

Second, the action phones home. The entrypoint calls getGithubOidcToken(), mints a token scoped to your repository, and passes it to uploadSkillPreviewFromGithubOidc(), which ships it to hol.org/registry/api/v1. The preview-upload parameter defaults to true. If you merge this, every push to main and every pull request mints an OIDC token and sends it to a third-party server.

Charlotte has no SKILL.md, no skill.yaml, no Hashgraph Online skill definitions. There is nothing for this workflow to validate.

I closed the PR and started pulling threads.

The account

internet-dot was created on April 14, 2025. It submitted its first PRs to Hashgraph Online repos the same day, then went dormant for 11 months. All campaign activity falls within a two-week window starting late March 2026. It has 1,599 public repositories, 7 followers, and a bio that reads "i'm just a small dot on the internet. infra and identity. open source." Its blog field links to hol.org.

hol.org is the website of Hashgraph Online (HOL), a blockchain organization in the Hedera ecosystem. Their GitHub org, hashgraph-online, has one public member: Michael Kantor (kantorcodes), who identifies himself as President.

On the same day the internet-dot account was created, it submitted its first PRs -- to hashgraph-online/standards-agent-kit. Three pull requests adding a plugin system. All merged. From there, internet-dot contributed documentation to hashgraph-online/hcs-improvement-proposals, fixed bugs in hashgraph-online/skill-publish (the action being mass-deployed), and pushed 10 PRs to hashgraph-online/ai-plugin-scanner in a single day. All merged rapidly.

All nine of internet-dot's public gists are HOL documentation -- UAID specifications, MCP agent discovery guides, HCS standards overviews.

This is not an independent contributor who happens to like HOL's tools. This is an operational account for the organization.

The campaign

The PR to Charlotte wasn't an isolated event. It was Phase 5 of a multi-stage operation that has been running since late March 2026.

Phase 1: Seed the ecosystem (March 25-31)

The campaign opens with 20+ pull requests to curated awesome-lists: awesome-ai-agents, awesome-a2a, awesome-agents, awesome-web3, awesome-decentralized, awesome-security, awesome-software-supply-chain-security. Each PR adds a Hashgraph Online entry. Also targeted: the dify-plugins and fastgpt-plugin registries.

No code execution. No OIDC tokens. Just getting the name "Hashgraph Online" into places developers browse when evaluating tools.

Phase 2: Build a contribution history (March 26-27)

A brief detour into what looks like blockchain bounty farming. PRs to SubTrackr, learnvault, EventHorizon, and Soroban-state-lens -- small fixes like "add keyboardShouldPersistTaps to ScrollView" and "add blur-based inline validation." All merged. The kind of activity that makes a GitHub profile look like an active contributor rather than a single-purpose bot.

Phase 3: Establish file presence (April 2-3)

This is where the scale changes. In two days, internet-dot submits 40+ PRs adding a codex-plugin.json manifest file to MCP repositories. No CI. No workflows. Just a static metadata file.

The target list reads like a directory of MCP infrastructure:

github/github-mcp-server
microsoft/playwright-mcp
cloudflare/mcp-server-cloudflare
elastic/mcp-server-elasticsearch
googleapis/genai-toolbox
hashicorp/terraform-mcp-server
containers/kubernetes-mcp-server
getsentry/XcodeBuildMCP
firecrawl/firecrawl-mcp-server
tavily-ai/tavily-mcp
exa-labs/exa-mcp-server
makenotion/notion-mcp-server

And about 30 more. This is the foot in the door. Get a file into the repo. It's just metadata, what's the harm?

Phase 4: Introduce CI execution (April 3)

For repos that merged the Phase 3 manifest, follow-up PRs arrive adding CI workflows that "lint" and "validate" the manifest. This is where id-token: write first appears in the permissions block. The repos that already accepted a harmless metadata file now get a workflow that executes on every push.

Several merged this too.

Phase 5: The payload (April 6-8, ongoing)

The mass wave. 200+ PRs in three days, all adding the hashgraph-online/skill-publish workflow with id-token: write. The YAML is pinned to commit 1c30734416d9, which kantorcodes merged into skill-publish at 23:46 UTC on April 6th -- a "docs: use universal logo in README" commit. Hours later, the campaign PRs started arriving.

The action has four modes: validate, monitor, quote, and publish. The campaign PRs use validate. The escalation path is built into the tool.

The pattern

Each phase escalates access and builds on the last:

Brand awareness (no code)
Contribution credibility (unrelated fixes)
File presence (static metadata)
CI execution (workflow permissions)
OIDC token access (identity exfiltration)

If you only look at Phase 5 in isolation, it looks like someone spamming CI workflows. Zoom out and it's a funnel.

The sockpuppet network

About a third of the Phase 5 targets aren't real repositories. They're controlled by the same operation.

I pulled account creation timestamps for the obviously fake targets -- the ones with generated usernames like Genuspogoniacubicmillimetre130, Immunogenic-prismspectroscope589, and Enfluranehighbloodpressure470. They were created in four distinct waves:

February 20-21: 3 accounts created within hours. Crinolineflexion823, Unfathomable-siren38, Genuspogoniacubicmillimetre130.

March 1: 7 accounts in two hours. Starting at 07:32 UTC with Shelflifegymnopilusvalidipes977 and ending at 09:05 with genetic-shopping832.

March 7: 6 accounts in 90 minutes. dublen5269 at 22:27, ending with Gindhar2112 at 23:48.

March 17-18: The big batch. 14 accounts in 12 hours. foliagerosebush512 and unhealthy-outlander317 were created one second apart at 21:50:33 and 21:50:34.

Every one of these accounts has exactly one repository. No followers. No prior activity.

There's a second pattern too: accounts with normal-looking usernames (Rares683, ANGEL37391, sproutn-iq123) that each have exactly two repos -- one project and one github.io page, both created within minutes of each other. Same structure, less obvious names.

In total, roughly 64 of the 200 target repos are sockpuppets. They make the campaign look more widespread than it is and pad the "number of repos using this workflow" metric. Counting only real projects, about 130 were targeted. Of those, 13 merged one or more campaign PRs.

The trust chain

The workflow runs pnpm install --frozen-lockfile --prod at runtime with no .npmrc, pulling dependencies from public npm. The flag --frozen-lockfile makes installs deterministic for a given commit. It also has --ignore-scripts to block postinstall hooks. That sounds careful.

But the dependencies are imported by entrypoint.mjs at runtime, so module-level code in any package still executes. And the entire supply chain is one person:

kantorcodes writes the action code (sole committer to hashgraph-online/skill-publish)
kantorcodes publishes the primary dependency @hol-org/rb-client to npm (150 versions, co-maintained with tmcc_patches)
kantorcodes commits the pnpm-lock.yaml that determines which package versions get installed
internet-dot submits the PRs choosing which commit hash to pin

--frozen-lockfile guarantees you get what's in the lockfile at that commit. The person writing the lockfile is the person writing the code it installs. A new campaign PR with a new commit hash pulls whatever new lockfile and new package versions that person decided to ship.

The scoped npm packages (@hol-org/*, @hashgraph/*) aren't vulnerable to dependency confusion. They don't need to be. One person already controls every link in the chain from the npm registry to the OIDC token upload.

What the OIDC token gets them

A GitHub OIDC token isn't a credential that grants access to your repo. You can't use it to push code or read secrets. But it's a signed assertion of identity that any server can verify.

If hol.org trusts GitHub's OIDC provider, that token tells them exactly which repository the request came from. The action's outputs include directory-topic-id, package-topic-id, skill-page-url. In publish mode, it writes to the Hedera Consensus Service -- that's a blockchain ledger. The record is immutable. Once your repo is "published" as a skill, the entry is on-chain and can't be removed by you.

Validate mode is the entry point. Get the workflow merged. Then a follow-up PR changes one line -- mode: validate becomes mode: publish -- and your repo is registered on a platform you never signed up for, with a permanent, verifiable record associating your project with their infrastructure.

What to do

If you maintain an MCP-related project, search your .github/workflows directory for hashgraph-online, skill-publish, hol-skill, or codex-plugin-scanner. If you find a workflow you didn't add intentionally, remove it.

Check your merged PRs from internet-dot. If you merged a codex-plugin.json manifest file, consider whether you want that there.

The permissions block in any workflow YAML is the first thing to check on any CI contribution. A workflow that validates local files needs contents: read at most. id-token: write on anything that isn't deploying to a cloud provider is a red flag.

The uncomfortable part

I run a series where I find small repos and submit PRs to strangers. I know what it looks like from the other side of the notification. The difference between what I do and what this campaign does is intent and effort -- I read the code, find real bugs, and write targeted fixes. This operation scrapes README descriptions, templates a YAML file, and submits it to 250 repositories backed by a network of fake accounts.

But to a busy maintainer glancing at their notification feed, both look like a stranger showing up with a pull request. That's what makes it effective. It exploits the same trust that makes open source collaboration work at all.

Thirteen maintainers merged this. They probably had busy days too.

Your Artifact Registry Doesn't Need 2 GB of RAM

Wes — Mon, 06 Apr 2026 11:12:12 +0000

Every team eventually needs an artifact registry. You need somewhere to push Docker images, host internal npm packages, or cache Maven dependencies so your builds don't break when a mirror goes down. The standard answer is Nexus or Artifactory. Both work. Both are also Java applications that need a JVM, a database, careful heap tuning, and at least 2 GB of RAM before they'll serve a single artifact. On a CI server that's already running builds, that memory budget hurts.

One developer decided the problem was simpler than the existing solutions make it look. The result is a 32 MB Rust binary that handles seven package protocols on less than 100 MB of RAM.

What Is Nora?

Nora is a lightweight artifact registry built by devitway (Pavel Volkov). It supports Docker/OCI, Maven, npm, PyPI, Cargo, Go modules, and raw file hosting in a single binary. It includes a web UI dashboard, Prometheus metrics, token auth with Argon2id password hashing, S3 or local storage backends, mirror/proxy mode for air-gapped environments, and garbage collection. You configure it with a TOML file and run it. That's it.

36 stars. Three months of focused solo development. About 19,600 lines of Rust across 45 source files. This project is doing real work and almost nobody knows about it.

The Snapshot


Project	Nora
Stars	36 at time of writing
Maintainer	Solo (devitway)
Code health	411 tests, proptest, fuzz targets, 61.5% coverage, CI that would make a team project jealous
Docs	Good README, CONTRIBUTING.md with build/test/PR instructions
Contributor UX	Same-day review on PRs, warm and specific feedback
Worth using	Getting there. Mirror mode and auth are solid. Watch this space.

Under the Hood

The architecture is what you'd hope for: each registry protocol gets its own module under src/registries/, storage is abstracted behind a trait (local filesystem or S3), and the HTTP layer is axum with tokio. Config is validated at startup. There's no clever metaprogramming, no macro soup. You can open the Docker registry module and understand what it does without reading three layers of indirection first.

The dependency list is restrained for a project this ambitious. axum and tokio handle the HTTP server. reqwest handles upstream proxy requests. serde and toml for config. The entire Cargo.toml reads like someone who picks dependencies on purpose rather than pulling in whatever shows up first on crates.io. For context: Nexus Repository Manager pulls in over 400 Maven dependencies. Nora's Cargo.lock has about 350 crate entries, but most of those are transitive deps from tokio and reqwest. The direct dependency count is small.

The CI pipeline is where Nora stands out from other solo projects. Most one-person repos have a test workflow and maybe clippy. Nora runs: cargo fmt, clippy with -D warnings, the full test suite, cargo-audit for vulnerability scanning, cargo-deny for license and supply-chain policy, Trivy for container scanning, Gitleaks for secret detection, CodeQL for static analysis, and OpenSSF Scorecard for security posture. That is not a typical solo developer setup. It's more thorough than plenty of team projects I've contributed to.

The test suite backs it up. 411 #[test] functions across 29 files. Proptest for parser fuzzing. Fuzz targets via cargo-fuzz with ClusterFuzzLite integration. Integration tests that spin up the actual binary and exercise all seven protocols. Playwright end-to-end tests for the web UI. Coverage measured at 61.5% via tarpaulin.

Security is taken seriously too. Credentials use Argon2id with zeroize to scrub secrets from memory after use. The token verification layer has a TTL cache so it's not re-hashing on every request. There are explicit TOCTOU race condition fixes in the storage layer and request deduplication for the proxy mode so concurrent pulls for the same image don't stampede the upstream registry.

What's rough? The web UI modules are untested. ui/api.rs (1,010 lines), ui/templates.rs (861 lines), and ui/components.rs (783 lines) have zero test coverage between them. That's 2,654 lines of code running the dashboard with no safety net. There's also a dead code problem: error.rs defines a full AppError type with an IntoResponse impl that's been sitting unused since v0.3. The comment says "wiring into handlers planned for v0.3" but they're on v0.4 now, and handlers still construct status code responses manually. The garbage collector has a subtler issue: collect_all_blobs scans all seven registries, but collect_referenced_digests only reads Docker manifests. Non-Docker blobs would look like orphans to the GC. None of these are deal-breakers, but they're the kind of gaps that matter as the project grows.

The Contribution

I was reading through the metrics code when I noticed detect_registry() had match arms for Docker, Maven, npm, PyPI, and Cargo, but Go and Raw requests fell through to an "other" catch-all. Every request to those two registries was invisible in Prometheus. The RegistriesHealth struct had the same gap: five fields for five registries, but Go and Raw weren't represented. The health endpoint would report them as down even when they were running fine.

The kicker was the test suite. There was already a test for Go registry path detection. It asserted that a Go module request should be labeled "other". The test was passing because it was checking for the wrong thing.

The fix was straightforward: add the missing match arms in detect_registry(), add the go and raw fields to RegistriesHealth and its construction in check_registries_health(), and fix the test to assert the correct label. PR #97.

Getting into the codebase was easy. The CONTRIBUTING.md lays out build and test commands clearly. The module structure maps directly to concepts: if you want to understand how Docker pushes work, you open src/registries/docker/. If you want metrics, you open src/metrics.rs. I found the bugs by reading, not by fighting the project layout. The whole thing compiled and tested cleanly on the first try.

This was the first external PR the project had ever received. The maintainer reviewed it, approved it, and merged it the same day. His comment:

"This is the very first community PR that NORA has ever received, and it means a lot. The fact that you not only noticed the missing Go and Raw registries in metrics, but took the time to write a clean fix with proper tests... Welcome to the team."

That response tells you something about whether this project has a future.

The Verdict

Nora is for teams that need artifact hosting without the Java tax. If you're running a small engineering team, managing a homelab, or working in an air-gapped environment where you can't reach public registries, a 32 MB binary that handles seven protocols on 100 MB of RAM is a compelling alternative to configuring Nexus heap flags.

The project's trajectory is strong. January was the initial scaffold with Docker support. February added six more protocols. March brought security hardening, proptest, integration tests, and a coverage push from 22% to 61.5%. April shipped v0.4 with mirror mode and air-gap support. That's a lot of ground to cover in three months, and the commit history tells a consistent story: conventional commits, one concern per commit, no monolithic dumps.

What would push Nora to the next level? Wire in the AppError type so error responses are consistent across registries. Fix the GC so it doesn't treat non-Docker blobs as orphans. Get some test coverage on the UI modules. And keep doing what's already working, because the foundation is solid.

Go Look At This

If you've ever winced at your artifact registry's memory usage, give Nora a look. The codebase is clean, the CI is thorough, and the maintainer merges good work the same day you submit it.

Star the repo. Try running it against your Docker workflow. If you want to contribute, the AppError wiring and the GC's Docker-only reference scanning are both waiting for someone to pick them up.

This is Review Bomb #12, a series where I find under-the-radar projects on GitHub, read the code, contribute something, and write it up. If you know a project that deserves more eyeballs, drop it in the comments.

This post was originally published at wshoffner.dev/blog. If you liked it, the Review Bomb series lives there too.

I Renamed All 43 Tools in My MCP Server. Here's Why I Did It Now.

Wes — Fri, 03 Apr 2026 23:53:10 +0000

Charlotte has 111 stars. That's not a lot. But it's enough that a breaking change will annoy real people.

I shipped one anyway.

The naming problem

When I started building Charlotte in February, I named every tool with a colon separator: charlotte:navigate, charlotte:observe, charlotte:click. It looked clean. It felt namespaced. Every tool call in every session used it.

The problem: the MCP spec restricts tool names to [A-Za-z0-9_.-]. The colon character isn't in that set. It never was. I either didn't check or didn't care at the time. The MCP SDK was lenient about it until v1.26.0, which started emitting validation warnings on every tool registration.

I had two options. Fix it now with 111 stars and a handful of active users. Or fix it later with more stars, more users, more documentation, more muscle memory, and more pain.

We renamed all 43 tools from charlotte:xxx to charlotte_xxx in a single commit. Breaking change. Documented in the changelog. Migration note in the release.

Here's the thing about MCP: clients discover tools dynamically at connection time. When an agent connects to Charlotte, it asks "what tools do you have?" and Charlotte sends the current list. The agent doesn't care what the tools were called yesterday. So for most users, the upgrade is invisible. The old names simply don't exist anymore and the new names appear automatically.

The people who get hit are the ones with custom prompts or configurations that reference tool names as strings. That's a small group right now. It will be a much larger group in six months.

Breaking changes are cheaper when you're small. That's the whole argument. Ship it early, pay the small cost, avoid the large one later.

What else is in 0.6.0

Batch form filling. This one matters for token economics.

Before 0.6.0, filling a 10-field contact form meant 10 separate tool calls: charlotte_type for each text field, charlotte_select for dropdowns, charlotte_toggle for checkboxes. Each call carries tool definition overhead (the MCP server sends its tool schemas on every API round trip). Ten calls at ~4,000 definition tokens each is 40,000 tokens just in overhead, before any actual content.

charlotte_fill_form takes an array of {element_id, value} pairs and fills everything in one call:

{
  "fields": [
    { "element_id": "inp-a3f1", "value": "Jane Smith" },
    { "element_id": "inp-b7c2", "value": "jane@example.com" },
    { "element_id": "sel-d4e8", "value": "Enterprise" },
    { "element_id": "chk-f9a0", "value": "true" }
  ]
}

One call. One set of definition tokens. The form is filled. It handles text inputs, textareas, selects, checkboxes, radios, toggles, date pickers, and color inputs. Type detection is automatic based on the element's role.

For a testing agent running form validation across 50 pages, this is the difference between 500 tool calls and 50. The token savings compound fast.

Lazy Chromium initialization. Charlotte used to launch a Chromium instance the moment the MCP server started. The problem: MCP clients like Claude Desktop and Cursor spawn all configured servers at startup, whether you're going to use them or not. If Charlotte is in your config but you're just writing code today, you had a headless browser burning RAM for nothing.

Now the browser launches on the first actual tool call. If you never browse, Chromium never starts.

Slow typing. charlotte_type gains slowly and character_delay parameters for character-by-character input. This sounds trivial until your agent tries to test a search-as-you-type field and the site's event handler only fires on individual keystrokes, not pasted text. Autocomplete, live validation, search suggestions. They all need real keystroke events.

Node.js 20 support. I was requiring Node 22 for no reason. No 22-only APIs were in use. Relaxing to >=20 opens Charlotte to the broader LTS user base.

The ASI bug, one last time

In v0.4.1, I found a bug where charlotte:evaluate silently returned null on multi-statement JavaScript. The cause was new Function('return ' + expr) combined with Automatic Semicolon Insertion. I wrote about it at the time.

I fixed it in evaluate.ts. Then found the same pattern in wait-for.ts and fixed it in 0.5.0.

0.6.0 found it a third time in pollUntilCondition, a utility function used by the wait system. Same bug. Same new Function('return ' + expr). Same migration to CDP Runtime.evaluate.

That's three separate files with the same broken pattern, discovered across three releases. Copy-paste bugs are persistent. If you find a pattern-level bug in your codebase, grep for every instance before you close the issue. I should have done that the first time.

7 strangers improved my code

When I started Charlotte in February, it was a solo project. 100% of commits from one person. An external evaluation in early March rated sustainability 2 out of 5 and flagged "97% single-developer commits" as the primary risk.

Six weeks later, seven people I've never met have merged code into Charlotte:

Teoman Yavuzkurt contributed three PRs: fixing the default viewport (800x600 was unrealistically small), solving a stale compositor frame bug in screenshots on SPA transitions, and fixing macOS symlink resolution in tests. Three different areas of the codebase. That's someone who read the code deeply enough to find problems across modules.

clawtom submitted two PRs: an O(1) lookup optimization for the snapshot store (replacing a linear scan with a Map index) and proper error logging for CDP failures in layout extraction. Both unsolicited. Both performance or reliability improvements that I hadn't prioritized.

Sandy McArthur, Jr. joined as a new contributor this cycle. Nuno Curado did the original security hardening back in February. kai-agent-free picked up the "read version from package.json" issue I had tagged as "good first issue." Nestor Fernando De Leon Llanos added the issue templates and community links.

I didn't recruit any of them. They found the project, read the code, and decided it was worth contributing to. The issue templates, the "good first issue" labels, the CONTRIBUTING guide, the test suite that gives contributors confidence their changes don't break things. All of that infrastructure exists to make contributing feel safe and worthwhile. It seems to be working.

The sustainability rating would look different today.

What's next

Charlotte is at 43 tools, 519 tests, and a 1.07:1 test-to-source line ratio. The structural tree view from 0.5.0 gives agents a full page map in under 2,000 characters. Iframe extraction handles embedded content. File output keeps large responses out of the context window. And now batch form fills collapse multi-step interactions into single calls.

The focus for the next cycle is the connect-to-browser feature: attaching Charlotte to an already-running Chrome instance instead of launching its own. This unlocks screen recording of agent sessions, live debugging, and the kind of demo videos that are worth more than any blog post.

Try it

npx @ticktockbent/charlotte@latest

Works with any MCP client: Claude Desktop, Claude Code, Cursor, Windsurf, Cline, VS Code, Amp.

GitHub | npm | Charlotte vs Playwright MCP

Open source, MIT licensed. If you're running browser-heavy agent workflows, I'd like to hear how it holds up.

Your System Is Not a State Machine

Wes — Fri, 03 Apr 2026 12:10:55 +0000

We were all taught the same thing. A system has states. It has transitions. Something happens, the system moves from State A to State B. You can draw it on a whiteboard. You can enumerate the possibilities. You can write tests for each branch.

That was true for a long time. It's not true anymore.

Do the math

Take a small transformer. 117 million parameters, each stored as a float32. The raw state space of just the weights is 2^(3.7 billion). The number of atoms in the observable universe is around 2^266.

That's before you add activations, attention matrices, the KV cache growing with every token. And that's one model sitting idle. Not a system. Not an architecture. Just one small model.

Now build something real. An orchestrator spawns four sub-agents. One is browsing a website. One is querying a database. One is calling an external API. One is doing a computation. Each has its own latency, its own failure modes, its own ability to return something you didn't expect.

What state is that system in?

You don't know. I don't know. Nobody knows, because the space of possible configurations is so absurdly vast that calling it "astronomical" is generous. You can't draw this on a whiteboard. You can't enumerate the branches. The flowchart is a lie.

It's not random either

Your first instinct might be to reach for probability. If we can't predict the exact state, maybe we can describe the distribution of likely states. Stochastic modeling. Markov chains. The math is right there.

But that framing is wrong too, because these systems aren't rolling dice. An agent returning a useful summary of a web page isn't a random event. It's the result of a goal-directed process that evaluated and corrected itself on a token-by-token basis across thousands of sequential decisions. The output is useful precisely because it isn't random.

So you're stuck in a third space. Not deterministic. Not stochastic. Something else.

Convergent but underdetermined

Here's the framing I keep coming back to.

An LLM doesn't select an output from a distribution and accept whatever comes up. Every token is an evaluation. The weights encode something like "given everything generated so far, what moves me closer to a coherent completion?" The model is steering. Continuously. At the lowest level of its operation.

That's already not a state machine. But zoom out.

Your orchestrator has four sub-agents running. Each one is internally converging toward its own useful output. The orchestrator is monitoring returns in real time, and each return reshapes how it evaluates the others. Agent 3's result might make agent 2's task irrelevant. Agent 1's failure might mean re-dispatching agent 4 with different parameters.

You have nested convergence loops running at different scales, different speeds, none following a predetermined path, all goal-directed. The system isn't transitioning between states. It's navigating toward coherence through a space that only reveals itself as the system moves through it.

The closest analogy isn't computer science. It's biology. A cell responding to chemical gradients isn't executing a flowchart or rolling dice. It's resolving toward a functional configuration through continuous interaction with an environment it can't fully predict.

Why this matters practically

If this framing is right, then designing agentic systems with state-machine thinking isn't just imprecise. It's architecturally wrong. You're imposing discrete checkpoints on a system whose fundamental operation is continuous convergence. You're fighting the nature of the thing.

The alternative might look something like designing around convergence envelopes. Not "what state should the system be in at step 3" but "what region of outcome space should this process be converging toward, and what boundaries should it not cross while getting there."

Under that model, an orchestrator isn't a state manager. It's a convergence auditor. Its job isn't to track which step the system is on. Its job is to monitor whether the system is still heading toward a useful result, and intervene when it drifts outside acceptable bounds.

I don't have the answers

I want to be clear that this is half a thought. I don't have a formal model. I don't have a replacement for the state machine abstraction that you can hand to a junior engineer and say "use this." I'm not sure one exists yet.

But I know the old model is broken. If you've tried to draw a flowchart for an agentic system and felt like you were lying, you were. The system you're building doesn't have states. It has trajectories. It doesn't transition. It converges.

Somebody smarter than me will figure out the formalism. I just wanted to point at the gap.

Anthropic Leaked Its Own Source Code and May Not Own It

Wes — Thu, 02 Apr 2026 12:33:39 +0000

On March 31st, Anthropic shipped version 2.1.88 of Claude Code to npm with a 60MB source map file that was supposed to stay internal. That file pointed to a zip archive on Anthropic's Cloudflare R2 bucket containing the entire TypeScript source. 1,900 files. 512,000 lines of code. The full architectural blueprint of one of the most commercially successful AI coding tools ever built.

Security researcher Chaofan Shou spotted it within hours. By the time Anthropic pulled the package, the codebase had been mirrored, forked over 41,500 times on GitHub, and archived on decentralized platforms that don't respond to takedown notices.

What followed was a 12-hour chain reaction that may have permanently changed the legal landscape for AI-generated code.

Anthropic's response was a DMCA blitz. GitHub disabled over 8,100 repositories. The original mirror and its entire fork network went dark. Lawyers moved fast.

But a Korean developer named Sigrid Jin moved faster. Jin, previously profiled by the Wall Street Journal as the heaviest Claude Code user in the world (25 billion tokens consumed), woke up at 4am and rewrote the entire core architecture in Python before sunrise. The repo hit 30,000 GitHub stars faster than any repository in GitHub history. He then rewrote it again in Rust.

Anthropic can't touch those rewrites. They're clean-room reimplementations. New code, new language, new creative expression. Copyright protects specific expression, not ideas, not architectures, not design patterns. Anyone can study how a system works and build their own version from scratch. That's how the entire software industry has always operated.

But the legal problems go deeper than clean-room rewrites. The DMCA takedowns themselves rest on a copyright claim that Anthropic's own public statements may have already destroyed.

The authorship problem

On March 18, 2025, the DC Circuit issued a unanimous opinion in Thaler v. Perlmutter holding that the Copyright Act requires human authorship. The case involved a computer scientist who tried to register copyright for artwork generated by his AI system. The court didn't just deny the registration on narrow grounds. It reasoned that the word "author" throughout the Copyright Act is structurally incoherent when applied to a non-human entity. Applying it to a machine would produce absurdities, like referring to a "machine's children" or a "machine's nationality." The Supreme Court declined to hear the appeal.

Some important caveats. That case was about visual art, not code. It dealt with a specific scenario where someone listed an AI as the sole author. It's one circuit, not a Supreme Court ruling. The court deliberately left the door open for "AI-assisted" works where a human contributes meaningful creative input.

The reasoning matters more than the narrow holding. The court established that "author" throughout the Copyright Act is structurally tied to human beings. That's not a ruling about art. That's a philosophical foundation that any future court addressing AI-generated code will find persuasive, even if it's not technically binding.

And then Anthropic's leadership walked up to that open door and welded it shut.

In January 2026, Boris Cherny, the head of Claude Code, posted on X that 100% of his code is written by Claude. No manual edits. Not even small ones. He shipped 22 pull requests in one day and 27 the next, each one "100% written by Claude." Across Anthropic, he said the figure is "pretty much 100%."

An Anthropic spokesperson softened that to 70-90% company-wide. For Claude Code specifically, about 90% of its code is written by Claude Code itself.

These aren't offhand comments. They're timestamped, attributed, reported by Fortune, The Register, CNBC, and others. They're discoverable evidence. And they directly undermine any claim of human authorship over the leaked codebase.

The court in Thaler left Anthropic a door. Their marketing team closed it.

You can't copyright an idea

There's a common response to this argument: "But the humans designed the system. They architected it. The AI just wrote the implementation."

This gets the law exactly backwards. Copyright protects specific expression, not ideas, not designs, not architectures. You can design a system all day long. That design isn't copyrightable. Patents can protect novel functional inventions, but that's a completely different legal regime with a completely different process and standard.

The part that copyright actually covers (the specific code) is the part Anthropic says the AI wrote. And the part the humans contributed (the design and architecture) is the part copyright doesn't protect.

So when someone rewrites the architecture in a different language, there's nothing to claim. The ideas are free. The original expression is AI-generated. And the new expression belongs to whoever wrote it.

The transitive authorship problem

Here's where it gets speculative, and where the implications extend far beyond one leaked npm package.

If 90% of Claude Code's source was written by Claude, then the training pipeline code that produces the next generation of Claude models was also substantially written by Claude. The model weights that come out of that pipeline are the output of an AI-authored system.

Can you copyright the product of a system that was built by AI?

The existing precedent only addresses direct AI outputs. Nobody has litigated whether a work produced by an AI-coded system inherits the copyright problem of the system that created it. But the logic is hard to avoid. If human authorship is the prerequisite for copyright, and the authorship chain passes through a substantially non-human link, the claim gets weaker at every generation.

Nobody knows where the line is. No court has addressed it. But every frontier AI company should be thinking about it, because the answer affects whether their core asset is protectable at all.

The moat that isn't

Trade secret is the last legal defense in this analysis. Trade secret protection doesn't require human authorship. It doesn't care who or what created the information. It only requires that the holder took "reasonable measures" to keep it secret.

Anthropic is not having a great month on that front either.

Days before the Claude Code leak, Fortune reported that descriptions of Anthropic's upcoming model (internally called "Mythos" or "Capybara") were sitting in a publicly accessible data cache along with close to 3,000 other files. Then the Claude Code source went out on npm. Two major exposures in one week.

If this ever went to court, opposing counsel would argue that Anthropic's operational security doesn't meet the "reasonable measures" threshold for trade secret protection. A single incident might be forgivable. A pattern is harder to defend.

The protections collapse one by one. Copyright requires human authorship, and Anthropic publicly says AI writes the code. Trade secret requires maintained confidentiality, and Anthropic keeps accidentally publishing things. Patent requires specific novel invention claims and a formal process, not something you can retroactively blanket over a leaked codebase. DMCA takedowns require a valid underlying copyright, and they only work on centralized platforms anyway.

What's left is practical barriers: the cost of compute, the difficulty of assembling training data, the head start of an established product, brand trust, enterprise relationships. Those are real advantages. But they're business moats, not legal ones. They can't be enforced in court. They erode as compute gets cheaper, as open-source models close the gap, and as competitors absorb architectural insights from leaks exactly like this one.

The product demo

There's an irony at the center of this whole story that's hard to overstate.

Anthropic built Claude Code. They told the world it was so good that their own engineers stopped writing code entirely. Then a packaging error exposed the source. The world's heaviest Claude Code user used what was almost certainly Claude to rewrite Claude Code in Python overnight. The result is legally untouchable, it's the fastest-starred repo in GitHub history, and it demonstrates exactly the capability Anthropic has been selling.

Anthropic's own product, used by Anthropic's own power user, to neutralize Anthropic's own IP. Made possible because the product is exactly as good as they said it was.

That's not a leak. That's a product demo.

What this means

The Claude Code leak is entertaining. The legal questions it surfaces are not. Every frontier AI company that uses its own models to write production code is building on the same unstable ground. The more they market AI autonomy to sell products, the more they undermine the legal frameworks that protect those products. Every press quote about AI writing 100% of the code is a future exhibit in a case they hope never gets filed.

The law hasn't caught up. Congress hasn't acted. The courts have addressed one narrow question in one circuit. But the trajectory is clear, and every company in this space is exposed to it.

The question isn't whether AI-generated code is copyrightable. The court already answered that. The question is whether anyone in the industry is willing to admit what that answer means for them.

I'm not a lawyer. This article is speculative analysis based on public reporting, public court opinions, and public statements by Anthropic leadership. If you're making business decisions about AI-generated IP, talk to an actual attorney.

Sources:

This post was originally published at wshoffner.dev/blog. If you liked it, the Review Bomb series lives there too.

Your Encrypted Backups Are Slow Because Encryption Isn't the Bottleneck

Wes — Thu, 02 Apr 2026 11:46:35 +0000

If you encrypt files before pushing them to backup storage, you've probably assumed the encryption step is what makes it slow. That's what I assumed too. Then I looked at the numbers. On any modern x86 chip with AES-NI, AES-256-GCM runs at 4-8 GB/s on a single core. ChaCha20-Poly1305 isn't far behind. The CPU is not the problem. The problem is that your encryption tool reads a chunk of data, encrypts it, writes it out, then reads the next chunk. It's serial. The disk sits idle while the CPU works, and the CPU sits idle while the disk works.

One person decided to fix that by applying the same async I/O technique that powers modern databases to file encryption. The result hits GB/s throughput on commodity NVMe hardware, and the whole thing is about 900 lines of Rust.

What Is Concryptor?

Concryptor is a multi-threaded AEAD file encryption CLI built by FrogSnot. It encrypts and decrypts files using AES-256-GCM or ChaCha20-Poly1305 with Argon2id key derivation, and it does it fast by overlapping disk I/O with CPU crypto using Linux's io_uring interface. It handles single files and directories (packed via tar), runs entirely in the terminal, and installs with cargo install concryptor.

73 stars. One month of focused development. A six-file core with 67 tests. It deserves more.

The Snapshot


Project	Concryptor
Stars	73
Maintainer	Solo (FrogSnot)
Code health	Clean architecture, 67 tests, clippy and fmt now enforced in CI
Docs	Excellent README with honest perf analysis and full format spec
Contributor UX	Fresh templates and CI, small codebase, easy to navigate
Worth using	Not yet for production (author's own disclaimer), but the architecture is real

Under the Hood

The centerpiece is a triple-buffered io_uring pipeline in engine.rs. The idea is simple: keep three sets of buffers rotating through three stages. While buffer A's encrypted contents are being written to disk by the kernel, buffer B is being encrypted in parallel by Rayon worker threads, and buffer C's plaintext is being read from disk. Every component stays busy. Nothing waits.

The implementation is tighter than you'd expect from a month-old project. Each io_uring submission queue entry carries bit-packed metadata in its user_data field: the low two bits identify which buffer slot, bit 2 flags read vs. write, and the upper bits store the expected byte count for short-I/O detection. When completion queue entries come back, the pipeline routes them to per-slot counters without any hash lookups or allocations. The whole loop runs num_batches + 2 iterations to let the pipeline drain cleanly at the end.

The file format is designed around O_DIRECT. Every encrypted chunk is padded to a 4 KiB boundary. The header is exactly 4096 bytes (52 bytes of data plus KDF parameters plus zero padding). Buffers are allocated with explicit 4096-byte alignment via std::alloc. This lets Concryptor bypass the kernel's page cache entirely, talking directly to NVMe storage via DMA. It's the same technique databases use to avoid double-buffering, and it's a big part of why the throughput numbers are real.

The security model is more careful than I expected from a solo hobby project. The full 4 KiB header is included as associated data in every chunk's AEAD tag, so modifying any header byte invalidates all chunks. There's a TLS 1.3-style nonce derivation scheme where each chunk's nonce is the base nonce XOR'd with the chunk index, preventing nonce reuse without coordination. A final-chunk flag in the AAD prevents truncation and append attacks. The 4032 reserved bytes in the header are authenticated too, so you can't smuggle data into them. The test suite covers chunk swapping, truncation (two variants), header field manipulation, reserved byte tampering, KDF parameter tampering, and cipher mismatch. These aren't afterthought tests. Someone thought about the threat model.

What's rough? The project is Linux-only. io_uring doesn't exist on macOS or Windows, and there's no fallback backend. If you try to build it on a Mac you'll get errors that don't explain why. The README is upfront about the experimental status, which is honest and appreciated, but it does mean you shouldn't point this at anything you can't afford to lose yet. The rand dependency is still on 0.8 (0.10 is current), and until recently clippy warnings and formatting drift had been accumulating unchecked. None of these are architectural problems. They're the kind of rough edges you get when one person is focused on making the core work first.

The Contribution

CONTRIBUTING.md asks you to run clippy and cargo fmt before submitting, but CI only ran cargo test. No enforcement. The result was predictable: 7 clippy warnings had accumulated across engine.rs and header.rs, and formatting had drifted in almost every source file.

I fixed all seven lints. Three were manual div_ceil reimplementations (the (a + b - 1) / b pattern that Rust now has a method for), one was a min/max chain that should have been .clamp(), one was a manual range check, and two were too_many_arguments warnings on internal pipeline functions where every parameter is essential and restructuring would just add noise. I also wired up KdfParams::DEFAULT via struct update syntax to eliminate a dead-code warning, ran cargo fmt --all, and added clippy and fmt checks to the CI workflow so they stay clean going forward.

Getting into the codebase was straightforward. Six files, clear responsibilities: engine.rs handles the pipeline, crypto.rs handles primitives, header.rs handles the format, archive.rs handles tar packing. The code is dense but not clever. You can follow the pipeline loop without needing to hold too much in your head at once. I had the PR ready in under an hour.

PR #10 is open as of this writing.

The Verdict

Concryptor is for people who encrypt files regularly and want it to be fast. If you're backing up to cloud storage, encrypting disk images, or just moving sensitive data between machines, the throughput difference between a serial encryption tool and a pipelined one is real. On NVMe, it's the difference between saturating your drive and leaving most of its bandwidth on the table.

The project is early. One maintainer, one month old, Linux-only, self-labeled experimental. It could stall. But the commit history tells a story of deliberate progression: the initial mmap approach was replaced with io_uring in the same day, security hardening followed within a week, the format was upgraded to v4 with full header authentication, and directory support landed before the first month was out. That's not hobby-project pacing. That's someone building something they intend to use.

What would push Concryptor to the next level? A fallback I/O backend for macOS and Windows would be the single biggest improvement. Even a plain pread/pwrite loop, slower than io_uring but functional, would open the project to most Rust developers who want to try it. Stdin/stdout streaming for pipe composability would help too. And the rand 0.8 to 0.10 migration is a real breaking change that Dependabot can't auto-fix. That's a contribution waiting to happen.

Go Look At This

If you care about I/O performance, encryption, or io_uring, Concryptor is worth reading. The codebase is small enough to understand in an afternoon, and the pipeline implementation is one of the cleaner io_uring examples I've seen in the wild.

Star the repo. Try encrypting a large file and watch the throughput. If you want to contribute, the rand 0.8 to 0.10 migration is sitting there waiting for someone to pick it up.

This is Review Bomb #11, a series where I find under-the-radar projects on GitHub, read the code, contribute something, and write it up. If you know a project that deserves more eyeballs, drop it in the comments.

This post was originally published at wshoffner.dev/blog. If you liked it, the Review Bomb series lives there too.

Your Package Manager's Installer Doesn't Know Fish Exists

Wes — Tue, 31 Mar 2026 14:58:19 +0000

You find a new CLI tool on GitHub. The README looks good. You scroll to "Installation" and see the magic one-liner: curl -sSL https://... | sh. You run it. The script downloads a binary, drops it somewhere sensible, and adds it to your PATH by appending a line to your .bashrc.

Except you use fish. And fish doesn't understand export PATH=. So the binary is on your disk, but your shell can't find it. You open the install script, figure out where it put things, and manually write set -gx PATH ~/.local/bin $PATH into your config.fish. You've done this before. You'll do it again.

This is a small problem. But it's a revealing one. The kind of developer who installs CLI tools from GitHub release pages, who tries new package managers, who runs fish instead of bash because they actually thought about their shell choice, that's your target user. And your installer just told them you didn't think about them.

What Is parm?

parm is a binary package manager for GitHub Releases, written in Go by alxrw. You give it a repo (parm install owner/repo) and it finds the latest release, picks the right binary for your platform, downloads it, and symlinks it onto your PATH. No root access, no system package manager, no registry to maintain. It queries GitHub directly.

It handles updates (parm update), version pinning (parm pin), removal (parm remove), and has a search command that queries GitHub's API. It's pre-release (v0.1.6) but functional, with a clear roadmap toward v0.2.0. About 138 stars and one very active maintainer.

The interesting design choice: there is no curated package registry. Homebrew has formulae. asdf has plugins. parm has GitHub's API and your judgment. The README is upfront about this: "Users are responsible for vetting packages." That's a tradeoff, and it's a deliberate one.

The Snapshot


Project	parm
Stars	~138 at time of writing
Maintainer	Solo developer, actively releasing
Code health	Clean Go with standard stack (Cobra, Viper, go-github), 32% test file ratio
Docs	Good README with usage table, disclaimers, and package compatibility guide
Contributor UX	Merged my PR next-day with "lgtm"
Worth using	Yes, for grabbing CLI tools from GitHub without the Homebrew ceremony

Under the Hood

parm follows the standard Go CLI layout. Commands live in cmd/ (one file per subcommand, Cobra-based). Core business logic lives in internal/core/ with separate installer and updater packages. The GitHub client lives in internal/gh/ using go-github v74. Configuration is TOML-based via Viper.

The dependency list is heavier than you'd expect for a tool this focused. Eight direct dependencies: go-github for the API, cobra and viper for CLI and config, semver for version comparison, mpb for progress bars, gopsutil for platform detection, filetype for binary type detection, oauth2 for GitHub authentication. None of these are unreasonable individually, but it's a lot of moving parts for "download a binary and symlink it." Compare to tools like ubi or eget that do the same thing with fewer dependencies. That said, the extra weight buys real features: progress bars, proper semver handling, and platform detection that works on all three major OSes.

The architecture within internal/ is well-separated. The installer handles asset selection (matching your OS and architecture against release asset names), archive extraction (tar, zip, and raw binaries), and symlink management. The manifest tracks what's installed, where, and at what version. The verification package handles binary validation. Each concern has its own package and its own tests. 19 test files out of 59 Go files is a decent ratio for a project this age.

What the Go code gets right: cross-platform support. The build targets include linux/darwin/windows on both amd64 and arm64. Platform detection via gopsutil picks the correct release asset. The asset name matching is smart enough to handle the inconsistent naming conventions across GitHub repos (linux-amd64, Linux_x86_64, linux-x64, etc.).

What the Go code doesn't cover: the shell. The install script (scripts/install.sh) that handles the curl | sh onboarding path was bash/zsh-only. It wrote export PATH=... into .bashrc, .zshrc, or .profile. Fish, the third most popular interactive shell, was completely unsupported. The binary would install, but the user's shell couldn't find it. For a tool whose entire value proposition is "install any program from your terminal," having the install script fail on a common terminal is a gap.

The Contribution

Issue #49 reported the fish problem in February 2026, and it was on the v0.2.0 roadmap. I picked it up.

The fix was about 70 lines added to scripts/install.sh. Fish uses set -gx PATH instead of export PATH=, and its config file lives at ~/.config/fish/config.fish (or wherever $XDG_CONFIG_HOME points). The implementation detects fish by checking whether the config file exists or whether fish is available on PATH, resolves the config path respecting XDG conventions, creates the directory if needed, and writes the PATH entry in fish syntax. It also handles GITHUB_TOKEN persistence (parm uses this for GitHub API rate limits) with the fish equivalent. If a user has both bash and fish installed, both configs get updated. The deduplication logic (grep for the bin directory before appending) follows the same pattern the script already used for bash/zsh.

Getting into the codebase took about fifteen minutes. The install script was self-contained, and the existing bash/zsh code was a clear template for the fish additions. PR #51 was approved with "lgtm" and merged the next day with "Merged, thank you for your contribution!" No review rounds, no changes requested. Clean in, clean out.

The Verdict

parm is for developers who install CLI tools from GitHub and are tired of the manual download-extract-chmod-symlink dance. If you've ever navigated to a GitHub releases page, scrolled through thirty assets to find the right one for your platform, downloaded it, extracted it, figured out which binary inside the tarball is the one you actually want, chmod'd it, and moved it to somewhere on your PATH, parm automates all of that.

The no-registry approach is either a feature or a concern depending on your threat model. There's no vetting, no review process, no curated list. You point parm at a repo and trust the maintainer's releases. The README is honest about this. For tools you already trust (ripgrep, fd, bat, delta), it's faster than Homebrew. For tools you've never heard of, you're on your own.

The project has momentum. Version pinning just shipped. Fish shell support (that's us) just landed. Windows shim support is on the roadmap. The maintainer is responsive and the codebase is clean enough that contributions land quickly. What would push parm further: a parm doctor command that validates your setup, shell completions for the major shells, and better error messages when a release doesn't have a compatible asset. But the core works today, and it's already replaced a chunk of my manual workflow.

Go Look At This

If you install CLI tools from GitHub, try parm. parm install junegunn/fzf and see how it feels. If you use fish, the installer now works thanks to PR #51.

Star the repo. Check the open issues. The v0.2.0 milestone has clear feature requests if you want to contribute.

This is Review Bomb #10, a series where I find under-the-radar projects on GitHub, read the code, contribute something, and write it up. If you know a project that deserves more eyeballs, drop it in the comments.

This post was originally published at wshoffner.dev/blog. If you liked it, the Review Bomb series lives there too.

The Blackwall Between Your AI Agent and Your Filesystem

Wes — Mon, 30 Mar 2026 12:38:57 +0000

Every AI coding agent you run has the same permissions you do. Claude Code, Cursor, Codex, Aider. They can read your SSH keys, write to your shell config, and run any command your user account can. We accept this because the alternative is setting up Docker containers and dealing with volume mounts and broken toolchains every time we want an agent to help with a project.

That trade-off has always felt wrong to me. Not because I think my AI agent is malicious, but because I know it executes code from dependencies I haven't read, runs shell commands it hallucinated, and sometimes rms things it shouldn't. The blast radius of a mistake is my entire home directory.

I went looking for something between "full trust" and "Docker wrapper," and I found a project named after the barrier between humanity and rogue AIs in Cyberpunk 2077.

What Is greywall?

greywall is a container-free sandbox for AI coding agents. It uses kernel-level enforcement on Linux (bubblewrap, seccomp, Landlock, eBPF) and Seatbelt profiles on macOS to isolate your agent's filesystem access, network traffic, and syscalls. Deny by default. No Docker, no VMs. One binary, four direct dependencies.

It ships with built-in profiles for 13 agents (Claude Code, Cursor, Codex, Aider, and more), and it has a learning mode that traces what your agent actually touches and generates a least-privilege profile from the results. The project is three weeks old, has about 110 stars, and the sole maintainer merges external PRs within hours.

The Snapshot


Project	greywall
Stars	~109 at time of writing
Maintainer	Solo (tito), mass-committing daily
Code health	17,400 lines of Go, 151 tests, clean layered architecture
Docs	ARCHITECTURE.md, CONTRIBUTING.md, 18 doc files, a full docs site
Contributor UX	Merged my PR same-day, CI catches lint, good first issues labeled
Worth using	Yes if you run AI agents on Linux or macOS

Under the Hood

The codebase is ~17,400 lines of Go with only four direct dependencies: cobra for CLI, doublestar for glob matching, jsonc for config with comments, and x/sys for kernel syscalls. Everything else is hand-rolled against the kernel API.

On Linux, greywall stacks five security layers, each covering what the others can't:

Bubblewrap namespaces (linux.go, 1,642 lines) handle the heavy lifting. In DefaultDenyRead mode, the sandbox starts from an empty root filesystem (--tmpfs /) and selectively mounts system paths read-only and your project directory read-write. Network isolation drops all connectivity, then three bridge types restore controlled access: a ProxyBridge for SOCKS5 traffic, a DnsBridge for DNS resolution, and a ReverseBridge for inbound port forwarding. All of them relay over Unix sockets via socat.

Seccomp BPF (linux_seccomp.go) blocks 30+ dangerous syscalls: ptrace, mount, reboot, bpf, perf_event_open. If your kernel doesn't support seccomp, greywall skips it and continues. This graceful fallback pattern repeats at every layer.

Landlock (linux_landlock.go) adds kernel-level filesystem access control. It opens paths with O_PATH and uses fstat to avoid TOCTOU races between checking a path and applying a rule to it. It handles ABI versions 1 through 5, stripping directory-only rights from non-directory paths to avoid EINVAL from the kernel.

eBPF monitoring traces violations in real time via bpftrace. Learning mode runs strace under the hood, captures every file your agent touches, and collapses the results into a reusable profile.

On macOS, greywall generates Seatbelt profiles for sandbox-exec with deny-by-default network rules and selective file access via regex patterns. macOS actually has a cleaner security model here. Seatbelt supports both allow and deny rules with regex, so you can write "allow ~/.claude.json*, deny everything else in home." Linux's Landlock is additive-only. Once you grant write access to a directory, you can't deny individual files inside it. This is the project's most interesting architectural tension, and it surfaces as a real bug: issue #62, where programs that do atomic file writes (create a temp file, rename over the target) break because the temp file and the target live on different filesystems inside the sandbox.

Command blocking (command.go, 524 lines) doesn't just match command names. It parses shell syntax: pipes, &&, ||, semicolons, subshells, and quoted strings. echo foo | shutdown gets caught. bash -c "rm -rf /" gets caught. It's more parser than filter.

The architecture makes sense for what it's doing. Each layer has a clear file, clear responsibility, and a fallback path. The build tags (//go:build linux, //go:build darwin) keep platform code separated without runtime conditionals. The test suite has 151 tests across 13 files covering command blocking, Landlock rules, Seatbelt profile generation, learning mode, and config validation. For a three-week-old project, that's unusually disciplined.

What's rough: the project is pre-1.0 and moving fast. Eight releases in 23 days. The DefaultDenyRead mode is ambitious and still has edge cases (the atomic writes bug, WSL DNS issues, AppArmor conflicts with TUN devices). The documentation is comprehensive but assumes you already know what bubblewrap and Landlock are. If you're new to Linux security primitives, the onboarding curve is steep.

The Contribution

Issue #5 asked for a greywall profiles edit command. The learning mode generates JSON profiles and saves them to ~/.config/greywall/learned/, but there was no way to edit them without hunting for the file path and hand-validating the JSON. The maintainer wanted an editor command that validates on close.

Getting into the codebase was straightforward. The existing profiles list and profiles show commands were right there in main.go, following the standard cobra subcommand pattern. The config validation was already built: config.Load() parses JSON (with comments via jsonc) and runs Validate(). I just needed to wire up an editor loop.

The implementation opens the profile in $EDITOR (splitting on whitespace to support code --wait and emacs -nw), saves the original content for rollback, and after the editor closes: detects no-change exits, validates the JSON, and on failure prompts to re-edit or discard. Discard restores the original file. About 95 lines total.

CI caught two lint issues I couldn't test locally (the project requires Go 1.25, I had 1.22): gocritic flagged an append to a different variable, and gofumpt wanted explicit octal syntax (0o600 instead of 0600). Pushed the fix, and the maintainer merged the whole thing within hours of submission. Approved the code immediately, just asked for the lint fix. That's a three-week-old project with a same-day merge for a first-time contributor. PR #64.

The Verdict

greywall is for anyone running AI coding agents who wants more than trust and less than Docker. If you use Claude Code or Cursor on a machine with real credentials, SSH keys, or cloud configs, this fills a gap that nothing else does at this weight class.

The project is young and moving fast. Three weeks old, 109 stars, eight releases. The maintainer is clearly using it daily and fixing bugs as they surface. The contributor experience is excellent: labeled issues, fast merges, CI that catches real problems. The Landlock limitation (no per-file deny inside a writable directory) is a genuine technical constraint that will shape the project's future, and the maintainer's detailed write-up on issue #62 shows someone who understands the problem deeply and isn't reaching for shortcuts.

What would push greywall to the next level? Solving the atomic writes problem would unblock a lot of real-world usage. A guided setup wizard (instead of requiring users to understand profiles and config files) would lower the barrier for non-security-minded developers. And more built-in profiles for common development workflows beyond AI agents could widen the audience. But the foundation is solid, the security model is sound, and the code is cleaner than most projects ten times its age.

Go Look At This

If you run AI agents on your dev machine, go install greywall and try greywall -- claude or greywall -- cursor. The built-in profiles work out of the box. If you want tighter control, run greywall --learning -- <your-agent> to generate a profile from actual usage, then greywall profiles edit to fine-tune it.

Star the repo. Try the learning mode. If something breaks in your setup, open an issue. The maintainer responds fast and the codebase is navigable enough that you might end up fixing it yourself.

This is Review Bomb #9, a series where I find under-the-radar projects on GitHub, read the code, contribute something, and write it up. If you know a project that deserves more eyeballs, drop it in the comments.

This post was originally published at wshoffner.dev/blog. If you liked it, the Review Bomb series lives there too.

Why Your SFTP Transfer Is Stuck at 2 MB/s (and the Fix Is a Protocol from 1983)

Wes — Sun, 29 Mar 2026 16:14:56 +0000

Two minutes to copy a 274 MB file to a VM running on localhost. Not over the internet. Not to a cloud instance across the country. Localhost. The same machine, loopback, zero network latency.

That was the experience a user reported in issue #290 on cubic, a lightweight CLI for managing QEMU/KVM virtual machines. The maintainer reproduced it, traced the problem to the upstream russh-sftp crate, and posted a comment asking if anyone had ideas about where the bottleneck was. I did. The answer turned out to be a protocol design decision that limits every Rust project using this crate to about 2 MB/s on file transfers, regardless of how fast the link is.

The fix was to stop using SFTP entirely and fall back to a simpler, older protocol.

What Is cubic?

cubic is a CLI tool for creating and managing lightweight virtual machines on Linux and macOS. Think of it as the middle ground between running Docker containers and spinning up full VMs in libvirt. You run cubic create myvm --image debian and get a cloud-init provisioned VM with SSH access, a dedicated disk, and port forwarding. cubic ssh myvm drops you into a shell. cubic scp file.tar.gz myvm:~/ copies files in. It's about 7,000 lines of Rust, built on QEMU/KVM with cloud-init for provisioning.

Under 40 stars. The maintainer (rogkne) commits daily and reviews external PRs within hours.

The Snapshot


Project	cubic
Stars	~37 at time of writing
Maintainer	Solo developer, committing daily
Code health	~7,000 lines of clean Rust, 104 unit tests, clap + thiserror + tokio
Docs	Good README, CONTRIBUTING.md with conventional commit rules
Contributor UX	Fast reviews, specific feedback, merged shell completions PR in multi-round review
Worth using	Yes, if you want lightweight VMs without libvirt's complexity

Under the Hood

cubic has a clean layered architecture. CLI commands live in src/commands/ (one file per subcommand, clap with derive macros). Business logic lives in src/actions/. The instance model, serialization (TOML and YAML), and storage live in src/instance/. Image fetching and distro definitions live in src/image/. SSH and file transfer live in src/ssh_cmd/.

The dependency list is lean. Four crates handle the heavy lifting: russh for SSH connections, russh-sftp for file transfers, clap for CLI parsing, and reqwest for image downloads. Everything else is standard library or small utility crates. The Cargo.toml is not trying to be clever.

One pattern that caught my eye: the project is async internally (tokio, russh) but sync at the CLI boundary. An AsyncCaller struct wraps a tokio multi-threaded runtime and exposes a call() method that blocks on a future. Every command creates one, runs its async work through it, and returns a sync result. It's simple and it works. No async bleeding into the CLI layer.

The image pipeline is solid. cubic fetches cloud images from distro mirrors, verifies SHA-256/SHA-512 checksums against the upstream checksum file, shows a progress bar during download, and caches images locally. Adding a new distro means adding one entry to the DISTROS static in image_factory.rs. Rocky Linux was added in a recent PR following this exact pattern.

The rough edges are in the SSH layer. The SFTP implementation delegates to russh-sftp, which turned out to be the source of the performance bug. The progress bar during file transfers is coupled to the async read wrapper (AsyncTransferView), which works but makes it hard to swap the underlying transfer mechanism without touching the view layer. The test coverage is good for models and serialization but thin for the SSH and QEMU interaction code, which is typical for tools that depend on external services.

The Contribution

The performance issue (#290) reported that cubic scp transferred files at roughly 2 MB/s on loopback. I dug into the russh-sftp internals to find out why.

The answer is in how russh-sftp implements AsyncWrite. Every call to poll_write() creates a one-shot channel, sends an SFTP write request, and blocks until the server responds with an acknowledgment. One write in flight at a time. No pipelining. The SFTP protocol (RFC 4253) explicitly supports pipelining: clients can send many write requests with different IDs and collect the responses asynchronously. OpenSSH's sftp client does exactly this with 64 outstanding requests by default. russh-sftp doesn't. The upstream issue (#70) has been open since June 2025 with no fix.

For a 274 MB file at the default 255 KB max write size, that's roughly 1,075 round-trips, each waiting for an ACK. Even on loopback, the per-request overhead adds up to minutes.

Wrapping the writer in a BufWriter wouldn't help. It coalesces small writes into larger ones, but each poll_write() still blocks on the ACK. You'd go from many small round-trips to fewer large ones, but the bottleneck is the same.

The fix was to bypass SFTP for single-file transfers and use SCP instead. SCP is a much simpler protocol: open an SSH exec channel with scp -t <path>, send a one-line header (C0644 <size> <filename>\n), stream the raw bytes, send a null byte, done. No request IDs, no per-packet ACKs during data transfer. Just a header and a byte stream.

I added a new scp.rs module (~170 lines) that implements SCP upload and download over a raw russh channel via channel.into_stream(). The async_copy function in russh.rs now detects single-file host-to-guest transfers and routes them through SCP. Directory copies and guest-to-guest transfers still use SFTP. Guest-to-host tries SCP first and falls back to SFTP if it fails (which it will for directories).

The review was thorough. The maintainer requested eight changes, all cleanups: use BufReader.read_line() instead of byte-by-byte loops, add error message prefixes, reuse the ack-reading function in the download path, validate the end-of-transfer marker byte. All reasonable, all addressed. He also asked (politely) whether the PR was AI-generated. I explained my workflow and he was satisfied. The PR went through two review rounds over 12 days and merged. PR #311.

The Verdict

cubic is for developers who want lightweight VMs without the weight of libvirt or the constraints of Docker. If you're testing deployment scripts, need an isolated Linux environment for a project, or just want to spin up a Debian box and SSH into it without thinking about Vagrant files, this does the job.

The project is young (v0.19.0, solo maintainer) but the trajectory is good. New distros get added regularly. The contributor experience is above average: specific review feedback, no ego, merged with thanks. The maintainer is clearly using this tool daily and fixing things as they surface.

What would push cubic to the next level? The SFTP performance fix helps, but the bigger opportunity is user experience. A cubic init that scaffolds a project config file, better error messages when QEMU isn't installed, and a Homebrew formula for macOS users would all lower the barrier. The foundation is clean. It just needs more people kicking the tires.

Go Look At This

If you manage VMs from the command line, try cubic. cubic create myvm --image debian and you're running in under a minute. If you've been burned by slow file transfers to VMs before, the SCP fix in PR #311 is worth a look for the protocol analysis alone.

Star the repo. The codebase is small enough to read in a sitting, and there are open issues at every difficulty level.

This is Review Bomb #8, a series where I find under-the-radar projects on GitHub, read the code, contribute something, and write it up. If you know a project that deserves more eyeballs, drop it in the comments.

This post was originally published at wshoffner.dev/blog. If you liked it, the Review Bomb series lives there too.

Finding Blocking Code in Async Rust Without Changing a Single Line

Wes — Wed, 18 Mar 2026 12:52:00 +0000

You know the symptoms. Latency spikes under load. Throughput that should be higher. A Tokio runtime that's doing less work than it should be, and you can't see why. Something is blocking a worker thread, starving the other tasks, and nobody's throwing an error about it.

The standard advice is tokio-console. Add console-subscriber to your dependencies, rebuild, redeploy, reproduce the problem, and look at task poll times. It works well. It also requires code changes, a rebuild, and a redeployment, which means it's not what you reach for when staging is melting and you need answers now.

The other option is perf. Attach to the process, collect stack traces, generate a flamegraph, and interpret a wall of unsymbolized frames. It'll tell you everything that's happening on every thread. The signal-to-noise ratio for "which Tokio worker is blocked and by what" is not great.

There's a gap between those two. A tool that attaches to a running Tokio process, finds the blocking code, and shows you the result, without touching your source.

What Is hud?

hud is an eBPF-based profiler for Tokio applications, built by cong-or. You give it a process name or PID, and it hooks into the Linux scheduler via eBPF tracepoints to detect when Tokio worker threads experience high scheduling latency. When a worker is off-CPU longer than a configurable threshold (default 5ms), hud captures a stack trace, resolves it against DWARF debug symbols, and shows you what was on the stack. No recompile, no instrumentation, no code changes.

It runs as a real-time TUI or in headless mode with Chrome Trace JSON export. About 147 stars. For the problem it solves, it should have more.

The Snapshot


Project	hud
Stars	~147 at time of writing
Maintainer	Solo developer, 178 commits and 15 releases in 3 months
Code health	Clean workspace, good module boundaries, well-documented internals
Docs	Five dedicated doc files (architecture, development, exports, troubleshooting, tuning)
Contributor UX	Both PRs merged within minutes. Would contribute again.
Worth using	Yes, if you run Tokio on Linux and have ever wondered "what's blocking?"

Under the Hood

The project is a Rust workspace with three crates. hud-ebpf (~400 lines, #![no_std]) runs inside the kernel: a sched_switch tracepoint for off-CPU detection and a perf_event hook sampling at 99 Hz for stack traces. hud-common (~330 lines) defines the shared types that cross the kernel/userspace boundary. hud (~8,700 lines) is the userspace application: event processing, DWARF symbol resolution, a ratatui TUI, and Chrome Trace export. The whole thing builds with cargo xtask build-ebpf for the eBPF side and a regular cargo build for userspace.

The interesting engineering starts with worker discovery. Tokio worker threads need to be identified before hud can filter events to just the runtime. This turns out to be harder than it sounds. The first problem is /proc's 15-character TASK_COMM_LEN limit, which truncates tokio-runtime-worker-0 to tokio-runtime-w. The second is custom runtimes: if you called thread_name("my-pool"), the default prefixes don't match. hud handles this with a 4-step fallback chain: explicit prefix via --workers, default Tokio prefixes, stack-based classification (sample for 500ms and look for Tokio scheduler frames), and a largest-thread-group heuristic. That last one just picks the biggest group of threads following a {name}-{N} naming pattern.

Frame classification has its own complexity. Rust statically links dependencies into the main binary, so being "inside the executable" doesn't distinguish your code from tokio's code from serde's code. hud uses a 3-tier classifier: file path patterns first (.cargo/registry/ means third-party, .rustup/toolchains/ means stdlib), then function name prefixes (tokio::, std::, hyper::), then memory range as a last resort. The TUI highlights user code in green and dims everything else.

The README is refreshingly honest about limitations. It measures scheduling latency, which is a symptom of blocking, not the blocking itself. It captures the victim's stack, not the blocker's. System CPU pressure can cause false positives. The comparison table with tokio-console and Tokio's built-in detection doesn't oversell hud. It positions it as a triage tool: narrow down the suspects, then dig deeper with instrumentation if needed.

The rough spots are minor. Test coverage is decent for the core modules (classification, worker discovery, hotspot analysis) but thin for the event processing pipeline and TUI rendering. The project is three months old and iterating fast (15 releases), so some gaps are expected. The docs make up for it: five dedicated files covering architecture, development workflow, export format, troubleshooting, and threshold tuning. That's unusual care for a project at this scale.

The Contribution

I submitted two PRs, targeting different layers of the stack.

The first was test coverage for the blocking pool filter. Tokio's spawn_blocking creates threads that share the same Inner::run function at the bottom of their stacks as actual worker threads. This is because Tokio bootstraps workers through the blocking pool mechanism. The distinguishing factor is that workers also have scheduler::multi_thread::worker frames higher up the stack. The is_blocking_pool_stack() function filters on this distinction to suppress spawn_blocking noise from the TUI.

This function went through four release iterations (v0.4.2 through v0.5.0) in response to a bug report where spawn_blocking tasks were showing up as false positives. The maintainer shipped multiple fix releases in rapid succession. But the function had zero test coverage. I added 9 tests covering the core logic: genuine blocking pool stacks, genuine worker stacks, empty stacks, partial matches, closure wrappers, and two realistic deep-stack scenarios. I bundled in doc fixes where TROUBLESHOOTING.md listed 3 worker discovery steps instead of the actual 4, and where the README said "x86_64 architecture" while every other doc said "x86_64/aarch64."

The second PR was an eBPF fix. The get_cpu_id() function in the kernel-side code always returned 0, with a TODO comment saying "aya-ebpf doesn't expose bpf_get_smp_processor_id directly yet." It does. The helper is re-exported through pub use gen::* in the aya-ebpf helpers module, but it's #[doc(hidden)], so it never shows up in the generated docs. The fix was adding an import and replacing the stub with the real call. Three lines changed. Every exported trace event was silently reporting the wrong CPU core.

Both PRs were merged within minutes. The codebase was easy to navigate: clear module boundaries, descriptive file names, good internal documentation. The eBPF side requires nightly Rust and bpf-linker, which adds setup friction, but the build process is documented and worked on the first try.

The Verdict

hud is for Rust developers running Tokio on Linux who want to understand what's blocking their runtime without adding instrumentation. The workflow is sudo hud my-app and you're looking at results. If you've ever stared at a flamegraph trying to figure out which of those Tokio frames is yours, hud does that filtering for you.

The project is young (three months) and solo-maintained, but the trajectory is strong. The commit history shows a developer who responds to bug reports with same-day fix releases, who writes honest documentation about tradeoffs, and who merges external contributions without friction. The codebase is clean enough that I was reading eBPF kernel code within an hour of cloning the repo. That doesn't happen by accident.

What would push hud further? More metrics in the TUI (per-CPU breakdown, timeline visualization of blocking events), broader async runtime support beyond Tokio, and CI integration for the headless export mode (pipe the JSON through jq for regression detection). The architecture supports all of this. The Metric approach is indirect by design, and the project is honest about that. What it offers in return is zero-friction access to information that would otherwise require a rebuild.

Go Look At This

If you run Tokio on Linux, try hud. Download the pre-built binary, point it at a running process, and see what shows up. If nothing does, your runtime is clean. If something does, you just saved yourself a rebuild.

Star the repo. Here are the tests I added for the blocking pool filter and the eBPF fix for the cpu_id stub. Both small, both merged.

This is Review Bomb #7, a series where I find under-the-radar projects on GitHub, read the code, contribute something, and write it up. If you know a project that deserves more eyeballs, drop it in the comments.

This post was originally published at wshoffner.dev/blog. If you liked it, the Review Bomb series lives there too.