DEV Community: Arun Soman

Idiomatic Concurrency in Rust: Stop Writing Accidental Microservices

Arun Soman — Fri, 25 Sep 2026 14:44:36 +0000

The Problem

A common pattern in Rust async applications looks like this:

Use the Tokio runtime.
Break the app into small “microservices” that each run in their own task.
Give each service a run function that loops over a tokio::select!.
Add a branch for CancellationToken::cancelled().
Wire the root token to ctrl_c.
Let services communicate through channels: mpsc, oneshot, watch, broadcast.
When one service fails, bubble the error up and let the whole application fail.

At first this feels clean. Each component is isolated. Each has a lifecycle. Each has a cancellation path.

But then the bugs start.

One service returns Err. Nothing cancels its siblings. The sibling is still waiting on a channel receive. The sender is gone. The receiver parks forever. The app does not crash. It does not shut down. It just hangs.

The developer adds more select! branches. More cancellation checks. More boilerplate. The same failure mode appears somewhere else.

The real issue is not Tokio. The real issue is lifecycle ownership.

This is not microservices. It is a modular monolith with in-process tasks. That label matters because it changes the failure model. There is no network partition. There is no independent fate. There is shared fate. If one component dies, the process is still alive, and the rest of the system may be stuck waiting for it.

tokio::spawn detaches. Dropping a JoinHandle does not cancel the task. A failed or panicked task does not automatically cancel its siblings. A select! inside one component does not fix global lifecycle semantics.

The result is exactly the pain described: redundant code, unhandled paths, stopped services, and other services waiting forever for a response that will never come.

The Solution

Treat the application as a supervised set of in-process tasks in a monolith.

Own task lifetimes. Define failure policy. Make every long-lived await cancel-aware. Keep simple things simple.

1. Use structured concurrency

Use JoinSet to own child tasks. Use TaskTracker if you need to track graceful shutdown completion.

Do not detach tasks unless you truly mean fire-and-forget.

Rust and Tokio do not have built-in structured concurrency. JoinSet and TaskTracker approximate it. TaskTracker is for close() + wait() — “everyone is done.” It does not propagate failure. Pair it with JoinSet when you care why things ended.

2. Use one root cancellation token

Create one root CancellationToken from ctrl_c. Give each component a child token.

Child tokens propagate parent → child only. A failing child does not cancel the root. The supervisor must observe the failure via join_next and call root.cancel().

3. Decide failure policy per component

Fail-fast? Restart with backoff? Degrade? Shut down cleanly?

Only restart if the component is safe to restart. Otherwise, bubble the error up and shut down the whole application deliberately.

4. Make every long-lived await cancel-aware

This is the actual fix for “nobody responding.”

CancellationToken does not wake a task parked in rx.recv().await. If a service is blocked on a channel receive that is not racing the token, the drain loop hangs forever.

Every long-lived await must either:

select! on token.cancelled(), or
rely on sender-drop so recv() returns Err / None.

Owners should drop their senders during shutdown.

5. Use the right channel for the job

watch for state/config
broadcast for events
mpsc for commands
oneshot for replies
bounded channels for backpressure

Unbounded channels hide backpressure bugs.

6. Do not make everything a task

Only spawn when you need independent lifetime, async I/O, or state isolation. If it is just a function call, call it. Message passing everywhere makes simple things harder.

7. Know your runtime defaults

#[tokio::main] is multi-threaded by default. worker_threads defaults to the number of CPUs. #[tokio::test] is current-thread by default.

More worker threads do not make blocking calls safe. Use spawn_blocking for blocking work.

Example Shape

use std::time::Duration;
use tokio::task::JoinSet;
use tokio_util::sync::CancellationToken;

async fn service_a(token: CancellationToken) -> anyhow::Result<()> {
    // Every long-lived await should be cancel-aware.
    loop {
        tokio::select! {
            _ = token.cancelled() => return Ok(()),
            // _ = rx.recv() => { ... }
        }
    }
}

async fn service_b(token: CancellationToken) -> anyhow::Result<()> {
    loop {
        tokio::select! {
            _ = token.cancelled() => return Ok(()),
            // _ = rx.recv() => { ... }
        }
    }
}

#[tokio::main]
async fn main() -> anyhow::Result<()> {
    let shutdown = CancellationToken::new();
    let mut tasks = JoinSet::new();

    tasks.spawn(service_a(shutdown.child_token()));
    tasks.spawn(service_b(shutdown.child_token()));

    let mut first_error: Option<anyhow::Error> = None;

    tokio::select! {
        _ = tokio::signal::ctrl_c() => {}
        res = tasks.join_next() => {
            match res {
                Some(Ok(Ok(()))) => {
                    // Normal exit. For a forever-service this is unexpected.
                    // For finite work, you may want to continue instead of cancelling.
                }
                Some(Ok(Err(e))) => first_error = Some(e),
                Some(Err(join_err)) => first_error = Some(join_err.into()),
                None => {}
            }
        }
    }

    shutdown.cancel();

    // Drain remaining tasks. Don't use `?` here; collect/log instead.
    let drain = async {
        while let Some(res) = tasks.join_next().await {
            match res {
                Ok(Ok(())) => {}
                Ok(Err(e)) => {
                    if first_error.is_none() {
                        first_error = Some(e);
                    }
                }
                Err(join_err) => {
                    if first_error.is_none() {
                        first_error = Some(join_err.into());
                    }
                }
            }
        }
    };

    if tokio::time::timeout(Duration::from_secs(5), drain)
        .await
        .is_err()
    {
        tasks.abort_all();
        while tasks.join_next().await.is_some() {}
    }

    if let Some(e) = first_error {
        return Err(e);
    }

    Ok(())
}

Notes:

JoinSet forces one output type across all spawns. Heterogeneous services need a common error type like anyhow::Error or a mapped enum.
After cancellation, a task may legitimately return Err. Log it instead of double-reporting the first error.
If you need actor semantics, crates like ractor exist. Only reach for them if you actually need actors.

Rules of Thumb

Rename the mental model: supervised tasks in a monolith, not microservices.
Own task lifetimes with JoinSet.
Use one root CancellationToken and child tokens.
Remember: cancellation flows parent → child, not child → parent.
Make every long-lived await cancel-aware.
Define failure policy per component.
Use bounded channels for backpressure.
Do not spawn everything.
Keep plain function calls as plain function calls.
Heartbeats are for separate processes. In-process, await task completion.
Coroutines will not solve lifecycle or failure semantics.

One-Liner

Stop writing accidental microservices: treat Tokio tasks as supervised components in a monolith, own their lifetimes with JoinSet, cancel through a root token, and make every await cancel-aware.

I Built a Flight Recorder for AI Agents (Because “What Did It Do?” Shouldn’t Be a Mystery)

Arun Soman — Fri, 25 Sep 2026 12:49:20 +0000

Introduction

AI agents now run your shell, edit your files, and ship to prod—often with zero humans in the loop. The tooling explosion of 2026 made software agent-native… and produced exactly one new question:

"What did my agent actually do while I was away?"

AGENTBOX is the answer. Strap a flight recorder to any command or agent—no SDK, no code changes, no cloud—and get a tamper-evident tape of everything it did: every tool call, every file touched, every URL hit.

I recently came across this project and it solves a problem I didn't know I had. Let me break down why it matters and how you can try it in 10 seconds.

The Problem: Agents Have Root. Who's Watching?

If you've used Claude Code, Codex CLI, or any autonomous coding agent, you know the drill: you give it a task, it runs for a while, and then something breaks. Maybe it deleted a file, maybe it hit an unexpected API, or maybe it just did something… odd.

Traditional logging doesn't cut it. You need a forensic record—a black box that captures everything in a way that can't be quietly edited afterward.

That's exactly what AGENTBOX does.

What Is AGENTBOX?

AGENTBOX is a zero-dependency, 100% local flight recorder for AI agents. It wraps any command or agent and produces a session file (JSONL) containing every event: tool calls, shell commands, file operations, URLs hit, even human keystrokes.

The tagline says it all:

"npm test for your agent's behavior · tamper-evident · 100% local · zero dependencies"

Key Features

🎥 The Replay: Security Footage for Your Terminal

agentbox replay opens an interactive scrubber—like security footage for your terminal.

Timeline bar with markers: ▲ tool call · $ shell · ✎ file op · i human input
Play / pause / speed (1×–64×)—skip to 00:47.2, the moment it dropped the table
Every keystroke the human typed is on the tape (HUMAN lines)
--headless renders a static frame for CI, GIFs, and GitHub

🔒 The Tape Cannot Lie

Every event is hash-chained. Each line commits to the previous one:

{ i, t, type, data, prev, hash }
hash = sha256(prev ‖ i ‖ t ‖ type ‖ data)

Edit one line—even a single character—and agentbox verify pins the exact event:

$ agentbox verify
✗ hash mismatch at event 7 — event was tampered with or forged

That makes the session file evidence, not a log: post-mortems, compliance, "the agent did it / no it didn't" arguments in PRs—settled.

🛡️ Secrets Never Land on the Tape

Every string written to a session file is scrubbed before it is hashed and appended. The chain commits to the redacted form, so the original secret is not recoverable from the file, the receipt, or a shared clip. Default patterns catch API tokens, cloud keys, auth headers, private keys, and connection strings.

You can extend the redaction patterns via AGENTBOX_REDACT_EXTRA or project-local config.

Quickstart: See It in 10 Seconds

# 1. Watch a scripted agent get recorded
npx agentbox-cli demo

# 2. Wrap anything — your agent, a script, any CLI
agentbox wrap -- claude "refactor auth.js"
agentbox wrap --name eval-run -- python evaluate.py --suite prod

# 3. Read the tape
agentbox list        # all sessions
agentbox receipt     # newest session, one page
agentbox replay      # scrub the footage
agentbox verify      # tamper check

# 4. Share a moment, not a dump
agentbox clip --from 30 --to 75   # → self-contained .clip.html

# 5. Or skip the wrapper entirely — passive mode
agentbox init claude    # hooks → every claude session, recorded
agentbox mcp -- npx -y @modelcontextprotocol/server-everything

No install, no config, no accounts. Sessions land in ./.agentbox/sessions/ next to your repo—commit them if you want receipts in git history.

Passive Mode: No Wrapper Needed

Wrapping is for flights you know about in advance. Adapters are for the ones you don't.

For Claude Code, you can merge hooks into your settings with a single command:

agentbox init claude         # merges hooks into .claude/settings.json
agentbox init claude --local # .claude/settings.local.json instead
agentbox init claude --remove # clean uninstall

From the next session on, Claude Code quietly feeds every event to agentbox hook claude.

The Receipt: Real Output from `agentbox demo`

Every session ends with a one-page flight receipt. Here's real output from the demo:

⬢ M A Y D A Y  R E C E I P T
┌──────────────────────────────────────────────────┐
│ session          demo-deploy                      │
│ command          node examples/fake-agen…         │
│ started          2026-09-24 18:40:52              │
│ duration         5.4s                             │
│ exit code        0 (clean landing)                │
├──────────────────────────────────────────────────┤
│ tool calls       7                                │
│ · bash("git status --s…                           │
│ · write(src/deploy.sh …                           │
│ · bash("./deploy.sh --…                           │
│ … +4 more                                         │
│ shell commands   1                                │
│ · rm -rf /tmp/old-buil…                           │
│ files touched    2 · 1 written · 1 edited         │
│ · src/deploy.sh wrote                             │
│ · config.yaml edited                              │
│ urls hit         1                                │
│ output volume    808 B across 17 lines            │
│ stderr lines     0                                │
│ humans consulted 0 (unsupervised flight)          │
├──────────────────────────────────────────────────┤
│ events recorded  19                               │
│ tamper chain     sha256 · intact                  │
└──────────────────────────────────────────────────┘
uneventful flight. the best kind.

Why This Matters

Agentic tooling is moving fast, and trust is the bottleneck. Before you let an agent run terraform apply or kubectl delete, you need to know you can audit exactly what happened—and prove it wasn't altered.

AGENTBOX fills that gap with a beautiful, minimal approach:

No cloud dependency — everything stays on your machine
No SDK integration — wrap any command, or use passive hooks
Cryptographically verifiable — hash-chained events make tampering detectable
Human-readable — receipts and replay make the data accessible, not just forensic

It's the kind of tool that should be in every agent operator's toolkit.

Getting Started

Head over to the repo and try the demo:

👉 github.com/arunsoman/agentbox

npx agentbox-cli demo

If you're running agents in production (or just want to know what they're really doing), give it a star and let the author know what you think.

Have you used AGENTBOX or a similar auditing tool for your agents? Share your experience in the comments below.

Nirdosha: a language that treats correctness, durability, and identity as compiler problems

Arun Soman — Sat, 22 Aug 2026 18:03:51 +0000

Every language promises safety. Nirdosha's example suite reads like an audit of that promise — each file exists to demonstrate one guarantee, with the rejected programs living in the test suite.

Ownership without a garbage-collector personality. box moves on assignment; a second use is a compile-time error. & borrows read without taking. The same affine discipline applies to every resource handle — files, sockets, database connections, message queues, sandboxes — so "forgot to close it" is a type error too.

nir
`fn consume(b: box i64) -> i64 {
    return *b
}

fn main() {
    let a: box i64 = box 10
    print(consume(a)) // moves `a`; using it again would not typecheck
}`

Effects you don't annotate until you want to. Effects are inferred by default; if you declare effect(pure) and the body does I/O, that's a compile error, not a linter warning.
Concurrency that can't alias. spawn's arguments move exactly like a normal call's, so two concurrent computations can never hold the same affine value. Channels are multi-producer, multi-consumer; sending a box moves it across the wire. The same chan/send/recv syntax works across an OS-process boundary via sandbox — the transport changes, the source doesn't.
Transactions as a language construct. transact generates an idempotency key before anything runs, records intent durably, verifies before committing, compensates on failure, and leaves non-terminal entries for replay after a crash. If the retry budget runs out, it traps rather than guessing.
Identity in the type system. Nirdosha is an OIDC relying party: it validates JWTs against JWKS and never mints identity tokens. A requires(role: "physician") function's value is unobtainable without proof — acquire is the only path, and calling it directly is a compile-time error.
Math for the mission. Dense linear algebra, a linear Kalman filter, and WGS84 geodesy are builtins. The benchmark suite isn't "solve one huge system" — it's 200,000 predict/update steps of a constant-velocity tracker, because that's the actual workload shape.
Apps, not toys. nirdosha serve runs real applications from the examples folder: an online store (SQLite-backed CRUD, checkout against a mock payment gateway, order events on a message queue), a revenue-assurance tool (reconciliation as a single SQL statement, dashboards derived by naming convention), and an enterprise trade-finance platform with maker-checker approval governance. The web UIs are derived from structs via emit-ui — not hand-written.
The examples are candid about what the language can't do yet — no string concatenation, affine handles without borrowing, fixed entity counts in actor simulations. That honesty is the point: every limitation is documented in the file that demonstrates the feature.
try it out