DEV Community: Irfan Wani

Sidekick part 5 (finale): what four parts added up to

Irfan Wani — Wed, 07 Oct 2026 14:06:07 +0000

Sidekick Part 5 (finale): what four parts added up to

This series started with a bet — a terminal companion on your hardware, no account, no bill — and spent four parts defending it piece by piece. This is the last planned part: what it all added up to, what I deliberately left out, and where new parts come from here.

The arc in four sentences

Part 1: overview — sk is chat, voice, and 19 tools running locally via Ollama, with cloud as opt-in per key rather than structural. The decision: deterministic grounding over prompt instructions.

Part 2: grounding — small models ignore rules, so code injects facts before the model sees the prompt: hardware snapshot, auto directory listings, fetched URLs, recency-triggered search. Every past hallucination is locked in test_eval.py as a regression test.

Part 3: voice — push-to-talk transcribed on your CPU by int8 faster-whisper, recordings deleted after every take. Privacy as architecture: no upload path exists.

Part 4: safety — approvals at dispatch, a deception-assuming shell denylist, egress deny-by-default, one trust chokepoint, and sk audit as proof. Every ceiling documented with issue numbers instead of oversold.

What the parts share

Three through-lines showed up in every installment whether I planned them or not. First, local-first is load-bearing, not branding — grounding works because sysinfo is real, voice privacy holds because there's no server, audit means something because the ledger is yours. Remove "runs on your hardware" and most claims in this series collapse. That's how you test a principle: delete it and see what breaks.

Second, honesty about ceilings is a feature. The series documented guessed caps, dumb regex triggers, an unfenceable shell-curl path, and a 980-line neighbor file (that was another project, but same instinct). Readers trust "here's where it breaks" more than "it just works," and future maintainers — including me — need the ceilings written down more than the victories.

Third, everything is sedimentary. Grounding exists because instructions failed. Egress rules exist because prompts failed. Eval tests exist because answers failed in production. Nothing here was designed top-down; it accreted around real failures, each layer dated by the bug that forced it.

What's deliberately missing

Memory and skills, the background daemon, the MCP server beyond a mention, providers and spend caps, the TUI internals. Not oversights — scope control. Each deserves the same treatment (mechanism, tradeoff, breakage) and each gets a part when there's something real to say about it.

The deal going forward

This series is complete as planned. But software doesn't stand still: when new features land, or old ones break interestingly, they'll show up here as new parts — same contract (evidence, mechanism, what broke), same numbering continuing upward. A series that ends is better than one that trails off; a series that resumes for cause is best.

Thanks for reading all five. Go run it locally: uv tool install sidekick-agent and sk init.

Built by the Sidekick community. Repo: https://github.com/Faisal-Fayaz/sidekick

Sidekick part 4: safety as honestly labeled ceilings

Irfan Wani — Mon, 05 Oct 2026 05:51:34 +0000

Sidekick Part 4: safety as a stack of ceilings, each honestly labeled

Most agent safety I've seen is a paragraph in the system prompt: "be careful with destructive commands." Small models ignore system prompts — we established that in Part 2. So Sidekick's safety model assumes the model will disobey and enforces the boundaries in code instead. This is Part 4: approvals, hard refusals, egress control, and the audit ledger — plus the ceilings, stated rather than hidden.

Layer 1: approval sets, enforced in the dispatcher

Reads auto-run. Everything else sits in APPROVAL_TOOLS — writes, deletes, general shell — gated behind inline [y/N] prompts, with /yolo, /confirm, and /readonly modes. Plan mode and read-only mode go further with deny sets (PLAN_DENIED_TOOLS, READONLY_DENIED_TOOLS) enforced at dispatch. The comment in the code says the quiet part out loud: hard gates exist because "prompt text alone does not stop disobedient models." Even exec, the read-only shell, dies in these modes — inventory commands have no place in a mode that promises zero side effects.

Layer 2: the shell denylist that assumes deception

SHELL_BLOCK_PATTERNS in src/sk/tools/shell.py reads like a museum of bad days: recursive rm at /, ~, $HOME, bare system roots — matched after quote-stripping, so rm "-rf" / doesn't slip through. mkfs, dd to devices, fork bombs by structural shape, chmod -R 777 /, curl … | sh. And crucially, patterns match against de-obfuscated variants, never the raw string alone — because attackers obfuscate and careless users quote.

My favorite entry: find / -delete. It destroys trees without any rm flag, so it sailed past the rm pattern for a while. The comment admits it: already blocked in exec, but the shell tool had no coverage. Every gap found becomes a pattern; every pattern carries the story of the gap.

Layer 3: egress deny-by-default

src/sk/egress.py splits network traffic into operator-configured (your model endpoint, MCP servers you set up — always allowed) and model-chosen (a page the model decided to read, a URL a provider handed back — the destinations an attacker reaches through prompt injection). The second category is deny-by-default: empty allowlist means no network, wildcards like * are refused at parse time, and every decision lands in the ledger.

And then the ceiling, in the module docstring: this covers fetches through the agent's tools. A shell command running curl is not covered — "a shell string denylist cannot be a network boundary." The README states it explicitly so the control is not oversold. I respect this enormously: most projects hide their ceilings; this one documents them with issue numbers (#288, #329).

Layer 4: trust fences + the ledger

src/sk/trust.py funnels five independent injection paths (repo docs, web text, search results, @path inlines, MCP results) through one chokepoint — because a fence applied at only some paths "is not a boundary." Markers are deliberately stable rather than random nonces, since rotating markers would destroy the prompt-cache discipline the test suite locks in; invisible-character smuggling is handled by a sanitizer instead. Same honesty here too: "prompt framing is advisory, not a control" — fences raise the floor; deny rules give the guarantee.

Everything lands in sk audit: tool runs, approve/deny decisions, local-vs-egress rows. The ledger is the point where safety becomes proof — the same trail from Part 1's privacy claim.

What broke

The find -delete gap above. The shell-curl ceiling, which no pattern layer can fix and which is therefore documented instead of patched. And the general lesson across all four parts of this series: each layer exists because the layer above it failed in production — instructions failed, so grounding; refusals failed, so injection; per-tool prompts nagged, so plan review up front. Safety here is sedimentary. That's the whole series: overview, grounding, voice, safety. Memory, skills, and the daemon are future parts if you want them.

Built by the Sidekick community. Repo: https://github.com/Faisal-Fayaz/sidekick

Sidekick part 3: voice that never leaves the machine

Irfan Wani — Sat, 03 Oct 2026 12:24:15 +0000

Sidekick Part 3: voice that never leaves the machine

Cloud voice assistants have a dirty secret everyone knows and nobody says: your voice goes to a server. Every "hey assistant" is an upload. Sidekick's sk talk does the whole loop on your hardware — OS-native capture, local transcription, transcript dropped editable into the prompt. Part 3 of this series is how that pipeline works, and the unglamorous bugs inside it.

The pipeline in 30 seconds

Enter starts recording, Enter stops it. Underneath: the OS-native recorder (arecord/ALSA on Linux, sox/CoreAudio on macOS, ffmpeg fallback) writes 16kHz wav, then local faster-whisper transcribes it — int8 quantized, CPU-only, model downloaded once and cached:

if model_size not in _model_cache:
    _model_cache[model_size] = WhisperModel(model_size, device="cpu", compute_type="int8")
model = _model_cache[model_size]
segments, _ = model.transcribe(wav_path, beam_size=5)
text = " ".join(s.text.strip() for s in segments).strip()
if not text:
    raise RuntimeError(
        "heard only silence — speak louder/closer, or run `sk mic-test` to check levels"
    )

No hard dependencies: arecord ships with the OS, sox/ffmpeg come via brew, and faster-whisper installs itself into the running environment on first use (uv pip install faster-whisper, with a "still not importable — restart and retry" path for the inevitable edge case). In the TUI, ctrl+g or the mic pill does the same thing. --stt-model base trades accuracy for speed when words vanish.

One technical decision: errors written for the speaker, not the log

Look at the failure messages: "recording is nearly empty — mic may be muted, run sk mic-test." "Heard only silence — speak louder/closer." Each one names the likely physical cause and the next command. That came from watching real sessions: transcription failures are almost never model failures, they're room failures — muted mics, wrong devices, whisper-quiet laptops. So sk mic-test records three seconds and returns a verdict — silent, quiet, good — with peak dB and an alsamixer hint when it's the OS mixer lying to you.

The privacy side is structural, not promised. Recordings are temp files deleted after every take — there's nowhere for them to accumulate because nothing in the pipeline has a server to send them to. The strongest privacy guarantee is an architecture with no upload path.

What broke

Two favorites from src/sk/voice.py. First: Python 3.13 removed audioop, so mic-level measurement needed a hand-rolled PCM stats function with struct — including a comment about endianness placement in format strings that reads like someone lost an afternoon to it.

Second, the spawn guard. faster-whisper's dependency tree can start a multiprocessing child, and on macOS recycled file descriptors made CPython reject the spawn with ValueError: bad value(s) in fds_to_keep. The fix monkeypatches spawnv_passfds to sanitize the fd list — dedupe, sort, drop closed ones — installed once and only around STT work. And because callers reduce failures to one-line messages, which hid the genuine crash site, full tracebacks now append to ~/.sidekick/voice-errors.log. The bug that teaches you your error reporting is broken is always two bugs.

Part 4 goes into the tool safety model: 19 tools, approval gates, hard refusals, and the audit ledger.

Built by the Sidekick community. Repo: https://github.com/Faisal-Fayaz/sidekick

Sidekick part 2: grounding beats instructions

Irfan Wani — Fri, 02 Oct 2026 05:04:34 +0000

Sidekick Part 2: grounding beats instructions — facts injected before the model sees the prompt

Small local models have a habit nobody warns you about: they ignore system-prompt rules. Tell a 3B model "always check the hardware before recommending," and it will confidently recommend anyway — from vibes. Our eval harness still carries the scar: an early version answered a hardware question with generic LLM advice, scoring 3/10 on our own quality check. Another told a user their ~/neural-hangar directory "does not exist." It existed. A third refused to fetch a URL at all: "I can't browse."

Three failures, one root cause: we were asking the model to go get facts. In Part 2 of this series, here's the design that fixed it — deterministic grounding in src/sk/agent.py, where code resolves facts and injects them before the model ever sees the prompt.

The four injectors

1. Real system snapshot, every turn. The system prompt carries a live hardware snapshot — CPU, RAM, GPU, disk — rendered fresh per turn. A mandatory rule sits next to it: if the question is about the user's device, the model MUST still call sysinfo so the trace shows grounding. Belt and suspenders: injected facts for correctness, a tool call for an auditable trail.

2. Auto local context. If your message mentions ~/anything, a regex pass lists that directory and reads its package.json/README.md/pyproject.toml — capped at 3 paths, ~2000 chars each:

def _auto_local_context(text: str) -> str:
    """Deterministic grounding: if user mentions ~/X, list it + read manifests.
    This does NOT rely on the model calling tools — it injects facts so the
    model cannot hallucinate 'does not exist'."""
    paths += re.findall(r"(~/[\w\-./~]+)", text)
    ...
    listing = tool_list_dir(p)
    chunks.append(f"[path {p} -> {exp}]\n{listing[:1500]}")

The model never gets the chance to say "does not exist" — the listing is already in context. This directly killed the ~/neural-hangar hallucination.

3. Auto web context. Same idea for URLs: up to 2 URLs fetched, 3000 chars each — a deliberate cap protecting the 8k context window of small models. The "I can't fetch URLs" refusal died here, backed by a never-refuse rule.

4. Auto search context. Explicit "search the web for X" phrasing triggers a search whose top-5 hits get injected. More interesting: recency markers ("latest," "right now," "this week") trigger it automatically — unless the question is local (mentions ~/, "my device," "can I run"), in which case search stays out of the way. That local-vs-world branch matters: searching the web for "can I run this on my GPU" would actively make the answer worse.

Two guardrails wrap all of it. Tool honesty: the model may report results ONLY from tool-role messages — never invent search results, file contents, or prices, and say plainly when it didn't call the tool. Untrusted fences: anything captured from outside (files, web pages, search results) arrives in fenced blocks marked DATA, never instructions — repo docs can teach conventions, but nothing in a fence can order a command, read a secret, or exfiltrate. That's the prompt-injection defense, structural rather than wished-for.

What broke (the eval receipts)

tests/test_eval.py opens with a comment listing real bad answers seen live, each now a locked regression test — 40 and counting:

3/10 generic advice with no grounding → the sysinfo + never-guess rules
"does not exist" hallucination → auto local facts
"I can't fetch URLs" refusal → auto web facts + never-refuse rule
silent write execution → the approval gate

The pattern: every quality failure became an offline test asserting on prompt assembly, tools, and gates — no Ollama needed. Quality bugs you can reproduce in CI stay fixed; ones you can't, don't.

The honest limitation: caps are guesses. Three paths, two URLs, 3000 chars — tuned for 8k-context models, not derived from anything. On larger-context models we're likely under-feeding. And regex triggers are dumb by design: ~/ in a joke about home directories still lists a directory. Cheap, predictable, occasionally silly — I'll take that over clever and surprising in the prompt path.

Part 3 goes into the voice pipeline: OS-native capture, local faster-whisper transcription, and why recordings are temp files deleted after every take.

Built by the Sidekick community. Repo: https://github.com/Faisal-Fayaz/sidekick

Sidekick part 1: a terminal companion on your hardware

Irfan Wani — Thu, 01 Oct 2026 04:51:43 +0000

Sidekick: a terminal companion that runs on your hardware (Part 1: overview)

Every AI coding assistant I used had the same deal: create an account, pipe your code through someone's cloud, watch the meter run. The models kept getting better, but the shape never changed — your files leave your machine, always.

Sidekick is the opposite bet: a local-first terminal companion you talk to — chat, voice, and 19 tools — running on your own hardware. No cloud account required, no API bill by default. Your files, memory, and voice never leave your machine unless you hand it a key. This is Part 1 of a series: the overview. Later parts go deep on grounding, voice, and the tool safety model.

What it actually is

One command, sk, with several surfaces sharing everything:

Fullscreen TUI (sk) — streaming Markdown answers, slash-command autocomplete with fuzzy filtering, sessions drawer, model picker, themes, push-to-talk mic pill
Plain REPL (sk chat) — for dumb terminals, screen readers, broken TUIs
Voice (sk talk) — Enter to record, Enter to stop; transcribed on your CPU by local faster-whisper, never uploaded
Single-shot runs (sk run "task") — scriptable agent runs with --json, --plan, --bg for background jobs
MCP server (sk mcp) — all 19 tools over stdio, so Claude Desktop and friends can use it too

Real captures from the app running headless — slash autocomplete, then a grounded answer with live system data:

Sessions, compacting, and review all work across surfaces:

Underneath: long-term memory with full-text search, todos, shell history, background daemon with calendar schedules, skill packs in SKILL.md format — and an audit ledger (sk audit) that logs every tool run, approval, and network egress so you can prove your code never leaked. That last one is the point most cloud tools structurally can't match: their product is your code; ours never sees it.

One technical decision: grounding beats instructions

The design bet I'd defend: deterministic grounding injected before the model sees the prompt — your paths, sysinfo, URLs, resolved as code, not described in prose. Small local models routinely ignore system-prompt rules, but they can't argue with facts already in context. Same instinct elsewhere: FTS5 full-text search over vector embeddings (zero dependencies, instant, no embedding server eating RAM on a 4GB box), and capability profiles where known models declare their tool protocol instead of paying a probing round-trip to discover it.

Bring your own key when you want frontier models — one guided sk connect flow, a dozen providers plus any OpenAI-compatible endpoint, spend caps per session, keys chmod 600 and masked. Local stays the default; cloud is opt-in per key, never structural.

Safety is a feature, not a banner

Reads auto-run. Writes, deletes, and shell need approval; shell hard-refuses rm -rf /, mkfs, device writes, and fork bombs even if you approve them. URL fetching blocks localhost and private IPs. Multi-tool turns with destructive actions get one plan review up front instead of nag-per-tool. Plan mode (/plan, F4) blocks writes entirely for research sessions, and /rewind undoes agent file edits.

Try it

uv tool install sidekick-agent[voice]  # global `sk`, STT included
sk init     # guided first-run: hardware → model → verify
sk          # fullscreen chat — start here

Local path needs Ollama (ollama serve, pull qwen2.5-coder:7b for smarts or llama3.2:3b for speed). 834 tests pass with no Ollama needed. In Part 2 I'll go inside the grounding pipeline — what gets injected, when, and what broke along the way.

Built by the Sidekick community. Repo: https://github.com/Faisal-Fayaz/sidekick

I built a 3D system design simulator to stop hand-waving architectures

Irfan Wani — Tue, 29 Sep 2026 17:29:31 +0000

I built a 3D system design simulator to stop hand-waving architectures

System design interviews and architecture reviews share a problem: we draw boxes and arrows on a whiteboard and assert things like "the cache absorbs most of this" or "we'll just auto-scale." Nobody checks. I wanted a toy where you can actually watch those claims break — so I built one.

Give it a try: https://system-design-simulator-navy.vercel.app/

What it is

System Design 3D Simulator is a web app: you compose infrastructure out of components (DNS, CDN, load balancers, rate limiters, API gateways, app servers, WebSocket gateways, databases with shardable primaries, read replicas, caches, search clusters, queues, workers, object storage, monitoring), wire them together in a 3D scene, dial up traffic, and watch throughput, p50/p99 latency, error rate, availability, queue depth, and cost respond in real time.

It also ships one-click interview architectures — URL shortener, social feed with fan-out on write, chat with WebSockets, CDN-heavy video platform — plus demand patterns (steady, viral spikes, daily cycle, growth ramp) so you can capacity-plan for peaks instead of averages.

The stack is React 19 + Vite 8 + Three.js via React Three Fiber, with Zustand holding all simulation state and Lucide for icons. Four files do most of the work: store.js (881 lines — the whole simulation engine), Scene.jsx (980 lines — 3D models, failure visuals, packet animation), ControlPanel.jsx (388 lines), and componentInfo.js (458 lines).

One technical decision: a tick-based capacity model with magic numbers up front

The core of the simulation is a fixed-tick loop (TICK_S = 0.8 seconds simulated per tick) where every component type gets a per-node capacity constant:

const APP_RPS_PER_NODE = 150;
const LC_EFFICIENCY = 1.08;      // least-connections spreads better
const CDN_OFFLOAD_PCT = 0.6;
const DB_LOAD_CAP = 220;         // load units per primary before saturation
const CB_DROP_FRACTION = 0.35;   // requests shed while breaker is open
const MQ_BUFFER_CAP = 8000;      // buffered msgs per broker before drops
const WORKER_DRAIN_PER_NODE = 200;

The tradeoff is obvious: these numbers are invented, not measured. A real CDN offloads far more than 60% for cacheable content; a real app server's RPS depends on what the handler does. But the point of the tool was never absolute accuracy — it's relative behavior. Kill the only load balancer and you get a total outage. Add read replicas and the primary stops saturating. Switch the LB strategy from round robin to least-connections (that 1.08 efficiency factor) or consistent hashing and you can watch the hot-partition trap appear. The constants being rough doesn't matter as long as the relationships between components behave plausibly, and keeping them as named constants in one place means anyone can tune them toward realism later.

The same thinking applies to the cost model: every component has a monthly $/node figure, cost per 1M requests is derived live — and dead nodes still bill. That last detail is deliberate. Idle capacity costing money is the whole argument for auto-scaling, and the simulator lets you toggle auto-scaling and watch scaling events fire.

Chaos was the fun part

Each node accumulates damage under sustained overload and eventually crashes; auto-heal restarts it, modeled loosely on Kubernetes liveness probes. A circuit breaker opens when the primary DB saturates and sheds read pressure until recovery. You can kill any node by hand, or roll the chaos monkey. The failure I keep coming back to: search queries with no search cluster fall back to expensive scans on the app servers and DB, and p99 quietly degrades instead of erroring. Silent degradation is harder to notice than an outage, which is exactly why I kept it.

What broke / what's next

Honest status: the failure visuals and packet animation live in the same 980-line Scene.jsx as the component models, and that file is the hardest thing in the repo to change — it needs splitting before anything else. The 3D drag-and-drop interaction (added a few commits ago along with the info modals) still feels fiddly for precise topologies. And the capacity constants, as noted, are placeholders with opinions — calibrating even one path (say, queue drain behavior) against a real benchmark would make the whole thing more trustworthy.

Next up would be per-link latency visualization and a shareable-URL encoding of a topology so two people can argue about the same diagram.

Run it

npm install
npm run dev

Then pick the URL shortener scenario, switch demand to viral spike, kill a node, and watch what redundancy actually buys you. That's the whole pitch: fewer whiteboard assertions, more watching things break.

In action

Real captures from the running app — a healthy system at 850/850 rps, p99 at 302ms:

Built by Irfan Wani. Repo: https://github.com/Irfanwani/system-design-simulator