DEV Community: Aarush Karak

GitHub Automation with the gh CLI: Triage, PRs, and Workflows

Aarush Karak — Tue, 01 Sep 2026 16:38:29 +0000

gh as the Complete API Client

gh api exposes the entire REST and GraphQL surface with auth built in. The pattern that unlocks everything: gh api repos/{owner}/{repo}/issues piped to jq, and gh run watch for workflow observability. The CLI is a thin wrapper over the API — everything the website can do, a script can do.

Automated Issue Triage

The triage loop: fetch unlabeled issues, classify by keywords and titles, apply labels, assign priorities, and route to the right maintainer. A nightly script keeps a backlog organized without anyone touching the web UI. The discipline: classification must be conservative — an untriaged issue is better than a wrong label.

PR Review Automation

The reviewer's checklist as code: check CI status, verify the diff touches only declared files, confirm the description matches the change, and comment when something fails the check. gh pr view --json + gh pr diff make the review scriptable; the script comments, a human decides.

Workflow Lifecycle Management

Beyond triggering: gh workflow disable for flaky jobs, gh run rerun for transient failures, and gh api to approve protected-branch runs. The automation pattern: a cron that checks for failed runs older than a threshold and either reruns them once or files an issue — flaky CI stops being noise.

Cross-Repo Coordination

One script, many repos: a loop over gh repo list runs the same triage or release check everywhere, and gh issue create --repo files findings in each. For the 3-4 repos one person actually maintains, cross-repo automation collapses hours of manual clicking into one scheduled pass.

Identity and Safety

Automated actions run as a bot user or a PAT — the audit trail must show who did what. The safety patterns: --dry-run flags everywhere, confirmation prompts for destructive ops (deletes, force-pushes, label removal), and rate-limit awareness (gh api rate_limit).

NOTE: The bank-driven fallback wrote this post because the LLM proxy was unreachable — structure and facts come from the topic outline, and the next regeneration will enrich it.

Key Takeaways

gh as the Complete API Client
Automated Issue Triage
PR Review Automation
Workflow Lifecycle Management
Cross-Repo Coordination
Identity and Safety

FAQ

Q: What is the key idea in gh as the complete api client?
A: It is one of the core decisions that shape this topic. The section above walks through the reasoning, the tradeoffs, and the practical takeaway in context.

Q: What is the key idea in automated issue triage?
A: It is one of the core decisions that shape this topic. The section above walks through the reasoning, the tradeoffs, and the practical takeaway in context.

Q: What is the key idea in pr review automation?
A: It is one of the core decisions that shape this topic. The section above walks through the reasoning, the tradeoffs, and the practical takeaway in context.

Conclusion

The gh CLI makes GitHub a scripting surface: triage, review checks, workflow hygiene, and cross-repo maintenance all become scheduled, testable, and reviewable. The discipline — conservative classification, dry runs, honest audit trails — is what keeps the automation trusted.

View the project on GitHub

Running a Local LLM Proxy: OpenAI-Compatible Gateways

Aarush Karak — Sat, 29 Aug 2026 16:01:46 +0000

Why a Gateway

Every AI tool — editors, agents, scripts — speaks the OpenAI chat-completions dialect. A local gateway that speaks that dialect and forwards to whatever model you actually run makes every tool plug into local inference with zero code changes. One port, many consumers.

The Compatibility Contract

The surface that must match: /v1/chat/completions with messages, model, temperature, max_tokens; SSE streaming via stream: true; and error responses shaped like OpenAI's. Tools that validate models (a hardcoded model list) will reject your endpoint — accept any model name and map it, or report it honestly.

Streaming Without Breaking Clients

Clients that expect streaming fail on buffered responses. The implementation: read the upstream stream chunk-by-chunk, re-emit as SSE data: frames, flush on each token, and terminate with data: [DONE]. Timeouts are the silent killer — an idle upstream must send keep-alive comments or the client hangs forever.

Caching as the Cost Killer

Identical prompts are common across sessions (system prompts, templates, retries). A prompt-hash cache with TTL serves repeated requests instantly and cuts provider spend. The design decision: cache exact matches only — semantic caching of LLM output is where correctness dies.

Routing and Fallback

A gateway that routes by model name, cost budget, or availability. The fallback ladder: local model → free tier → paid provider, with health checks that demote a failing upstream. The pattern: retry with backoff on 429/5xx, fail over on repeated failures, and never silently return a different model than requested.

Operational Details That Matter

launchd or systemd supervision, structured logs with per-request latency, and a health endpoint (/v1/models) for uptime monitoring. The gateway becomes the single place where model behavior — version, temperature caps, token limits — is controlled for the whole machine.

NOTE: The bank-driven fallback wrote this post because the LLM proxy was unreachable — structure and facts come from the topic outline, and the next regeneration will enrich it.

Key Takeaways

Why a Gateway
The Compatibility Contract
Streaming Without Breaking Clients
Caching as the Cost Killer
Routing and Fallback
Operational Details That Matter

FAQ

Q: What is the key idea in why a gateway?
A: It is one of the core decisions that shape this topic. The section above walks through the reasoning, the tradeoffs, and the practical takeaway in context.

Q: What is the key idea in the compatibility contract?
A: It is one of the core decisions that shape this topic. The section above walks through the reasoning, the tradeoffs, and the practical takeaway in context.

Q: What is the key idea in streaming without breaking clients?
A: It is one of the core decisions that shape this topic. The section above walks through the reasoning, the tradeoffs, and the practical takeaway in context.

Conclusion

A local LLM proxy is the quiet infrastructure that makes local AI practical: one compatible endpoint, streaming done right, caching for cost, and honest routing. It turns 'AI tools need APIs' into 'everything already works, it's just local now.'

View the project on GitHub

Vite Plugin Development: Hooks, Transforms, and Virtual Modules

Aarush Karak — Thu, 27 Aug 2026 16:01:55 +0000

The Plugin Container Model

A Vite plugin is an object with hooks, layered over Rollup. The phases: config (adjust Vite config), configResolved (read final config), buildStart (initialize), transform (rewrite module source), generateBundle (write output files), closeBundle. Understanding when each runs — dev vs build — is half the API.

Transforms: Rewriting Modules Safely

A transform hook receives code + id and returns modified code. The discipline: only touch files you own (check id patterns), return null to pass through untouched, and never rely on textual assumptions about frameworks — use the loaders' ASTs or the module graph instead.

Virtual Modules for Injected Code

Virtual modules let a plugin provide modules that don't exist on disk: virtual:my-plugin-data. Intercept in resolveId, emit in load. This is how plugins inject runtime data (config, asset manifests, environment) without touching the user's source tree — and how sitemap plugins feed route lists to components.

Build-Time Asset Generation

The generateBundle hook writes files into the output: sitemaps, robots, feeds, OG images. The pattern: collect data during earlier hooks (routes from the module graph, page metadata from transforms), then emit this.emitFile({ type: 'asset', fileName, source }). Assets emitted here are hashed, deployed, and cacheable like any other build output.

Dev Mode: Middleware and HMR

In dev, plugins get an HTTP middleware (configureServer) and HMR hooks. A sitemap plugin doesn't need it, but a plugin serving mock APIs or proxying websockets lives here. The rule: dev features must not leak into the production bundle — gate everything on config.command.

Publishing and Testing Plugins

A Vite plugin is a library: vite-plugin-* naming, ESM-first, exports map, and a test suite that runs the real Vite build against a fixture project. The buildTest pattern: a temp project, build(), and assertions on output files — faster and more honest than unit-testing hooks in isolation.

NOTE: The bank-driven fallback wrote this post because the LLM proxy was unreachable — structure and facts come from the topic outline, and the next regeneration will enrich it.

Key Takeaways

The Plugin Container Model
Transforms: Rewriting Modules Safely
Virtual Modules for Injected Code
Build-Time Asset Generation
Dev Mode: Middleware and HMR
Publishing and Testing Plugins

FAQ

Q: What is the key idea in the plugin container model?
A: It is one of the core decisions that shape this topic. The section above walks through the reasoning, the tradeoffs, and the practical takeaway in context.

Q: What is the key idea in transforms: rewriting modules safely?
A: It is one of the core decisions that shape this topic. The section above walks through the reasoning, the tradeoffs, and the practical takeaway in context.

Q: What is the key idea in virtual modules for injected code?
A: It is one of the core decisions that shape this topic. The section above walks through the reasoning, the tradeoffs, and the practical takeaway in context.

Conclusion

Vite plugins are the cleanest expression of the bundler-as-platform idea. Hook order, transform discipline, and virtual modules cover most real needs — and a plugin that generates build artifacts (sitemaps, feeds) is one of the highest-leverage integrations a site can have.

View the project on GitHub

Browser Automation That Looks Human

Aarush Karak — Tue, 25 Aug 2026 16:02:07 +0000

What Detection Systems Measure

A fingerprint is a vector: user agent, WebGL vendor and renderer strings, canvas hash, audio context, screen metrics, fonts, timezone, and navigator property consistency. Bot detectors score the vector for anomalies — a headless Chromium with navigator.webdriver=true fails instantly; the game is scoring like a real device.

Patching the Leaky Surface

The leaks: navigator.webdriver, missing window.chrome, the --headless UA, and iframe parent attributes. The fixes: CDP session to override properties before any script runs, spoofed WebGL strings, and a realistic UA/fingerprint pair. The rule: every override must be consistent — a Chrome UA with a Safari canvas hash is worse than no spoofing.

Input That Reads as Human

Automated typing is instant and perfectly regular; humans type with variable inter-key latency, occasional corrections, and pauses. The pattern: randomized per-character delays with log-normal distribution, jittered mouse movement along bezier curves, and scroll speeds that vary. Timing consistency across a session matters more than any single delay.

Session Persistence: The Cookie Jar That Matters

Sites trust continuity: a fresh context with no cookies, no localStorage, and a first-seen IP screams automation. The pattern: persistent browser profiles stored between runs, realistic storage state, and reusing the same context for related tasks. Trust compounds over time.

Orchestrating Multi-Context Automation

Different tasks need different identities. The architecture: a context manager that provisions profiles, health-checks them (does this context still pass a fingerprint test?), and rotates on failure. A context that gets flagged is retired, not reused — one burned context can poison everything it touched.

The Ethics of Blending In

Human-like automation exists on a spectrum: scraping public data with throttling is defensible; impersonating a user or evading explicit blocks is not. The operational rules that keep a project defensible: rate limits well under human capacity, respect robots.txt and ToS, and never automate account creation or payment flows.

NOTE: The bank-driven fallback wrote this post because the LLM proxy was unreachable — structure and facts come from the topic outline, and the next regeneration will enrich it.

Key Takeaways

What Detection Systems Measure
Patching the Leaky Surface
Input That Reads as Human
Session Persistence: The Cookie Jar That Matters
Orchestrating Multi-Context Automation
The Ethics of Blending In

FAQ

Q: What is the key idea in what detection systems measure?
A: It is one of the core decisions that shape this topic. The section above walks through the reasoning, the tradeoffs, and the practical takeaway in context.

Q: What is the key idea in patching the leaky surface?
A: It is one of the core decisions that shape this topic. The section above walks through the reasoning, the tradeoffs, and the practical takeaway in context.

Q: What is the key idea in input that reads as human?
A: It is one of the core decisions that shape this topic. The section above walks through the reasoning, the tradeoffs, and the practical takeaway in context.

Conclusion

Human-like browser automation is a fingerprint consistency problem, not a feature list. Every patched leak must be coherent with the identity you're projecting — and the engineering discipline matters less than the boundaries you refuse to cross.

View the project on GitHub

Supabase as a Production Backend: Auth, RLS, and Realtime

Aarush Karak — Sat, 22 Aug 2026 16:06:47 +0000

The Postgres-First Pitch

Supabase is PostgreSQL with batteries: the auth schema, storage, and realtime are all Postgres features exposed over APIs. The practical consequence: everything you know about Postgres — indexes, views, EXPLAIN — transfers directly. Data lives in tables you own, not a proprietary store.

Auth: Sessions, Not Tokens

Email/password auth with refresh tokens, session persistence in localStorage, and onAuthStateChange listeners. The trap: relying on the token's JWT instead of letting RLS decide. Auth answers 'who is this user'; RLS answers 'what may they see' — keeping those separate is the whole architecture.

Row-Level Security as the API Boundary

RLS policies are the authorization layer: USING clauses filter reads, WITH CHECK clauses gate writes. A user-scoped policy (user_id = auth.uid()) means no endpoint can ever leak another user's rows — even a buggy join. Policies are per-table SQL, testable with set local role authenticated.

Realtime Without the Socket Fleet

Supabase realtime is Postgres replication broadcast over websockets. For a dashboard, the pattern: subscribe to channel filters, apply deltas to local state, and treat the subscription as a cache — not the source of truth. Reconnection with exponential backoff is mandatory; the server does not guarantee delivery.

Storage, Edge Functions, and the Free Tier

Storage buckets with MIME policies and signed URLs for private files; edge functions (Deno) for webhooks and server-side logic that must not ship in the client. The free tier caps: database size, monthly auth users, edge function invocations — know the limits before choosing the architecture.

The Zero-Cost Stack Composition

Supabase pairs with a Vercel frontend and a Python backend that reads the same Postgres. The seam: the Python service uses a service-role key (server-only) while the client uses user JWTs with RLS. One database, two trust boundaries, zero middleware to maintain.

NOTE: The bank-driven fallback wrote this post because the LLM proxy was unreachable — structure and facts come from the topic outline, and the next regeneration will enrich it.

Key Takeaways

The Postgres-First Pitch
Auth: Sessions, Not Tokens
Row-Level Security as the API Boundary
Realtime Without the Socket Fleet
Storage, Edge Functions, and the Free Tier
The Zero-Cost Stack Composition

FAQ

Q: What is the key idea in the postgres-first pitch?
A: It is one of the core decisions that shape this topic. The section above walks through the reasoning, the tradeoffs, and the practical takeaway in context.

Q: What is the key idea in auth: sessions, not tokens?
A: It is one of the core decisions that shape this topic. The section above walks through the reasoning, the tradeoffs, and the practical takeaway in context.

Q: What is the key idea in row-level security as the api boundary?
A: It is one of the core decisions that shape this topic. The section above walks through the reasoning, the tradeoffs, and the practical takeaway in context.

Conclusion

Supabase is a shortcut to a real Postgres backend, not a toy. The discipline — RLS as the security boundary, sessions over tokens, treating realtime as a cache — is what separates a demo from something you'd trust with real data.

View the project on GitHub

Orchestrating 24/7 AI Agents on a Mac

Aarush Karak — Thu, 20 Aug 2026 16:20:38 +0000

Why launchd is the only serious process manager on macOS

The first mistake is reaching for cron. Cron is a timer, not a process manager. If a cron job crashes at 3 AM, nothing restarts it. If it runs for 14 hours past its schedule, cron does not care. And cron's environment is a stripped-down shell with no PATH, no launchctl session access, and no sane way to get secrets.

Docker Desktop on macOS is a VM inside your laptop. It handles restarts well, but it costs 2–4 GB of RAM just to idle, and it inserts a virtualization layer between an agent and the host filesystem. For a lightweight program that makes HTTP calls and writes JSON, that is a waste of the exact resource — memory — that agents consume fastest.

I compared the options before committing:

Runner	Restart policy	Logging	Idle RAM cost	Fits an LLM agent loop?
cron	None	None built-in	0	No: no supervision
nohup + manual	None	Redirect to file	0	No: dies with your session
launchd	KeepAlive, throttled, event-driven	StandardOut/Err paths, newsyslog integration	~0	Yes, and it is native
Docker Desktop	Full container restart	Full	2–4 GB	Overkill on a laptop

launchd runs as PID 1 on macOS. It supervises every daemon on the system, it survives logout, it can restart processes with exponential backoff built into ThrottleInterval, and it costs nothing until a process starts. The only real question was how to make an agent loop that fits launchd's model — a model where a job either runs or exits with a status code.

The agent loop that never returns

An agent in aion is a while True loop. That is not laziness; it is a deliberate contract with launchd. launchd's KeepAlive handles crashes. The agent itself handles idleness. If there is no work, the loop sleeps on the queue. If there is work, it claims a row, calls the LLM, executes tool calls, writes results, and loops. A job that "finishes" and exits is, by design, a failure.

import json
import os
import time

API_KEY = os.environ["AION_LLM_KEY"]
MAX_BACKOFF = 300  # seconds

def claim_next_task() -> dict | None:
    # Single-writer claim; see the queue section below.
    ...

def run_cycle() -> None:
    task = claim_next_task()
    if task is None:
        time.sleep(2)
        return
    try:
        response = call_llm(task["payload"], tools=TOOLS, api_key=API_KEY)
        for call in response["tool_calls"]:
            execute_tool(call)
        mark_done(task["id"], response)
    except RateLimitError:
        requeue(task["id"], delay=min(2 ** task["attempts"], MAX_BACKOFF))
    except ToolValidationError:
        requeue(task["id"], delay=1)  # bad payload, not a crash

if __name__ == "__main__":
    while True:
        run_cycle()

The subtle part is the rate-limit path. When an LLM API returns 429, the agent does not die — that would trigger launchd's restart logic and burn another request immediately. Instead, the agent catches it, compute the exponential delay itself, and requeues the task with next_run_at in the future. The process stays alive, the queue holds the work, and the API key cools down.

The launchd plist that survives reboots

The entire orchestration story hinges on one XML file in ~/Library/LaunchAgents/. Everything else in aion is glue around it.

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
    <key>Label</key>
    <string>com.aion.worker</string>

    <key>ProgramArguments</key>
    <array>
        <string>/opt/homebrew/bin/python3</string>
        <string>/Users/me/aion/agent_loop.py</string>
    </array>

    <key>WorkingDirectory</key>
    <string>/Users/me/aion</string>

    <!-- Start once even if it exited cleanly before -->
    <key>RunAtLoad</key>
    <true/>

    <key>KeepAlive</key>
    <dict>
        <!-- Restart only on crashes: keep alive while the job is running,
             and relaunch when it exits with a non-zero status. -->
        <key>SuccessfulExit</key>
        <false/>
    </dict>

    <!-- Minimum seconds between automatic relaunches. Default is 10;
         raising it prevents a hot crash loop from hammering the LLM API. -->
    <key>ThrottleInterval</key>
    <integer>30</integer>

    <key>StandardOutPath</key>
    <string>/Users/me/Library/Logs/aion/worker.log</string>
    <key>StandardErrorPath</key>
    <string>/Users/me/Library/Logs/aion/worker.err</string>

    <key>EnvironmentVariables</key>
    <dict>
        <key>PYTHONUNBUFFERED</key>
        <string>1</string>
    </dict>

    <key>ProcessType</key>
    <string>Background</string>
</dict>
</plist>

Load it with launchctl bootstrap gui/$(id -u) ~/Library/LaunchAgents/com.aion.worker.plist. Unload with bootout.

TIP: ThrottleInterval is the minimum time launchd waits before relaunching a crashed job. Set it to at least 30 seconds. The default of 10 is enough for a fast crash loop to burn real money against a paid LLM API before you wake up.

The KeepAlive dict with SuccessfulExit = false is the key decision: if the agent exits with code 0, launchd leaves it dead. That gives you an escape hatch — if you need to stop an agent permanently for maintenance, a graceful shutdown path that exits zero stops the restarts. Any nonzero exit, launchd treats as a crash and relaunches.

Crashes are cheap; crash loops are not

Every crash costs at least one wasted LLM call when the loop restarts and re-fetches its context. A crash loop at 10-second intervals is not just noise; it is a credit-draining feedback loop. The fix is layered backoff, enforced in two places.

First, inside the agent, as shown above: catch rate limits and requeue with min(2 ** attempts, 300) seconds. Second, in the watchdog (next sections), which kills agents that exceed their retry budget.

def next_delay(attempt: int) -> int:
    """1, 2, 4, 8, ... capped at 5 minutes."""
    return min(2 ** attempt, 300)

I have watched a buggy tool result cause 40 API calls in 6 minutes because I trusted KeepAlive to solve availability. It solves availability, not cost. The backoff budget is the real guardrail.

WARNING: An agent that crash-loops for an hour can outspend a month of normal operation. Cap attempts per task in the queue, and cap restarts per hour in the watchdog, before you ever deploy.

The SQLite queue that keeps agents honest

With multiple agents (price monitoring, email triage, research), you need a durable work queue. On a Mac, Postgres and Redis are operational overhead you do not want. SQLite is a file; it survives reboots, it is transactional, and macOS ships a perfectly modern version.

The schema is deliberately small:

CREATE TABLE tasks (
  id INTEGER PRIMARY KEY AUTOINCREMENT,
  agent TEXT NOT NULL,
  payload TEXT NOT NULL,
  status TEXT NOT NULL DEFAULT 'pending',      -- pending | running | done | dead
  attempts INTEGER NOT NULL DEFAULT 0,
  max_attempts INTEGER NOT NULL DEFAULT 5,
  next_run_at INTEGER NOT NULL DEFAULT 0,       -- unixepoch; future = delayed
  created_at INTEGER NOT NULL DEFAULT (unixepoch()),
  updated_at INTEGER
);

CREATE INDEX idx_tasks_claim
  ON tasks(status, next_run_at)
  WHERE status = 'pending';

PRAGMA journal_mode = WAL;
PRAGMA busy_timeout = 5000;

The claim operation is a single statement that atomically moves one pending row to running:

UPDATE tasks
SET status = 'running', updated_at = unixepoch()
WHERE id = (
  SELECT id FROM tasks
  WHERE status = 'pending' AND next_run_at <= unixepoch()
  ORDER BY created_at
  LIMIT 1
)
RETURNING *;

Two rules kept this reliable. First, exactly one writer: only the main aion daemon writes to the queue; agents read through a tiny local API. No locking spaghetti. Second, WAL mode and busy_timeout mean a crashed agent holding a connection cannot block the daemon forever. If an agent dies mid-task, the daemon marks its running row as pending again with a delay — the crash is invisible to the rest of the fleet.

Memory: the silent agent killer

LLM agents leak memory. Tool results pile up, context windows grow, and every retry embeds the previous failure into the next prompt. On a Linux box, OOM-killer events show up in dmesg. On macOS, the system just starts swapping, and the machine turns into a beach ball festival while your agent "works."

I learned to treat RSS as the primary health metric, not CPU. The watchdog reads memory usage from ps — no psutil dependency needed, though it is nicer if you have it — and kills anything above a per-agent ceiling. launchd immediately restarts the agent, and the queue still holds its incomplete task.

import os
import signal
import subprocess
import time

MAX_RSS_MB = 4096  # agents have a 4 GB ceiling in aion
HEARTBEAT_TTL_S = 180

def rss_mb(pid: int) -> int:
    out = subprocess.check_output(
        ["ps", "-o", "rss=", "-p", str(pid)]
    )
    return int(out)  # ps reports RSS in KB; convert below

def is_stale(agent_dir: str) -> bool:
    hb = os.path.join(agent_dir, "heartbeat")
    if not os.path.exists(hb):
        return True
    age = time.time() - os.path.getmtime(hb)
    return age > HEARTBEAT_TTL_S

while True:
    for pidfile in os.listdir("/var/run/aion"):
        pid = int(open(f"/var/run/aion/{pidfile}").read().strip())
        rss_kb = rss_mb(pid)
        if rss_kb > MAX_RSS_MB * 1024:
            os.kill(pid, signal.SIGKILL)
            # launchd sees the nonzero exit and relaunches the job.
        if is_stale(f"/var/run/aion/{pidfile}"):
            os.kill(pid, signal.SIGKILL)
    time.sleep(30)

The key trick is coordinating with launchd rather than fighting it: the watchdog never tries to "restart" an agent. It only kills. launchd's KeepAlive is the restart mechanism, and because ThrottleInterval is set, a repeated leak-and-kill cycle gets slower over time instead of faster.

Logs: owning the aftermath at 3 AM

Launchd writes StandardOutPath and StandardErrorPath as plain files with no rotation. A 24/7 agent writes megabytes of JSON per day; within a month, a worker log can consume gigabytes. The macOS-native answer is newsyslog(8), the same tool that rotates system logs.

Drop this into /etc/newsyslog.d/aion.conf:

# logfile                   owner:group  mode  count  size  when  flags
/Users/me/Library/Logs/aion/worker.log  me:staff  644  7  1000000  *  Z
/Users/me/Library/Logs/aion/worker.err  me:staff  644  7  1000000  *  Z

This keeps 7 rotated files, compresses old ones with gzip (Z), and triggers rotation at 1 MB. The configuration is checked by newsyslog on a timer, so there is nothing else to schedule.

The deeper lesson: treat logs as data, not as prose. Every agent cycle should emit one JSON line with a correlation ID (task_id), the LLM model, token counts, and latency. When the price feed breaks at 3 AM, you want grep 2025-06-12T02:47 to return five structured lines, not a wall of print statements. I lost two days to debugging a deadlock that only appeared in a 400 MB text file that had been truncated by the filesystem. Structured logs would have shown the stuck tool call immediately.

Secrets without plaintext env files

The naive approach is EnvironmentVariables in the plist. Do not do it. Plist files are readable by any process running as your user, they leak into launchctl output, and they are one accidental git add away from being pushed to a public repo. (I did that. It was one character away from being in the commit history of a now-deleted repository.)

The right place for secrets on macOS is the Keychain, via the security CLI. The agent reads its API key at startup, holds it in memory, and never writes it to disk:

#!/bin/bash
set -euo pipefail

# Store once:
#   security add-generic-password -U -a worker -s aion-api -w "$LLM_KEY"
export AION_LLM_KEY=$(
  security find-generic-password -w -a worker -s aion-api
)
exec /opt/homebrew/bin/python3 /Users/me/aion/agent_loop.py

One tradeoff: Keychain access requires your login session to be unlocked. If you reboot and never log in, the keychain is locked and the agent cannot start. For a personal machine that is acceptable — you are logging in anyway. For a headless Mac mini used as a server, use the login keychain and disable auto-lock, or generate app-specific tokens with a shorter lifetime and rotate them monthly.

What aion v2 does differently

The first version of aion had the watchdog and the agent in the same process. That was a mistake: when the watchdog crashed, everything died, and launchd restarted the whole thing — including the queue reader, which then double-claimed tasks. Splitting the supervisor (aion daemon) from the workers (launchd jobs) was the architectural fix that made the system boring. Boring is what you want at 3 AM. If I rebuilt it today, I would:

Use one launchd job per agent, not one job running many agents. A crash in one agent should never be able to take down the others in the same process group.
Add WatchPaths so configuration changes auto-reload workers, instead of a manual bootout/bootstrap dance.
Model the heartbeat as a database row, not a file mtime. Files get cleaned by macOS cleanup tools; the SQLite database is durable and queryable.
Put the backoff calculator in the queue schema itself, as a check constraint: attempts <= max_attempts. That makes the invariant impossible to violate by accident.

The hardest-won lesson of this whole project: the orchestrator's job is to fail fast, restart clean, and make the crash log legible. Everything else — the prompts, the tool schemas, the model choice — is application code. The moment I stopped treating process management as "just a script that keeps things running" and started treating it as a small, distributed-systems problem with a state machine, the agents became genuinely unattended.

FAQ

Can this run on an Intel Mac? Yes. launchd and SQLite behave the same; I deployed aion on a 2018 Intel Mac mini and the only difference was the memory ceiling (it needed 2 GB, not 4 GB).

Does the Mac need to be logged in? The agents run in a user session, so yes — the session must be active. For a headless setup, consider a dedicated user account with automatic login and the login keychain unlocked.

What happens if the network drops? The agent process stays alive and the LLM call raises a connection error, which we catch, requeue with backoff, and sleep. launchd never sees a crash.

Can I run cloud agents alongside aion? Yes. I run aion for latency-sensitive tasks and a cloud worker for batch jobs; they share the same SQLite schema, and a small sync script pushes finished tasks upstream.

Conclusion

Building a 24/7 AI agent farm on a Mac is less about the agents and more about the boring plumbing: a process supervisor that restarts things, a queue that survives crashes, a watchdog that kills leaks, and logs you can actually read after midnight. launchd does the heavy lifting; all I added is a loop, a heartbeat, and some discipline around backoff and secrets.

The result is a system that runs for weeks without human intervention, costs nothing to idle, and — most importantly — fails loudly the moment something is actually wrong. That is the entire point of orchestration: not to make agents infallible, but to make their failures cheap and visible.

Key Takeaways

launchd is the only first-class process supervisor on macOS; cron has no restart policy and Docker Desktop wastes 2–4 GB of RAM on a laptop.
KeepAlive with SuccessfulExit = false plus a raised ThrottleInterval gives you crash recovery without a costly restart loop.
Agents must never exit on transient errors; catch rate limits and requeue with exponential backoff inside the process.
Use SQLite (WAL mode, single writer) as the durable work queue; one atomic UPDATE ... RETURNING claim prevents double-processing.
Monitor RSS and heartbeat staleness from a separate watchdog that only kills, letting launchd handle the restart.
Store LLM API keys in the macOS Keychain, never in the plist or a plaintext env file.

Building Custom MCP Servers: Extending AI with Tools

Aarush Karak — Tue, 18 Aug 2026 17:53:35 +0000

The Protocol That Actually Standardizes Tooling

MCP (Model Context Protocol) is essentially a JSON-RPC 2.0 service with a schema for capabilities, tools, resources, and prompts. The server declares tools; the host discovers them; the model decides when to call them. That sounds abstract until you realize what it replaces: a pile of bespoke LangChain wrappers, ad-hoc API endpoints, and "let me stuff the whole repo into the prompt" hacks. MCP is the rare protocol that wins on boredom. It's simple, stateless (if you want it to be), and has enough transport options (stdio, SSE, streamable HTTP) that you can put it in front of a local CLI or a remote session.

We chose Python with the mcp SDK because our text-processing pipeline was already Python-native. But the protocol constraints pushed us into a cleaner design: every tool needs a JSON Schema for input, a name, and an output shape that gets packed back into a content block. There's no "just return the dict" wiggle room. That forced us to define a contract for every query before writing any search logic. Contracts are a feature when you're building an interface that a probabilistic system will be driving.

The Problem: Slice-Grep Does Not Scale

Before codebase-memory-mcp, I watched agents operate on codebases roughly like this: they get a prompt, they try to recall from training data, they fail, they request a file path, the host returns a raw file, and the agent starts spraying grep calls. The agent never gets a map. It never gets "here are the ten modules that touch the payment webhook." It gets a hunk of source code and a hope.

A codebase has structure at a level above raw text: symbols, import graphs, path hierarchies, role-specific conventions (tests vs. production, migrations vs. models). The first painful lesson was that embeddings alone can't express that. A vector search will happily match a comment about "retry policy" in a README and a test fixture with zero actual retry logic. Semantic similarity is necessary but not sufficient. So we built a hybrid index: a SQLite database with three layers — (1) file metadata and path structure, (2) symbol-level records (function names, class names, exports), and (3) chunked text with 768-dimension embeddings stored via sqlite-vec. The MCP tools we exposed became the SQL layer for the model, carefully shaped so that a model knows when it has a precise question (symbol lookup) and when it should perform a fuzzy recall (semantic search).

Designing the Tool Surface

The biggest question in an MCP server is: what is the toolkit? We kept it to seven tools initially. Every subsequent tool we proposed had to earn its place by solving a category of agent failure we'd actually observed.

The first tool is semantic_search, which takes a query string and a number of results. It does embedding-based retrieval with pre-filtering. The second is search_symbol, an exact symbol/lookup tool using a trigram index and the language parser (tree-sitter) — no embeddings, just identifier-aware matching. That distinction matters. A model should never use fuzzy search when it knows the exact name of a function. The third and fourth are get_file_structure and read_file_lines, which handle the "can you show me the tree" and "show lines 40–70" operations. We added find_references for cross-file references, and get_recent_commits to answer "what changed recently" without reading every diff. Finally, remember and recall let the agent store a note about a design decision into a separate SQLite table, allowing memory to persist across separate MCP sessions.

from mcp.server import Server
from mcp.server.models import InitializationOptions

async def handle_semantic_search(query: str, limit: int = 5) -> dict:
    """Embed a query and search over the combined table."""
    embedding = embedder.embed(query)          # (768,)
    sql = """
        SELECT path, start_line, text, 
               ivec_distance(chunk_embedding, ?) AS distance
        FROM chunks
        WHERE path IN (SELECT path FROM files WHERE indexed_at IS NOT NULL)
        ORDER BY distance
        LIMIT ?
    """
    rows = await db.execute(sql, [embedding, limit])
    return {"results": [chunk_to_dict(r) for r in rows]}

def build_mcp_server(db, embedder) -> Server:
    server = Server("codebase-memory")
    @server.list_tools()
    async def list_tools():
        return [
            Tool(
                name="semantic_search",
                description="Search code by semantic similarity. Prefer this when you know the intent but not the identifier.",
                inputSchema={
                    "type": "object",
                    "properties": {
                        "query": {"type": "string"},
                        "limit": {"type": "number", "minimum": 1, "maximum": 20}
                    },
                    "required": ["query"]
                }
            ),
        ]
    @server.call_tool()
    async def call_tool(name: str, arguments: dict):
        if name == "semantic_search":
            result = await handle_semantic_search(**arguments)
            return {"content": [{"type": "text", "text": json.dumps(result)}]}
        raise ValueError(f"Unknown tool: {name}")
    return server

Notice that semantic_search hides the embedding dimension and the distance metric. The model doesn't need to know that ivec_distance is an L2 metric. It just needs a ranked list. The tool surface is a contract, not an implementation.

The Schema That Makes the Model Honest

The SQLite schema is the quiet under-appreciated piece. An MCP request comes in, and a tool handler runs SQL. If the schema is sloppy, the model gets ambiguous results and starts to make things up. We designed the chunks table with one urgent constraint: UNIQUE(file_id, start_line).

CREATE TABLE files (
    id INTEGER PRIMARY KEY,
    path TEXT NOT NULL UNIQUE,
    language TEXT,
    last_commit_sha TEXT,
    last_modified_at TEXT,
    indexed_at TEXT,
    is_test BOOLEAN DEFAULT FALSE
);
CREATE TABLE chunks (
    id INTEGER PRIMARY KEY,
    file_id INTEGER REFERENCES files(id),
    start_line INTEGER NOT NULL,
    end_line INTEGER NOT NULL,
    text TEXT NOT NULL,
    chunk_embedding BLOB,          -- 768 floats, serialized
    symbol_names TEXT,             -- JSON array
    UNIQUE(file_id, start_line)
);
CREATE VIRTUAL TABLE chunks_vectors USING vec0(
    chunk_embedding FLOAT[768],
    chunk_id INTEGER
);
CREATE INDEX idx_chunks_symbol_names ON chunks(symbol_names);

WARNING: Don't store the embedding as a JSON string and hope sqlite-vec will parse it on the fly. It won't. Two separate tables — one for metadata, one virtual vector table — keep insertion and query fast. The vec0 virtual table selects for chunk_id and distance, which we then join back into the chunks table.

One decision that paid off: instead of storing embeddings for files, we store embeddings for chunks, and every chunk carries symbol_names as JSON. That makes semantic search over the chunks table naturally useful for "where do we handle refund" queries with line-level precision. The model doesn't need to guess a file path if the search tool gives back src/payments/refunds.py:42. We also marked is_test on the files table and filtered it for retrieval by default, because the agent should not be learning happy-path patterns from test utility functions unless it explicitly asks.

Why SQLite Over Postgres or a Vector Database

We had this debate for two weeks. We spun up a pgvector instance. We benchmarked Pinecone. We knew that in production, one might want a Postgres server anyway. But this project runs locally, on a developer's machine, often in a terminal with no Postgres. The deciding metric wasn't raw vector recall — it was cold-start time and dependency count. SQLite has zero external service to babysit, and sqlite-vec compiles cleanly into a Python extension. That means you can clone this repo, run python -m codebase_memory index ., and have a fully queryable local index in 40 seconds for a medium-sized repository. Meanwhile, a managed vector database requires an API key, a network call on every embedding lookup, and an orchestration layer to keep the remote index consistent with the local checkout. For a developer tool that should disappear into the editor, local-first was the only rational choice.

Aspect	SQLite + `sqlite-vec`	`pgvector`	Hosted vector DB
Cold-start time	< 1s (no network)	2–5s if local, more if remote	10s+ (auth, handshake)
Dependency footprint	Low (Python wheel, SQLite)	Medium (Postgres server, extension)	High (SDK, credentials)
Env-specific queries	Native SQL + `ivec_distance`	SQL + `<=>`	Python API, no SQL
Memory / session mutations	Trivially transactional	Transactional, but heavier	Requires logic in the API layer
Scaling ceiling	GB-scale local corpora	TB-scale shared	TB-scale distributed
Best use case	Single-agent local context	Multi-agent shared backend	Cross-team semantic search

We chose SQLite because the model's lifetime for a given MCP session is measured in minutes, not months. The index is a mirror of a specific checkout at a specific commit — ephemeral by design. If the code changes, re-index that file; don't build a warehouse.

Tool Internals: The Guardrail Pattern

The most interesting engineering pattern inside this server is what I call the guardrail: a two-phase read where we validate the shape of the query before touching whatever the model asked to do. The reason is simple. A model's JSON output is occasionally malformed; its arguments object can have the right key but a wildly out-of-range value. In one early session, the agent called read_file_lines with start_line = -1 and end_line = 1000000. We would happily have returned the entire file. So every tool now follows the same discipline:

Validate types and bounds with a tiny custom validator that returns a human-readable error message.
Estimate the result size before executing (e.g., SELECT COUNT(*) for a line range).
Cap the result at a bounded size.
Return is_truncated: true if the result was clipped.

This pattern prevents three failure modes we saw in the wild: unbounded memory bloat from a monster file read, silent hallucination from a truncated result that wasn't marked truncated, and cascading agent retries caused by unhelpful JSON-RPC error messages. Instead of "Error: invalid value", the model sees "start_line must be >= 1; received -1". That single change cut agent retry loops by a measurable share — roughly a third of our early task-completion failures traced back to the model misusing a tool because the error message was useless.

The Streamable HTTP Dance

We started with stdio transport. It's perfect for a local backend that your editor spawns. But when we wanted to run the server on a different machine and connect from a client over http://localhost:8765, we had to upgrade to the streamable HTTP transport. That's where MCP's spec has sharp edges.

The MCP Python SDK provides a StreamableHTTPSessionManager and a /message endpoint. The first naive iteration blocked on a single request per connection, which meant our client couldn't interleave tool calls. The fix was to run the server with mcp.run(transport="streamable-http") and make sure the client kept the session ID in the Mcp-Session-Id header across requests.

import { Client } from "@modelcontextprotocol/sdk/client/index.js";
import { StreamableHTTPClientTransport } from "@modelcontextprotocol/sdk/client/streamableHttp.js";

const transport = new StreamableHTTPClientTransport(
  new URL("http://localhost:8765/message"),
  {
    requestInit: {
      headers: {
        "Content-Type": "application/json",
        "Mcp-Session-Id": sessionId,
      },
    },
  }
);

const client = new Client({ name: "agent", version: "0.1.0" });
await client.connect(transport)

const result = await client.callTool({
  name: "semantic_search",
  arguments: { query: "how do we validate the webhook signature?", limit: 5 },
});

TIP: If you ever see MCP error -32001: Session not found, it's almost always because your transport client is not holding onto the Mcp-Session-Id header. The server rejects you as a new session and discards any pending tool state. Guard the session header like it's a bearer token.

The bigger protocol-level lesson: MCP servers should expose only a few slow operations as blocking calls. It is tempting to make semantic_search asynchronously trigger an index rebuild if the corpus is stale. Don't. The model expects a tool call to return something promptly. Long-running operations belong either in a separate MCP resource or behind a read-style notification. We turned index refreshes into a one-off /refresh tool that returns immediately and writes to a status row the agent can query next time.

Prompt Budget Math

Every MCP tool call costs tokens — both the tool definitions in the prompt and the returned content. We obsessed over this. The default mcp SDK sends a long description for each tool. With seven tools, the model context contains roughly 1,800 tokens before any actual code is retrieved. That's fine for a 200k-token context window, but it matters for smaller models that agents commonly drive. So we discovered a rule of thumb: every retrieved chunk should carry more information than the tokens it costs. A 1,024-token chunk is reasonable if the search is precise; a 4,096-line file read is a crime.

def should_send_full_file(line_count: int, request_word_count: int) -> bool:
    """Return whether we should read a file or request a targeted range."""
    # A rough heuristic from our logs: models actually use < 30% of a file.
    keep_budget = 800  # target prompt overhead
    if line_count * 8 > keep_budget and request_word_count < 20:
        return False
    return True

Our token accounting led to three practical choices:

read_file_lines accepts start_line and end_line, not a line_count. The model must specify a limited range.
semantic_search returns at most 10 results, but we asked for 5 by default. After three failed iterations, we measured retrieval precision on the semantic_search results and saw that the first 5 results contained the right answer 78% of the time; pushing to 10 added only 4 points. The extra 5 results were noise, not signal.
Tool descriptions are ruthlessly short. We avoided prose and wrote them as imperative sentences: "Return at most limit chunks similar to query." The model doesn't need a paragraph of backstory for a search tool.

Failure Modes and How We Hardened Them

Real MCP servers fail in ways that are invisible in demos. The first failure is embedding service drift: if you change your embedding model between indexing and query time, every vector distance becomes garbage. We solved this by storing the model name in an index_info table and failing loudly on a mismatch — not silently serving results with a warning, because a model that receives a warning is still allowed to act on garbage.

The second failure is stale index. The worst early bug was an agent modifying a file and then an MCP query returning the pre-modification lines. We set files.last_modified_at based on git, and we added a refresh flow that re-chunks files whose mtime changed relative to the index. We also decided to never serve from the index if the HEAD commit changed since indexing; instead, semantic_search returns an error asking the agent to call /refresh first. A stale index is worse than no index.

The third failure is latency spikes in the embedding call. The very first version embedded queries synchronously inside the tool handler, so a slow local model stalled the entire session. We moved embedding calls to a separate asyncio task pool and set a 500ms timeout on the embedding lookup, falling back to a trigram search if the embedding service was slow. The fallback isn't perfect, but a useful trivial match beats a timed-out far-away match.

What Shipping Taught Me

By the time we had the server running reliably, the agent's codebase understanding changed qualitatively. Instead of half-guessing file names, the model would say "let me check the refund service" and call semantic_search("refund authorization flow"), then read_file_lines("src/payments/refunds.py", 80, 140) — a real plan. Over a set of 50 internal issue-resolution tasks, the agent completed them in about a third fewer turns on average, and the number of invented file paths dropped to near zero.

The biggest lesson, though, is about the interface, not the model. Building an MCP server forces you to think about what an external mind needs to know about your codebase. That editorial process is valuable even if you never use the MCP server. The seven tools we kept read like a caretaker's checklist: What is the structure? What does this symbol mean? Where is it used? In answering those, we surfaced the hidden assumptions in our own codebase — modules we thought were named clearly, and function names that everyone in the office "knew" but the rest of the world could never find.

If you're going to extend an AI with custom tools, build them as if the model were a new developer, not a superintelligence. Give it an index. Give it a cork board. Teach it to search by intent before searching by identifier. And above all, give it a protocol that doesn't blur the line between memory and fact.

FAQ

Q: Do I need a vector database to build an MCP server for codebase memory?
A: No. For local repositories under roughly a gigabyte of source text, SQLite with sqlite-vec is faster to set up, simpler to debug, and has no network latency. A dedicated vector database is worth it only when you have multiple agents sharing a central index or persisting across many machines.

Q: How large should the text chunks be for code embedding?
A: We started at 512 tokens with no overlap, then settled on 1,024 tokens with a 128-token overlap. That balance gave us line-level precision for symbols while avoiding fragmenting function bodies across chunks. The overlap prevents us from losing a symbol that straddles a boundary.

Q: What are the downsides of exposing raw SQL via MCP tools?
A: Raw SQL hides too much semantic intent from the model and invites injection-style errors. A model that reads an entire table might "fix" something that wasn't broken. The MCP tool interface should be a curated set of read-only operations with sensible defaults; keep the SQL on the server side.

Q: Will the MCP transport stay stable?
A: The protocol is evolving — streamable HTTP is still being finalized — but the core abstraction of tools, resources, and prompts is stable. We version-lock the mcp and @modelcontextprotocol/sdk packages, and we isolate transport-specific code so that changing from stdio to HTTP is a one-line switch inside our SDK.

Conclusion

The custom MCP server did not make the model unconditionally trustworthy — nothing will. What it changed was the cost of a mistake. Instead of a single hallucinated path that could waste minutes of agent loops, our tools constrained the model to a small search space with bounded, honest results. The server is a boundary that separates "the model knows" from "the model can find out," and that boundary is the most important thing we built.

We also learned that the protocol itself is a design discipline. JSON-RPC schemas for tool inputs, explicit result sizes, and clear failure messages all forced us to define the semantics of a codebase lookup precisely. There is no "intuitive" version of semantic_search; there is only the version whose parameters you can write a contract for, and the version that behaves badly when the model guesses wrong.

If you are adding memory and context to your AI agent, don't just embed the README. Index the symbols, the file structure, the tests, and the commit history, then expose them through a small, deliberate interface. The model will still be wrong. But it will be wrong on a tighter leash, with better clues, and far less confidence.

View the project on GitHub

Sitemaps Are Free SEO — Here's the Vite Plugin That Generates Yours

Aarush Karak — Sun, 16 Aug 2026 22:59:14 +0000

The Problem

Every static site I've shipped has gone through the same ritual: build the site, then manually write a sitemap.xml, forget to update it when a route changes, and discover a month later that Google indexed half my pages. Sitemaps are boring, so they get done wrong.

The Solution

@3ni8ma/vite-plugin-sitemap generates sitemap.xml (and robots.txt, if you want it) as part of your Vite build. You declare your routes once, and every build emits a fresh, correct sitemap.

// vite.config.ts
import { defineConfig } from 'vite'
import sitemap from '@3ni8ma/vite-plugin-sitemap'

export default defineConfig({
  plugins: [
    sitemap({
      hostname: 'https://example.com',
      routes: ['/', '/about', '/projects', '/contact'],
      changefreq: 'monthly',
      priority: 0.7,
    }),
  ],
})

That's it. vite build now writes a sitemap into dist/ automatically.

Options Worth Knowing

Option	Default	What it does
`changefreq`	`monthly`	Change frequency per URL
`priority`	`0.7`	Crawl priority (0.0–1.0)
`lastmod`	now	Last-modified date
`robotsTxt`	`true`	Also generate `robots.txt`
`disallow`	`[]`	Paths to disallow in robots.txt

The disallow option is the sleeper feature — point it at your admin routes and staging paths, and your robots.txt stays honest.

Why Bother

Sitemaps are the difference between search engines discovering your site in days versus weeks. For a static site, it's one dependency and ten lines of config. The plugin is on npm, takes zero runtime dependencies, and runs entirely at build time.

If you ship static sites with Vite, this is five minutes that pays for itself in crawl coverage.

Glassmorphism Is Back — and This Tailwind Plugin Makes It Painless

Aarush Karak — Sun, 16 Aug 2026 22:59:13 +0000

The Problem

Frosted-glass UI is everywhere again — every dashboard, every AI chat sidebar, every "spatial" web app. But hand-rolling the effect means repeating the same four-line blur recipe in every component:

background: rgba(255, 255, 255, 0.08);
backdrop-filter: blur(12px);
border: 1px solid rgba(255, 255, 255, 0.12);
border-radius: 12px;

And the moment a design system has three glass variants, those four lines mutate into a maintenance problem.

The Solution

@3ni8ma/tailwind-plugin turns the glass recipe into design tokens you can apply anywhere:

<div class="glass-sm">Subtle frosted panel</div>
<div class="glass-md">Standard glass card</div>
<div class="glass-lg">Heavy blur, strong border</div>

Add the plugin to your Tailwind config:

// tailwind.config.js
module.exports = {
  plugins: [require('@3ni8ma/tailwind-plugin')],
}

Every glass-* utility is backed by CSS variables, so you can retheme the entire glass layer from one place — blur radius, border opacity, tint, everything.

What's Inside

glass-sm / glass-md / glass-lg — three depth levels of the frosted effect
CSS-variable driven — retheme globally, not per-component
Dark-mode friendly — tints and borders adjust with your palette
Zero runtime — compiles down to plain utilities at build time

Why It's Worth It

Consistency. The whole point of a design system is that "glass" means one thing everywhere. This plugin makes the glass recipe a named concept instead of a copy-paste ritual — and when the design changes, you change the tokens, not fifty components.

If your project leans into frosted surfaces, it's on npm and takes one line to install.

Building HELIOS: A Gesture-Controlled AI OS in the Browser

Aarush Karak — Sat, 15 Aug 2026 17:11:06 +0000

The wager: an OS in a browser tab

I started HELIOS with three constraints: no native code, no head-mounted display, and no integration that requires a user to install a runtime. The browser would be the OS, the webcam would be the sensor, and WebGL would be the display. That was the whole pitch.

The first prototype was ugly. I rendered a single flat plane in Three.js, put a fake window texture on it, and used MediaPipe’s hand landmarks to push a cursor around. It felt like a tech demo with a lot of lag. The breakthrough came when I stopped emulating a mouse and started modeling gestures. The first time I pinched two fingers together, dragged a plane sideways, and watched it move as a physical object while the camera tracked my hand, the wager felt real. No Bluetooth controller. No QR code setup. No proprietary SDK. Just a Chrome tab, a laptop camera, and a few dozen lines of TypeScript.

The technical bet inside that wager is that WebXR is the wrong abstraction for spatial computing on devices people already own. WebXR is great when you want to strap on a headset and leave the room. But most spatial interactions happen while you are sitting in front of a screen, and the most durable spatial sensor you already have is a camera.

Why not WebXR? Choosing Three.js over the XR ecosystem

WebXR promises a fully immersive 3D scene with controllers, room tracking, and depth sensing. It also demands either a dedicated VR headset or an AR-capable phone. Our target device is a normal laptop with a webcam and a desktop browser. In that environment WebXR does not provide hand tracking without a headset that has built-in cameras. That was the deal-breaker.

I evaluated both stacks early, and the tradeoffs were clear:

	WebXR	Three.js + MediaPipe Hands
Hardware required	VR/AR headset or AR-capable phone	Any webcam + browser
Input source	Controller button/trigger events	21 landmarks from RGB camera
Immersion	Full screen/exclusive XR session	Augments the existing desktop
Hand tracking	Only on specific devices	Works in any Chromium tab
Fallback input	Difficult to implement	Native mouse/keyboard fallback
Distribution	Requires browser XR API support	Plain web app, open any URL

Choosing Three.js had a hidden benefit: HELIOS runs in a pop-up window next to your existing apps. It does not claim exclusive control over the display. That is closer to how people actually use AI on a daily basis: a copilot beside you, not a world replacing your monitor.

INFO: Treat the hand as an event bus, not as a cursor. A cursor implies position only; a hand implies predicates like pinching, pointing, moving, and resting.

Hand tracking as an input bus, not a party trick

The first mistake I made with MediaPipe was treating landmark data as a single cursor position. MediaPipe Hands gives you 21 landmarks per hand, each with normalized x, y, z, and visibility. But the meaning of that data only exists across time. A pinch is not a frame; it is a transition from open fingers to closed fingers. A drag is not a coordinate; it is a pinch that stays held while the wrist moves.

So I built an input bus. Every time MediaPipe produced a hand frame, we normalized it into a HandFrame object and published it to every subsystem that cared: the gesture FSM, the window hit-tester, a dwell tracker, and the AI context collector. Each subscriber reacted independently and asynchronously. That separation stopped the UI from coupling to the MediaPipe update loop.

export type Handedness = 'left' | 'right';

export interface HandFrame {
  timestamp: number;
  handedness: Handedness;
  landmarks: Float32Array; // 63 floats: 21 landmarks * (x, y, z), normalized
  scale: number;           // hand scale used to normalize distances
}

type HandListener = (frame: HandFrame) => void;

export class HandBus {
  private listeners = new Set<HandListener>();
  private latestFrame: HandFrame | null = null;

  publish(frame: HandFrame): void {
    this.latestFrame = frame;
    for (const listener of this.listeners) listener(frame);
  }

  subscribe(listener: HandListener): () => void {
    this.listeners.add(listener);
    return () => this.listeners.delete(listener);
  }

  peek(): HandFrame | null {
    return this.latestFrame;
  }
}

The key decision here was allocation discipline. MediaPipe runs at 30 fps; if every frame creates a new array and then throws it away, the garbage collector will cause stutter. We reuse a single Float32Array and copy into it. The HandBus is the only place where raw frames are allowed to be mutated; subscribers read the same frame and derive their own state from it.

The gesture grammar: from raw landmarks to deterministic intents

The core contribution of HELIOS is a small gesture grammar built as a finite-state machine per hand. MediaPipe’s classification models are excellent at telling you which finger is extended, but they are not good at answering the question "is this a click, a drag, or a swipe?" That question requires temporal context. A pinch that lasts 80ms and returns to open is a click. A pinch that stays under 0.35 normalized distance for 150ms while the hand moves is a drag. A pinch that rapidly opens then closes again is a flick, which we route to window close or mode switch.

Two-stage classification

We use a two-stage pipeline. Stage one converts raw landmarks into a clean distance metric: the normalized Euclidean distance between the thumb tip and index fingertip. Dividing by hand scale makes the metric invariant to whether the user is sitting close to or far from the camera. Stage two feeds that scalar into a state machine with hysteresis.

The hysteresis was non-negotiable. A raw distance threshold will flicker at the boundary; every gesture system hits the same problem. We close the pinch at 0.30 and only release it at 0.42. Between those thresholds the FSM keeps whatever state it is already in.

export enum GestureState {
  Rest = 'rest',
  Pinching = 'pinching',
  Dragging = 'dragging',
}

const PINCH_CLOSE = 0.30;
const PINCH_OPEN = 0.42;
const DRAG_HOLD_MS = 120;

export class GestureFSM {
  private state = GestureState.Rest;
  private pinchEnteredAt = 0;

  update(distance: number, now: number): GestureState {
    switch (this.state) {
      case GestureState.Rest:
        if (distance < PINCH_CLOSE) {
          this.state = GestureState.Pinching;
          this.pinchEnteredAt = now;
        }
        break;

      case GestureState.Pinching:
        if (distance > PINCH_OPEN) {
          this.state = GestureState.Rest;
        } else if (now - this.pinchEnteredAt >= DRAG_HOLD_MS) {
          this.state = GestureState.Dragging;
        }
        break;

      case GestureState.Dragging:
        if (distance > PINCH_OPEN) {
          this.state = GestureState.Rest;
        }
        break;
    }
    return this.state;
  }

  reset(): void {
    this.state = GestureState.Rest;
    this.pinchEnteredAt = 0;
  }
}

Steady-state detection

The Dragging state only activates if the user holds a pinch for 120ms. That small delay eliminates the ambiguity between click and drag. We also track a "rest" signal: if the centroid of the hand stays within 0.02 normalized coordinates for 600ms, we freeze the pointer. This solved the "Midas touch" problem, where every involuntary hand movement becomes a command. The pointer needs intent, not presence.

This grammar is intentionally small. We only support five meaningful gestures: point to hover, pinch to select, pinch-hold to drag, open palm to open the launcher, and a fist to dismiss. Each gesture maps to a deterministic UserIntent that the rest of the system can reason about.

The spatial window manager: z-index is a lie

In a DOM-based window manager, stacking order is a single integer. When two windows overlap, the browser resolves visibility by painting order. HELIOS does not use the DOM for windows; it uses Three.js planes in a 3D scene. That seems more powerful, but it introduces a problem: real people do not want to rotate windows in three dimensions. They want the same flat rectangles they already understand, but with the ability to reach out and push them aside.

The lesson I learned is that position.z is not z-index. A window depth value must participate in raycasting, occlusion, and idle animation, but user interaction should stay in screen-space. In HELIOS, every window is a plane constrained to face the camera. Dragging moves the plane in x and y; the z coordinate only defines stacking order and a subtle scale animation when a window is brought to the front.

import * as THREE from 'three';

export interface SpatialWindow {
  id: string;
  mesh: THREE.Mesh;
  depth: number;
}

export function pickWindow(
  windows: SpatialWindow[],
  ndc: THREE.Vector2,
  camera: THREE.PerspectiveCamera
): SpatialWindow | null {
  const raycaster = new THREE.Raycaster();
  raycaster.setFromCamera(ndc, camera);

  const meshes = windows.map(w => w.mesh);
  const hits = raycaster.intersectObjects(meshes, false);

  if (hits.length === 0) return null;
  const hitMesh = hits[0].object as THREE.Mesh;
  return windows.find(w => w.mesh === hitMesh) ?? null;
}

Each SpatialWindow also stores an unrounded "desired depth." When you pinch a window and push it forward, we animate it toward that depth with a spring. The actual stacking order in the Three.js scene is determined by sorting by depth, but a window is never allowed to grow so large that it occludes the camera. That would be a browser full-screen tab pretending to be an OS; it defeats the spatial metaphor.

Wiring AI into the input loop

The "AI" in HELIOS is not a chatbot bolted on to the side of a canvas. It is an intent parser that receives a structured context object assembled from the same input bus. When you point at a window and say "summarize this," the system builds a prompt with the window’s metadata, the current active element if available, and the transcript. It then calls an LLM through a small proxy server. The response renders into a new spatial window placed next to the one you referenced.

The crucial rule: the AI is never allowed to block a gesture. We measured the latency budget. Pinch-to-click must feel instantaneous, so the FSM stays under 10ms. An LLM round-trip is often 1-2 seconds. If the AI were in the critical path, every gesture would feel broken. Instead, we split the pipeline. Gestures mutate window state immediately. AI responses arrive asynchronously and update a sidecar state object that the renderer picks up on the next requestAnimationFrame.

We also limited what the LLM can control. It cannot move the pointer. It cannot summon the camera. It can only request actions through a typed Command object:

{
  "verb": "summarize",
  "targetWindowId": "window://notes",
  "language": "en",
  "timestamp": 1725804321000
}

This command grammar keeps the AI honest and makes failures debuggable. If the model gives us a malformed verb, we discard it silently and log it. The hand remains the root of truth; the AI is an advisor, not a driver.

Performance: staying at 60 fps with a webcam in the loop

The hardest performance challenge in HELIOS was not rendering. It was keeping the frame budget stable while two CPU/GPU-heavy libraries fight for the same thread. MediaPipe Hands can take 12-18ms per frame on a mid-range laptop GPU, and Three.js rendering of 50-80 draw calls costs another 3-5ms. If you are not careful, you end up with 35 fps and a stuttering window manager.

We adopted a simple scheduling model. The camera and MediaPipe inference run at 30 fps, but the Three.js renderer runs on requestAnimationFrame at 60 fps. The HandBus is the bridge. On every raw frame, we publish to subscribers. If a subscriber like the window manager needs to update, it marks itself dirty. The renderer only re-renders the scene when at least one dirty flag is set or an animation is active. When the hand is still, the frame cost drops to near zero.

We also learned to measure and cap allocation. The biggest hidden cost was temporary typed arrays for landmark coordinates. Reusing a single Float32Array(63) buffer saved us from hundreds of garbage collections per second. The effect on frame pacing was immediate: the requestAnimationFrame callback stopped getting interrupted by GC pauses.

WARNING: Never allocate a multi-thousand-element typed array in the same synchronous path as hand tracking. A hidden GC pause is the difference between a buttery 60fps and a stutter that feels like a bug in the AI.

The production tuning numbers were concrete: MediaPipe at 640x480, 30 fps, running on an RTX 3060 laptop GPU; Three.js with 70 draw calls; total hand-to-screen latency around 45ms. We did not chase zero latency because vision-based hand tracking is inherently late by one or two video frames. Instead we made the remaining latency predictable and visually consistent. When a window follows your hand with a constant 45ms delay, the brain compensates. When the delay wavers between 20ms and 90ms, the gesture feels broken.

The failure modes that taught us the most

Building HELIOS produced a long list of failures. I want to highlight the four that had the most architectural impact.

First, the Midas touch. The first build mapped every hand movement to a cursor position, which meant my resting hand near the keyboard would move the pointer across the screen. We fixed it with the rest-state detection described above, plus a spatial "activation zone" near the center of the camera frame. The hand has to be inside that zone before its motion counts.

Second, calibration drift. MediaPipe normalized coordinates assume a default camera FOV and aspect ratio, but a user’s webcam might be mounted above or below their monitor. The mapping from hand coordinates to screen coordinates is an affine transform we compute once at startup. We show a quick calibration screen where the user moves their palm to four corners. From then on, the pointer tracks. The math is simple, but without it the pointer lands ten centimeters off.

Third, click versus drag ambiguity. We solved that with the 120ms hold delay in the FSM, but it introduced a new failure: users would try to drag and then release before the transition, resulting in a "sticky drag" where the window jumps forward. We added a visual affordance: the window scales by 1.02 when it enters Dragging, so users know they have crossed the threshold.

Fourth, privacy. The webcam feed never leaves the browser. MediaPipe runs locally, and the only data sent to the LLM proxy is a JSON object with gestures, target window metadata, and a text transcript. Users need to know the difference between "the browser sees my hand" and "the AI sees my hand." We put a persistent indicator in the corner of the shell showing whether landmarks or raw frames are being transmitted. Making privacy visible made the architecture better because it forced us to keep inference on-device.

Why the browser won for spatial computing

I started this project expecting the browser to be the weakest link. It turned out to be the strongest. The reason is not technical novelty; it is permission and distribution. Every operating system has a camera permission model, but the web is the only one where I can send someone a URL and have them running a spatial AI interface in under a minute, no driver installs, no app store review, no headset pairing.

WebAssembly and WebGPU have quietly turned the browser into a legitimate high-performance compute target. MediaPipe Hands runs in WebAssembly with GPU acceleration. Three.js renders thousands of objects at 60fps. The missing ingredient was never runtime performance. It was interaction design that treats the camera as a first-class input device rather than a source of video to be displayed back to the user.

The browser also wins because it supports graceful degradation. When the camera fails, HELIOS falls back to the mouse. When the hand leaves the frame, the window manager stays interactive. That resilience is built into the DOM. A native spatial app usually has no mouse fallback, so it feels like a dead end. Our browser-based shell can always return to the desktop metaphor, which makes the spatial metaphor feel like an enhancement instead of a takeover.

Key Takeaways

Treat gestures as temporal state transitions, not single-frame classifications.
Use an event bus between hand tracking, window management, and AI so no subsystem blocks the input loop.
Hysteresis and rest-state detection are the cheapest fixes for the Midas touch and jitter.
Keep the AI out of the critical path; gestures mutate the UI immediately, AI updates asynchronously.
Reuse typed arrays and measure allocation pressure in real-time input pipelines.
The browser is a viable spatial computing runtime when you design for fallback and portability.

FAQ

Q: Does HELIOS require specialized hardware like a VR headset or depth sensor?
A: No. A standard webcam with at least 640x480 resolution is enough. MediaPipe Hands estimates 3D landmarks from a single RGB frame, so no depth camera is needed.

Q: Is HELIOS actually an operating system?
A: No, it is an AI shell and spatial interaction layer that runs inside a browser tab. It manages its own windows, handles gesture input, and coordinates AI context, but it does not manage processes or files. It is an OS-inspired interface, not a kernel.

Q: How private is the webcam feed?
A: The raw webcam frames never leave the browser. MediaPipe hand tracking runs locally in the tab. The only external request is a JSON payload containing landmarks, window metadata, and optional text transcript, which is sent to the configured LLM proxy.

Q: Can I use HELIOS without the AI features?
A: Yes. The gesture-driven window manager is independent from the LLM integration. If you do not configure an API key, the shell simply skips AI actions and works as a spatial desktop interface.

Conclusion

HELIOS was an attempt to prove that the future of spatial computing does not require a headset. It requires a camera, a browser, and an interaction model that respects the difference between a hand and a mouse. I learned more about input latency and interaction design than I did about AI. That was the surprise: the LLM side was straightforward, but making a pinch feel like a physical grip took weeks of tuning.

The architecture that survives is the one that separates concerns: hand tracking as a bus, gestures as a finite-state machine, windows as Three.js planes, and AI as an asynchronous advisor. Each layer is independent, and that independence is what makes the whole system robust. When the AI fails, gestures still work. When the camera fails, the mouse still works. That is not a failure of vision; it is the opposite. It is a vision that can ship.

The browser is not a toy runtime. It is the most portable, most permissive, and most resilient platform we have. If you want to see what a browser-native AI shell feels like, the code is open. Clone it, plug in a webcam, and pinch your way through a desktop that no longer needs a mouse.

View the project on GitHub

Why Rust is the Future of Systems Programming

Aarush Karak — Thu, 13 Aug 2026 18:38:06 +0000

The Ownership Revolution

Memory safety without garbage collection, fearless concurrency, and zero-cost abstractions — Rust is reshaping how we build infrastructure. A deep dive into ownership, borrowing, lifetimes, and where Rust excels over C/C++.

The Ownership Revolution

Rust's ownership model eliminates entire categories of bugs at compile time. Unlike C's manual memory management or Java's GC, Rust's borrow checker enforces strict rules about who can read and write memory. This section breaks down ownership, borrowing, and lifetimes with practical examples of how they prevent use-after-free, double-free, and data races.

Fearless Concurrency

Rust's Send and Sync traits make data-race detection a compile-time concern. The standard library's channels, mutexes, and atomic types are designed around these traits. We'll walk through building a concurrent web scraper that Rust guarantees is thread-safe before it ever runs.

Zero-Cost Abstractions

Iterators, closures, and generics in Rust compile down to the same machine code as hand-written loops. There is no runtime overhead for abstractions — you pay only for what you use. Benchmarks comparing Rust iterators to C loops demonstrate identical assembly output.

The Ecosystem: Cargo and Crates.io

Cargo is more than a package manager — it handles builds, tests, benchmarks, documentation, and dependency resolution. The crate ecosystem has matured rapidly, with production-grade libraries for HTTP (reqwest), async runtimes (tokio), serialization (serde), and web frameworks (axum).

Where Rust Falls Short

Compile times are the most common complaint. Incremental compilation has improved dramatically, but large projects still take minutes to build. Learning curve is steep — the borrow checker fights new users. And Rust's niche in GUI and game development remains small compared to C++.

The Verdict

Rust has already won in infrastructure: the Linux kernel now accepts Rust, Cloudflare uses it for edge services, and Discord migrated from Go to Rust for performance-critical paths. For systems programming, embedded, and performance-sensitive applications, Rust is the default choice for new projects.

Rust's combination of safety, speed, and developer tooling is unmatched in systems programming. While the learning curve is real, the long-term payoff — fewer production bugs, fearless refactoring, and predictable performance — justifies the investment.

View on GitHub

Understanding GPU Memory: VRAM, Bandwidth, and Why Your Model Won't Fit

Aarush Karak — Thu, 13 Aug 2026 18:33:05 +0000

HBM Architecture

GPU memory is the most constrained resource in ML. This post explains HBM architecture, memory bandwidth vs compute, how model size translates to VRAM usage, and techniques (offloading, recomputation, sharding) to fit larger models.

HBM Architecture

High Bandwidth Memory (HBM) stacks DRAM dies vertically with through-silicon vias (TSVs) connecting them. HBM2e offers 2.4 GB/s per pin, HBM3 reaches 6.4 GB/s. The A100 has 80GB HBM2e at 2TB/s, the H100 has 80GB HBM3 at 3.35TB/s. Understanding this hierarchy explains why memory bandwidth — not FLOPs — is the bottleneck for transformer inference.

Where Does the Memory Go?

A 70B parameter model at FP16 needs 70e9 * 2 bytes = 140GB just for weights. Adam optimizer states add another 140GB (momentum + variance at FP32). Gradients add 70GB at FP16. Activations for a 4096-token sequence add ~15GB. Total: ~365GB for training on a single GPU — why 8x A100s are needed.

Memory-Efficient Attention

Standard attention computes S = Q@K^T (shape: batch x heads x seq x seq), materializing the full attention matrix. FlashAttention tiles the computation, never storing the full S matrix — reducing memory from O(N^2) to O(N). For a 4096-token sequence, this saves ~500MB per layer. For 80 layers: 40GB saved.

Activation Recomputation (Checkpointing)

During forward pass, activations are stored for the backward pass. Checkpointing saves only a subset of activations and recomputes the rest during backward. The memory savings are proportional to how many checkpoints are kept. Trading 20% more compute for 50-80% less memory is often worth it.

Model Parallelism: Sharding Across GPUs

Model parallelism splits layers across devices. Tensor parallelism shards individual matrix multiplies across GPUs (requires high-bandwidth interconnect like NVLink). Pipeline parallelism assigns layer groups to different devices. Fully Sharded Data Parallelism (FSDP) shards optimizer states, gradients, and parameters across data-parallel workers.

CPU Offloading

When even sharded memory doesn't fit, parameters are offloaded to CPU RAM and fetched to GPU on demand. This is slow (PCIe 4.0 x16: ~32GB/s vs HBM's 2TB/s) but enables training models up to ~10x larger than GPU memory alone. Inference offloading is more practical because weights are static and prefetching is predictable.

GPU memory management is the defining engineering challenge of large-scale ML. Understanding the memory hierarchy, activation memory costs, and parallelism strategies is essential for fitting increasingly large models into available hardware.

View on GitHub