DEV Community: Todd Sullivan

I Built a Persistent AI Assistant That Runs on My Mac

Todd Sullivan — Fri, 24 Apr 2026 09:05:25 +0000

I got tired of AI assistants that forget everything the moment a session ends. So I built one that doesn't.

It runs 24/7 on my Mac, has access to my files, GitHub, iMessage, email, and calendar. It knows who I am, what I'm working on, and what I said to it last week.

The core problem with stateless AI

Every time you open a new Claude or ChatGPT session, you start from zero. You re-explain your context. You re-establish what you're working on. You paste in the same background info.

This is fine for one-off tasks. It's terrible for an ongoing working relationship.

The memory architecture

Instead of in-context memory, I use files:

MEMORY.md — long-term curated knowledge. What matters, distilled.
memory/YYYY-MM-DD.md — daily logs. What happened, decisions made, things to remember.
USER.md — who I am, my stack, my communication style.
TOOLS.md — local setup specifics.

Every session, the agent reads the relevant files before doing anything. This is the continuity layer.

MCP for real-world access

Model Context Protocol (MCP) is what lets the agent actually do things — not just talk about them.

I use it for:

Apple Mail, Calendar, Messages via a local MCP server
GitHub via gh CLI
File system access
Browser automation (Puppeteer via Chrome DevTools Protocol)

The result

It's not a chatbot. It's closer to a part-time assistant who's always available and never forgets anything. The most useful thing isn't any single capability — it's that context persists.

I can say "remember the JWT issue from last week" and it actually knows what I mean.

The hardest part isn't the AI. It's designing the memory and context system that makes it feel coherent over time.

On-Device AI: What Nobody Tells You About the Tradeoffs

Todd Sullivan — Fri, 24 Apr 2026 08:59:40 +0000

Everyone's building cloud AI. I've been building AI that runs with no internet, on a phone, in real-world conditions.

Here's what I've learned.

Model size vs accuracy in the wild

In the lab, your model hits 94% accuracy. In production, it's handling variable lighting, partial occlusion, camera shake, and phones that haven't been updated since 2021. Your 94% becomes something lower.

The instinct is to make the model bigger and more accurate. The problem: bigger models are slower, and on-device speed matters a lot when a person is standing in a room waiting for a result.

The real answer is usually: accept a lower accuracy threshold and design your UX to handle uncertainty gracefully. A confidence score + "tap to confirm" beats a slow high-confidence answer that times out.

Category-level beats object-level at scale

If you're doing object detection across thousands of SKUs, training a model to identify every individual product is a losing strategy. Too many classes, too many edge cases, constant retraining as products change.

Category-level detection — "this is a drinks product, this is a snack, this is a cleaning product" — is dramatically simpler and more stable. You can add object-level identification on top for high-value cases.

The feedback loop problem

On-device models don't automatically improve. You need a pipeline:

User makes a correction (override the model's output)
Correction is logged with context (lighting, device, conditions)
Flagged for review
Feeds the next training cycle

Without this, your model is frozen the moment you ship it. With it, field conditions become training data.

The data pipeline is harder than the model

Getting inference results off the device and into your backend — with context, without data loss, without requiring constant connectivity — is the actual hard problem. The model is the easy part.

Offline-first sync, conflict resolution, context preservation across sessions. That's where the real engineering lives.

On-device AI is genuinely exciting but it's a different discipline from cloud inference. The constraints are real and they change the design of everything.

Zero-Config Test Runner: JWT Auto-Gen and No Setup Docs

Todd Sullivan — Fri, 24 Apr 2026 08:59:39 +0000

Here's a thing I've built more than once: a test automation runner that works perfectly on my machine and is a complete mystery to everyone else.

The usual failure mode:

Private key has to live in a specific path
Three env vars need to be exported
You need to know which bundle and cloud flag to pass
There's a doc somewhere explaining this, probably out of date

I just fixed all of that. Here's the new interface:

./run-test.sh smoke-tests

That's it. Optional args for tag, bundle, and cloud target if you need them. Otherwise: just works.

How it works

JWT generated at runtime. The script finds jwt-private-key.pem in the repo root (RS256, no expiry needed for test runs). No env var. No "where do I put this file" question.

JWT=$(node tools/generate-jwt.js)

Firebase key resolved automatically. Checks the repo root first, falls back to ~/.ssh/. Works the same on a fresh clone as it does on my laptop.

const keyPath = fs.existsSync('./firebase-key.json')
  ? './firebase-key.json'
  : path.join(os.homedir(), '.ssh', 'firebase-key.json');

One optional arg. Need to target a specific bundle or cloud env? Pass it. Otherwise the defaults are sane.

The Claude catch

Built this pairing with Claude Opus. It caught something I'd missed: my JWT generation had an expiry set that would silently break test runs longer than the token lifetime. Not a crash — just stale auth, failing tests, no obvious error.

That's the kind of thing that only shows up at 2am. Fixed it before it shipped.

Result

Any engineer can clone the repo and run tests. No setup doc. No "ask Todd what the env var is called." No onboarding friction.

Zero-config should be the default, not the goal you work toward.

Claude is in My Commit History

Todd Sullivan — Fri, 24 Apr 2026 08:58:33 +0000

My recent commits have a new co-author:

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

It started as accurate bookkeeping. Now it's just how I work.

What I'm actually using it for

I build a lot of tooling — test runners, CI pipelines, API integrations. Claude doesn't help me write README files or summarise meetings. It's in the code, catching the things I miss when I'm moving fast.

A few recent examples:

JWT expiry logic. I was generating tokens for a test runner. Claude flagged that my expiry config would silently break long test runs — the kind of failure that only shows up at 2am. Fixed before it shipped.

Firebase key resolution. I wanted the tool to find credentials automatically. Claude suggested the fallback chain: check the repo root first, then ~/.ssh/. Obvious in hindsight. I'd have hardcoded the path.

API endpoint shape differences. Caught a mismatch between dev and staging environments before it became a test failure.

What the pattern looks like

I describe the problem. Claude asks questions — usually good ones. We work it out together. Then I write the code, or we write it together if it's straightforward.

The "AI writes your code" framing is wrong. It's closer to: pair programming with someone who's read everything, never has meetings, and doesn't care about tabs vs spaces.

Why I credit it in commits

Because it's accurate. If a human pair programmer caught the JWT bug, I'd mention them. Same logic applies. It also makes the history honest when I'm reviewing it six months later.

The commit history doesn't lie. If Claude is in it, it earned the attribution.