DEV Community: Todd Sullivan

My On-Device ML Feature Was Registered, But Never Scheduled

Todd Sullivan — Tue, 11 Aug 2026 08:06:21 +0000

I spent part of this week fixing a mobile AI feature where the model code was not the problem.

The app has a small on-device personalization loop: collect daily logs, merge in HealthKit metrics, retrain a local model periodically, and use that state to drive predictions and nudges. Nothing huge. Just the kind of background ML plumbing that makes a feature feel alive instead of static.

The bug was simpler and more annoying:

I had registered the background tasks.

I had handlers for them.

I had rescheduling inside each handler.

But the first task was never submitted.

So nothing ever ran.

Registration is not scheduling

This is the shape that matters with BGTaskScheduler:

func application(
    _ application: UIApplication,
    didFinishLaunchingWithOptions launchOptions: [UIApplication.LaunchOptionsKey: Any]? = nil
) -> Bool {
    registerBackgroundTasks()
    submitBackgroundTasks()
    return true
}

That second line is the one I was missing.

registerBackgroundTasks() tells iOS which identifiers this process can handle. It does not put work on the queue. Each handler can reschedule itself after it runs, but something has to submit the first request or the chain never starts.

The final version is deliberately boring:

private func submitBackgroundTasks() {
    scheduleHealthRefresh()
    scheduleMLRetrain()
    scheduleEngagementDaily()
    scheduleWeeklyInsight()
}

And the retrain request is just a normal BGProcessingTaskRequest:

func scheduleMLRetrain() {
    let request = BGProcessingTaskRequest(
        identifier: AppConstants.mlRetrainTaskID
    )
    request.earliestBeginDate = Date(
        timeIntervalSinceNow: 86400 * Double(AppConstants.retrainingIntervalDays)
    )
    request.requiresExternalPower = true
    request.requiresNetworkConnectivity = false
    try? BGTaskScheduler.shared.submit(request)
}

That is the whole fix. Not a new scheduler abstraction. Not a custom job runner. Just submit the work at launch and let iOS deduplicate by identifier.

The other trap: background queues and SwiftData

The second issue was actor isolation. BGTaskScheduler invokes handlers on a background queue. My persistence layer uses container.mainContext, so every handler that touches SwiftData needs to hop back to the main actor.

private func handleMLRetrain(task: BGProcessingTask) {
    task.expirationHandler = { task.setTaskCompleted(success: false) }

    Task { @MainActor in
        let context = PersistenceController.shared.container.mainContext
        let descriptor = FetchDescriptor<DailyLog>(
            sortBy: [SortDescriptor(\DailyLog.date)]
        )
        let logs = (try? context.fetch(descriptor)) ?? []
        try? await MLModelManager.shared.trainPersonalisedModel(from: logs)
        scheduleMLRetrain()
        task.setTaskCompleted(success: true)
    }
}

Again, boring. But this is exactly where AI features usually break: not in the model, but in the lifecycle around the model.

One shared path for HealthKit updates

I also pulled the HealthKit merge into one shared function so the scheduled refresh and HealthKit background delivery cannot drift apart:

@MainActor
static func fillTodaysLogFromHealthKit() async {
    let metrics = await HealthKitManager.shared.fetchDailyMetrics(for: Date())
    let context = PersistenceController.shared.container.mainContext
    let log = DailyLog.findOrCreate(for: Date(), in: context)

    if log.hrv == nil               { log.hrv = metrics.hrv }
    if log.restingHeartRate == nil  { log.restingHeartRate = metrics.restingHeartRate }
    if log.stepCount == nil         { log.stepCount = metrics.stepCount }
    if log.sleepDuration == nil     { log.sleepDuration = metrics.sleepDuration }

    try? context.save()
}

The important detail is that manual user-entered values win. Background sync should fill gaps, not overwrite what someone typed by hand.

The tests that caught the boring failures

A few regression tests now pin the things that looked too small to test before:

one DailyLog row per calendar day, because duplicate daily rows poison the training set
future-dated rows must not be mistaken for today
CloudKit-backed SwiftData schema must load, because a bad relationship can crash the app on launch
engagement notifications default to enabled, because UserDefaults.bool(forKey:) returns false for an unset key
pending Watch entries drain by payload type instead of clearing the whole queue

That last one matters because the Watch queue contains two shapes in one array: symptom logs and wellbeing check-ins. The old reconciliation path could drain symptoms and silently delete check-ins that had not been processed yet.

Silent data loss in a health-adjacent logging app is not a UX bug. It is a trust bug.

The takeaway

On-device AI features are mostly normal mobile engineering with a model in the middle.

The model can be fine while the product is still broken because:

the background task was registered but never scheduled
the handler touched persistence from the wrong execution context
sync overwrote user-entered data
duplicate daily rows polluted the training input
a queue drain deleted the wrong payload type

None of that is glamorous. All of it matters.

If a model retrains locally but the scheduler never wakes up, you did not ship personalization. You shipped a nice code path nobody calls.

That is the bar I keep coming back to with practical AI work: not "does the model exist?" but "does the system around it actually keep running after the app leaves the foreground?"

The mAP50 That Lied to Me: A Debugging Story About On-Device Safety AI

Todd Sullivan — Tue, 04 Aug 2026 08:02:43 +0000

I'm building GroundCheck, an offline-first field inspection app for construction safety managers. One of the things we bet on early was on-device hazard detection — a YOLO model that runs entirely on the phone, no internet required, spotting missing hardhats and safety vests in a photo the moment you take it. No competitor in this space does that; everyone else ships cloud-only detection that falls over the second you lose signal, which on a job site is often.

This is the story of a metric that looked great, was actually badly broken, and what it took to find out — plus a training run that got derailed twice by things that had nothing to do with the model.

A good score, hiding a broken detector

Our first real model, trained on a 10-class taxonomy (hardhats, vests, masks, cones, and a few others), hit 0.510 mAP50. That cleared our internal bar of 0.50. Model shipped, feature declared done, on to the next thing.

Then I ran it against a simple test photo — six construction workers, standard job-site shot, some in hardhats, all in hi-vis vests. The model found zero vests. Not "low confidence," zero. It also hallucinated a hardhat on someone who wasn't wearing one.

That's a bad look for a safety app, and it directly contradicted the "0.510, above target" number sitting in my training log. So what happened?

mAP50 is a metric that's easy to be misled by, if you only ever look at the macro-average. Ours was averaged across 10 classes with wildly different underlying performance:

Class	mAP50
Hardhat	0.924
Face-guard	0.850
Safety Vest	0.343
Mask	0.267

A model that nails hardhats and face-guards can drag a broken vest detector up to a respectable-looking average. The number wasn't wrong, exactly — it just wasn't answering the question I actually cared about, which was "does this work for the classes that matter most."

The lesson: a single aggregate metric is a summary, not a verdict. If your classes aren't roughly equally easy and equally important, look at the per-class breakdown before you trust the headline number — and if you can, run the model against a real example and look at it. The zero-vest failure took me thirty seconds to spot with my own eyes; it was invisible in the metric that was supposed to be measuring exactly that.

Fixing it

Two changes: I dropped a pair of classes (NO-Safety Vest / NO-Mask) that had too little consistent training data to ever work — the app now infers "no vest" from a person detected with no vest box overlapping them, rather than trying to detect the absence directly, which turned out to be a much harder learning problem than it needed to be. Then I went and found three new open datasets specifically to beef up the underrepresented classes, roughly tripling the validation volume for the worst offenders.

Retrained. Same architecture (YOLOv8s), same target: beat 0.510, and this time, verify the classes that actually matter.

The training run that fought back

This part has nothing to do with the model and everything to do with running long training jobs on hardware that occasionally has other ideas. Two things went wrong that weren't ML problems at all:

First, the machine rebooted mid-training (unrelated maintenance), killing the process at epoch 134 of a planned 200. Recoverable — checkpoints survive a kill, just resume from last.pt.

Second, and more interesting: the auto-resume script I'd written specifically to handle crashes like that had a bug. yolo detect train resume=True model=<checkpoint> is supposed to pick up exactly where you left off. In my environment, it silently didn't — it fell back to Ultralytics' built-in 4-image toy dataset instead of my real 32,000-image training set, and happily reported success. No error, no warning. Just a training run quietly doing nothing useful for however long it took me to notice the loss curves looked suspiciously clean.

I caught it because the process count looked wrong — one training job should mean one Python process, and I was staring at twenty-five. Followed that thread back to the argument parser silently discarding my dataset config and substituting its own defaults.

The fix that actually worked: stop trusting the "smart" resume, and do a plain, boring, explicitly-specified restart every time — spell out every argument, verify the log shows my actual dataset path before walking away, don't rely on a flag that's supposed to infer everything for you. Boring and verified beats clever and silent, every time a background process fails quietly instead of loudly.

Knowing when to stop

The retrained model plateaued around epoch 177 at 0.682 mAP50, still short of a stretch goal of 0.80. I tried a short "continuation" run — warm-starting a fresh 40-epoch pass from the final weights, hoping for a few more easy points.

It made things worse, immediately and consistently — accuracy dropping every single epoch. Resetting the full learning-rate schedule on top of a model that had already converged over 178 epochs was too big a shock; instead of fine-tuning, it started forgetting. Killed it after 4 epochs once the trend was unambiguous, and kept the original checkpoint. Sometimes the right move with a training run — like with a lot of optimization problems — is recognizing you've already found a good stopping point, and further tinkering is more likely to hurt than help.

Validating for real, this time

Same two checks as before, run properly this time:

Per-class mAP50 on the full validation set:

Class	mAP50	Before
Hardhat	0.855	0.924
Safety Vest	0.684	0.343
Mask	0.555	0.267
vehicle	0.425	(new weakest)

The test photo. Same six workers as before. This time: 4 of the visible vests correctly detected, all 3 worn hardhats correctly detected, bare heads correctly flagged. Not perfect — a couple of vests still missed, confidence scores are moderate rather than sky-high — but a real, visible fix. The classes that were quietly broken before are now genuinely functional.

The takeaway

Two unglamorous habits saved this retrain from shipping a broken detector twice: refusing to trust a single aggregate metric without a per-class breakdown, and refusing to trust an automated recovery mechanism without checking, concretely, that it actually did what it claimed to do. Neither is exciting. Both are the difference between a safety feature that works and one that just looks like it does in a dashboard nobody double-checked.

GroundCheck is an offline-first inspection app for construction safety managers, built around on-device AI hazard detection that works with zero connectivity — because job sites don't always have signal, and safety checks shouldn't wait for a cloud API.

My Local AI Assistant Got Worse When I Remembered Too Much

Todd Sullivan — Tue, 21 Jul 2026 08:06:35 +0000

My Local AI Assistant Got Worse When I Remembered Too Much

I moved a personal AI assistant onto a small local model last week and immediately hit a boring problem: the model was fine, but my memory layer was not.

The old version persisted raw conversation history and replayed it back into the prompt. That worked well enough with hosted models. Then I pointed the same app at a local OpenAI-compatible server running Qwen3-4B-4bit through Swama on the Mac mini.

After 196 accumulated messages, the assistant started doing the classic small-model failure mode: parroting its own previous replies, over-weighting stale context, and sounding less useful the more “memory” I gave it.

The fix was not a vector database. It was deleting most of the memory.

I split memory into two different things:

short-term conversation state
long-term user facts

Short-term history now stays in RAM only. It resets after an idle gap, and it has a hard cap so a marathon session cannot poison every future turn.

self.session_memory: Dict[str, ConversationBufferMemory] = {}
self._last_activity: Dict[str, float] = {}

idle_limit = int(os.getenv("SESSION_IDLE_MINUTES", "120")) * 60
last = self._last_activity.get(user_id)

if last is not None and now - last > idle_limit:
    del self.session_memory[user_id]

Long-term memory is not chat logs. It is a small list of distilled facts: preferences, people, devices, recurring activities, that kind of thing. Maximum 30 facts per user.

_FACT_EXTRACTION_PROMPT = """
Update the fact list. Add only stable, personal facts worth remembering across
conversations: preferences, interests, people, pets, places, devices, recurring
activities. Ignore small talk, one-off requests, and anything the assistant said
about itself.

Return ONLY a JSON array of strings.
"""

The fact extraction runs in a background thread after each exchange. The chat path should not wait for memory housekeeping.

threading.Thread(
    target=self._extract_facts,
    args=(user_id, msgs[-2].content, msgs[-1].content),
    daemon=True,
).start()

I also deliberately use a hosted model for the distillation step. The local 4B model is good enough for fast interaction, but long-term memory cleanup is one of those places where quality matters more than latency. It is off the response path anyway.

The other local-model tweak was tool bias. Small models are much more likely to answer from stale weights even when tools exist, especially if the system prompt says anything like “use your knowledge first.” So the Swama handler adds a blunt override for live data:

_TOOL_BIAS = (
    " IMPORTANT OVERRIDE: for anything happening NOW - weather, sea or"
    " kitesurfing conditions, device/home status, prices, news, live data"
    " of any kind - you MUST call the matching tool. Never answer those"
    " from memory. /no_think"
)

Qwen3 also emits <think> blocks even when asked not to, including mid-stream after tool calls, so the streaming handler strips those tags incrementally. Not glamorous, but necessary if you do not want raw reasoning markup leaking into a voice/chat UI.

The useful lesson was this:

Memory is not “more previous tokens.”

For a personal assistant, raw transcript replay is the cheapest thing to build and one of the easiest ways to make the system worse. The assistant needs enough recent context to hold the current conversation, plus a tiny set of stable facts that survive across sessions.

Everything else is prompt pollution with a better name.

Source: Recent personal assistant backend work: Swama local model support, Qwen3-4B-4bit via an OpenAI-compatible endpoint, RAM-only session history, 2-hour idle reset, 200-message cap, background fact extraction, and 30 persisted user facts.
Tags: ai, python, llm, devops
Status: published

I Made My Voice Agent Feel Faster by Streaming Sentences, Not Audio

Todd Sullivan — Tue, 14 Jul 2026 08:05:39 +0000

The annoying thing about voice agents is that “the model is fast” does not mean the experience is fast.

I had a small voice assistant running on a local device, talking to a hosted chat backend. The actual LLM call was only one part of the wait. The full path looked more like this:

wake word detection
speech recognition
authenticated /chat call
model response
local TTS synthesis
audio playback

If you wait for step 4 to finish before starting step 5, the user hears nothing until the entire reply is done. That feels dead, even when the backend is technically fine.

So I changed the contract. The hardware client now calls the chat endpoint with stream_tts: true:

response = self.session.post(
    f"{CHAT_API_BASE}/chat",
    json={"message": message, "stream_tts": True},
    timeout=30,
    stream=True,
)

The backend yields text chunks as they arrive from the model. The device keeps a small buffer, splits complete sentences, and starts synthesizing each sentence immediately:

_SENTENCE_BOUNDARY_RE = re.compile(r'(?<=[.!?])\s+')

def split_complete_sentences(buffer: str) -> tuple[list[str], str]:
    *sentences, remainder = _SENTENCE_BOUNDARY_RE.split(buffer)
    return [s.strip() for s in sentences if s.strip()], remainder

That is deliberately boring. Not phoneme streaming. Not a custom audio protocol. Just sentence-level pipelining.

The next useful bit was overlapping synthesis and playback. A single background worker waits for synthesized WAV files and plays them in order, while a one-worker ThreadPoolExecutor starts rendering the next sentence as soon as it is complete.

for sentence in sentences:
    future = synth_executor.submit(self.tts_engine.synthesize, sentence)
    playback_queue.put((sentence, future))

That removed the worst gap: “sentence one finished playing, now start thinking about sentence two’s audio.” The hardware now does the obvious thing a human expects — keep talking.

I also cut backend time-to-first-token by doing less. For this conversational path, I turned off extended model thinking:

_NO_THINKING = types.ThinkingConfig(thinking_budget=0)

And I stopped advertising Google Search on every request. The search tool is only added when the prompt smells like it needs current/external information. Most turns do not.

if self._needs_search(built_contents):
    all_functions.append(self.google_search)

The result was about a 5x cut in chat time-to-first-byte for the common path, plus a much better perceived response because speech starts before the full answer exists.

The lesson was not “stream everything.” It was smaller than that:

stream at the boundary the product can actually use
overlap the slow local work with the slow network work
do not give the model tools or reasoning budget unless the turn needs them
log chunks, sentence counts, gaps, and total time so you can see where the pause moved

Voice agents do not need heroic architecture to feel better. Sometimes the fix is a regex, a queue, and deleting the expensive defaults.

Silent Data Loss Is Worse Than a Failed Export

Todd Sullivan — Tue, 07 Jul 2026 08:04:31 +0000

This week I fixed a set of bugs in a mobile export pipeline where the scary part was not that the export could fail.

It was that it could succeed while quietly omitting data.

The pipeline builds an internal assessment model, validates it, serializes it into an RdSAP XML export, packages evidence, writes a checksum, and records an audit log. The deterministic exporter work was already in place. Same assessment in, same bytes out.

But deterministic output only helps if the model you serialize has not already lost meaning.

A code review turned up a few places where valid survey answers were being collapsed into “nothing to export”.

The dangerous `null`

The first bug was a token mapper:

fuelToken(value): string | null

A valid answer, Other, was returning null. That meant mandatory fields like main fuel and water heating fuel could disappear from the XML instead of being exported as the existing fallback token.

The fix was boring:

Other -> "other"

Boring is good. Boring means the field is present and downstream validation can do its job.

The second version of the same bug was draught proofing. Unknown was a valid answer in the survey UI, but there is no clean numeric RdSAP value for it. The old behavior silently omitted the value. The fixed behavior raises a non-blocking validation warning instead.

That distinction matters:

missing because the assessor never answered: omit
present but not representable in this export format: warn
present and representable: export

Those are three different states. Treating them all as null is how data loss gets a polite API.

`false` is not “unanswered”

The more subtle bug was boolean handling.

The original helper effectively collapsed explicit false and “never answered” into the same output. So a recorded “no” for fields like cylinder present, immersion heater, or insulation could become indistinguishable from a question that was never asked.

That is a bad trade. In an assessment/export system, “no” is data.

The helper now returns a tri-state value:

boolean | null

And the serializer emits true/false for real answers, only omitting the element when the value is genuinely unanswered.

The test is tiny, which is exactly the point:

it("distinguishes an explicit false answer from never-answered", () => {
  const a = buildAssessment({
    inspection,
    responses: mkResponses({ ...validValues, epc_cylinder_present: false }),
    media: [],
  });

  expect(a.hotWater.cylinderPresent).toBe(false);
});

No giant test harness. Just one assertion that prevents the bug from coming back.

Do not guess from raw JSON

There was also a LiDAR floor-area parser that tried too hard.

If the JSON had a total, use it. Fine.

If total was missing, the old fallback scanned the raw JSON for a number. That could accidentally grab an individual room area and treat it as the property's total floor area.

That is worse than failing.

The fixed version returns null when total is missing. No heroic guessing. No “probably 20m² because I found 20 somewhere in the blob”.

expect(a.property.floorAreaM2).toBeNull();

Again: boring test, useful guarantee.

Audit failures before they escape

One more bug was outside serialization. getExporter() could throw before the main try/catch, which meant a format lookup failure skipped the audit-trail write that other failures recorded.

That is the kind of edge case that makes production debugging annoying. The export failed, but the failure path forgot to leave evidence.

The fix was just moving the lookup inside the guarded path so the audit log is written consistently.

The AI-assisted part

This is where AI-assisted development is actually useful for me.

Not “write the whole exporter and hope”. More like:

build the deterministic seam
have Claude review the boring mapping logic
turn every discovered ambiguity into a narrow test
keep the deliberate deferrals explicit

This pass added 6 focused tests. The export suite moved from 119 to 125 tests, with the targeted export tests passing locally: 3 suites, 23 tests.

There is still one known bigger issue left alone: validation is currently too RdSAP-specific for every export format. That needs per-format validation profiles. I did not jam it into this fix because it is a different change.

That is another useful rule when working with AI in real codebases: fix the silent data loss now, leave the larger design change visible, and do not let the model turn a bug fix into a rewrite.

The best export pipeline is not the cleverest one.

It is the one that refuses to quietly lie.

The Exporter Was Easy. Making It Deterministic Was the Work.

Todd Sullivan — Tue, 30 Jun 2026 08:02:27 +0000

The Exporter Was Easy. Making It Deterministic Was the Work.

I spent this week building a mobile export pipeline for UK energy assessment data.

The output is not glamorous: take completed survey responses, turn them into an RdSAP XML file, package the evidence, write a manifest, calculate a checksum, and keep a local audit trail.

The interesting part was not generating XML. The interesting part was making the whole thing deterministic and testable before adding more export formats.

The shape ended up like this:

responses
  -> buildAssessment()
  -> validateAssessment()
  -> exporter.serialize(assessment)
  -> sha256
  -> write package
  -> persist audit log

The exporter itself is deliberately boring:

serialize(assessment): string

No file system. No clock. No database. No random IDs. Same assessment in, same bytes out.

That one rule makes the rest of the pipeline much easier to reason about. If the XML changes, it changed because the input changed or the serializer changed. Not because a timestamp moved, a native module behaved differently, or a test environment had a different document directory.

Side effects at the edge

The orchestrator is the only layer allowed to do side effects:

const body = exporter.serialize(assessment);
const checksum = await deps.sha256Hex(body);
const pkg = await writeExportPackage(
  { inspectionId, exporter, assessment, body, checksum },
  deps.fs
);
await deps.persistLog(log);

Those dependencies are ports:

export type ExportDeps = {
  fs: FileSystemPort;
  sha256Hex: (input: string) => Promise<string>;
  persistLog: (entry: ExportLogEntry) => Promise<void>;
};

In production, they map to Expo file system, Expo crypto, and a local SQLite audit table.

In tests, they are an in-memory file system, Node crypto, and an array log sink.

That was not architectural theatre. Under jest-expo, native modules are often partially unavailable. documentDirectory can be undefined. Crypto enums can be missing. If the export logic directly imports and touches those modules, the “end-to-end” test either becomes a mock festival or stops covering the actual path.

With ports, the test runs the real pipeline:

const { deps, files, logs } = memoryDeps();
const run = await runExport(input(), deps);

const body = files.get(expectedPath)!;
expect(run.result.checksum).toBe(
  createHash("sha256").update(body, "utf8").digest("hex")
);

That test proves a few things at once:

the package path is correct
the written XML is exactly the pure exporter output
the checksum is a real SHA-256 of the bytes
the manifest references the same checksum
one audit entry is written

There are also tests for evidence attachment copying, missing attachment sources, validation-blocked exports, checksum reproducibility, and a second JSON archive exporter running through the same pipeline.

Validation blocks, but still logs

One design choice I care about: failed validation writes an audit record too.

If a mandatory field is missing, the export does not create files. But the attempt is still logged with success: false, validation counts, schema version, and no checksum.

That makes the export feature behave like a real operational system rather than a button that either emits a file or silently refuses.

Why this matters for AI-built software

This is the seam I want when using AI heavily in engineering work.

LLMs are very good at producing a first version of “convert this shape into that schema.” They are much less reliable if the codebase lets schema mapping, file IO, logging, hashing, and UI state collapse into one blob.

The fix is not to use less AI. The fix is to give the code stronger boundaries:

pure transformations for the model or human to edit safely
deterministic outputs that can be snapshot-tested or checksummed
injected IO so tests exercise the pipeline without native dependencies
audit trails for both success and blocked paths
format-specific exporters behind a small registry

Once those seams exist, adding the next export format is not a rewrite. It is another serializer plus a few tests.

That is the part I keep finding in real AI-assisted development: the model can help move fast, but the architecture has to make fast changes safe.

Source: Recent mobile export module work: RdSAP XML export pipeline, pure serializers, injectable IO ports, SHA-256 packaging, local audit logging, and 24 export tests.
Tags: ai, testing, typescript, mobile
Status: published

AI Features Need Product Edges, Not Just Better Prompts

Todd Sullivan — Tue, 23 Jun 2026 08:03:02 +0000

Most AI features don't fail because the model is bad.

They fail because everything around the model is treated like a demo.

This week I was tightening an iOS workout app that uses Claude through Supabase Edge Functions. The model part is straightforward: send training context, get a structured exercise or plan back, validate it, write it into SwiftData.

The less glamorous work was the part that makes it feel like an actual product:

monthly AI credit balance
offline handling
auth token storage
disabled states while generation is running
different rules for “add exercise” vs “swap this exercise”
tests for the boring edge cases

That is where most AI app quality lives.

One button, several states

The UI has a small “fill with AI” affordance on the exercise editor. Underneath it, the button is not just “call endpoint”. It has to know whether suggestion is currently allowed:

var canSuggest: Bool {
    guard !isLoadingAI,
          NetworkMonitor.shared.isOnline,
          (creditsRemaining ?? 1) > 0
    else { return false }

    // Swap mode can use the original exercise as context.
    // Add mode needs the typed name as a hint.
    return isSwapMode || !name.trimmingCharacters(in: .whitespaces).isEmpty
}

That little predicate is doing a lot of product work.

If the user is offline, don't let them tap into a doomed network request.
If a generation is already running, don't double-spend.
If they have zero credits, don't pretend the feature is available.
If they are swapping an existing exercise, don't force them to type a name because the old exercise is already useful context.

The model does not care about any of this. The user does.

Credits should be part of the response

The suggestion response includes the updated credit count:

struct SuggestedExercise: Decodable {
    let name: String
    let briefDescription: String
    let muscleGroup: String
    let sets: Int
    let repTargetLow: Int
    let repTargetHigh: Int
    let restSeconds: Int
    let isDualDumbbell: Bool
    let creditsRemaining: Int
}

Then the view model updates local state immediately after a successful fill:

creditsRemaining = result.creditsRemaining
aiFilledFields = true

That avoids the classic AI-product weirdness where the backend knows the user has spent a credit but the UI keeps showing stale allowance until the next refresh.

It is also easier to test. In the app tests, the credit transition is explicit:

func testAIDisabledWhenCreditsReachZero() {
    let vm = ExerciseEditViewModel(mode: .add(makeDay()), context: container.mainContext)
    vm.name = "Row"
    vm.creditsRemaining = 1
    XCTAssertTrue(vm.canSuggest)

    vm.creditsRemaining = 0
    XCTAssertFalse(vm.canSuggest)
}

There are 13 tests just around the exercise edit view model, plus separate coverage for offline error mapping. Not because this is academically interesting, but because this is the stuff that breaks in front of real users.

Offline is not a server error

Another small detail: connectivity failures are mapped separately from backend failures.

static let connectivityCodes: Set<URLError.Code> = [
    .notConnectedToInternet,
    .networkConnectionLost,
    .timedOut,
    .cannotConnectToHost,
    .dataNotAllowed,
]

The resulting message is intentionally plain:

"You're offline. Reconnect to use AI features."

No “unexpected server response”. No fake intelligence. Just tell the user what happened.

The actual lesson

Shipping AI features is mostly normal software engineering with a probabilistic dependency in the middle.

The model call matters, but the surrounding contract matters more:

Can the user invoke it right now?
What happens if the network dies?
Is usage counted consistently?
Does the UI reflect server state immediately?
Are the edge cases testable without calling the model?

Once those pieces are in place, the AI feature stops feeling like a prompt wired to a button and starts feeling like part of the app.

That is the bar I keep coming back to: not “does the model answer?” but “does this survive normal product reality?”

I Replaced My Old AI Agent Stack with Hermes — and It Finally Feels Like Infrastructure

Todd Sullivan — Fri, 19 Jun 2026 11:16:31 +0000

I recently moved my personal AI automation setup over to Hermes.

Not because I wanted another chatbot. I already had one of those.

The useful shift was that Hermes feels less like “an AI you talk to” and more like a small operating layer for personal automation: persistent memory, scheduled jobs, skills, gateway routing, tool access, and enough system awareness to actually do maintenance work instead of just narrating it.

That distinction matters.

Most AI assistant demos stop at the prompt. Real usage starts after the third week, when you need the assistant to remember your preferences, avoid stale work contexts, run scheduled checks, post to the right channel, and not accidentally resurrect some old service you migrated away from months ago.

That last one is not theoretical.

After the migration I noticed stale references to my old agent stack still appearing in messages. So I asked Hermes to verify the machine state rather than just reassure me.

It checked:

the live launchd services
the running gateway process
old LaunchAgent plists
cron job delivery metadata
process names and command lines
stale references in config/state

The important part: it found the actual answer, not just the obvious one.

Hermes itself was running cleanly through launchd as ai.hermes.gateway. The old gateway labels were disabled. But there were still two leftover MCP-related LaunchAgents from the previous setup:

com.clawmcp.server
com.clawmcp.tunnel

One of them was still using an old SSH key path from that previous world. Not catastrophic, but exactly the kind of residue that turns into weird behaviour later.

Hermes stopped them, disabled the launchd labels, moved the plist files into a timestamped disabled folder, and re-verified that only the Hermes gateway remained active.

That is the bit I care about.

A useful AI agent is not just a text generator. It needs to be able to close the loop:

inspect the real system
distinguish stale text from live processes
make a reversible change
verify the change actually worked
remember the durable lesson for next time

Hermes also gives me a cleaner mental model than my previous setup.

Skills are reusable procedures. Memory is for stable facts. Cron jobs are durable scheduled agents. The gateway lets the same assistant operate from Discord, email, terminal, or wherever I am. Profiles keep different contexts isolated. Toolsets make capabilities explicit.

That sounds boring, which is why I like it.

Boring infrastructure is what makes automation trustworthy.

The more I use AI agents, the less interested I am in “look what this model can say” and the more interested I am in whether the surrounding system can behave predictably over time.

Can it run every morning and only notify me when something needs attention?
Can it publish an article without leaking secrets?
Can it check the actual machine state before making a claim?
Can it avoid old work contexts I no longer want referenced?
Can it improve its own operating instructions when a workflow changes?

That is where the productivity gain is.

The model matters, obviously. But the scaffolding around the model matters more than people admit: state, tools, memory, permissions, scheduling, logs, profiles, and recovery paths.

Hermes is starting to feel like the right layer for that: not a magic assistant, not a toy, not a demo — just a practical agent runtime I can keep shaping around my own workflows.

That is probably the highest compliment I can give an automation system:

I am starting to trust it with boring jobs.

I Put TestFlight Behind a Preflight Gate Before It Can Touch the Repo

Todd Sullivan — Tue, 16 Jun 2026 08:04:53 +0000

I added a preflight gate to an iOS + watchOS app build this week because the build script had a small but annoying property: it mutated the repo before it proved the repo was safe to build.

The script stamps CFBundleVersion with a timestamp before generating the Xcode project and archiving for TestFlight. That is useful because every upload gets a monotonically increasing build number.

It is also exactly the wrong thing to do before quality checks.

If formatting or tests fail after that stamp, the working tree is now dirty for a reason unrelated to the change I was trying to ship. Small thing, but this is how build scripts become suspicious. You run them, they fail, then you have to separate your actual diff from the build system's leftovers.

So I moved the build into a fail-fast shape:

echo "==> Running preflight checks..."
"$PROJECT_DIR/scripts/preflight.sh"

BUILD_NUMBER=$(date +%Y%m%d%H%M)
sed -i '' "s/CFBundleVersion: \"[^\"]*\"/CFBundleVersion: \"$BUILD_NUMBER\"/" "$PROJECT_DIR/project.yml"

The important bit is ordering. preflight.sh runs before the build number is touched.

The gate currently has four checks:

# 1. no local surprises
git -C "$PROJECT_DIR" status --porcelain

# 2. formatting is enforced, not auto-mutated
swiftformat --lint \
  "$PROJECT_DIR/SessionzApp/" \
  "$PROJECT_DIR/SessionzKit/Sources/" \
  "$PROJECT_DIR/SessionzKit/Tests/" \
  "$PROJECT_DIR/SessionzWatch/" \
  --config "$PROJECT_DIR/.swiftformat"

# 3. local Swift package tests
swift test --package-path "$PROJECT_DIR/SessionzKit" --parallel

# 4. backend edge-function unit tests
deno test "$PROJECT_DIR/supabase/functions/_lib/"

That last line is the one I care about most.

The app has a Claude-backed plan generation flow. The mobile app sends goals, equipment, available weights, and a small training context to a backend function. The backend validates auth/subscription state, calls Claude, then returns structured JSON that becomes local SwiftData models.

I do not want the only tests for that boundary to live in an app simulator.

So I pulled the pure logic out of the Edge Function handlers into _lib/ modules and tested that directly with Deno:

// subscription.ts
export function isActiveStatus(status: string | null): boolean {
  return status === "active" || status === "trialing";
}

export function validateProductId(productId: unknown): string | null {
  return typeof productId === "string" && productId.trim().length > 0
    ? productId
    : null;
}

Same idea on the Swift side. The AI coach needs compact summaries of recent training, not an unbounded dump of every logged set. PerformanceSummary condenses the last 42 days into short lines like:

Dumbbell Floor Press: best 20 kg × 8, trained 3×
Push-up: best 20 reps (bodyweight), trained 2×

That now has regression coverage for the edge cases that tend to rot quietly: bodyweight vs weighted sets, zero-weight entries, sorting by training frequency, limiting output size, and ignoring sets outside the six-week window.

The result is not fancy CI. It is just a local quality gate that refuses to make a TestFlight archive unless the repo is clean, formatted, and both sides of the AI boundary pass tests.

This is the kind of infrastructure I like for AI features: boring checks around the non-boring part.

The model can be probabilistic. The system around it should not be.

Source: Recent Sessionz commits: scripts/preflight.sh, TestFlight build integration, Swift regression tests for plan/performance logic, and Deno tests for backend Edge Function helpers.

Tags: ai, swift, testing, devops

Status: published

I Replaced Hardcoded Workouts with a Claude-Generated Plan System

Todd Sullivan — Tue, 09 Jun 2026 08:03:48 +0000

I spent this week ripping out the fixed-routine part of a SwiftUI workout app and replacing it with a small AI planning system.

Not a chatbot bolted onto the side. The app asks for goals, reads a compact set of real constraints, calls Claude through a backend function, then stores the resulting 7-day plan locally in SwiftData.

The old version was simple: two hardcoded push/pull routines and a fixed exercise library. Fine for a prototype, but too rigid. If the user only has dumbbells up to 25kg, trains three days a week, and had a rough sleep week, the app needs to know that before it suggests anything.

The new flow is:

user configures equipment once
user enters goals, capped at 300 chars
app reads a small HealthKit summary if permission exists
Supabase Edge Function verifies auth and subscription state
Edge Function calls Claude
app parses the returned JSON into SwiftData models
watchOS session tracking uses the generated plan day

The important bit is the prompt shape. I did not want prose back from the model. I wanted app data.

const userMessage = `Goals: ${(goals as string).slice(0, 300)}
Equipment: ${(equipment as string[]).join(", ")}
Available weights (kg): ${(weights as number[]).join(", ")}${healthContext}
Schema: ${PLAN_JSON_SCHEMA}`;

The system prompt is deliberately boring:

Output ONLY valid JSON matching the schema provided.
No explanation. No markdown fences.
Exercise descriptions: 1 sentence max.
Metric units only.
Include warmup and rest seconds.

That boring prompt is the product decision. The UI does not need motivational filler. It needs PlanDay, WarmupExercise, and PlannedExercise rows that can be rendered, edited later, and tracked on the watch.

The response schema is inline and maps directly onto the local model:

{
  "planName": "string",
  "days": [
    {
      "dayNumber": 1,
      "isRestDay": false,
      "warmup": [{ "order": 1, "name": "string", "duration": "string" }],
      "exercises": [
        {
          "order": 1,
          "name": "string",
          "sets": 3,
          "repTargetLow": 8,
          "repTargetHigh": 12,
          "suggestedWeightKg": 20.0,
          "restSeconds": 90
        }
      ]
    }
  ]
}

I also put the API call behind a server-side gateway instead of calling Claude directly from iOS. That gives me three useful controls:

JWT verification before generation
subscription check before spending tokens
monthly usage tracking per user

The current limit is 10 plan generations per month. With the compact prompt, the target budget is roughly 800 input tokens and 600 output tokens per generation. That keeps AI cost predictable instead of letting a mobile text field become an unbounded invoice generator.

On-device still matters here. The generated plan, session logs, equipment inventory, and active plan state live locally in SwiftData. The network is only used to create a new plan. Running a workout should not depend on an API being online.

The lesson: AI features get much easier to ship when you stop treating the model response as content and start treating it as a typed boundary.

Prompt in. JSON out. Validate, store, render, track.

That is less magical than a chat UI, but it is much closer to software.

Source: Recent SwiftUI/watchOS app work: replaced hardcoded routines with a Claude-backed 7-day plan generator, HealthKit summary input, Supabase Edge Function gateway, SwiftData local storage, and monthly usage limits.

Tags: ai, swift, ios, claude

Status: published

The iOS 26 Deep-Link Bug That Only Happened When the App Was Already Open

Todd Sullivan — Tue, 02 Jun 2026 08:04:23 +0000

I spent a chunk of this week chasing a bug that looked like auth, then routing, then Supabase, then Expo Router.

Actual root cause: iOS 26 was receiving the magic link, but the running app never got the warm-start URL in JavaScript.

Cold start worked:

app not running
tap myapp://auth/callback?code=...
iOS launches the app
Expo Router mounts auth/callback
PKCE code exchange succeeds

Warm start failed:

app already open
tap the same magic link
app comes foreground
no JS Linking event
user is still logged out

No crash. No useful exception. Just a login flow that works from killed state and fails from running state, which is exactly the kind of mobile bug that eats a day.

The Native Part: SceneDelegate Still Matters

On iOS 26, warm-start URLs are delivered through the scene lifecycle:

func scene(_ scene: UIScene, openURLContexts URLContexts: Set<UIOpenURLContext>) {
  for context in URLContexts {
    forwardURL(context.url)
  }
}

The Expo app already had an AppDelegate, but warm-start delivery needed a UIWindowSceneDelegate as well. The slightly non-obvious bit was how to forward the URL.

Calling RCTLinkingManager directly looked reasonable, but it only posts the React Native URL notification. expo-linking was not listening to that path in this app. Routing through AppDelegate.application(_:open:options:) let ExpoAppDelegate forward the URL to all of its subscribers, including Expo's linking module:

private func forwardURL(_ url: URL) {
  let app = UIApplication.shared
  if let appDelegate = app.delegate as? AppDelegate {
    _ = appDelegate.application(app, open: url, options: [:])
  } else {
    RCTLinkingManager.application(app, open: url, options: [:])
  }
}

That got the URL into JS.

Then iOS 26 added one more trap.

The Key Window Trap

The system log had the clue: key window is null.

The app creates its UIWindow in didFinishLaunchingWithOptions, before the UIWindowScene exists. startReactNative calls makeKeyAndVisible(), but at that point the window is not attached to a scene yet.

On iOS 26, that meant scene:openURLContexts: never fired for warm-start links.

The fix was to associate the existing window with the scene and call makeKeyAndVisible() again after windowScene is set:

func scene(
  _ scene: UIScene,
  willConnectTo session: UISceneSession,
  options connectionOptions: UIScene.ConnectionOptions
) {
  guard let windowScene = scene as? UIWindowScene else { return }

  if let appDelegate = UIApplication.shared.delegate as? AppDelegate,
     let window = appDelegate.window {
    window.windowScene = windowScene
    window.makeKeyAndVisible()
  }
}

After that, scene:openURLContexts: fired reliably. Verified on the iOS 26 simulator: native SceneDelegate log, AppDelegate openURL log, then the JS linking handler.

The JS Part: Don't Route During Foreground Animation

Once the URL reached JS, the first instinct was to navigate to /auth/callback and let that screen do the exchange.

That was fragile for two reasons.

First, during warm start the app is still moving through UISceneActivationStateForegroundInactive. I saw the URL arrive at 10:56:09.749; foreground transition finished at 10:56:10.140. Navigation commands during that window can be silently dropped.

Second, auth/callback may already be mounted. A useEffect([]) on that screen will not re-run just because another link arrived.

So I moved warm-start exchange into the always-mounted root layout:

const linkSub = Linking.addEventListener("url", ({ url }) => {
  const parsed = Linking.parse(url);
  const code = typeof parsed.queryParams?.code === "string"
    ? parsed.queryParams.code
    : null;

  if (code) {
    supabase.auth.exchangeCodeForSession(code).catch(() => {});

    const unsub = useAuthStore.subscribe(({ session, signingIn }) => {
      if (session && !signingIn) {
        unsub();
        router.replace("/");
      }
    });

    setTimeout(unsub, 30_000);
  }
});

The important part is the split:

native code guarantees warm-start URLs reach Expo linking
root layout handles every warm-start URL event
auth callback remains as the cold-start/race fallback
navigation waits until the auth store has a settled session

The Takeaway

Deep links are not one flow. They are at least two:

cold start: launch app into a URL
warm start: deliver URL into an already-running app

If you only test the cold path, your auth flow can look perfect while the common real-world case is broken.

For Expo + React Native apps on newer iOS versions, I now treat warm-start deep links as their own integration test: app open, link tapped, native URL delivery observed, JS Linking event observed, auth state settled, navigation after state settle.

That is more ceremony than a one-line Linking.addEventListener, but it is the difference between a demo login flow and a production one.

Source: Recent Expo / React Native auth work: iOS 26 warm-start deep-link fix, SceneDelegate URL forwarding, PKCE exchange moved to root layout.
Tags: ios, reactnative, devops, javascript

The Silent Code Path: When Your AI Runs on Camera But Not on Gallery

Todd Sullivan — Fri, 29 May 2026 08:17:16 +0000

Here's a bug that's easy to miss and harder to debug: your AI runs perfectly on one input path, silently does nothing on another, and there's no error — just missing results.

I hit this recently in an on-device inspection app. The flow is straightforward: capture a photo (camera or photo library), run AI hazard detection, overlay bounding boxes, trigger violation alerts if needed.

Camera captures worked great. Gallery picks saved the photo fine. But no bounding boxes appeared, no alerts fired. The AI was just... not running.

What Actually Happened

Here's the stripped-down structure:

// Camera capture handler
const handleRequestCapture = async (uri: string) => {
  await savePhoto(uri);
  await detectAndSave(uri);   // ✅ AI runs
  checkViolationAlerts();
};

// Library pick handler  
const handleLibraryPick = async (uri: string) => {
  await savePhoto(uri);
  // ❌ detectAndSave was never called
};

The library path was added later, modelled on the save logic but not the full inference pipeline. No crash. No warning. Just missing detections.

The fix was six lines — copy the detectAndSave + alert block into the library handler. But finding it took longer than writing the fix.

Why This Pattern Is Easy To Ship

On-device AI inference tends to get wired in during the happy path. You build the camera flow first, the AI gets integrated there, everything works in demos. Then you add the gallery pick as a "nice to have" — and because you're only thinking about the file I/O, you copy the save logic but not the inference call.

There's no type error. No lint warning. The function name (handleLibraryPick) doesn't imply inference should happen. The photo saves successfully, so from the app's perspective, nothing broke.

Lessons From The Fix

1. Treat every input path as a first-class citizen.
If AI inference is core to your feature, it should run regardless of how the image arrived. Camera, gallery, deep link, background upload — all of them.

2. Extract inference into a shared pipeline.
After the fix I refactored toward a single processPhoto(uri) function that both handlers call. Now if the pipeline changes, it changes in one place.

const processPhoto = async (uri: string) => {
  await savePhoto(uri);
  await detectAndSave(uri);
  checkViolationAlerts();
};

const handleRequestCapture = (uri: string) => processPhoto(uri);
const handleLibraryPick = (uri: string) => processPhoto(uri);

3. End-to-end tests that cover input variants.
Unit tests on detectAndSave wouldn't have caught this — the function worked fine, it just wasn't being called. What you need is an integration test that exercises each entry point and asserts that inference results exist.

4. Visibility into what ran.
This is the sneaky part: when AI silently doesn't run, you need observability to know it happened. Adding a log line (AI inference: skipped | ran on <uri>) to your inference call sites makes this class of bug immediately obvious in development.

The Broader Pattern

This isn't unique to AI. Any side-effect that only gets wired to one code path — analytics events, permission checks, cache invalidation — can silently miss inputs added later. But with AI inference it's particularly painful because the failure mode is invisible and the debugging surface is narrow (you're looking at model output, not a thrown error).

Build the pipeline once, call it everywhere.

DEV Community: Todd Sullivan

My On-Device ML Feature Was Registered, But Never Scheduled

Registration is not scheduling

The other trap: background queues and SwiftData

One shared path for HealthKit updates

The tests that caught the boring failures

The takeaway

The mAP50 That Lied to Me: A Debugging Story About On-Device Safety AI

A good score, hiding a broken detector

Fixing it

The training run that fought back

Knowing when to stop

Validating for real, this time

The takeaway

My Local AI Assistant Got Worse When I Remembered Too Much

My Local AI Assistant Got Worse When I Remembered Too Much

I Made My Voice Agent Feel Faster by Streaming Sentences, Not Audio

Silent Data Loss Is Worse Than a Failed Export

The dangerous null

false is not “unanswered”

Do not guess from raw JSON

Audit failures before they escape

The AI-assisted part

The Exporter Was Easy. Making It Deterministic Was the Work.

The Exporter Was Easy. Making It Deterministic Was the Work.

Side effects at the edge

Validation blocks, but still logs

Why this matters for AI-built software

AI Features Need Product Edges, Not Just Better Prompts

One button, several states

Credits should be part of the response

Offline is not a server error

The actual lesson

I Replaced My Old AI Agent Stack with Hermes — and It Finally Feels Like Infrastructure

I Put TestFlight Behind a Preflight Gate Before It Can Touch the Repo

I Replaced Hardcoded Workouts with a Claude-Generated Plan System

The iOS 26 Deep-Link Bug That Only Happened When the App Was Already Open

The Native Part: SceneDelegate Still Matters

The Key Window Trap

The JS Part: Don't Route During Foreground Animation

The Takeaway

The Silent Code Path: When Your AI Runs on Camera But Not on Gallery

What Actually Happened

Why This Pattern Is Easy To Ship

Lessons From The Fix

The Broader Pattern

The dangerous `null`

`false` is not “unanswered”