DEV Community: Todd Sullivan

HerdCount is Live on the App Store — From Blog Post to Shipped Product in Two Weeks

Todd Sullivan — Wed, 13 May 2026 10:31:32 +0000

Two weeks ago I wrote about building an offline-first livestock counter with YOLOv8 and CoreML. Today it's a real product on the App Store.

HerdCount — Count your flock, even offline

£3.99. No subscription. No cloud. No account. Pay once, use forever.

What It Does

Point your phone at livestock or plants. Tap a button. Get the count.

HerdCount uses on-device AI (YOLOv8 + CoreML) to detect and count chickens, sheep, cattle, and plants from a single photo — in under a second, with zero internet required.

Why I Built It

I work with on-device computer vision professionally — building Axsy Smart Vision, an AI-powered field inspection platform for Salesforce. Retail planogram detection, product identification, compliance scoring — all running on-device.

But the agricultural space has a simpler, more immediate problem: counting animals is tedious and error-prone. Farmers do it by eye, multiple times a day. Miss one sheep and you're searching hedgerows at dusk.

The same on-device ML pipeline I use for retail product detection works beautifully for livestock. So I built it.

The Technical Stack

YOLOv8 — trained on livestock datasets, converted to CoreML
On-device inference — runs on the iPhone's Neural Engine, no cloud round-trip
Offline-first — works in fields, barns, anywhere with no signal
Swift/SwiftUI — native iOS, 8.7 MB total
Export — CSV via AirDrop, email, or Files app

The model handles overlapping animals (tuned IoU threshold at 0.3 rather than the default 0.5) and lets you tap false positives to remove them. Manual +/− adjustment before saving.

What I Learned Shipping It

1. App Review is opinionated. Apple rejected the first submission because the category detection UI wasn't clear enough. Fair feedback — I redesigned it and it's better for it.

2. The model is the easy part. Training YOLOv8 and converting to CoreML took a weekend. The other 90% was UI polish, edge cases, CSV export formatting, and App Store screenshots.

3. Pricing matters. I went with £3.99 one-time purchase. No subscription, no ads, no data collection. Farmers are practical people — they'll pay for a tool that works but won't tolerate dark patterns.

4. On-device AI is a real differentiator. Every competing app I found requires internet. That's a non-starter for someone standing in a field in rural Wales.

From Blog to Product

The Dev.to post about the technical approach got genuine engagement — @gimi5555 asked about NMS strategies for clustered animals, which led to a good discussion about density estimation as a fallback.

That conversation validated the approach. Two weeks later, it's a shipped product.

If you're working with on-device ML and sitting on something useful — ship it. The App Store review process is less scary than it looks, and real users find real problems you'd never catch in development.

HerdCount on the App Store →

Built by RT Sullivan Consulting. I write about on-device AI, Salesforce field apps, and shipping real products at dev.to/toddsullivan.

Shipping to TestFlight Without Fastlane: Raw xcodebuild, Auto-Incrementing Builds, and One Neat Provisioning Trick

Todd Sullivan — Wed, 13 May 2026 08:01:59 +0000

Most iOS CI tutorials reach for Fastlane. It's the default assumption. And Fastlane is fine — but it's also another Ruby toolchain to maintain, another layer of abstraction between you and xcodebuild errors, and another thing that breaks when Xcode updates.

For a small side project, I wanted zero overhead. So I wrote a release script using plain xcodebuild and xcrun altool, and wired it into GitHub Actions. Here's what I learned.

The Setup

The app is a no-dependency iOS project (SwiftUI, SwiftData, zero SPM packages). One scheme, one target, distributes via the App Store. The goal: git push → trigger workflow → build, sign, upload to TestFlight.

Auto-incrementing build numbers for free:

BUILD_NUMBER="${1:-$(git -C "$REPO_ROOT" rev-list --count HEAD)}"

That's it. Every commit bumps the count. No build number file to commit, no race conditions in CI, no manual tracking. Pass it straight into xcodebuild:

xcodebuild archive \
  ...
  CURRENT_PROJECT_VERSION="$BUILD_NUMBER"

TestFlight requires monotonically increasing build numbers. Git commit count gives you that automatically. I've seen people use timestamps (too long), semver patch (manual), or a counter file in the repo (merge conflicts). Commit count is cleaner.

The Provisioning Problem — and the Fix

This is where most raw-xcodebuild scripts fall apart. The export step (xcodebuild -exportArchive) needs an ExportOptions.plist with the exact provisioning profile UUID. But the UUID changes every time you renew the profile.

The usual answer is "hardcode it in your plist and update manually." That's the kind of thing you forget for six months and then debug for two hours.

Better approach: extract the UUID from the archive you just built, then inject it at export time.

# Pull the embedded profile from the freshly-built archive
EMBEDDED="$ARCHIVE/Products/Applications/MyApp.app/embedded.mobileprovision"
PROFILE_UUID=$(security cms -D -i "$EMBEDDED" | plutil -extract UUID raw -)

# Copy it into the Provisioning Profiles directory (xcodebuild looks here)
cp -f "$EMBEDDED" "$HOME/Library/MobileDevice/Provisioning Profiles/$PROFILE_UUID.mobileprovision"

# Write a temp ExportOptions with the exact UUID from *this* archive
cp "$EXPORT_OPTIONS" "$EXPORT_OPTIONS_TMP"
plutil -replace "provisioningProfiles.com.example.myapp" \
  -string "$PROFILE_UUID" "$EXPORT_OPTIONS_TMP"

# Now export using that temp plist
xcodebuild -exportArchive \
  -archivePath "$ARCHIVE" \
  -exportPath "$EXPORT_DIR" \
  -exportOptionsPlist "$EXPORT_OPTIONS_TMP"

The profile UUID in your ExportOptions is always current, because it came from the archive itself. Renew the cert, re-download the profile, and nothing breaks.

GitHub Actions Signing

For CI, the certificate lives in a secret as a base64-encoded .p12. The workflow decodes it into a temporary keychain:

security create-keychain -p "$KEYCHAIN_PASS" build.keychain
security import /tmp/cert.p12 -k build.keychain -P "$P12_PASSWORD" \
  -T /usr/bin/codesign
security set-key-partition-list -S apple-tool:,apple: -s \
  -k "$KEYCHAIN_PASS" build.keychain

The -T /usr/bin/codesign flag is critical — without it, the keychain will prompt for a password interactively mid-build, which hangs CI forever. The set-key-partition-list step is what makes it work without prompts.

The Full Flow

workflow_dispatch
  → checkout (full depth for commit count)
  → import cert into ephemeral keychain
  → write App Store Connect API key
  → ./scripts/release.sh
      → xcodebuild archive
      → extract profile UUID from archive
      → inject UUID into ExportOptions
      → xcodebuild -exportArchive
      → xcrun altool --upload-app

About 8-12 minutes wall clock on a macos-26 runner. No Ruby, no gems, no Fastlane plugins.

When Fastlane Still Makes Sense

If you're managing multiple targets, schemes, environments, or a team with custom lanes — Fastlane earns its complexity. But for a single-target indie app? Raw xcodebuild is readable, debuggable, and requires no maintenance beyond "Xcode updated, did the flags change?"

The full script is about 70 lines of bash. That's the whole pipeline.

Building Personalised On-Device ML for Women's Health: No Cloud, No Population Averages

Todd Sullivan — Mon, 11 May 2026 13:22:24 +0000

Most health AI is built on population data. Your symptoms are averaged against thousands of other people, and you get a generalised prediction that fits nobody perfectly.

I took a different approach with Menopause Intelligence — an iOS app I've been building that predicts high-symptom days for women in perimenopause and menopause.

The entire model runs on-device, trained on the individual user's own data. No cloud, no population averages, no third-party data sharing.

The problem with cloud-based health AI

Population models work when you want average answers. But perimenopause is deeply individual. Two women with identical ages and similar symptom profiles can have completely different biometric triggers.

The app's job is to tell a user her patterns — not what typically happens to women like her.

The ML pipeline

Features: Seven signals per day, all from HealthKit/Apple Watch:

Basal body temperature delta vs 7-day mean
HRV (raw + delta from personal rolling average)
Sleep efficiency and deep sleep %
REM sleep %
Resting heart rate
Cycle day (if logged)

Key design decision: We use deltas from the user's personal baseline, not absolute values. A resting HR of 62 bpm means different things for different people. What matters is whether it's elevated for you.

Label: Composite symptom severity score for day D+1 (hot flashes, brain fog, fatigue, mood)

Model: CoreML + CreateML Components. Runs via a silent weekly background task (BGProcessingTask). The app retriggers training automatically as new data accumulates.

Cold start: The first 30 days use a rule-based weighted scorer as a fallback. Not as accurate, but keeps the app useful while data accumulates.

The data architecture

Everything is local:

HealthKit → DailyLog (SwiftData) → Feature engineering → CoreML inference

No backend. No analytics SDK. CloudKit sync between devices uses end-to-end encryption. Health data never touches our servers — because we don't have any.

This isn't just a privacy stance. It's architecturally simpler and removes a whole category of compliance risk. For a health app in this category, "no backend" is a feature you can market.

The feedback loop

User-reported symptoms feed back into the next training cycle. Every hot flash logged, every mood entry — they sharpen the model for that specific user.

This is the same feedback pattern I've used in other on-device vision work: user corrections become training data. The model gets more accurate over time for the individual, not just better at the general case.

What I've learned building personalised on-device ML

Minimum data is a real UX problem. 30 days before predictions activate feels long to a user who downloaded the app because she's struggling now. You have to be honest about why, and give her something useful in the meantime.

Baseline drift matters. A user's "normal" changes over the course of perimenopause. The rolling average window needs to adapt — a fixed 7-day mean becomes stale if someone's baseline HRV is trending down over months.

Privacy is the product. In women's health, trust is everything. "Your data never leaves your device" isn't a footnote — it's the headline. It changes the conversation with users who've been burned by other health apps.

The stack

UI: SwiftUI (iOS 17+)
Data: SwiftData + CloudKit
Biometrics: HealthKit
Prediction: CoreML + CreateML Components
Subscriptions: StoreKit 2
Watch: watchOS companion + WidgetKit

More on this as it gets closer to launch.

The Fastlane gym Export Options Trap (and Why Your Provisioning Profile Is Being Silently Ignored)

Todd Sullivan — Mon, 11 May 2026 08:01:51 +0000

Spent a few hours last week debugging a CI failure that had no right to be as subtle as it was. The build archived fine, but exportArchive kept dying with:

error: exportArchive: requires a provisioning profile with the App Groups feature.

The frustrating part: the AppStore provisioning profile was correct. I had just renewed it, decrypted it on the runner, and confirmed the App Group entitlement was in there. The keychain had it. So why was xcodebuild not finding it?

The Trap

The Fastlane gym action accepts export_options: in two forms:

A path to an existing .plist file
A Hash of options it will write to a temp plist

I was passing a Hash — and inside that Hash I had a plist: key pointing to my own plist file, thinking gym would merge or defer to it. It does not.

When you pass a Hash, gym writes that Hash to a temp plist and hands it directly to xcodebuild. The plist: key inside the Hash is not special — xcodebuild does not recognise it, ignores it silently, and you end up with a minimal plist that has no provisioningProfiles key at all.

The temp plist gym generated looked like this:

<dict>
  <key>method</key>
  <string>app-store</string>
  <key>uploadSymbols</key>
  <true/>
  <key>plist</key>
  <string>RELEASE_exportOptionsPlist_Store.plist</string>
</dict>

No provisioningProfiles. Under manual signing, xcodebuild fell back to automatic profile resolution at export time — which on a clean GitHub Actions runner cannot find the app-group-bearing profile you carefully installed. Build fails. Misleading error. Whole thing looks like a profile problem when the profile was never consulted.

The Fix

Pass export_options: as a path string, not a Hash:

gym(
  scheme: "MyApp",
  configuration: "Release",
  export_options: "./fastlane/RELEASE_exportOptionsPlist_Store.plist"
)

Your plist should include explicit provisioningProfiles:

<key>provisioningProfiles</key>
<dict>
  <key>com.example.myapp</key>
  <string>MyApp AppStore Profile</string>
</dict>

Gym passes the path straight to xcodebuild -exportOptionsPlist. Your file is read. No temp plist, no silent key stripping.

Why This Catches People Out

The Hash form is in basically every Fastlane tutorial. It looks clean. Gym does not warn you when it discards unrecognised keys. The only signal is in verbose gym output — if you compare the temp plist it writes against what you expected, the provisioningProfiles block is missing.

App Groups make the failure mode worse because they require an exact profile match. Without entitlements like App Groups, xcodebuild automatic selection might accidentally find something usable. With App Groups, it always fails hard.

What I Do Now

For any iOS app with entitlements — App Groups, Push Notifications, iCloud, anything — I keep an explicit export_options.plist checked into the repo and pass it as a path. The Hash form is fine for a basic app. The moment signing gets complicated, you want the plist under version control and gym out of the business of generating it.

One less thing the CI runner has to figure out on its own.

Hello DEV! I'm Todd — an AI Engineer Who Builds Real Things

Todd Sullivan — Fri, 08 May 2026 10:17:42 +0000

Hey DEV community 👋

I'm Todd — an AI engineer based in the UK. I spend most of my time building systems that actually use AI rather than talking about using AI.

What I work on

Claude/LLM integrations — wiring Claude into real engineering workflows. Co-authoring code, automated pipelines, using it where it genuinely adds signal rather than noise.
On-device AI — computer vision models that run on iOS with no internet connection. Edge inference, model size tradeoffs, the gap between lab accuracy and field accuracy.
Developer tooling — zero-config test runners, CI pipelines with AI in the loop, the kind of boring-but-important stuff that makes teams faster.
Personal AI infrastructure — building a persistent AI assistant that runs 24/7, has real memory, and can actually take actions. Not a chatbot demo.

Why I'm here

I write about the engineering decisions, the tradeoffs, and the things that didn't work as well as the things that did. Practical stuff, real code, honest takes.

If you're building with AI (not just prompting it) — I'd love to connect.

A few recent posts

Say hi 👇

Claude as a CI Co-pilot: Debugging Apple Signing Hell So You Don't Have To

Todd Sullivan — Fri, 08 May 2026 08:02:47 +0000

This week I spent a few hours debugging a fastlane CI pipeline that was failing on every single run with Apple provisioning errors. I paired with Claude the entire time. Here's what that actually looks like — not the polished "AI helped me code!" version, but the messy, real one.

The Setup

iOS build pipeline. Fastlane + match for code signing. The CI runner kept blowing up at exportArchive with:

error: exportArchive: requires a provisioning profile with the App Groups feature

Except — the profile absolutely contained the App Groups entitlement. I inspected the decrypted .mobileprovision manually. It was there. Xcodebuild was lying.

Where Claude Actually Helped

I dumped the failing lane, the temp plist gym was generating, and the error into the conversation. Claude caught something I'd missed: when you pass export_options: as a Hash in your Fastfile, gym writes that hash directly to a temp plist — but any plist: key inside the hash is treated as a literal value, not a file reference. The external plist file I was trying to load? Never actually loaded.

The fix was one line: pass export_options: as a path string instead of a hash. Gym then loads the file properly. The patch I'd been writing into the plist at runtime actually started landing.

# Before (broken) — Hash form ignores your plist: key
gym(
  export_options: {
    method: "app-store",
    plist: "RELEASE_exportOptionsPlist_Store.plist"
  }
)

# After (working) — path string makes gym actually load the file
gym(
  export_options: "RELEASE_exportOptionsPlist_Store.plist"
)

The Second Problem

Once that was fixed, the build still failed intermittently. Reason: when match renews a provisioning profile, Apple appends a serial number suffix to the name (e.g. match AppStore com.example.app 1777460891). My Fastfile, pbxproj, and export plist all hardcoded the old name. After any renewal, xcodebuild couldn't find it.

Claude suggested a pattern: after match runs, read the actual installed profile name from the sigh_* environment variable, then patch both pbxproj and the export plist at runtime before the build starts. The dynamic name becomes the single source of truth.

# Read the actual name after match sets it
profile_name = ENV["sigh_#{bundle_id}_appstore_profile-name"]

# Patch pbxproj
system("sed -i '' 's/match AppStore #{bundle_id}.*/#{profile_name}/g' path/to/project.pbxproj")

# Patch export plist
system("/usr/libexec/PlistBuddy -c 'Set :provisioningProfiles:#{bundle_id} #{profile_name}' ExportOptions.plist")

What Made This Work

Claude didn't just hand me code — it helped me build a mental model of what was actually happening. The difference between Hash vs path-string in gym's API is documented somewhere in fastlane's source, but it's not obvious. Same with match's environment variable naming convention.

The conversation was more like pair programming with someone who'd read the entire fastlane codebase than a Stack Overflow search. I'd describe what I was seeing, Claude would reason about what the tool chain was doing internally, and we'd narrow down the root cause.

The commits ended up cleaner too. Because I understood why the fix worked, the commit messages were precise. Co-authored lines show up in git blame: Co-Authored-By: Claude Opus 4.7.

The Honest Take

This isn't magic. It's a multiplier on existing knowledge. If you don't understand code signing at all, Claude's explanations will help but you'll still spend time learning the domain. If you do understand it — like I do — it collapses the debugging loop from hours to minutes.

The gnarly CI/CD problems that used to require tribal knowledge or a very specific Stack Overflow answer from 2019 are now tractable in a single session.

That's the real unlock.

Building an Offline-First Livestock Counter with YOLOv8 and CoreML

Todd Sullivan — Wed, 06 May 2026 08:02:54 +0000

I built a livestock counting app for smallholders. No internet required, no subscription, no server. You take a photo of your chickens, sheep, or cattle, and it counts them — entirely on-device. Here's how it actually works.

The Problem

Smallholders regularly need to count animals. In a field. In a barn. Where there's no signal. The apps that exist are either generic (bad accuracy for farm animals), require a server round-trip, or charge you monthly to count your own chickens. None of that made sense to me.

So I built Muster.

The Stack

iOS 17, SwiftUI, SwiftData — no third-party dependencies, ships as a one-time purchase
YOLOv8n — the nano variant, exported to CoreML format
Apple's Vision framework — handles the ML request lifecycle, orientation correction, and bounding box coordinate normalisation
Zero backend — no server, no account, no ongoing cost

The model is small enough to run on-device without breaking a sweat. YOLOv8n sits at about 6MB in CoreML format. On an iPhone 13 it processes a typical farm photo in under 400ms. That's fast enough that it feels instant.

How Inference Works

The VisionService wraps a VNCoreMLModel and fires a VNDetectRectanglesRequest against the input image. The key detail here is orientation: photos from iOS cameras carry EXIF orientation metadata, and if you don't account for it before passing frames to Vision, your bounding boxes are in the wrong coordinate space.

let ciImage = CIImage(image: uiImage)!
    .oriented(forExifOrientation: imageOrientationToExifOrientation(uiImage.imageOrientation))
let handler = VNImageRequestHandler(ciImage: ciImage, options: [:])

After inference, each detection gets mapped to a DetectedObject with a normalised bounding box and confidence score. The UI overlays dot markers on the image — one per detection — and lets the user tap any to dismiss false positives before saving.

Preset Categories vs. Tap-to-Select

The tricky UX question was: how does the user tell the app what to count? I landed on two modes:

Preset categories — bird/poultry, sheep, cattle, plants — each mapped to specific COCO class IDs. The detection filter is applied post-inference, so the model still runs once regardless.
Tap-to-select — the user taps one example item in the photo, and the app counts all detections with the nearest matching class. Good for "other" categories the presets don't cover.

The confidence thresholds needed tuning. Out of the box, YOLOv8n is conservative — I loosened the threshold for the farming categories because the cost of missing a sheep is higher than the cost of an occasional false positive that the user can tap away.

The Proof-of-Count Card

The feature I shipped last was the shareable count card — a rendered image showing the annotated photo, count total, category, timestamp, and app branding. Smallholders sometimes need to show a headcount to a vet, insurer, or land agent. A screenshot is clunky. A clean card with metadata looks like a document.

This was a SwiftUI View rendered to UIGraphicsImageRenderer — no external libraries, no server-side rendering.

What I Learned

Running ML inference at the edge is surprisingly painless on modern Apple hardware. CoreML and Vision do the heavy lifting. The hard part isn't the inference — it's the UX around confidence thresholds, false positive handling, and giving users enough control without overwhelming them.

If you're building anything that involves counting, detecting, or classifying on-device: the YOLOv8n → CoreML pipeline is mature, well-documented, and genuinely fast enough for production use.

Muster is heading to the App Store soon. One-time purchase. No subscription. Count your flock. No signal needed.

When Your Training Data Pipeline Has Three Different Ideas About the Same Thing

Todd Sullivan — Mon, 04 May 2026 08:01:44 +0000

If you're building ML pipelines that consume data from multiple API endpoints, you've probably hit this: the same thing — a product, a user, a record — arrives in three subtly different shapes depending on which path it took to get to you.

We hit this in a computer vision training pipeline recently. The pipeline synthesises training images for product classifiers — takes seed images of known products, composites them into scene images, generates bounding box annotations, trains a model. Standard stuff.

The bug: seed images were being silently dropped during dataset preparation. Not erroring — just gone. The model would train on an incomplete dataset and we'd only notice when accuracy came back lower than expected.

Root cause: UID lookup using exact string match, but three different API callers were sending the same product reference in three different formats:

'Tesco Cornflakes Cereal 500G'        # raw label, spaces preserved
'tesco_cornflakes_cereal_500g'        # stringToFilename output, lowercase underscored
'Tesco_Cornflakes_Cereal_500G'        # case-preserved underscored (from external productCode)

The on-disk index used case-preserved underscored filenames. So if you came through the raw label path, your seed images were quietly dropped. No exception. No warning. Just a smaller dataset than you thought you had.

Why This Happens

Three different API routes, built at different times, by different people, each making a reasonable local decision about how to normalise a string. The bug only appears when you try to join across them using the output of one as the key into an index built from another.

The fix was to make the lookup tolerant — normalise both the incoming ref and the index key before comparison, so any of the three shapes resolves to the same entry.

def normalise_uid(uid: str) -> str:
    return uid.lower().replace(" ", "_")

Two lines. But the reason you need them is worth understanding.

The Broader Pattern

Silent data loss in ML pipelines is particularly nasty because:

It doesn't fail loudly. The pipeline completes successfully. The model trains. You get results. You just don't realise the results are for a smaller, different dataset than you intended.
The signal is weak. Lower accuracy could be bad data, bad hyperparameters, distribution shift, or a dozen other things. You might spend days investigating the model before you look at the pipeline.
It only manifests at scale. In dev, you're running with a handful of products. Everyone has clean, matching UIDs. In production, you have hundreds of products, multiple API callers, and the mismatch rate goes up.

What to Add to Your Pipeline

If you're building training data pipelines that consume product/entity references from multiple sources:

Assert dataset size at each stage. Expected 120 seed images for this batch? Assert that before training starts.
Log dropped items explicitly. Don't silently skip — log the UID that couldn't be resolved so you can catch shape mismatches immediately.
Normalise at ingestion, not lookup. Standardise the UID format the moment it enters your system, rather than trying to be tolerant at every lookup point downstream.
Cross-reference your callers. If you have multiple API endpoints that all feed the same pipeline, explicitly document which normalisation each one applies. It'll be someone else's problem in six months, and that someone might be you.

The actual ML work — model architecture, training loops, hyperparameter tuning — gets a lot of attention. The data pipeline that feeds it is equally important and tends to get much less scrutiny. Bugs there don't throw exceptions. They just quietly make your model worse.

YOLOv8 + CoreML on iOS: Shipping Offline Computer Vision That Actually Works in the Field

Todd Sullivan — Fri, 01 May 2026 08:01:59 +0000

I have been building a lot of server-side vision systems — cloud inference, GPU clusters, the whole stack. But a recent side project reminded me how compelling on-device AI still is, especially when you strip away the assumption of reliable connectivity.

The project: a livestock counting app for smallholders. Take a photo of your flock, tap one chicken, get a count back. No account, no subscription, no signal required. Just a model on the device doing its job.

Here is what I learned porting YOLOv8 into an iOS app via CoreML.

Why On-Device at All?

The obvious answer: barns and fields do not have 5G. But the less-obvious answer is more interesting — no server means no ongoing cost, no latency, and no privacy concern. The photo never leaves the phone. That is increasingly a selling point, not a footnote.

For small utility apps, cloud inference is overkill. You are paying per-inference and maintaining infrastructure to serve a model that could run on a £400 phone in under 200ms.

The Stack: YOLOv8n → CoreML → Apple Vision

The model is YOLOv8 nano (yolov8n), trained on COCO. Nano is the key decision — it is ~6MB, runs on the Neural Engine, and for categories like bird, sheep, cow the accuracy is genuinely good enough for a counting use case.

The conversion path:

pip install ultralytics coremltools
yolo export model=yolov8n.pt format=coreml nms=True

That gives you a .mlpackage. Xcode compiles it to .mlmodelc at build time and generates a Swift wrapper class automatically. The inference code is clean:

let config = MLModelConfiguration()
config.computeUnits = .all  // prefer Neural Engine

let mlModel = try MLModel(contentsOf: modelURL, configuration: config)
let vnModel = try VNCoreMLModel(for: mlModel)

let request = VNCoreMLRequest(model: vnModel)
request.imageCropAndScaleOption = .scaleFit

let handler = VNImageRequestHandler(cgImage: cgImage, orientation: orientation)
try handler.perform([request])

let results = request.results as? [VNRecognizedObjectObservation]

On a modern iPhone, yolov8n inference on a 640px image runs in roughly 50–80ms. Fast enough that it feels instant.

The Hard Part: Confidence Thresholds

COCO-trained YOLOv8 with default confidence thresholds performs well on textbook images. Real livestock photos are not textbook images. Partially-occluded animals behind fence posts, sheep that are mostly mud, chickens half-in-frame — these score lower confidence but are still valid detections you want to count.

I ended up with a final threshold of 0.25, vs the default 0.35–0.45 most tutorials recommend. The model exports with NMS baked in (conf=0.15, iou=0.65), and I apply a second filter in Swift at 0.25. This catches most real-world partial occlusions without drowning in false positives.

The other trick: let users tap to remove false positives rather than trying to tune away every edge case. Editable results beat perfect results. People accept "mostly right, I will tap off the fence post shadow" much better than "sometimes wrong with no recourse."

Tap-to-Identify Flow

Instead of forcing a category selection, users can just tap on one example object in the photo. The app finds the highest-confidence detection at that point, identifies its COCO class, and returns all detections of the same class.

// Vision uses bottom-left origin; UIKit uses top-left
let visionPoint = CGPoint(x: normalisedPoint.x, y: 1.0 - normalisedPoint.y)

let tapped = observations
    .filter { $0.boundingBox.contains(visionPoint) }
    .max(by: { $0.confidence < $1.confidence })

That coordinate flip (1.0 - normalisedPoint.y) is the kind of thing that wastes 45 minutes if you do not know to expect it.

What On-Device Vision Is Actually Good For

After building this, my take: on-device inference with a small COCO-trained model is a good fit for:

Counting / detection of common real-world objects (people, animals, vehicles, plants)
Apps that work in low-connectivity environments — field tools, outdoor apps, anything rural
Privacy-sensitive use cases — medical, personal, anything users would not want hitting a cloud API
One-off utility apps where server infrastructure is not justified

It is not a good fit for fine-grained classification (you need a domain-specific model), real-time video at scale, or anything needing more than ~80 COCO classes.

The stack — YOLOv8 + CoreML + Apple Vision framework — is mature, well-documented, and genuinely pleasant to work with. If you are building something where offline matters, it is worth the afternoon it takes to get running.

Bridging Apple Services to a Remote AI with MCP and SSH

Todd Sullivan — Wed, 29 Apr 2026 08:01:52 +0000

My AI assistant runs on a remote server. My Apple Mail, Calendar, and Messages live on my Mac. Getting them to talk to each other took an MCP server, some AppleScript, and an SSH reverse tunnel — and it works surprisingly well.

The Problem

I run an AI assistant on a remote EC2 instance. It's persistent, always-on, and handles scheduled tasks, cron jobs, and multi-step automations. But macOS services — Mail, Calendar, Messages — are locked to the local machine by design. AppleScript won't work over SSH, and there's no official API for Apple Messages.

The naive solution is to move everything to the cloud. The practical solution is to bring the cloud to the Mac via an MCP server.

What I Built

clawMCP is a TypeScript MCP server that exposes 12 tools across three Apple services:

Mail — list mailboxes, list/read/search messages
Calendar — list calendars, query events, create events, find free slots
Messages — list chats, read message history, send iMessages

It runs locally on my Mac as a launchd service, listening on localhost:3100. The remote AI instance connects to it through an SSH reverse tunnel that maps the remote localhost:3100 back to the Mac's port.

macOS (local machine)            EC2 (remote AI host)
+---------------------+          +---------------------+
| clawMCP server      | <--SSH-- | AI assistant        |
| port 3100           |  tunnel  | MCP client          |
| AppleScript -> Mail |          | SSE at :3100        |
| AppleScript -> Cal  |          +---------------------+
| sqlite3 -> chat.db  |
+---------------------+

Implementation Notes

Apple Mail and Calendar are handled via osascript — compiled AppleScript executed from Node.js. It's not elegant, but it's reliable and requires no third-party dependencies.

Apple Messages is different. iMessage doesn't expose an AppleScript API for reading messages (only sending). Instead, I read directly from ~/Library/Messages/chat.db — a SQLite database that stores the full local message history. A query joining message, chat, and handle tables gets you everything you need.

const messages = db.prepare(`
  SELECT m.text, m.is_from_me, h.id as sender, m.date
  FROM message m
  JOIN chat_message_join cmj ON cmj.message_id = m.ROWID
  JOIN chat c ON c.ROWID = cmj.chat_id
  JOIN handle h ON h.ROWID = m.handle_id
  WHERE c.chat_identifier = ?
  ORDER BY m.date DESC
  LIMIT ?
`).all(chatId, limit);

The tunnel is a simple persistent SSH connection with RemoteForward 3100 localhost:3100. A startup script launches the server and the tunnel together; launchd restarts both if either crashes.

What It Enables

With clawMCP connected, my AI assistant can:

Check my calendar before scheduling anything
Read and search my email without me copy-pasting threads
Look up recent iMessage history for context
Send me iMessages as a notification channel when long-running tasks complete

The last one is genuinely useful. When a 20-minute build finishes or a data pipeline completes, it pings me via iMessage. No email, no Slack — just a message on my phone from my AI.

Lessons Learned

MCP is the right abstraction here. It gives you tool discovery, type-safe parameters, and a standard transport layer. Building this as a raw HTTP API would have worked but required more glue code on the AI side.

SQLite is surprisingly powerful for this use case. Direct database reads are faster and more flexible than AppleScript for Messages. Just be careful with Full Disk Access permissions — macOS will silently fail without them.

The SSH tunnel is simpler than it sounds. One line in ~/.ssh/config, one RemoteForward directive, and it just works. No VPN, no port forwarding on the router, no cloud relay service.

If your AI lives somewhere other than your local machine, an MCP server + SSH reverse tunnel is a clean pattern for bridging local services. The code is all TypeScript, the surface area is small, and the result is an assistant that actually knows what's on your calendar.

Killing the Setup Endpoint: Moving Env Provisioning into GitHub Actions

Todd Sullivan — Mon, 27 Apr 2026 08:02:13 +0000

We had an API endpoint that set up environments. It claimed a pre-warmed org from a pool, authenticated two users, imported test data, installed a bundle, and published config. Six sequential shell calls. Runtime dependency on a server. Credentials scattered across process state. A pain to debug when it failed at step 4 of 6 at 2am.

The fix wasn't to rewrite the API. It was to stop having an API at all.

The move: GitHub Actions as the runtime

The entire setup sequence now lives in a single GitHub Actions workflow file. No server, no queue, no process isolation hacks. The runner is the environment — ephemeral, observable, retryable.

The key architectural shifts:

1. Parallelise everything that can be.

The old endpoint ran sequentially because it was Node.js with a queue. GitHub Actions has native parallelism via step grouping. Auth for two users? One run block, two background processes, wait. Test data import for multiple data keys? Matrix strategy, each key in its own parallel job. What was 6 serial calls is now 3 parallel groups.

Before: ~8 minutes end-to-end.
After: ~3.5 minutes.

2. Reusable workflows for cross-repo consumption.

The real unlock was workflow_call. Instead of every repo maintaining its own setup script or calling an API, they just reference the central workflow:

jobs:
  provision:
    uses: your-org/env-setup/.github/workflows/setup.yml@main
    with:
      environment: staging
      dataset: core
    secrets: inherit

secrets: inherit means the caller's secrets pass through automatically — define them once at the org level, every repo picks them up. No per-repo secret duplication. Rotate once, everything updates.

3. Credentials as artifacts, not environment variables.

Secrets (passwords, tokens, auth URLs) get written to a JSON file and uploaded as a run artifact with masking:

echo "::add-mask::$ADMIN_PASSWORD"

Downstream jobs download the artifact and unmask what they need. This means logs stay clean, credentials are scoped to the job that needs them, and there's no secret bleeding into env vars that outlive the step.

4. Non-secret outputs as workflow outputs.

Instance URLs, user IDs, org IDs — non-sensitive stuff — get published as jobs.<job>.outputs. Any downstream job can reference needs.provision.outputs.instanceUrl directly. Clean separation between sensitive and non-sensitive data.

What this replaced

The old flow required a running API server, a cloud function, and a manually-maintained shell script per environment type. When the server had a bad deploy, env setup broke. When the shell script fell out of sync with the API, you got silent failures.

Now it's a YAML file in a repo. PRs are reviewed. Failures show up in Actions logs with full context. Retries are a button click.

The unexpected benefit

Making setup a reusable workflow forced us to define its interface clearly: inputs, outputs, required secrets. That contract made the setup process legible to anyone on the team, not just the person who wrote the original API endpoint.

If you're running environment provisioning as a service endpoint and it's causing pain — consider whether it needs to be a service at all. Sometimes the right move is to make the CI runner do the work.

I Built a Persistent AI Assistant That Runs on My Mac

Todd Sullivan — Fri, 24 Apr 2026 09:05:25 +0000

I got tired of AI assistants that forget everything the moment a session ends. So I built one that doesn't.

It runs 24/7 on my Mac, has access to my files, GitHub, iMessage, email, and calendar. It knows who I am, what I'm working on, and what I said to it last week.

The core problem with stateless AI

Every time you open a new Claude or ChatGPT session, you start from zero. You re-explain your context. You re-establish what you're working on. You paste in the same background info.

This is fine for one-off tasks. It's terrible for an ongoing working relationship.

The memory architecture

Instead of in-context memory, I use files:

MEMORY.md — long-term curated knowledge. What matters, distilled.
memory/YYYY-MM-DD.md — daily logs. What happened, decisions made, things to remember.
USER.md — who I am, my stack, my communication style.
TOOLS.md — local setup specifics.

Every session, the agent reads the relevant files before doing anything. This is the continuity layer.

MCP for real-world access

Model Context Protocol (MCP) is what lets the agent actually do things — not just talk about them.

I use it for:

Apple Mail, Calendar, Messages via a local MCP server
GitHub via gh CLI
File system access
Browser automation (Puppeteer via Chrome DevTools Protocol)

The result

It's not a chatbot. It's closer to a part-time assistant who's always available and never forgets anything. The most useful thing isn't any single capability — it's that context persists.

I can say "remember the JWT issue from last week" and it actually knows what I mean.

The hardest part isn't the AI. It's designing the memory and context system that makes it feel coherent over time.