DEV Community: (ハオ)

How I put Apple Intelligence into my app Look

(ハオ) — Sun, 28 Jun 2026 13:30:00 +0000

So this is a little story about how I added on-device AI to my macOS app, Look, using Apple's Foundation Models framework. I wanna share what was good, what was painful, some real code, and why I think building on this now is actually a good chance even if the model is still small.

First, what is Look?

Look is a keyboard launcher for macOS. You press a hotkey, you type, and it find your apps, files and folders super fast. The search core is written in Rust so it is quick.

Look Repository

But here is the thing, people don't really think in keywords. They think in questions. Like "that pdf i downloaded last week", or "what is 18% of 240", or "who painted Guernica". Normal keyword search can't really help with that.

So I added a small AI layer to Look with Apple Intelligence. My one rule was simple: AI is best effort and it must never block or slow down the search. If the model is not available, not ready, or it just fails, then Look works exactly like before. Nothing breaks.

Why on-device and not a cloud LLM?

For a launcher, three things matter a lot. It should be private (your file names never leave the Mac), it should be free (i hit this tool like 100 times a day, i can't pay per token), and it should start fast (no network round trip). Apple's Foundation Models gives you a ~3B model running locally with a clean Swift API. For a launcher that is just the perfect shape.

Ref: https://developer.apple.com/documentation/foundationmodels

How i actually integrated it into Look

One important decision first. In Look, ALL the Foundation Models code lives behind one provider type. The rest of the app never import FoundationModels directly, it only talks to my own little abstraction. So the on-device model is just one swappable "tier", and later i can add a cloud model without touching everything.

1. Always check availability first

Apple Intelligence is not always there. Maybe the user have old hardware, maybe they didn't turn it on, maybe the model is still downloading. You can read all of this from SystemLanguageModel.default.availability, and you must handle every case.

#if canImport(FoundationModels)
import FoundationModels
#endif

var availability: AIProviderAvailability {
    #if canImport(FoundationModels)
    guard #available(macOS 26, *) else {
        return .unavailable(.requiresNewerOS)
    }
    switch SystemLanguageModel.default.availability {
    case .available:
        return .available
    case .unavailable(.deviceNotEligible):
        return .unavailable(.requiresNewerOS)
    case .unavailable(.appleIntelligenceNotEnabled):
        return .unavailable(.appleIntelligenceNotEnabled)   // "Turn on Apple Intelligence in System Settings."
    case .unavailable(.modelNotReady):
        return .unavailable(.modelNotReady)                 // "The on-device model is still downloading."
    case .unavailable(let other):
        return .unavailable(.other("\(other)"))
    @unknown default:
        return .unavailable(.other("Unknown availability state"))
    }
    #else
    return .unavailable(.requiresNewerOS)
    #endif
}

The trick here is two layers. #if canImport(FoundationModels) let the app compile on a lower deployment target, and #available(macOS 26, *) guard the runtime. In Look every AI entry point check availability.isAvailable and just return nil if not. The AI toggle in Settings show a green check or a reason, but the search itself never break.

Ref: https://developer.apple.com/documentation/foundationmodels/systemlanguagemodel

2. Structured output with @Generable (this is the best part)

Okay this is where Foundation Models really shine. Instead of parsing the model text by hand, you just describe the shape of the output with @Generable and @Guide, and the framework give you back exactly that type. In Look i use it to turn a vague query like "the folder where i keep invoices" into a typed plan that my Rust engine can run.

@available(macOS 26, *)
@Generable
private struct EngineQueryPlan {
    @Guide(description: "The kind of thing the user wants to find.")
    let kind: PlanKind

    @Guide(description: "Just the keywords to search for, with filler words removed.")
    let searchText: String

    @Generable
    enum PlanKind: String {
        case app, file, folder, recent, any
    }
}

// guided generation, no string parsing. the result IS an EngineQueryPlan
let session = LanguageModelSession(instructions: Self.instructions)
let response = try await session.respond(
    to: query,
    generating: EngineQueryPlan.self
)
return response.content.asIntent()   // -> Rust prefix grammar, e.g. app -> a"..., file -> f"...

No JSON prompt engineering, no parsing, no begging the model to "please answer in valid JSON". You just get a typed and validated struct. This remove a whole category of bugs honestly.

In Look this only run as a rescue. When the fast local search return zero result, then i rewrite the natural language query into engine grammar and search again. If local search already found something, the model never run. So you never pay latency for the queries that were already fine.

Ref: https://developer.apple.com/documentation/foundationmodels/generating-swift-data-structures-with-guided-generation

3. Streaming short answers

For the inline answer card (like "what is the capital of Norway"), i stream the response so the text appear word by word, and i cap the length so the answer stay small and launcher sized.

func answer(query: String) -> AsyncThrowingStream<String, Error>? {
    AsyncThrowingStream { continuation in
        let task = Task {
            do {
                let session = LanguageModelSession(instructions: Self.answerInstructions)
                let options = GenerationOptions(maximumResponseTokens: 220)
                for try await snapshot in session.streamResponse(to: query, options: options) {
                    if Task.isCancelled { break }
                    continuation.yield(snapshot.content)   // snapshot.content is cumulative
                }
                continuation.finish()
            } catch {
                continuation.finish(throwing: error)
            }
        }
        continuation.onTermination = { _ in task.cancel() }
    }
}

Two small details worth stealing. Each streamed snapshot.content is the cumulative answer (not a delta), and the on-device model is the last fallback in Look. Calculator and web sources are tried first, the model only run when nothing else hit.

Ref: https://developer.apple.com/documentation/foundationmodels/languagemodelsession

4. Prewarm to hide the latency

The first call to the model have a noticeable startup cost. A launcher is opened and closed all the time, so i prewarm() a resident session the moment the window open or the query start to look like a sentence.

@MainActor
final class AppleIntelligenceWarmer {
    static let shared = AppleIntelligenceWarmer()
    private var session: LanguageModelSession?

    func prewarm() {
        let warm = session ?? LanguageModelSession()
        session = warm
        warm.prewarm()
    }
}

Ref: https://developer.apple.com/documentation/foundationmodels/languagemodelsession/prewarm()

The advantages (what is genuinely great)

Private by default. Everything run on the Mac. For a launcher that see every file name you type, this is not a nice to have, this is the whole reason i felt okay to ship AI at all.
Zero marginal cost. No API key, no billing, no rate limit. I can let the model run on every qualifying query and not watch a meter.
Really nice Swift API. The @Generable guided generation is the star. Getting a typed and validated struct straight from the model is so good.
It is already on the device. I don't bundle a multi gigabyte model in my app, the OS ship and update the weights.
Clean and composable. Sessions, instructions, streaming, options, prewarm. The pieces just snap together into a real feature.

The disadvantages (what still hurt, honestly)

Availability is a moving target. It need macOS 26 and eligible hardware, the user have to enable it, and the model can still be downloading. A big chunk of your users just won't have it. So you must design the no-AI path as the default, not the exception.
It is a small model. ~3B is great for classification, rewriting and short factual answers, but it is clearly weaker than big cloud models at reasoning, long writing or niche knowledge. Scope your feature to what a small model do well.
First use latency. Without prewarm the first token is slow. Even with it, on-device inference is not free. For a sub 100ms tool this is a real thing to budget around.
Guardrails can surprise you. Generation can get refused or interrupted by safety guardrails. You have to catch those error and degrade nicely. In Look it just fall back to the raw query.
macOS 26 only and the API is young. You are building on a brand new surface that will keep changing. The #if canImport plus #available gymnastics is the price of being early.

Why building on it now is a chance, not a risk

Okay here is the main point I wanted to make. The local model only get better, and my code don't have to change. Apple ship and upgrade the on-device weights through OS updates. The same LanguageModelSession call i write today will, one year from now, be backed by a smarter and faster model. For free. With zero work from me. The @Generable plan i defined keep returning the same type, it just get filled in more accurate.

That really change the math. Adopting Foundation Models early is not a bet that today's small model is good enough for everything, because it clearly is not. It is a bet that the floor rise under you. The integration work (the provider abstraction, the availability handling, the prewarm, the fallback design) is the hard part, and it is a one time cost. Once Look speak fluent Foundation Models, every OS update become a silent capability upgrade.

So I am shipping on-device AI in Look today. Scoped tight to what a 3B model do well, always optional, never blocking. And i treat it like infrastructure that i grow into, not a feature i have to perfect right now. If you are building a Mac app, that is the chance. Get the plumbing right while the API is still young, and ride the model improvements for free.

Thanks for reading. If you wanna try it, go grab Look and turn on the AI toggle in Settings.

Reference links

Foundation Models framework: https://developer.apple.com/documentation/foundationmodels
SystemLanguageModel: https://developer.apple.com/documentation/foundationmodels/systemlanguagemodel
LanguageModelSession: https://developer.apple.com/documentation/foundationmodels/languagemodelsession
Guided generation (@Generable / @Guide): https://developer.apple.com/documentation/foundationmodels/generating-swift-data-structures-with-guided-generation
HIG: Generative AI: https://developer.apple.com/design/human-interface-guidelines/generative-ai
WWDC25, Meet the Foundation Models framework: https://developer.apple.com/videos/play/wwdc2025/286/

I built look because Spotlight didn’t match how I work.

(ハオ) — Wed, 08 Apr 2026 08:24:09 +0000

An open-source, community-driven launcher for macOS. A lightweight alternative to Spotlight and Raycast.

Github: Repository

Some projects start from a big idea. This one started from a hundred tiny annoyances.

Open app -> Switch -> Search file -> Open Finder -> Copy something -> Paste the wrong thing -> Search again -> Open Terminal for one quick command. Go back.

Nothing was broken, but my attention kept getting sliced into small pieces.
After enough of those slices, even simple work feels heavy. I wanted one small window that could keep me in flow.

That window became Look.

And yes, I tried built-in tools first. Spotlight works for many people, but in my daily workflow I kept hitting the same friction:

slower than I wanted when moving quickly
ranking that often missed my actual intent
no single place for app search, file flow, clipboard reuse, and quick utility actions So this was not “reinvent for fun.” It was a practical response to repeated friction.

The feeling I was chasing

I did not want the "most feature-rich launcher."

I wanted a launcher that feels:

quiet: does not pull me into UI complexity
fast: responds quickly enough that I trust muscle memory
local-first: does not depend on cloud round trips for basic actions
honest: predictable behavior, clear shortcuts, no mystery state

That is the core of look.

What `look` does in real daily usage

1) Search apps, files, and folders from one place

The default flow is intentionally simple:

open launcher
type
Enter

When I need precision, I use scoped prefixes:

a"term -> apps only (great when app names collide with file names)
f"term -> files only (for docs, notes, exports)
d"term -> folders only (for project navigation)

r"pattern -> regex (for structured matching)

This tiny prefix system sounds small, but in practice it saves a lot of "did I search the wrong thing" moments.

Path-style search also works (git/books-pc/readme style fragments), so navigation feels closer to how developers actually think.

2) Clipboard history that is practical, not gimmicky

Clipboard history is one of those features that either becomes daily muscle memory or gets ignored.

I built it to be useful in normal work:

c" -> show recent clipboard entries
c"word -> filter by remembered keyword

Enter -> copy selected history item back to clipboard
delete in preview -> remove sensitive item from look history

The right-side preview matters. I can inspect long clipboard content before reusing it, instead of guessing from a short line.

This sounds minor, but if you work across Slack/email/code/docs all day, it removes a lot of repeated re-copying.

3) Command mode for utility actions in the same flow

Sometimes I do not want to open a new app for a quick operation.

Cmd+/ opens command mode.

Current built-ins:

calc for quick expression checks
shell for short one-off command execution
kill for force-killing a stuck app
sys for a quick system snapshot

Keeping these inside launcher means fewer context switches and less "open this app for one tiny action" behavior.

4) Keyboard details that save time

These shortcuts are where the day-to-day value shows up:

Cmd+F: reveal selected app/file/folder in Finder at exact location
Cmd+C: copy selected file/folder to pasteboard so I can paste directly
Cmd+Enter: send current query to browser search when local result is not enough
Cmd+H: open full in-window help for quick reminder

I also made sure the launcher does not keep aggressively forcing itself to front after opening another app, because that behavior breaks trust quickly.

5) Better behavior for Vietnamese typing

One personal quality-of-life detail: accented input.

I added normalization so matching is friendlier for Vietnamese typing patterns, including handling cases like đ -> d and combining marks.

Practical example: tẻrminal still matches Terminal.

It is a small implementation detail, but it makes the launcher feel like a tool for real people, not just demo text.

Under the hood: why this architecture

I wanted native UX quality and tight performance control. The project is split into three layers:

SwiftUI/AppKit shell (macOS integration + interaction)
Rust core engine (indexing, matching, ranking)
FFI bridge (small stable boundary between the two)

.
├── apps/
│   └── macos/
│       └── LauncherApp/
├── core/
│   ├── engine/
│   ├── indexing/
│   ├── matching/
│   ├── ranking/
│   └── storage/
├── bridge/
│   └── ffi/
├── docs/
├── scripts/
└── assets/

This split helps in practice:

UI iteration stays quick on native macOS side
ranking/matching logic is testable and deterministic in Rust
bridge API remains narrow and easier to reason about (search, record_usage, reload_config, translate)

Ranking, relevance, and "why this result is on top"

Search quality is always the hardest part of launcher UX.

look currently combines multiple signals:

exact/prefix/fuzzy text matching
contains/token matching
path-aware scoring
result kind bias (app/folder/file)
usage frequency and recency influence

Blank query has dedicated browse scoring so launcher is useful before typing.

One interesting real-world behavior: if a settings entry has been used heavily, it can stay high even when a new app was just opened. That is not random; it follows the current weighting model.

I added regression tests around these ranking edge cases after seeing them in real usage data.

Clipboard tradeoffs (the honest version)

Clipboard monitoring uses a lightweight polling strategy plus burst sampling for rapid copy sequences.

This gives a good responsiveness/performance balance, but it is still a practical engineering tradeoff, not a perfect event-capture system under extreme bursts.

I prefer this kind of explicit tradeoff over pretending something is "real-time perfect" when it is not.

Installation

Homebrew

brew tap kunkka19xx/tap
brew install --cask look

Update:

brew upgrade --cask kunkka19xx/tap/look

Common install and first-run issues (please read)

Gatekeeper blocks first launch

If macOS blocks launch:

right-click Look.app -> Open -> confirm
or System Settings -> Privacy & Security -> Open Anyway

[IMAGE_PLACEHOLDER_GATEKEEPER]

`Cmd+Space` conflict with Spotlight

By default, Spotlight owns that shortcut.

To use Cmd+Space with look:

open System Settings -> Keyboard -> Keyboard Shortcuts... -> Spotlight
disable or rebind Show Spotlight search

Hotkey does not work after fully quitting app

Relaunch once:

open "/Applications/Look.app"

Then hotkey behavior returns.

Configuration: users can tune a lot

I did not want a "my defaults are your destiny" app.

look supports both settings UI and config-file tuning.

Config file:

~/.look.config

Reload command:

Cmd+Shift+;

What users can configure:

indexing roots/depth/limits (file_scan_roots, file_scan_depth, file_scan_limit)
exclusion rules (file_exclude_paths, app_exclude_paths, app_exclude_names, skip_dir_names)
privacy/network controls (translate_allow_network)
backend logging level (backend_log_level)
launch behavior (launch_at_login)
UI customization (tint, blur, opacity, font, border, background mode)

This is important to me because people work differently. Some want strict minimalism, some want broader indexing, some care deeply about visual tuning.

Source, docs, and project links

GitHub: https://github.com/kunkka19xx/look
User guide: docs/user-guide.md
Backend guide: docs/backend-guide.md

Long-term goal (and honest constraint)

If the project gets enough traction, one of my top priorities is proper Apple signing/notarization for a smoother install experience.

Right now, users may need manual Gatekeeper "Open Anyway" steps. I want to remove that.

But this requires real cost (Apple Developer program) and extra release pipeline work. If community adoption grows, that is where I will invest first.

Closing

I built look because I was tired of losing focus to small, repeated friction.

This is still evolving, but it already gives me what I hoped for: less switching, less noise, and more actual work done.

If you try it, I would love candid feedback:

where ranking feels great
where ranking feels wrong
what feature would make this your default launcher

Repo: https://github.com/kunkka19xx/look