Your iPhone Is an OpenAI-Compatible LLM Server Now

TL;DR: Local LLM Server turns an iPhone, iPad, or Mac into an LLM server: Apple's on-device Foundation Models exposed over an OpenAI- and Ollama-compatible HTTP API. Any existing client library points at your phone's LAN address and just works. Free with ads; Pro is $1.99 without them.

Apple Intelligence ships a ~3B-parameter language model on every recent iPhone, running locally on the neural engine. It's genuinely capable for structured tasks—classification, extraction, summaries—and it costs nothing per token. The catch: Apple exposes it as a Swift framework. If your tooling speaks HTTP—Python scripts, LangChain, DSPy, a curl one-liner, literally the entire LLM ecosystem—there's no door in.

Local LLM Server is that door. Open the app, pick a port, hit Start Server, and the device serves at http://<your-lan-ip>:8080 with the two API shapes everything already understands: OpenAI's (/v1/chat/completions and friends) and Ollama's. Your OpenAI client library doesn't know it's talking to a phone.

The phone is the server.

What "compatible" actually covers

The parts I'm most pleased with, because they're the parts most wrappers skip:

• Structured outputs — response_format with a JSON schema drives Apple's guided generation, so you get valid JSON back, not JSON-shaped prose.
• Tool / function calling — the standard OpenAI client-executed loop.
• Real token counts — usage comes from the actual tokenizer, with a context_length_exceeded preflight instead of a mystery truncation.
• Embeddings — /v1/embeddings served from Apple's contextual embedding model, also on-device.
• A browser playground at GET /, optional Bearer auth, LAN or localhost-only binding, and App Intents so Shortcuts can start and stop the server.

The compatibility test I actually care about: existing chat clients talking to the phone.

There's a 90-test suite behind the API surface. One user wired the server into DSPy for prompt optimization against the on-device model, which is exactly the kind of thing I hoped people would do and did not want to explain was impossible.

Why bother, when you could call a cloud API?

Three answers, in my order of caring: it's private (prompts never leave the device—for some data that's not a preference, it's the requirement); it's free at the margin (bulk classification jobs cost zero tokens); and it's always with you (the server is in your pocket, on a plane, in a dead zone). The honest counterweight: it's a 3B model. It will not write your novel. For extraction, tagging, structured summaries, and glue-code intelligence, it's shockingly serviceable—and those are most of what scripts actually need.

The important line: on-device processing, no cloud round-trip.

Free vs Pro

The free app is ad-supported; Local LLM Server Pro is $1.99, one-time, no ads. Same server, same API. Pro briefly hit #5 on the US paid Developer Tools chart—a thin, volatile chart, so I won't pretend that's fame, but it did tell me this audience would rather pay two dollars than look at a banner, which matches how I feel about banners.

Tiny chart, still a real signal: Local LLM Server Pro at #5 in Developer Tools.

Requires iOS/iPadOS/macOS 26+ on Apple Intelligence-capable hardware (that's where the on-device model lives). If you've got a spare device on your desk, it's a free local inference endpoint you didn't know you owned.