On-Device AI in React Native with Expo: Local LLM, Speech-to-Text, Vision & Fully Local Agents

There are billions of phones out there that can already run AI models locally, but most apps don't use that at all. Sending every AI request to a server is wasteful, especially for simple ones, and there are a lot of those. It costs money, it adds latency and the user's data leaves their phone. At scale it also adds up in the energy and water data centers use. I think it's important that we start making use of what these devices can already do. expo-ai-kit is my small contribution to that.

It's an Expo module for running AI on the device. This post shows how to use it for text generation, speech-to-text and vision in an Expo/React Native app. Everything runs locally on the phone, using the models and frameworks that iOS and Android already ship with (Apple Foundation Models, Apple Speech and Vision, ML Kit on Android).

saidkaban / expo-ai-kit

On-device AI for Expo and React Native. Local LLM, speech-to-text, vision, and embeddings. No API keys, no cloud.

expo-ai-kit

On-device AI for React Native. Local LLM, speech-to-text, vision, and embeddings for Expo apps and React Native apps with Expo modules installed. No API keys, no cloud.

Documentation · Examples · npm · Changelog

• Private and offline. Inference runs on the device. Prompts, audio, and images never leave it.
• No API key, no per-request cost. Nothing to provision, meter, or proxy.
• One TypeScript API for both platforms. Apple Foundation Models, Speech, and Vision on iOS ML Kit and MediaPipe on Android; LiteRT-LM for downloadable models such as Gemma, Qwen, and Phi.
• Pay only for what you use. Every feature is opt-in, so unused native code, models, and permissions never ship. Zero runtime JavaScript dependencies.

Install

npx expo install expo-ai-kit

Turn on the features you use in your app config, then make a native build. Until you enable a feature, none of its native code is compiled into your app…

View on GitHub

Setup

npx expo install expo-ai-kit

Features are off by default. You enable the ones you need in app.json, and only those get compiled into your app:

{
  "expo": {
    "plugins": [
      ["expo-ai-kit", { "llm": true, "speech": true, "vision": true }]
    ]
  }
}

You need Expo SDK 54+ and a development build. Expo Go won't work since this is native code.

On-device LLM

On iOS 26+ this uses Apple Foundation Models (Apple Intelligence devices). On Android it uses the ML Kit Prompt API on supported devices.

import { isAvailable, prepareBuiltInModel, sendMessage } from 'expo-ai-kit';

if (await isAvailable()) {
  await prepareBuiltInModel();

  const { text } = await sendMessage([
    { role: 'user', content: 'Explain gravity in one sentence.' },
  ]);
}

prepareBuiltInModel() matters on Android because the model may need to download the first time.

Streaming:

import { streamMessage } from 'expo-ai-kit';

const { promise, stop } = streamMessage(
  [{ role: 'user', content: 'Write a very short story.' }],
  (event) => setText(event.accumulatedText)
);

const { text } = await promise;

The calls are stateless, so you send the whole message history each time. You're not limited to the built-in models either. You can download capable models like Gemma 4 and run them on both iOS and Android with setModel(). There's also generateObject for JSON output, generateText for tool calling and a Vercel AI SDK provider under expo-ai-kit/ai.

Speech-to-text

iOS 26+ uses Apple's SpeechAnalyzer, Android 12+ uses ML Kit speech recognition.

import {
  getSpeechRecognitionAvailability,
  prepareSpeechRecognition,
  requestSpeechPermissionsAsync,
  streamTranscription,
} from 'expo-ai-kit';

const availability = await getSpeechRecognitionAvailability({ locale: 'en-US' });
if (availability.status === 'downloadable') {
  await prepareSpeechRecognition({ locale: 'en-US' });
}

const permission = await requestSpeechPermissionsAsync();
if (!permission.granted) return;

const { promise, stop } = streamTranscription(
  (update) => setTranscript(update.text),
  { locale: 'en-US' }
);

// later
stop();
const { text } = await promise;

For recorded audio files there's transcribe({ audio: { uri } }). On iOS it also returns timestamps. On Android, files are processed at real-time speed, so it's better for short voice notes than long recordings.

Vision

iOS uses the Vision framework and needs no config. Android uses ML Kit.

OCR:

import { recognizeText } from 'expo-ai-kit';

const result = await recognizeText({ uri: photo.uri });
console.log(result.text);

Image labels:

import { labelImage } from 'expo-ai-kit';

const labels = await labelImage({ uri: photo.uri }, { maxResults: 5 });
// [{ label: 'Dog', confidence: 0.97 }, ...]

Background removal:

import { removeBackground } from 'expo-ai-kit';

const cutout = await removeBackground({ uri: photo.uri });
// cutout.uri is a PNG with a transparent background

URIs from expo-image-picker or expo-camera work directly. There's also detectFaces for face bounding boxes.

Combining speech and the LLM

Speech and text generation don't block each other, so you can transcribe and then send the result to the model:

const { promise, stop } = streamTranscription((u) => setTranscript(u.text));

// when the user is done talking
stop();
const { text: question } = await promise;

const { text: answer } = await sendMessage([
  { role: 'system', content: 'Keep answers short.' },
  { role: 'user', content: question },
]);

Some limitations

• Not every device supports every feature. Check availability before calling and have a fallback.
• Some vision features don't work in the iOS simulator, so test on a real device.
• Only one LLM generation can run at a time. A second call throws INFERENCE_BUSY.
• These are small models. They work well for things like summarizing, extracting info and short replies, less so for long or complex tasks.

Where this is going

The part I'm most excited about is on-device agents. Most of the pieces are already in the library. The model can call your functions with generateText, it can hear the user with speech-to-text and it can see with vision. Put those together and you get an agent that lives on the phone and does things for the user, like setting a reminder or finding a photo, without a single request going to a server.

Small models aren't great at long multi-step tasks yet, but they're getting better fast. That's the direction I'm taking expo-ai-kit.

Links

• Docs: https://expo-ai-kit.dev
• GitHub: https://github.com/saidkaban/expo-ai-kit
• npm: https://www.npmjs.com/package/expo-ai-kit

If you try it, I'd like to hear how it goes. Issues and PRs are welcome. A star on GitHub also helps the vision I mentioned at the beginning.

Comments

[Odd Eggs Studio]

We fall back to Apple's on-device speech after a daily cloud allowance in our app, so the availability check with a "downloadable" state is really handy. Does SpeechAnalyzer report locales like Vietnamese as downloadable through it too, or do some just come back unsupported?

[Dave Kurian]

On Apple's side the split is supported versus installed. SpeechTranscriber.supportedLocales includes locales that are downloadable but not on the device yet, installedLocales only what is already there, and supportedLocale(equivalentTo:) returns nil for a locale it does not support. So some locales can come back unsupported rather than downloadable. I'd check the target device's supported list for vi-VN before promising offline transcription, and treat an unsupported result as the cue to stay on your cloud path.

[Odd Eggs Studio]

Supported versus installed is the distinction I was missing, thanks. A nil from supportedLocale(equivalentTo:) for vi-VN is the check to run before the app ever offers offline transcription.