DEV Community

Cover image for Prompt API: 3 Steps to On-Device AI in Chrome Extensions
ExtensionBooster
ExtensionBooster

Posted on Originally published at extensionbooster.net

Prompt API: 3 Steps to On-Device AI in Chrome Extensions

Originally published on ExtensionBooster.

The Chrome Prompt API lets you run natural-language prompts against Gemini Nano directly on the user's device — no API keys, no per-request billing, and no data leaving the browser. If your extension's AI feature is stuck behind cloud costs and latency, this is the fastest way out. Here's what it takes to ship your first on-device prompt.

Why on-device beats a cloud call

Adding AI to an extension through a cloud provider means five problems that scale with every new user: per-token costs, user text sent to external servers, dead features offline, rate limits, and round-trip latency. The Prompt API removes all five, because the model runs locally.

Check the hardware floor before promising anything. Gemini Nano needs: Windows 10/11, macOS 13+, Linux, or ChromeOS (Chromebook Plus); 22 GB of free disk for the model; 16 GB of RAM in CPU mode (or 4+ GB VRAM in GPU mode); an unmetered connection for the initial download; and Chrome 138 or newer. Users below that floor should still get your extension — with a graceful fallback instead of AI.

Your first prompt in 3 steps

  1. Check whether the model is available:
const available = await LanguageModel.availability({
  expectedInputs: [{ type: "text", languages: ["en"] }],
  expectedOutputs: [{ type: "text", languages: ["en"] }],
});
Enter fullscreen mode Exit fullscreen mode

availability() returns "available", "downloadable", "downloading", or "unavailable" — the last one means the hardware doesn't qualify.

  1. Create a session:
const session = await LanguageModel.create();
Enter fullscreen mode Exit fullscreen mode
  1. Prompt it:
const result = await session.prompt("Summarize this in one sentence: " + articleText);
Enter fullscreen mode Exit fullscreen mode

No SDK to install, no credentials to manage, no billing dashboard.

What you can build with it

  • Page Q&A — seed the session with page text via initialPrompts, then let users ask "What are the pricing tiers?"
  • Content classification — a system prompt that replies with a single category turns any article into a labeled record.
  • Contact extraction — ask for names, emails, and phone numbers as JSON from selected text.
  • Moderation — rate user-generated content "safe", "warning", or "block" before it reaches your backend.

Two production habits matter from day one. First, stream anything longer than a sentence with session.promptStreaming() so users see text appear instead of staring at a spinner. Second, clean up: call session.destroy() when done, and watch session.inputUsage against session.inputQuota — a contextoverflow event means it's time for a fresh session or a summarized history.

The fallback pattern that keeps reviews clean

On-device AI isn't everywhere yet, so wrap every call in a check: if availability() returns "unavailable", fall back to a cloud API, a simpler heuristic, or hiding the feature. The other non-ready states deserve different handling — while the model is "downloadable" or "downloading", LanguageModel.create({ monitor(m) { ... } }) can report download progress to the user. Most users decide within the first few minutes whether an extension is worth keeping, and an AI feature that fails silently will cost you the install.

One honest caveat: the Prompt API is currently in an origin trial (Chrome 138+), so the surface may change before a stable release. Build against it, but keep fallbacks tight.

Key takeaways

  • The Prompt API runs Gemini Nano on-device: zero per-request cost, no data leaving the browser, no network required.
  • Three lines of code — availability check, session, prompt() — ship a working AI feature.
  • It accepts text, images, and audio input out of the box.
  • Always stream long outputs and destroy sessions to avoid memory leaks and bad reviews.
  • Check availability on every load and fall back cleanly on unsupported hardware.

The full guide covers multimodal input, TypeScript setup with @types/dom-chromium-ai, and session cloning in more detail — read the complete Chrome Prompt API guide for extension developers, and if you're getting an AI-powered extension ready for the store, ExtensionBooster's extension growth tools can help with launch and optimization.

Which on-device feature are you shipping first — page Q&A, classification, or moderation? Tell us what you're building and how the Prompt API handles it.

Top comments (0)