We've all typed a reply, hit post, and regretted the tone ten minutes later. I built Comment Vibe, a Chrome extension that shows a small badge next to any comment box (LinkedIn, X, YouTube, generic contenteditable) with the detected tone of what you're typing — and offers a kinder rewrite when you sound harsh.
The interesting part isn't the product. It's the constraint: everything runs on-device via Chrome's built-in Gemini Nano. No server, no API key, no analytics, no network requests. The whole extension is a handful of raw MV3 files — no npm, no bundler. When your pitch is "your comments never leave your machine," the best privacy audit is cat content.js.
This post is everything I wish I'd known about the Prompt API before starting.
The Prompt API is actually stable now
The LanguageModel API went stable for extensions in Chrome 138 and stable on the open web in Chrome 148. No flags, no origin trial tokens. Chrome downloads Gemini Nano (~2–4 GB) automatically on first use, on eligible hardware (desktop, 16 GB+ RAM, >4 GB VRAM, 22 GB free disk).
That hardware floor is the real trade-off: you get zero infra cost and perfect privacy, but a chunk of your users simply can't run it. Your UX must degrade gracefully — mine shows an availability check in the popup and otherwise stays silent.
Lesson 1: You still have to probe two namespaces
Older builds expose the legacy window.ai.languageModel; newer ones the global LanguageModel. Different Chrome builds ship different namespaces depending on where they were in the origin-trial timeline, so I probe both — and feature-detect initialPrompts support with a retry ladder:
async function constructSession() {
if (typeof LanguageModel !== 'undefined') {
const avail = await LanguageModel.availability();
if (avail === 'unavailable') throw new Error('unavailable');
const initialPrompts = [{ role: 'system', content: SYSTEM_PROMPT }, ...FEW_SHOT];
try {
return sessionMetadata(await LanguageModel.create({ initialPrompts, ...LANGUAGE_OPTIONS }), true);
} catch (error) {
if (!supportsReducedOptionsRetry(error)) throw error;
try {
return sessionMetadata(await LanguageModel.create({ initialPrompts }), true);
} catch (reducedError) {
if (!supportsReducedOptionsRetry(reducedError)) throw reducedError;
// older builds without initialPrompts support
return sessionMetadata(await LanguageModel.create({ systemPrompt: SYSTEM_PROMPT }), false);
}
}
}
if (typeof window !== 'undefined' && window.ai?.languageModel) {
const { available } = await window.ai.languageModel.capabilities();
if (available === 'no') throw new Error('unavailable');
return sessionMetadata(await window.ai.languageModel.create({ systemPrompt: SYSTEM_PROMPT }), false);
}
throw new Error('Chrome AI not found');
}
Unsupported options throw TypeError or NotSupportedError, so each fallback strips one capability and retries. Ugly, but it's what "works on every Chrome since 127" costs.
Lesson 2: Sessions are stateful — clone, don't reuse
This one bit me. A Prompt API session keeps its conversation history. If you prompt the same session repeatedly, every analysis appends to its context: each call gets slower, and eventually you overflow the session quota.
The fix: keep one warm base session (creating a session is expensive — it can load the model), and clone() a throwaway per analysis. A clone starts fresh from the initial prompts and is cheap:
const acquired = await getSession(); // cached, warm base session
const session = acquired.cloneCapable
? await acquired.session.clone() // fresh context, no model reload
: acquired.session;
try {
// ... prompt on the clone
} finally {
session.destroy?.();
}
Bonus: baking the system prompt and few-shot examples into initialPrompts means they survive the clone — you don't re-send them with every request.
Lesson 3: Prefill the assistant response to force JSON
Gemini Nano is a small model. Ask it politely for JSON and you'll get JSON most of the time, wrapped in markdown fences some of the time, and a friendly paragraph on bad days. Two tricks got me to reliable structured output:
Few-shot examples in the session's initial prompts, plus an assistant prefill — you hand the model the beginning of its own answer and it continues from there:
const messages = [
{ role: 'user', content: `Classify the tone of this comment:\n"${text}"` },
{ role: 'assistant', content: '{"sentiment":', prefix: true }, // ← prefill
];
const raw = await session.prompt(messages);
const result = normalize(parseResponse('{"sentiment":' + raw));
The prefix: true flag means the model must start its response mid-JSON. As a side effect, sentiment is guaranteed to be the first field — which enables the streaming trick below.
Lesson 4: Stream, and act on the first parseable field
With promptStreaming() the badge colors in as soon as the sentiment value is readable, while the reason and rewrite are still generating:
async function streamPrompt(session, messages, onEarlySentiment, signal) {
let raw = '';
let signaled = false;
for await (const chunk of callPrompt(session, 'promptStreaming', messages, signal)) {
// older builds stream the cumulative text, newer ones stream deltas
raw = (chunk.length > raw.length && chunk.startsWith(raw)) ? chunk : raw + chunk;
if (!signaled && onEarlySentiment) {
const m = raw.match(/"(positive|neutral|negative|toxic)"/);
if (m) { signaled = true; onEarlySentiment(m[1]); }
}
}
return raw;
}
Note the chunk handling: older Chrome builds stream the cumulative text, newer ones stream deltas. That one line handles both.
Lesson 5: Never trust the schema — normalize everything
Even with prefill, a small model improvises key names. Ask for reason, receive tone, analysis, description, or explanation. So the parser maps every synonym the model has ever invented back to the expected shape:
const reasonRaw = source.reason ?? source.tone ?? source.analysis
?? source.description ?? source.explanation ?? '';
const rewriteRaw = source.rewrite ?? source.suggestion
?? source.alternative ?? source.improved_version ?? null;
And when JSON parsing fails entirely, there's a last-ditch fallback: regex the raw text for sentiment keywords. A wrong-but-plausible badge beats a broken one.
Lesson 6: Debounce like you mean it
On-device inference is free but not fast. I wait 900 ms after the last keystroke before prompting, wire an AbortController through so a new keystroke cancels the in-flight analysis, and tag requests with an id so a slow stale response can't overwrite a newer result.
Lesson 7: The API zoo is uneven — check before you depend
Chrome ships seven built-in AI APIs, at very different maturity levels:
| API | Status | Worth using? |
|---|---|---|
Prompt (LanguageModel) |
✅ Stable | Yes — the workhorse |
| Translator / Language Detector | ✅ Stable | Yes — they compose beautifully |
| Summarizer | ✅ Stable | Yes |
| Rewriter | 🧪 Dev trial | No — ran a full origin trial, then went back to a flag |
| Writer | 🧪 Dev trial | Not yet |
| Proofreader | 🧪 Origin trial | Not yet |
The Rewriter story is the cautionary tale: an API can complete an origin trial and still not graduate. Don't build a hard dependency on anything pre-stable — I do rewrites with the Prompt API instead.
The stable trio composes nicely though: Language Detector identifies what language you're typing in, Translator localizes the feedback and the rewrite into it (cached per language pair, falling back to English), so the suggestion is directly pasteable.
Was "no build tooling" worth it?
Honestly: yes, and more than I expected. No supply chain, nothing to audit but the shipped files, Load unpacked straight from the repo, and the Web Store zip is just zip -r with excludes. The cost is no TypeScript and no bundler-era conveniences — for ~1000 lines of content script, that trade is easy.
Try it
- Extension: Comment Vibe on the Chrome Web Store (Chrome 138+, desktop)
- Playground for all seven built-in AI APIs — live availability check for your browser plus an interactive demo of each: https://dzienko.dev/comment-vibe/
- Source (MIT): [your repo link]
If you're building on the Prompt API and hitting weirdness I didn't cover, drop a comment — I've probably seen it.
Top comments (0)