Quentin Merle

Posted on Oct 14 Edited on Nov 4 Originally published at vibrisse-studio.dev AI-assisted

Chatbots Are Lazy Design: Building an 'Invisible UI' with In-Browser SLMs & WebGPU

#ai #webdev #ux #javascript

What In-Browser SLMs, WebLLM, and WebMCP Change for Web Interfaces

A PoC, real-world engineering hurdles, and code running directly in your browser.

🚀 Live Demo:invisible-ui-demo.vercel.app(Requires Chrome 113+, Edge, Brave, or Safari 18+ on macOS)
📦 GitHub Source:github.com/QuentinMerle/invisible-ui-demo

I recently spent five frustrating minutes typing prompts into an e-commerce chatbot just to filter a camera lens by focal length and mount.

The bot asked vague questions. I typed full paragraphs. It misunderstood what I wanted. I corrected it. Six messages later, my cart was still empty. Two standard checkboxes would have solved my problem in four seconds.

Dropping a floating chat bubble in the bottom-right corner of a website is not smart AI integration. It is just lazy design.

Instead of building clear visual interfaces, teams dump the effort back onto the user: “Here is an empty text box. You figure out what you want, write a good prompt, and wait.”

What if we did the opposite? Instead of forcing users to talk to a bot, what if a lightweight local AI model running inside the browser quietly noticed user actions, removed friction, and adapted the interface without any typing?

Here is what happened when I built an on-device "Invisible UI" with SmolLM2-360M and WebGPU—what worked, what broke, and the real-world limits we hit.

1. The Core Problem: Why Chatbots Fail

The Restaurant Example

Traditional chatbots have a big flaw: they are completely passive. They assume the user:

Knows exactly what they want.
Knows how to write a good prompt.
Actually wants to have a conversation with an online store. In reality, almost nobody wants to chat with a website:

1. The Chatbot (The Lazy Waiter) Hands you an empty notepad: "Write down what you want to eat."
👉 High mental effort. The user has to do all the work.
2. Direct Morphing (The Over-eager Waiter)
Takes your plate away while you are eating and swaps it for another dish without asking.
👉 Panic and confusion: "Wait, did I click something? Is this page broken?"
3. Explicit Assent (The Attentive Sommelier) Notices you looking at the wine list and quietly suggests: "If you like dry white wine, this bottle goes great with your fish."
👉 Helpful and polite. The AI suggests, but the user decides.

When you replace simple buttons with a chat window, you replace quick visual clicks with slow, frustrating typing.

The Cloud Trap: Cost and Latency

When developers try to make pages "smart", their first instinct is to send user clicks to big cloud models like GPT-4o or Claude 3.5 Sonnet.

For real-time UI changes, this makes no sense:

Too slow: A cloud roundtrip takes 800 to 2,500 ms. An interface that takes two seconds to update feels broken.
Too expensive: Paying $15 per million tokens just to decide which button to show burns thousands of dollars every month for a basic task.
Privacy issues: Sending what users read, click, or paste to a third-party cloud creates major GDPR problems. If an interface is going to adapt smoothly, the model must run directly on the user's computer with 0 ms network lag and $0 cloud bills.

2. The Architecture: Detecting Intent Without Asking

In our prototype (LiquidStore), we tested a product page for an espresso machine with two main types of visitors:

The Engineer: looks for technical numbers (boiler size, PID temperature stability, extraction pressure).
The Lifestyle Buyer: looks at kitchen photos, colors, and premium design. ### Step 1: Normal Code First, AI Second

Our golden rule: never call an AI model if normal code can do the job. Standard JavaScript handles the easy cases first:

// useImplicitIntent.js — Fast JavaScript logic
// Viewport presence + Mouse hover + Text selection

const scores = { expert: 0, lifestyle: 0 };

interval = setInterval(() => {
  // 1. Viewport: what is visible on screen right now (weight 0.35)
  visibleElements.forEach(({ profile, ratio }) => {
    scores[profile] += ratio * 0.35;
  });

  // 2. Mouse: active cursor attention (weight 0.95)
  if (hoveredProfile) {
    scores[hoveredProfile] += 0.95;
  }

  // If one score is clearly higher, lock the choice immediately without AI
  const delta = Math.abs(scores.expert - scores.lifestyle);
  if (delta >= 2.8) {
    commitLockIn(scores);
  }
}, 400);

When the user's interest is obvious (delta >= 2.8), pure JavaScript picks the profile in less than 1 ms.

When it's not clear—for example, if someone spent equal time on the specs and the photos—the local model (SmolLM2-360M via WebLLM & WebGPU) steps in to make the call:

// Local WebGPU call — runs in under 90 ms with 0 network calls
async function arbitratePersona(scores) {
  const response = await engine.chat.completions.create({
    messages: [
      { role: "system", content: "You are a binary classifier. Answer ONLY 'expert' or 'lifestyle'." },
      { role: "user", content: `Scores: expert=${scores.expert.toFixed(1)}, lifestyle=${scores.lifestyle.toFixed(1)}.` }
    ],
    temperature: 0.1,
    max_tokens: 5
  });

  return response.choices[0].message.content.trim().toLowerCase();
}

SmolLM2-360M (~190 MB download, ~376 MB memory) answers in under 90 ms on WebGPU. No waiting, no extra words.

What Broke During Testing

1. The "Scroll-Only" Mistake

At first, I tried to track everything with IntersectionObserver. That did not work.

IntersectionObserver only fires when elements move into view. The moment a user stops scrolling to actually read a table, the API stops sending events. Reading time was not counted at all.

Also, just because a section is on screen doesn't mean the user is reading it. They could be looking at another window.

On a computer, the best indicator of attention is the mouse. Studies show that mouse position matches eye gaze about 75% of the time.

That is why we combine three signals:

Scroll position: passive view on screen (weight 0.35).
Mouse hover: active pointer focus (weight 0.95). If the cursor stays on PID ±0.5°C, the user is reading technical details.
Text selection: highlighting words (boost +1.2). Highlighting text is a very clear sign of interest. > 📱 What about Mobile? On phones, there is no mouse pointer. The engine has to rely on scroll speed and long-presses. Touch data is much noisier than mouse data, which is a real challenge for mobile devices.

2. The Screen Morphing Trap

When I first hooked up the system to change the page automatically, the result was awful.

The AI figured out the user's intent, and the page immediately rearranged itself: headings changed, reviews swapped places with technical charts, and colors switched while I was reading.

The user reaction was immediate frustration:

"Wait, did I click something by accident?"
"Why did my screen just jump?"
"Where did the text I was reading go?"

Changing the layout while someone is reading breaks a basic rule of good UX: User Control and Freedom.

The Fix: The Sommelier Model (Explicit Assent)

We separated the AI detection from the visual changes.

The local model does its work quietly in the background. Once it has a clear idea, it displays a small, quiet badge at the bottom of the screen:

“Looking for technical details? [Switch to Workshop View ⚙️]”

If the user clicks, the page updates cleanly. If they ignore it, the badge fades away. The AI suggests, but the user stays in charge.

♿ Accessibility Note (WCAG):
Moving things around on a live page can completely confuse screen readers and keyboard users. Using a clear button to confirm changes lets us notify screen readers properly using aria-live.

A Real Use Case: Filling Checkout Forms Automatically

Where does a small local AI model actually save real time? Fixing checkout forms.

Around 85% of mobile shopping carts are abandoned, often because filling out 12 address fields on a phone is painful. Standard browser autofill often fails when a website uses unexpected field names (like street_name instead of address_line1).

We replaced the 12 fields with one single paste box:
“Paste your email signature or address here.”

Slides vs. Reality

At tech conferences, people talk about Google's FunctionGemma 270M. But when you actually open a terminal and try to use it with npm install @mlc-ai/web-llm:

There is no ready-to-run WebGPU build for FunctionGemma 270M.
WebLLM's built-in tool calling requires large models (3B to 7B parameters). Forcing a user to download 3 GB of data just to parse an address makes no sense. #### The Simple Solution: Grammar Constraints (**xgrammar**)

Instead of heavy tool-calling frameworks, we pair our tiny model (SmolLM2-360M) with strict JSON grammar rules (**@mlc-ai/web-xgrammar**).

The WebGPU engine forces the model to follow the JSON schema token by token:

// Clean JSON extraction using local WebLLM with grammar rules
const response = await engine.chat.completions.create({
  messages: [
    { role: "system", content: "You are a JSON extractor. Output valid JSON matching the shipping schema." },
    { role: "user", content: `Extract shipping fields from this text:\n"""${rawText}"""` }
  ],
  temperature: 0.1,
  response_format: { type: "json_object", schema: shippingSchema }
});

const shippingData = JSON.parse(response.choices[0].message.content.trim());

Speed: ~450 ms on local GPU.
Privacy: Zero data leaves the browser. Everything stays on the device.
Confidence check: The model outputs a confidence score for each field. Clear fields get a green check; uncertain fields are highlighted in yellow so the user can quickly double-check before paying.

3. The Takeaway: What This Changes

1. The Cost & Speed Comparison

Setup	How it runs	Speed	Cloud Cost	Privacy	Browser Support
Option 1: On-Device (Best)	WebLLM + SmolLM2-360M (WebGPU)	< 90 ms	$0.00	100% on device	Chrome, Safari 18+, Edge
Option 2: Edge Model	Gemini Flash-Lite / Groq on Edge Worker	~100 ms	~$0.00005	Sent over HTTPS	All browsers
Option 3: Big Cloud Model	GPT-4o+ / Claude X.X Sonnet	1,500+ ms	~$0.015	Sent to third-party	All browsers

2. WebMCP: Making Sites Readable for AI Agents

While building this for human visitors, another question came up: how should external AI agents interact with this page?

Instead of forcing AI agents to guess button names and scrape HTML, the page directly shares its tools using WebMCP (navigator.modelContext):

// Register tools for external AI agents
navigator.modelContext.registerTool({
  name: "switch_product_profile",
  description: "Changes the layout between expert and lifestyle views",
  parameters: {
    type: "object",
    properties: {
      profile: { type: "string", enum: ["expert", "lifestyle"] }
    },
    required: ["profile"]
  },
  execute: async ({ profile }) => {
    setProfile(profile);
    return { status: "success", activeView: profile };
  }
});

For humans: the site quietly adapts behind the scenes.
For AI agents: the site provides clean, typed tools.

3. Smarter A/B Testing

Standard A/B testing can be clumsy: you show Variant A to half your traffic and Variant B to the other half, wait three weeks, and try to find a pattern in noisy averages.

With quick intent detection:

The site figures out what a user is interested in within the first 5 seconds.
Technical layouts are shown only to verified technical users.
Results are much cleaner, and teams can test specific ideas in days instead of weeks. ## Conclusion: Quiet AI is Better AI

The big mistake of early AI products was thinking that because LLMs use language, every feature had to be a chatbot.

In reality, a chat box is often just a sign of lazy design.

Small language models work best when they stay invisible: figuring out user intent, parsing messy text, and removing friction before the user even needs to ask.

The best interface isn’t the one you talk to.
It’s the one that gets out of your way.

Proudly crafted in Beauce, Québec 🇨🇦. Interested in the alliance between immersive web engineering and local AI sovereignty? Let's connect on LinkedIn or via Vibrisse Studio!

DEV Community