Bringing AI to the Browser: Transform Your Web App with On-Device Models

TL;DR

Tired of bleeding money on AI API costs? Here’s the fix: run AI models directly in your users’ browsers. I’ve built Browser.AI, a prototype that lets you tap into on-device AI through a simple window.ai API — think localStorage, but for AI. No more API fees, better privacy, and faster performance. Users get control over their data and which models to run, while you get to scale your AI features without scaling costs. Best part? It works offline and isn’t locked to any specific vendor. Check out the prototype, contribute to the W3C proposal, and let’s make AI more accessible for developers everywhere.

Key points:

• 🚫 No more API costs
• 🔒 Privacy-first (data stays on device)
• ⚡ Better performance (no server roundtrips)
• 🔌 Works offline
• 🎮 Users control their models
• 💻 Simple window.ai API

Intro

You’ve built something awesome — an AI-powered app for your startup or passion project. At first, you lean on OpenAI’s API to get it out the door quickly, and things go well. You gain users, feedback is positive, and it feels like you’re on the right track. But then reality hits: the API costs are ballooning, and each new user brings you closer to burning through your budget. Sound familiar?

This is the problem every small team building AI apps eventually faces: powerful AI capabilities come with a steep price tag. You can’t keep paying these rising costs without killing your runway. But what’s the alternative? Do you compromise on features, slow down your roadmap, or drop out of the AI race altogether?

No. There’s a smarter solution — on-device AI models.

By shifting AI workloads directly onto your users’ devices, you can eliminate those crushing API costs, speed up your app, and give users more control over their data. Best of all, this approach makes your product more scalable, with zero trade-offs in performance. You keep your hard-earned dollars, your users keep their privacy, and everyone wins.

In this article, I’m going to break down the why and how of on-device AI, walk you through a prototype I built called Browser.AI, and show you how this approach is set to transform the way developers integrate AI into their web apps.

The Big Idea

Imagine tapping into powerful AI models directly from the browser, using an API as familiar as **window.fetch** or **window.localStorage**. No more server-side processing, no more exorbitant API fees. Just seamless, on-device AI that you control.

Here’s what that could mean for you:

• Unified API Access: A standard window.ai API that works across all browsers.
• Model Flexibility: Support for open-source models, so you’re not locked into any vendor.
• User Empowerment: Users can choose which models they enable, enhancing privacy and control.

Just as the MediaDevices API provides you access to hardware like the camera or microphone, the window.ai API could provide access to AI models installed on the user's device.

Who Benefits

This approach benefits anyone building AI-powered apps, from indie developers to large engineering teams, along with their users. Here’s how:

Developers Get:

• Cost Efficiency: Say goodbye to skyrocketing API bills. By running AI models directly in the user’s browser, you’re offloading processing to their hardware — at zero extra cost to you.
• Improved Performance: No more latency from server round-trips. On-device AI can make your app snappier, providing a better user experience.
• Scalability: With the heavy lifting done on the client side, scaling your app to more users doesn’t mean increased server costs.
• Flexibility and Control: You’re not tied to a specific AI provider. Use open-source models and switch them out as you see fit.

Users Get:

• Enhanced Privacy: Data stays on their device, reducing concerns about data being sent to third parties.
• Offline Functionality: Features work even without an internet connection — think editing a document on a plane with full AI assistance.
• Control Over Resources: Users can decide which models to enable, giving them more say in their experience.

You cut costs, and ditch third-party dependencies. Users get data control and a smoother experience.

How It Works

Think of this like accessing speech synthesis or camera features through browser APIs, but for AI models. With processors now optimized for on-device AI, local models like LLMs and CNNs can power features directly on users’ devices, much like Apple, Microsoft, and Google are already doing.

• Browser-Level Integration: The idea is for an API, like window.ai, built right into the browser to access AI models on the user’s device.
• Model Management: Users manage which AI models are installed and control site access, similar to camera or microphone permissions.
• Cross-Browser Compatibility: A standardized API means developers write code once, and it works across all browsers, regardless of the specific AI model in use.

Technical Details

• JavaScript API: The window.ai API provides methods for permissions, model info, and creating sessions.
• Permissions: Before accessing a model, your app requests permission from the user.
• Sessions: Once permission is granted, you establish a session with the model to perform tasks like text completion or embeddings.

When This Could Happen

Progress is already underway — I’ve submitted a proposal to the W3C, and if this idea resonates, I’d love your feedback. Google Chrome is also exploring this space, but I see some gaps in their approach, which I’ll get into later.

Alternatives

Before we dive into Browser.AI, let’s look at existing options and where I see their limitations:

window.ai Extension

I really like this project — they’re doing a lot of smart things, especially in giving users control over which models they use. Definitely worth checking out.

Pros:

• Gives users control over which models to use.
• Supports both local and remote models.

Cons:

• Adoption Barrier: Users need to install an extension — not ideal for mass adoption.
• Technical Complexity: Geared towards users comfortable with setting up API keys and local models.
• Mobile Limitations: Extensions aren’t available on mobile browsers.

Chrome window.ai Implementation

What It Is: An experimental API built into Chrome, offering basic AI features.

Pros:

• Integrated directly into the browser — no extension required.

Cons:

• Limited Models: Uses Google’s own models, which tend to perform poorly compared to alternatives.
• Ecosystem Fragmentation: Risks each browser supporting only their preferred models.
• Lack of Flexibility: Limited control for developers and users over which models to use.

transformers.js

What It Is: A JavaScript library that runs AI models in the browser using WebAssembly.

Pros:

• Runs entirely in the browser — no server required.
• Leverages open-source models.

Cons:

• Redundant Downloads: Each website must package and serve the model, leading to duplicate downloads.
• Performance Overhead: Large models can impact load times and performance.
• Complex Integration: Managing models and performance optimization can be challenging.

While I do think there are downsides, it’s still a great project. If you’re interested in experimenting with this approach, you can check out my article, Run Models in the Browser With Transformers.js

Introducing Browser.AI

The future of AI on the web isn’t about relying on costly third-party services. It’s about putting power back in the hands of developers and users. That’s where Browser.AI comes in.

It’s designed around three key principles:

• Keep it simple: An intuitive API that feels natural for JavaScript developers.
• Flexibility and control for users and developers: Users choose which models to allow, and developers pick the models that fit their apps — ensuring cross-browser compatibility and avoiding vendor lock-in. a.k.a., model-agnostic.
• Boost performance and privacy: Running AI locally reduces latency, keeps data private, and offline functionality for a smoother experience.

Browser.AI is a working prototype that demonstrates how developers can run AI models directly on users’ devices through an API on the window object. This approach enhances privacy, speed, and offline capabilities for web apps.

Developing with the API

Try it firsthand, you can download the app, or clone the repo.

For more information on the architecture, see the README in the project repo. The API is as follows:

Browser.AI API

Permissions API

The Permissions API allows developers to request permission for using specific AI models and see which models are already enabled.

• Request permission: window.ai.permissions.request({ model: 'name', silent: true }) Prompts the user to grant permission for a model (e.g., 'llama3.2'). Returns true if approved.
• List enabled models: window.ai.permissions.models() Retrieves a list of models the user has enabled.

Model API

The Model API allows developers to connect to a specific AI model and retrieve information.

• Get model info: window.ai.model.info({ model: 'name' }) Provides details about the specified model.
• Connect to model: window.ai.model.connect({ model: 'name' }) Establishes a session with the model, enabling further interaction.

Session API

Once connected to a model, the Session API lets you interact with the model via chat or embeddings.

• Chat: session.chat({ messages: [{ role: 'user', content: 'hello' }] }) Sends a message to the AI model and gets a response.
• Embed: session.embed({ input: 'text to embed' }) Generates embeddings from the input text.
• More coming soon

Simple Code Example

if(await window.ai.permissions.request({ model: 'llama3.2', silent: true })) {
    const session = await window.ai.model.connect({ model: 'llama3.2' });
    const response = await session.chat({ messages: [{ role: 'user', content: 'hello' }] })
    console.log(response)
}

Next Steps

• Try the Prototype: Download the Browser.AI prototype and integrate AI features like chat or text completion into your projects. You’ll find instructions and examples in the README. Hands-on experimentation with on-device models will show you how they can enhance your apps.
• Share Feedback on the WICG Proposal: Check out the WICG proposal for Browser.AI and share your thoughts on the technical implementation, privacy, and cross-browser compatibility. Your feedback will help refine the API’s design, so weigh in on key areas like model permissions and session handling.
• Contribute to the Project: If you’re a developer, contribute to Browser.AI on GitHub — improve the code, suggest features, or fix bugs. Non-coders can help by testing the prototype, reporting issues, or suggesting ideas for improve user control. Every contribution counts.

Final Thoughts

AI is transforming how we build and interact with web applications, but the costs and dependencies on third-party providers can be a huge barrier for small teams and indie developers.

On-device models offer a way out — a chance to cut costs, reduce latency, and give users more control over their data. Browser.AI is just one approach to making this vision a reality. By providing a simple, standardized API, we can empower developers to scale their AI-powered features without relying on expensive services, all while prioritizing performance and privacy for users.

This is fertile ground. There’s a ton of potential here and with feedback from the community, we can build something that works for both developers and users. Whether you’re just getting into AI or trying to scale your product without breaking the bank, jump in, test out the prototype, and let me know what you think. Let’s make this happen together.

Resources

• WICG/proposals
• browser.christophermckenzie.com
• GitHub - kenzic/browser.ai

To stay connected and share your journey, feel free to reach out through the following channels:

• 👨‍💼 LinkedIn: Join me for more insights into AI development and tech innovations.
• 🤖 JavaScript + AI: Join the JavaScript and AI group and share what you’re working on.
• 💻 GitHub: Explore my projects and contribute to ongoing work.