If you used a computer in the late 90s or early 2000s, you probably remember the sheer joy (and chaos) of virtual desktop pets. Whether it was a little sheep walking on your taskbar, a Shimeji dropping from the top of your screen, or the infamous Clippy judging your Word documents—they added a ton of personality to our screens.
I really missed that feeling, so I decided to bring it back for the modern web.
But I didn't want to just make a looping GIF. I wanted to build a virtual companion that actually "reads the room."
Meet Arcrawls — an open-source, context-aware browser mascot.
https://arcrawls.com
The Tech Stack
Building a browser extension that interacts with the DOM is tricky enough, but adding AI into the mix made it a really fun engineering challenge. Here is what I used:
Core Extension: TypeScript & Vite
Landing Page: Vue 3 & TailwindCSS
Extension API: Manifest V3
AI & Machine Learning: Chrome's Built-in AI (Gemini Nano) & ONNX Runtime (WebAssembly)
The Challenge: 100% Local Offline AI
The core feature of Arcrawls is that it analyzes the text and sentiment of the webpage you are currently reading. If you are reading a depressing news article, the pet acts sad. If your developer console is throwing a bunch of red errors, it puts on a hard hat and looks confused.
The easiest way to do this would have been to just grab the document.body.innerText and fire it off to an OpenAI API endpoint. But I firmly believe browser extensions should not be sending your private reading habits to the cloud. Instead, the entire AI sentiment analysis engine runs 100% locally and offline directly inside your browser. I achieved this using a hybrid approach:
Chrome's Built-in Gemini Nano: Arcrawls taps into the new, experimental
window.aiAPI built directly into modern Chrome browsers. This allows the extension to leverage the local Gemini Nano model to perform complex sentiment and context analysis without downloading massive model weights.ONNX Runtime for WebAssembly: For fallback support and lighter, faster inference tasks (like quick zero-shot classification), the extension runs on-device machine learning via ONNX Runtime compiled to WebAssembly (WASM). This runs directly within the extension's sandboxed Service Worker.
Because everything happens locally, zero data leaves your device, inference is incredibly fast, and there are absolutely zero recurring cloud API costs to worry about.
Bringing it to life with Physics
I implemented a custom spring physics engine that runs via a requestAnimationFrame loop. The pet recalculates its target position based on where you are scrolling and uses spring dampening to crawl, jump, and interact with the edges of your viewport naturally. Getting the physics to run buttery-smooth in Vanilla TS while injecting it into a shadow DOM (to prevent the host site's CSS from bleeding into the pet's UI) was a massive learning experience!
It's Open Source!
Arcrawls is a passion project, and it is 100% free and open-source. Building for Manifest V3 (especially trying to run WASM inside Service Workers) was a huge learning curve with plenty of weird CSP and fetching bugs to overcome.
I'd absolutely love for you to try it out! If you are interested in extension development, local AI, or just want a little crab keeping you company while you debug, check out the code.
Website: https://arcrawls.com
GitHub Repo: fujiDevv/context-aware-browser-pet https://github.com/fujiDevv/context-aware-browser-pet
Let me know what you think in the comments! Has anyone else experimented with Chrome's Gemini Nano or ONNX Runtime inside browser extensions? I'd love to swap notes on how you handled the memory constraints!
Top comments (2)
Curious about the local AI choice for continuous interaction.
Hey @frank_signorini, I went with local AI because it guarantees absolute privacy for the user's browsing data while completely eliminating network latency and the insane cloud API costs of continuously analyzing active webpages!