🐾 Sanctuary: Building a Living AI Companion with Gemma

This is a submission for the Hacktoberfest Weekend Challenge: Build for a Friend

What I Built

I built Sanctuary — a living AI companion featuring a virtual roommate pet named Mochi, powered by Google's open-weight Gemma 2B model and optional ElevenLabs voice synthesis.

Instead of being another chatbot that waits for a prompt inside a text box, Mochi is an autonomous creature designed to share your digital workspace.

• 🐾 Continuous Autonomous Life Simulation: Mochi has an ongoing internal state model including energy, happiness, curiosity, restfulness, boredom, and social need. It wanders, stretches, loafs, naps, and reacts to the user without continuously calling an AI model.
• 👁️ Passive Workspace Awareness: Mochi tracks cursor position, movement, proximity, and user activity. When the user is actively typing, it suppresses unnecessary interruptions and gives them space.
• 🧠 Gemma-Powered Reflection: When the user wants to talk or needs guidance, Mochi can activate Google's open-weight Gemma 2B model for CBT-inspired reflection. It can help identify patterns such as catastrophizing or all-or-nothing thinking and turn them into more balanced perspectives and practical grounding steps.
• 🗣️ Warm Voice Interaction: Mochi can speak using ElevenLabs Flash v2.5, with the browser's Web Speech API available as an alternative for users who want a local/offline voice option.
• 🪟 Floating Companion Mode: Mochi can move around the Sanctuary viewport and can be popped out into a browser-managed floating window using the Document Picture-in-Picture API.
• 🫁 Grounding Tools: Sanctuary includes guided breathing and ambient calming experiences, including procedural purring and rain sounds.

I built Sanctuary for my friend Alex, a software engineer who struggles with late-night burnout, insomnia, and spiraling self-criticism during difficult work periods.

The problem I wanted to solve was simple:

When someone is overwhelmed, why should they have to open a blank chatbot and figure out what to type?

Mochi is designed to provide quiet digital companionship first. It can simply sit nearby, wander around, take a nap, or react to what you're doing. When you actually want to talk, it becomes an intelligent conversational companion.

Sanctuary is intended as an emotional-support and mindful-grounding companion, not a clinical treatment or replacement for professional therapy.

Demo

🚀 Live Web App: https://sanctuary-0ib8.onrender.com/

In supported Chromium browsers such as Chrome and Edge, use the Pop-out PiP button to detach Mochi into a browser-managed floating companion window while you continue working.

Code

PawanTheGod / sanctuary

🌱 Sanctuary — Living AI Roommate Pet & Supportive Companion

Sanctuary is a supportive companion and living roommate pet ("Mochi") built for a friend navigating burnout, late-night insomnia, and acute anxiety spirals.

(Note: Sanctuary is an emotional support and mindful grounding companion, not a clinical treatment or replacement for professional therapy.)

---

✨ What Is Sanctuary?

Sanctuary is built around Mochi — an autonomous creature that inhabits your digital workspace.

Unlike traditional chatbot assistants that wait passively for text prompts or send intrusive pop-up notifications, Mochi runs a continuous 20Hz local life simulation:

• 🐾 Living Presence: Wanders across your screen, stretches, loafs, sleeps with floating Zzz particles, and gets playful micro-zoomies.
• 👁️ Cursor Awareness: Dynamic eye pupil tracking and proximity dwell detection relative to your cursor.
• 🧠 Open-Source Gemma Brain: Leverages Google's open-weight Gemma LLM (running on-device via WebGPU / WebLLM or local Ollama) for CBT-inspired cognitive distortion reflection and…

View on GitHub

📂 GitHub Repository: https://github.com/PawanTheGod/sanctuary

Tech Stack:

• Vanilla JavaScript (ES Modules)
• HTML5
• CSS3
• Web Audio API
• WebGPU
• WebLLM (@mlc-ai/web-llm)
• Google Gemma 2B
• Ollama
• ElevenLabs API
• Document Picture-in-Picture API

How I Built It

Sanctuary is built around a decoupled architecture where Mochi's body and cognition are separate systems.

1. MochiLifeEngine — The Body

The core mochi-engine.js runs a continuous local life simulation.

Instead of asking an LLM what Mochi should do every few milliseconds, the engine maintains internal biological-style variables such as:

• energy
• happiness
• curiosity
• boredom
• sleepiness
• socialNeed

A priority arbitrator uses these values and the current context to select behaviors such as wandering, approaching the cursor, playing, loafing, stretching, affection, or sleeping.

This means Mochi can feel alive without continuously consuming LLM tokens.

2. ContextProvider — The Senses

context-provider.js acts as Mochi's sensory layer.

It observes local interaction signals such as:

• Cursor position and movement
• Cursor proximity to Mochi
• User activity and inactivity
• Typing activity
• Browser focus
• Session duration
• User interaction with Mochi

These events allow Mochi to respond to its environment without requiring an LLM for every decision.

One important design rule is that active typing suppresses unnecessary proactive interruptions.

3. Gemma — The Cognitive Layer

When an interaction actually requires reasoning, Sanctuary escalates from the local life engine to Gemma.

The primary local path uses:

Gemma 2B → WebLLM → WebGPU

with the model:

gemma-2-2b-it-q4f16_1-MLC

Sanctuary also supports a local Ollama path using:

gemma2:2b

Gemma is therefore not responsible for Mochi's continuous physical behavior.

Instead:

Mochi's local life engine → meaningful interaction → Gemma reasoning → Mochi response

This separation was one of the most important architectural decisions in the project.

4. CBT-Inspired Reflection

When the user shares a worry, Mochi can use Gemma to structure the conversation around CBT-inspired reflection.

For example, it can help identify patterns such as:

• Catastrophizing
• All-or-nothing thinking
• Mind reading
• Negative assumptions

It then helps turn the thought into a more balanced perspective and a small practical grounding action.

The goal is not to replace therapy, but to make supportive reflection feel more natural and accessible.

5. Voice Layer

For users who want spoken interaction, Sanctuary can route Mochi's responses through ElevenLabs Flash v2.5.

The browser's Web Speech API provides an alternative voice path, allowing the core companion experience to remain usable without requiring ElevenLabs.

6. Floating Companion Surface

Mochi was originally designed as a website companion, but I wanted the pet itself to feel more persistent.

Sanctuary therefore includes a dedicated floating companion layer and a Document Picture-in-Picture surface.

This allows Mochi to leave the main Sanctuary interface and remain visible in a browser-managed floating window while the user works elsewhere.

It is deliberately browser-based rather than pretending to provide a native OS-wide overlay.

Why Does Open Innovation Matter?

Open innovation made an important part of Sanctuary possible: local AI cognition.

Instead of designing Sanctuary around a mandatory cloud LLM API, I could build the cognitive layer around an open-weight model that can run locally.

With Gemma 2B running through WebLLM/WebGPU or Ollama, users can run the reasoning component on their own machine.

That changes the architecture significantly.

The continuous life simulation doesn't need a cloud service at all. Mochi's behavior engine, state, context processing, and interaction logic can operate locally, while Gemma can also run locally when the user chooses a local inference path.

This makes the companion:

• Less dependent on network availability
• More responsive to local interactions
• More controllable by the developer
• Better suited to privacy-sensitive conversations when local inference is used
• Free from per-token cloud LLM costs in local mode

Open-weight AI also allowed me to experiment with an architecture where the LLM is not the entire product.

Gemma is the cognitive layer.

The rest of Mochi — its body, senses, behavior, memory/state, and personality — is software I designed around it.

That was the part that made Sanctuary possible as more than another AI chat interface.

My Agent Session

I built Sanctuary through an intensive pair-programming workflow with an autonomous AI coding agent.

The agent helped scaffold and iterate on:

• Mochi's life simulation
• ContextProvider
• Behavior prioritization
• Gemma/WebLLM integration
• Ollama fallback
• Voice integration
• Floating companion architecture
• Document Picture-in-Picture support
• Security and secret-leak checks
• Documentation and architecture files

The development process was iterative: build → run → test → inspect → harden → repeat.

The important architectural constraint throughout the process was to keep the 20Hz life engine independent from Gemma, so the companion would remain responsive and autonomous rather than turning every movement into an LLM request.

Prize Categories

🏆 Best Use of Gemma

Gemma is the cognitive core of Sanctuary.

The project uses Google's open-weight Gemma 2B locally through WebLLM/WebGPU, with Ollama providing another local inference path.

Rather than using Gemma simply as a chatbot, I integrated it into a larger autonomous companion architecture where it is selectively activated when meaningful reasoning is required.

🔊 Best Use of ElevenLabs

ElevenLabs provides Mochi's expressive voice layer through ElevenLabs Flash v2.5.

The voice is designed to make Mochi feel more like a companion than a conventional text-based assistant, while Web Speech provides an alternative voice path.

🟣 Best Use of Render

Sanctuary is deployed on Render, providing a publicly accessible live application that anyone can try without setting up the project locally.

🚀 Live Demo: https://sanctuary-0ib8.onrender.com/