What I Built
I built Jarvis Control Center, a local-first browser agent that lets users control their computer with natural language.
You can ask it to open websites, search the web, play YouTube videos, manage tasks, and use connected tools. For example:
“Open YouTube and play 11K by Seedhemaut.”
Jarvis uses Gemma locally to understand the request, decide whether a tool is needed, and execute the browser action through Playwright.
It saves users from repeatedly opening tabs, searching manually, and performing routine browser workflows.
Demo
The launch demo is available in the repository README:
View the Jarvis Control Center repository
The demo shows Jarvis receiving a YouTube request, opening the browser, searching for “11K — Seedhemaut,” and playing the result.
Code
How I Built It
Jarvis is built with:
- Next.js and TypeScript for the application
- Gemma 4 E2B running locally through Ollama
- Playwright for browser automation
- Convex for the database, authentication, realtime synchronization, and server-side functions
- ElevenLabs as an optional text-to-speech provider
- Browser speech APIs for voice input
- Composio for connecting external tools and services
The project is designed to work locally by default. Users can run Gemma through Ollama without sending their conversations to a hosted AI API. Optional cloud services can be enabled through environment variables when voice output or external integrations are needed.
Why Does Open Innovation Matter?
Open innovation made it possible to build an AI assistant that users can run and customize on their own machines.
Using an open-weight model through Ollama means the core assistant can work locally, with more privacy and control than a closed hosted API. Developers can inspect the system, change the agent logic, add new tools, and adapt the model to their own workflows.
The project also combines multiple open technologies instead of locking the entire experience to one provider.
The Lazy Payoff
Jarvis turns multi-step browser workflows into a single natural-language request.
Instead of opening YouTube, searching for a song, checking the result, and pressing play manually, the user can give one instruction and let the browser agent handle it.
The same pattern can be extended to web research, task management, calendar actions, email workflows, and other repetitive browser tasks.
Prize Categories
- Best Use of Gemma 4
- Best Use of GitHub Copilot
- Best Use of ElevenLabs
GitHub Copilot helped bootstrap and accelerate the initial implementation. The project was then cloned into this repository and further developed with a local-first Gemma architecture, browser automation, Convex, and optional ElevenLabs voice output.
Gemma 4 is the core reasoning model used by Jarvis through Ollama, while ElevenLabs provides optional natural-sounding text-to-speech when cloud services are enabled.
Top comments (0)