DEV Community

Cover image for Building a Privacy-First Autonomous Agent with Gemini and LiveKit: Introducing Ved AI 🤖⚒️
Ishaan
Ishaan

Posted on

Building a Privacy-First Autonomous Agent with Gemini and LiveKit: Introducing Ved AI 🤖⚒️

Intro Snippet: Hey Dev Community! I've spent the last few months building a personal assistant that doesn't just chat—it acts.

Today, I’m open-sourcing Ved AI. It’s a local-first autonomous agent that uses Gemini Pro for reasoning and LiveKit for lightning-fast voice interactions.

Why I built this: Most assistants today feel like silos. Ved is designed to reach into your workflow—Gmail, Drive, System Control—and get things done without your data leaving your control.

The Tech Stack:

Backend: FastAPI & Python
AI: Google Gemini & Groq
Voice: LiveKit Realtime API
Memory: Knowledge Graph (NetworkX)
I’d love for you guys to check out the source code, play with the "Setup-First" UI, and let me know your thoughts!

GitHub logo Ishaan-Sharma-tech / ved-ai

Next-generation Autonomous AI Assistant with Real-time Multi-modal Voice, Agentic Swarms, and full Google Workspace integration. Built for Performance & Privacy.

🚀 Ved AI: Your Advanced Autonomous Personal Agent

Ved is a next-generation AI assistant designed to handle your digital life through voice and text. Built with a "Privacy-First" approach, Ved runs locally and connects to cutting-edge AI models to provide a seamless, sharp, and highly capable agentic experience.


🌟 Key Features

  • 🎙️ Real-time Multi-modal Voice: Powered by LiveKit and Gemini Realtime API for natural conversations.
  • 🐝 Agentic Swarms: Spawns parallel agents for Research, Synthesis, and Data Retrieval.
  • 🛠️ Integrated Toolset: Full control over your local System, Gmail, Google Drive, Google Tasks, YouTube, and the Web.
  • 📱 Telegram Integration: Secure remote control of your agent from anywhere.
  • ⚙️ Setup-First UI: Configure everything (Identity, Keys, Paths) directly from the Web UI.

⚡ One-Click Quick Start (Windows)

The easiest way to get started is using our automated launcher:

  1. Clone the repo: git clone https://github.com/your-username/ved-ai.git
  2. Double-click start_ved.bat: This will automatically
    • Create your .env

Top comments (0)