DEV Community: Roshni

Local-First AI Astrologer: Building a Private, Multimodal Vedic AI Agent🔮

Roshni — Sun, 01 Mar 2026 14:06:03 +0000

Where Ancient Wisdom Meets Local AI

What if your birth chart could be interpreted by an AI that actually reads classical Vedic texts, understands planetary mathematics, analyzes your palm through a camera — and does it all without sending your data to the cloud?

As part of the Vision Possible Hackathon by VisionAgents AI, I explored a bold idea:

Can we build a culturally intelligent, multimodal AI agent that runs locally and respects user privacy?

The result is Local-First AI Astrologer — an open-source voice + vision AI system that delivers personalized Vedic astrology readings completely on-device.

The Vision Behind the Project
Core Features
Birth Chart Intelligence
Kundli Matching
AI Palm Reading
Tech Stack Deep Dive
How Local RAG Ensures Privacy
Setup & Installation
Demo
Challenges & Learnings
Why This Matters
Conclusion

The Vision Behind the Project

Most astrology platforms today are:

Static
Generic
Cloud-dependent
Non-explainable

Vedic astrology, however, is deeply structured — Nakshatra systems, Dashas, planetary transits, Guna Milan compatibility — yet few tools provide contextual reasoning grounded in authentic texts.

I wanted to build something different:

Retrieval-grounded reasoning
Real-time voice interaction
Vision-powered palm reading
Fully local architecture ## Core Features

Birth Chart Intelligence

The AI generates personalized analysis including:

Nakshatra interpretation
Rashi (Moon sign) explanation
Mahadasha timeline breakdown
Planetary transit insights
Traditional remedies

Powered by:

ephem for astronomical calculations
Local RAG over curated Vedic astrology PDFs
FAISS vector search
Sentence-transformer embeddings

Instead of hallucinating, the system retrieves knowledge from classical texts and explains it conversationally.

Kundli Matching

The compatibility engine performs:

Guna Milan scoring
Dosha detection
Contextual compatibility reasoning

It doesn’t just output numbers — it explains the relationship dynamics in natural language.

AI Palm Reading (Vision + RAG)

Hold your hand up to the camera and the system:

Detects major lines
Identifies mounts
Classifies hand shape
Maps features to palmistry knowledge base

No NVIDIA APIs.
No heavy cloud inference.

Just lightweight vision integration combined with local retrieval.

Tech Stack Deep Dive

Built 100% in Python.

Layer - Technology
Embeddings - sentence-transformers
Video Stream - GetStream.io
Vector Store - FAISS
Knowledge Base - Vedic Astrology PDFs + Palmistry for All
Astronomy Engine - ephem
Voice - Gemini Realtime / Deepgram / ElevenLabs
Vision - Camera-based processing
RAG Setup - setup_rag.py
Agent Runtime - agent.py

The architecture supports:

Buffered conversational responses
Optional voice/video modes
Fully local document indexing

How Local RAG Ensures Privacy

Most AI tools send user inputs to cloud APIs.

This system:

Uses local embeddings
Runs FAISS on-device
Performs scoped document retrieval
Avoids external knowledge calls

Your:

Birth date
Time & location
Compatibility inputs
Palm images

Never leave your device.
This is privacy-first multimodal AI.

Setup & Installation

Clone the repository:

git clone https://github.com/SpandanM110/Local-First-AI-Astrologer
cd Local-First-AI-Astrologer
pip install -r requirements.txt

Then:

Add API keys in .env
Place PDFs inside /knowledge
Run:

python setup_rag.py
python agent.py run

Enable your camera for palm reading

Text-only mode works as well.

Demo

Watch the full demo here: https://youtu.be/q6vUcWZL22E?si=wQS-jXx4LDpHFer7

The demo showcases:

Live birth chart reasoning
Voice interaction
Real-time palm detection
Smooth buffered responses

Challenges & Learnings

Precision in Astronomical Computation: Small calculation differences significantly impact Dasha timelines.

2️. Reducing Hallucinations: RAG dramatically improved factual grounding.

3️. Lightweight Vision: Building palm reading without expensive GPU dependencies was critical.

Big takeaway: Multimodal AI becomes powerful when retrieval is precise and scoped.

Why This Matters

This project explores three important AI trends:

Local-First AI → Privacy by design
Multimodal Agents → Voice + Vision + Retrieval
Cultural Intelligence Systems → Domain-specialized reasoning

Instead of one giant generalized assistant, the future may belong to hyper-specialized, privacy-preserving agents.

Conclusion

Local-First AI Astrologer is more than a hackathon project — it’s a prototype for what personal AI agents can become:

Context-aware
Culturally grounded
Multimodal
Privacy-respecting

If you're excited about:

AI agents
RAG architectures
Local-first systems
Multimodal experimentation

⭐Star the repo
🍴Fork it
💬Share feedback

GitHub: https://github.com/SpandanM110/Local-First-AI-Astrologer

Beyond Prompt Engineering: Building Reliable AI Systems with Google Gemini

Roshni — Thu, 26 Feb 2026 14:32:25 +0000

This is a submission for the Built with Google Gemini: Writing Challenge

What I Built with Google Gemini

For the Gemini 3 Kaggle competition, I built a structured reasoning and evaluation framework using Google Gemini through Google AI Studio.

Instead of developing a traditional chatbot or UI-based AI tool, I focused on designing a multi-stage prompting pipeline that improves reasoning reliability, reduces hallucinations, and enforces structured outputs.

The Problem

In most real-world applications, developers rely on one-shot prompts:
Prompt → Output → Done.

But in production systems, this approach often leads to:

Inconsistent reasoning
Output variability
Hallucinations
Formatting instability

I wanted to explore whether we could move beyond prompt engineering and treat AI interactions as system design problems.

The Solution

I built a layered prompting architecture:

Primary reasoning prompt (step-by-step problem solving)
Self-reflection prompt (model critiques its own logic)
Correction layer (identify and fix inconsistencies)
Final structured output (validated, formatted response)

This approach significantly improved logical consistency and reduced unstable outputs.

Google Gemini played the central role as the reasoning engine.

I experimented with:

Temperature and top-p tuning
Constraint-based formatting (JSON enforcement)
Multi-run comparisons
Edge-case stress testing

Rather than treating Gemini as a black box, I treated it as a component inside a structured evaluation loop.

Demo

You can view the Kaggle notebook here:https://www.kaggle.com/competitions/gemini-3/writeups/new-writeup-1765179354652

The notebook includes:

Structured prompt experiments
Parameter sensitivity testing
Multi-step reasoning comparisons
Self-reflection correction pipeline

While this project was not deployed as a UI product, it was built as a reproducible experimental framework for systematic AI evaluation.

What I Learned

1. Prompt Determinism Is Fragile

Small changes in temperature, wording, or context length dramatically impact reasoning stability. I learned that reliability comes from system structure, not clever phrasing.

2. AI Performs Better with Feedback Loops

When Gemini was asked to review its own reasoning before producing a final answer, it frequently corrected logical flaws. This showed me the power of self-evaluation layers.

3. System Thinking > Prompt Tricks

Instead of chasing the “perfect prompt,” I started focusing on:

Repeatability
Controlled variability
Evaluation metrics
Constraint enforcement

This shifted my mindset from experimentation to engineering.

4. Debugging AI Is a Skill

I learned how to:

Compare multi-run outputs programmatically
Identify hallucination patterns
Document AI behavior systematically
Treat LLM responses as testable components
That discipline changed how I approach generative AI projects

Google Gemini Feedback

What Worked Well

Strong multi-step reasoning capability
Good adherence to formatting with explicit constraints
Clear logical structure when guided step-by-step
Stable performance in controlled environments
Gemini 3 felt significantly more consistent compared to earlier iterations.

Where I Faced Friction

Longer contexts sometimes reduced reasoning sharpness
Slight temperature changes increased variability
Creativity settings impacted output stability
Prompt sensitivity required careful constraint wording

The biggest takeaway for me:

Model capability matters but system design around the model matters even more.

Thanks for participating!

DEV Community: Roshni

Local-First AI Astrologer: Building a Private, Multimodal Vedic AI Agent🔮

Where Ancient Wisdom Meets Local AI

Table of Contents

The Vision Behind the Project

Birth Chart Intelligence

Kundli Matching

AI Palm Reading (Vision + RAG)

Tech Stack Deep Dive

How Local RAG Ensures Privacy

Setup & Installation

Demo

Challenges & Learnings

Why This Matters

Conclusion

Beyond Prompt Engineering: Building Reliable AI Systems with Google Gemini

What I Built with Google Gemini

The Problem

The Solution

Demo

What I Learned

Google Gemini Feedback

Where I Faced Friction