How I Built a Local AI Orchestrator and City AI: My Journey as a Developer

#ai #webdev #javascript #python

Hi, I'm Raj Patil (known online as Dream / lostxmusafir), an AI Engineer and Full-Stack Developer. In this article, I want to break down how I developed two of my most impactful projects: Local AI Orchestrator and City AI.

You can find my full interactive portfolio here: Raj Patil AI Portfolio.

1. Local AI Orchestrator: Privacy-First Offline AI

With growing data privacy concerns, relying on cloud-based LLM APIs isn't always feasible for enterprise applications. I built the Local AI Orchestrator to solve this. It's a completely offline, privacy-first AI system that runs locally on consumer hardware like my NVIDIA RTX 3050.

Tech Stack & Architecture:

Frameworks: LangChain, FastAPI (Python)
Local Models: Ollama (Llama 3, Mistral, Phi-3)
Hardware Integration: NVIDIA CUDA & TensorRT optimization
Cache Layer: Redis (to store local session contexts)

Key Learnings:

Optimizing LLM quantization (GGUF formats) was critical to achieving sub-100ms token generation times on a local laptop GPU. It proved that robust, responsive AI applications don't always need expensive cloud servers.

2. City AI: AI-Powered Civic Administration

City AI is a platform built to automate municipal feedback loops. When citizens submit complaints (like street light outages or road damage), the system categorizes, prioritizes, and routes them to the correct local government departments automatically.

Tech Stack:

Core Orchestrator: Google Gemini API
Backend: Node.js, Express.js
Database: MongoDB
Authentication: JSON Web Tokens (JWT)

How it Works:

A user submits a complaint text or image.
The Gemini API analyzes the semantic meaning, extracts key descriptors, and assigns a severity score.
The Express backend uses custom routing logic to dispatch the ticket to the respective agency's dashboard.

Why I Specialize in Web + AI Integration

I believe the future of software lies at the intersection of high-performance web applications and intelligent AI orchestrations. Whether it's building a full-stack food delivery system (like my Swiggy Clone) or engineering cross-platform apps using Flutter (like my mobile civic client Place.ai), my objective is to make software feel alive, smart, and exceptionally fast.