Aegis is a healthcare companion designed to help senior citizens with medication, safety, and accessing information.
The project combines AI, computer vision, Wi-Fi sensing, real-time communication, and mobile development into a single application.
Architecture
Aegis is built with Next.js, React, TypeScript, and Tailwind CSS, packaged for Android using Capacitor.
The backend is built with Python and FastAPI, exposing REST and WebSocket APIs for the different AI and ML services.
Next.js + React
|
Capacitor
|
Android
|
FastAPI
|
┌─────┼──────────┐
| | |
Gemini Groq CSI + ANN
| | |
Medical Chat Fall Detection
AI-Powered Prescription Scanner
One of the main features is the medical document scanner.
A user can photograph a prescription and send it to the backend. Gemini 2.5 Flash analyzes the image and converts it into structured information such as:
- Medication name and dosage
- Medication schedule
- Instructions
- Locations for procedures
- Safety restrictions
Instead of displaying a raw AI response, this information is converted into structured data that the frontend can use.
Prescription Image
↓
Gemini 2.5 Flash
↓
Structured JSON
↓
Medication + Safety Information
Context-Aware Medical Chat
The extracted prescription can then be used as context for a chatbot.
The backend uses Llama 3.3 70B through Groq to answer questions using the available prescription context.
Prescription
↓
Structured Context
↓
User Question
↓
Llama 3.3 70B
↓
Context-Aware Answer
This makes the chatbot specific to the user's uploaded medical information instead of functioning as a completely generic assistant.
Camera-Less Fall Detection
The most experimental part of Aegis is its fall-detection system.
Instead of relying on cameras, the system uses Wi-Fi Channel State Information (CSI) to detect changes caused by movement.
The backend processes CSI data from 20 subcarriers and extracts statistical and signal-processing features such as variance, standard deviation, signal energy, skewness, and kurtosis.
These features are passed through a scaler and an ANN classifier.
Wi-Fi CSI
↓
Signal Window
↓
Feature Extraction
↓
Scaler
↓
ANN
↓
FALL / NO_FALL
The result is streamed to the frontend using WebSockets, allowing the application to monitor the prediction in real time.
When a fall is detected, Aegis can trigger an emergency notification to a configured caretaker.
AI-Powered Web Intelligence
Aegis also includes a browser extension that can extract webpage content and send it to the backend.
The backend generates a summary using Llama 3.3 70B. That summary can then be used as context for a chatbot.
Webpage
↓
Browser Extension
↓
AI Summary
↓
Context
↓
Chat
This allows users to ask questions about the webpage rather than manually searching through it.
Accessibility
Since the application is designed with senior citizens in mind, the UI focuses on:
- Large touch targets
- High-contrast text
- Simple navigation
- Voice output
- Multilingual support
Aegis also includes AI-powered text-to-speech and translation functionality.
Tech Stack
| Component | Technology |
|---|---|
| Frontend | Next.js, React, TypeScript, Tailwind |
| Mobile | Capacitor |
| Backend | Python, FastAPI |
| Vision AI | Gemini 2.5 Flash |
| LLM | Llama 3.3 70B via Groq |
| Fall Detection | Wi-Fi CSI + ANN |
| Real-time | WebSockets |
| TTS | Orpheus via Groq |
What I Learned
The main challenge wasn't integrating individual AI models. It was connecting them into useful workflows.
Aegis combines:
Vision → Structured Data → Chat
Wi-Fi → ML → Real-Time Alert
Webpage → AI Summary → Contextual Chat
The project taught me that building an AI application is less about adding as many models as possible and more about connecting intelligence to a meaningful user workflow.
The source code is available on GitHub:
Top comments (0)