This is a submission for the GitHub Finish-Up-A-Thon Challenge
🩺 Agent Baymax V2 — From a Local AI Assistant to a Full Healthcare Platform
What I Built
Some projects don't need to be replaced—they need to be completed.
Agent Baymax started as a simple terminal-based healthcare companion built during a hackathon. The original version could answer basic health-related questions, log calories, track hydration, and maintain short-term conversational context using a locally running language model.
While the idea worked, it never fully matched the vision I had in mind.
Users had to manually enter meals, there was no personalization across sessions, no visual interface, no image understanding, and no way to retrieve information from trusted healthcare resources.
For the Finish-Up-A-Thon Challenge, I decided to revisit the project and transform it into something much more capable.
The result is Agent Baymax V2 — an AI-powered healthcare companion that combines:
- 🧠 Conversational AI
- 📸 Multimodal meal analysis
- 🍽️ Personalized nutrition planning
- 🧮 Dynamic BMR and TDEE calculations
- 📚 Retrieval-Augmented Generation (RAG)
- 💾 Long-term memory extraction
- 🔐 Secure authentication
- ☁️ Cloud-based storage and data management
What started as a simple terminal application evolved into a modern healthcare platform designed around personalization, accessibility, and intelligent health tracking.
Demo
This project is currently under active development and is being demonstrated through screenshots.
Dashboard
The redesigned dashboard provides users with a modern healthcare experience featuring personalized health metrics and quick access to Baymax's capabilities.
AI Healthcare Assistant
Users can chat naturally with Baymax and receive contextual health guidance powered by Gemini.
Vision-Based Meal Analysis
Users can upload meal images and receive calorie estimates, macro breakdowns, health scores, and nutritional insights.
Personalized Health Metrics
Baymax dynamically calculates BMR, TDEE, protein targets, and daily nutrition recommendations based on the user's profile.
Knowledge Base & RAG
Healthcare documents can be uploaded, embedded, and searched to provide evidence-based responses grounded in trusted resources.
The Comeback Story
The original Agent Baymax was built as a lightweight proof of concept.
Version 1 used Google's Gemma 2B instruction-tuned model running locally through Hugging Face Transformers. Users interacted through a terminal interface where they could ask basic health questions, track hydration, and log food consumption.
Although functional, several limitations quickly became apparent:
- No graphical user interface
- No authentication system
- No persistent memory
- No image understanding
- No document retrieval
- No personalization engine
- No scalable architecture
Rather than abandoning the project, I chose to rebuild it from the ground up.
The first major step was moving from a terminal application to a modern web platform powered by Next.js and Supabase.
Next came multimodal AI capabilities. Instead of manually logging food, users can now upload meal images and receive structured nutritional analysis.
To improve personalization, I implemented a memory extraction system that identifies important long-term user preferences, dietary restrictions, and lifestyle choices and stores them for future interactions.
I then introduced a Retrieval-Augmented Generation pipeline using embeddings and vector search, allowing Baymax to retrieve information from uploaded healthcare documents instead of relying entirely on model knowledge.
Finally, I redesigned the user experience with glassmorphic interfaces, smooth animations, streaming AI responses, cloud storage, authentication, and scalable backend infrastructure.
Before vs After
| Agent Baymax V1 | Agent Baymax V2 |
|---|---|
| Terminal-Based Application | Modern Web Platform |
| Local Gemma 2B Model | Gemini-Powered AI |
| Text-Only Inputs | Text + Image Understanding |
| Local File Storage | Cloud Database |
| Session Context Only | Persistent Long-Term Memory |
| Manual Food Logging | AI Meal Recognition |
| No Knowledge Retrieval | RAG + Vector Search |
| Single-User Local Tool | Scalable Full-Stack Platform |
This challenge became less about finishing unfinished code and more about finishing the original vision behind Baymax.
My Experience with GitHub Copilot
As the project grew from a small Python prototype into a full-stack AI application, development complexity increased significantly.
GitHub Copilot helped speed up implementation by assisting with:
- API integrations
- Authentication flows
- Database operations
- TypeScript interfaces
- React component scaffolding
- Utility functions
- Backend route generation
For larger features such as memory extraction workflows and vector search integrations, Copilot provided useful starting points that accelerated experimentation and iteration.
Rather than replacing problem-solving, it reduced repetitive work and allowed more focus on architecture decisions, feature design, and user experience.
🛠️ Tech Stack
Frontend
- Next.js 16
- React 19
- Tailwind CSS v4
- Framer Motion
Backend & Database
- Supabase PostgreSQL
- pgvector
- Supabase Auth
- Supabase Storage
AI & Machine Learning
- Google Gemini
- Google Embedding Models
- Vercel AI SDK
Infrastructure
- Upstash Redis
- Vector Search
- Retrieval-Augmented Generation (RAG)
- Long-Term Memory Engine
Final Thoughts
Revisiting Agent Baymax taught me an important lesson.
Not every unfinished project deserves to be abandoned. Sometimes the most rewarding challenge is returning to an old idea and finally building it the way you originally imagined.
Agent Baymax began as a small hackathon experiment.
Agent Baymax V2 represents the vision that experiment was always meant to become.
And this challenge gave me the perfect excuse to finally finish it.
Top comments (0)