*This post is my submission for [DEV Education Track: Build Apps with Google Ai
What I Built
I built Gemini HealthBot, a multimodal AI-powered doctor applet designed to provide reliable, accessible medical consultations to people worldwide, especially in underserved areas. The core idea is to leverage Gemini's capabilities to create a "self-reliable" doctor—meaning it cross-verifies its responses with built-in fact-checking prompts and user feedback loops to improve accuracy over time—acting as a virtual physician for all mankind. This bot addresses the global healthcare gap by offering preliminary diagnoses, symptom analysis, and preventive advice without needing in-person visits.
The problem it solves: Billions lack timely medical access due to geography, cost, or shortages. Gemini HealthBot democratizes health info, using multimodal inputs (images of symptoms, voice descriptions, text queries) to deliver empathetic, evidence-based responses. It shapes the future by envisioning a network of specialized "Gemini bots" (e.g., dermatology bot, mental health bot) that evolve via community data, fostering a proactive, AI-augmented healthcare ecosystem.
Built as a web applet deployed on Cloud Run, it's user-friendly: users input symptoms via text, upload photos/videos of issues (e.g., rashes, wounds), or speak aloud, and get tailored advice with disclaimers to consult professionals.
Demo
Deployed applet: https://gemini-healthbot-2025.run.app (Hosted on Google Cloud Run for easy access).
Here's a quick demo video showcasing the bot in action (2-minute walkthrough): Watch on YouTube.
Screenshots:
Home Interface: Clean chat UI with options for text, image upload, audio recording. (Image: User greets bot, selects "Symptom Check".)
Multimodal Input: User uploads a photo of a skin rash, types "Itchy red spots on arm," and records audio describing onset. (Image: Upload modal with image preview and waveform for audio.)
Output Response: Bot analyzes: "Based on the image, this resembles eczema. Audio suggests allergy trigger. Recommendations: Moisturize, avoid irritants. Confidence: 85% (self-verified via medical sources)." Includes visual summary chart. (Image: Response card with image annotation and advice list.)
Feedback Loop: Post-consult, user rates accuracy; bot logs for self-improvement. (Image: Thumbs up/down buttons.)
Since I used Gemini 2.5 Flash Image during the free trial (Sept 6-7), the video demonstrates full functionality, ensuring judges can see it even if trial features are limited post-submission.
How I Used Google AI Studio
I used Google AI Studio to prototype and deploy the entire applet rapidly, starting from prompt engineering in the studio's interface. I leveraged Gemini 2.5 Pro for core reasoning and multimodal processing, integrating the Live API for real-time chat sessions (up to 3 concurrent free-tier sessions). The applet is built as a prompt-based system where user inputs are fed into a structured prompt chain: first for input parsing, then multimodal analysis, and finally response generation with reliability checks.
Deployment was seamless via Cloud Run integration directly from AI Studio—no custom code needed beyond prompt tuning. I tested iterations in the studio's playground, using sample images/videos/audio to refine prompts for accuracy (e.g., "Analyze this image for dermatological issues, cross-reference with WHO guidelines"). This allowed quick pivots, like adding audio transcription for voice inputs. Overall, AI Studio handled 100% of the backend logic, making it accessible for solo devs to build production-ready multimodal apps.
Multimodal Features
The applet shines with Gemini's multimodal capabilities, enhancing UX by making consultations intuitive and comprehensive—like talking to a real doctor via phone/video.
Image Understanding (Gemini 2.5 Pro/Flash): Users upload photos of visible symptoms (e.g., skin conditions, injuries). The bot describes and diagnoses (e.g., "The irregular borders suggest possible melanoma—seek urgent care"), annotating the image in responses. This boosts reliability by visual evidence, reducing miscommunication from text alone; UX win: Immediate, visual feedback builds trust, especially for non-verbal symptoms.
Audio Processing (Live API): Voice inputs for describing symptoms (e.g., "I've had chest pain for two days"). Gemini transcribes and analyzes tone/stress for emotional context (e.g., detecting anxiety). Enhances accessibility for low-literacy users or those multitasking; UX: Feels conversational, like a telehealth call, with transcribed summaries for review.
Combined Modalities: Prompts fuse inputs (e.g., image + audio + text) for holistic analysis: "Integrate rash image, voice description of fever, and query on travel history to assess tropical disease risk." Self-reliability via prompt-enforced citation (e.g., "Based on CDC data...") and feedback (users flag errors, bot adjusts future prompts).
These features create an empathetic, future-shaping experience: The bot isn't just reactive but proactive (e.g., suggesting lifestyle bots for follow-up), empowering users globally while emphasizing it's not a substitute for professionals. This multimodal fusion makes health advice more accurate and engaging, potentially saving lives in remote areas.
Top comments (0)