Endo.AI – Multimodal Endocrinology Assistant (Google AI Studio Challenge Submission)
Endo.AI – Multimodal Endocrinology Assistant
🌍 Overview
Endo.AI is an intelligent medical platform for endocrinology diagnosis and decision support.
It combines medical image analysis (DICOM/JPG) with clinical reasoning and AI-powered conversations.
The app leverages Google Gemini for true multimodal understanding (image + text).
⚡ Features
- 📂 Patient Management – View, edit, and securely store patient data.
- 🩻 Medical Image Analysis – Automatic diagnosis from ultrasound, CT, or MRI images (DICOM/JPG).
- 💬 Clinical Assistant – Converse with an AI agent to analyze cases and receive suggestions.
- 📑 Guidelines – Retrieve medical guidelines per disease or specialty.
🛠️ Tech Stack
- Frontend: Next.js + Tailwind CSS
- Backend: FastAPI (Python)
- Database/Storage: Supabase
- AI: Google Gemini Pro (Multimodal API)
- Deployment: Dockerized for cloud hosting
📸 Screenshots
🚀 How it Works
- A doctor uploads an image (DICOM or JPG).
- The AI assistant analyzes the scan and correlates findings with lab results.
- The user can chat with the Clinical Assistant for further reasoning.
- Medical guidelines are integrated dynamically to support clinical decision-making.
🎯 Why it matters
Endocrinology requires combining imaging, lab tests, and clinical judgment.
With multimodal AI, doctors benefit from:
- Faster and more accurate diagnosis
- Holistic view of the patient’s case
- Improved decision support
🧑💻 Repository & Demo
- GitHub Repo: https://github.com/micktomi/endocrinology-backend https://github.com/micktomi/endocrinology-frontend
- Live Demo (optional): [insert link here]
✅ Submission
This project is submitted for the Google AI Studio Multimodal Challenge.
Overall Assessment
Of course. Here is a professional medical analysis of the provided thyroid ultrasound image, structured as a report for an endocrinologist.
Endocrinology Ultrasound Report
Patient Information:[Not Provided]Date of Exam:03/11/2018**Modality:Thyroid Ultrasound
1. Image Quality:
The provided image is a single, static, high-resolution B-mode ultrasound view of the right thyroid lobe, obtained with a high-frequency linear transducer (ML6-15, 11.0 MHz). The image quality is adequate for the assessment of the visualized nodule and adjacent parenchyma.
- Limitations:This single view precludes a comprehensive assessment. Evaluation is limited by the absence of images in orthogonal planes (transverse and longitudinal), cine loops, measurements, Color Doppler imaging for vascularity assessment, and evaluation of the contralateral lobe, isthmus, and regional lymph nodes. **2. Endocrine Glands & Related Structures:
Thyroid Gland:The image focuses on the right thyroid lobe ("RT LOBE"). A prominent nodule is seen occupying a significant portion of the field of view. The surrounding thyroid parenchyma appears to have a normal, homogeneous echotexture, though only a small area is visualized for comparison.* **Adjacent Structures:Overlying strap muscles are visible superficially. Deeper and surrounding anatomical structures (e.g., carotid artery, jugular vein, trachea) are not included in this specific image.
**3. Pathological Findings:
A solitary thyroid nodule is identified within the right lobe with multiple suspicious sonographic features:Composition:Predominantly solid.* *Echogenicity:Markedly hypoechoic relative to the adjacent thyroid parenchyma. *Shape:The nodule has a "taller-than-wide" morphology, with its anteroposterior dimension appearing greater than its transverse dimension. *Margins:The margins appear irregular and are not well-circumscribed. *Echogenic Foci:Multiple punctate echogenic foci are scattered throughout the nodule, highly suspicious for microcalcifications. **Size:Based on the provided scale, the nodule's largest dimension is estimated to be approximately 1.0 - 1.2 cm. Precise three-dimensional measurements are required.
**4. Impression / Differential Diagnosis:Primary Impression:The constellation of findings (markedly hypoechoic, solid, taller-than-wide, irregular margins, and microcalcifications) is **highly suspicious for malignancy. This sonographic pattern is most characteristic of Papillary Thyroid Carcinoma (PTC). * ACR TI-RADS Classification:This nodule would be classified as **TR5 (Highly Suspicious), with a score ≥7 points.
Differential Diagnosis:
Papillary Thyroid Carcinoma (most likely)
Medullary Thyroid Carcinoma
Poorly differentiated thyroid cancer
An atypical benign nodule (very unlikely given the multiple high-risk features)
5. Recommendations:
- **Fine Needle Aspiration (FNA):Ultrasound-guided FNA biopsy of this nodule is strongly recommended for definitive cytopathological diagnosis.2. **Complete Neck Ultrasound:A comprehensive diagnostic neck ultrasound should be performed to include: * Formal three-plane measurement of the nodule.
- Thorough evaluation of the entire thyroid gland (both lobes and isthmus).
- Detailed survey of the central (Level VI) and lateral (Levels II-V) cervical lymph node compartments for evidence of metastatic disease.
- Assessment of nodule vascularity with Color Doppler imaging.
- Laboratory Evaluation:Baseline thyroid function tests (TSH, Free T4) should be obtained. **6. Urgency:
- *HIGH PRIORITY.The sonographic features are highly predictive of malignancy. While not a medical emergency, the diagnostic workup, particularly the FNA biopsy, should be expedited to facilitate timely diagnosis and management planning. The patient should be evaluated by an endocrinologist or thyroid surgeon promptly. ** **Disclaimer:This analysis is based on a single medical image and is for educational and informational purposes only. It is not a substitute for a formal radiological report or a consultation with a qualified healthcare professional. Clinical correlation and a complete evaluation are essential for patient diagnosis and management. 📋 Recommendations • ⚠️ URGENT: Immediate medical evaluation is required based on this analysis. • AI analysis requires verification by a qualified healthcare professional.
Confidence: 85.0% | Processing: 27.6s
Top comments (0)