How did you reduce Hallucination?
Hallucination was reduced using Fact-Grounded Distillation Objective (FGDO).
Standard models optimize for fluent text only.
FGDO adds a penalty when generated facts contradict or aren’t supported by UMLS (a trusted medical knowledge base).
This forces the model to prefer factually grounded outputs over plausible-sounding but false ones.
As a result, it achieved higher factual accuracy than BioGPT and GPT-Neo.
What is Hallucination?
AI models sometimes make up medical facts confidently — called hallucination. In healthcare, a wrong fact can literally harm someone. So your project's goal: build a medical AI that doesn't lie.
What is fine-tuning here?
After MediGPT is pre-trained (decoder-only, GPT-style, on the biomedical corpus), it goes through supervised fine-tuning using FGDO — this is the stage where the dual-loss (cross-entropy + factual loss) is applied, using AdamW optimizer, a warmup + cosine decay learning rate schedule, gradient clipping, and FP16 mixed-precision training.
example:
Without FGDO (just fluent text):
"Asthma is caused by eating spicy food."
→ Sounds plausible, but it’s factually wrong.
With FGDO (fact-grounded via UMLS):
"Asthma is a chronic condition where the airways become inflamed and narrow, often triggered by allergens or exercise."
→ This matches medical knowledge and avoids unsupported claims.
So FGDO acts like a fact-checking filter during training — penalizing unsupported statements and rewarding medically accurate ones.
Vector Database
Stores UMLS facts as embeddings so the system can quickly find the closest matching medical fact to verify what the model said.
UMLS = Unified Medical Language System, a big trusted database of medical terms, concepts, and relationships (like a giant medical dictionary + knowledge graph). "UML" (no S) is a totally different thing — a software design diagram notation. In your project, you mean UMLS embedding.
What is a UMLS embedding?
An embedding is just a way of turning text into a list of numbers (a vector) that captures its meaning, not just its spelling. Similar meanings → similar numbers.
So a UMLS embedding means: taking each medical concept/fact from UMLS (e.g., "Metformin treats Type 2 Diabetes") and converting it into a numerical vector that represents its meaning.
React
A chat-style UI where users type medical questions and get answers with a disclaimer to consult healthcare professionals. React would be the natural choice to build that client-side chat UI — handling message state, rendering conversation history, and making calls (via Fetch API, tying back to your earlier question) to your backend/model API to get responses.
GPT — Full form
Generative Pre-trained Transformer — and MediGPT's architecture is decoder-only, following this GPT family design, since decoder-only models suit autoregressive generation (ideal for summarization, report generation, medical Q&A).
MediGPT uses this decoder-only architecture (predicts text word-by-word).
One-liner for interviews:
"I built a medical AI and created FGDO — a training method that checks facts against a trusted medical database using vector search, and penalizes the model for wrong answers, making it more reliable than RAG-based or plain fine-tuned models."
The Problem
AI models sometimes make up medical facts confidently — called hallucination. In healthcare, a wrong fact can literally harm someone. So your project's goal: build a medical AI that doesn't lie.
What is MediGPT?
A custom AI language model (like ChatGPT, but decoder-only GPT-style architecture) specifically trained for medical/clinical text — answering health questions, summarizing clinical notes, etc.
Step 1: Data
Trained using real medical text:
PubMed — medical research papers
MIMIC-III/IV — real (anonymized) ICU patient records
UMLS — a trusted medical knowledge base (used to check facts, not as training text)
MedQuAD / MIMIC-IV-Ext-BHC — used to create practice questions during training
Step 2: Custom Tokenizer
Normal AI models break medical words like "electroencephalogram" into weird chopped-up pieces. You built a custom tokenizer (BPE) trained on medical text so complex terms are understood as single, clean units — more efficient and accurate.
Step 3: Model Architecture
Decoder-only Transformer (same family as GPT) — good at generating text step-by-step. Also uses ALiBi, a technique that helps it handle very long clinical documents without losing track of context.
Step 4: The Big Innovation — FGDO
This is your main contribution, and it's how you reduce hallucination:
Model answers a question
The answer is broken into small individual facts.
Each fact is checked against UMLS (using a vector database for similarity search — matching the fact's meaning to trusted medical entries).
If a fact is false or unverifiable → penalty. If true → no penalty (or reward).
This penalty ("factual loss") is added to normal training loss, so the model is directly trained to avoid making things up — not just fine-tuned to sound fluent.
RAG vs FGDO — the key distinction
RAG = look up facts at answer-time and hand them to the model as a cheat sheet. Doesn't fix the model itself.
FGDO = teach the model during training to stop making false claims. Fixes the model's actual behavior.
Your project uses FGDO, not RAG — that's the novel part.
The App
A React-based chat interface (like a chatbot) where users ask medical questions and get answers — always with a disclaimer to consult a real doctor.
Results
Compared to BioGPT and GPT-Neo, MediGPT scored higher on:
ROUGE-1 & ROUGE-L (text quality/similarity to correct answers)
Factual Accuracy (0.95 vs 0.85 and 0.77)
Ethics built in
Never replaces doctors — just assists ("human-in-the-loop")
Follows HIPAA/GDPR for patient privacy
Flags uncertainty instead of guessing when unsure
Checked for bias across different patient groups
One-line summary you can say in an interview:
"I built a medical AI model and proposed a new training method called FGDO, which checks the model's facts against a trusted medical database during training and penalizes it for wrong information — making it more factually reliable than existing models like BioGPT, without depending on RAG."
Top comments (0)