HealthBridge: Engineering an Offline, CPU-Only LLM to Bridge Nigeria’s 1:4,000 Doctor-to-Patient Gap

Victor — Sun, 06 Sep 2026 16:12:47 +0000

This is a submission for Weekend Challenge: Generosity Edition

What I Built

I built HealthBridge; an offline, CPU-only AI assistant designed to tackle Nigeria's severe doctor shortage and connectivity barriers.

Specifically, here is exactly what I delivered:

A fully offline LLM-powered health education tool
I packaged a quantized large language model (Qwen2.5-1.5B-Instruct in GGUF Q4_K_M) so it runs entirely on a standard laptop's CPU. No internet, no GPU, and no recurring cloud costs making it viable for rural and peri-urban Nigerian communities where connectivity is spotty or unaffordable.
A triage and patient education interface
I built an interactive assistant (working through the Streamlit interface challenges you saw in the report) that does four specific jobs for community health workers, patients, and caregivers:

. Explains common symptoms in plain language.

. Describes basic treatments (e.g., oral rehydration therapy for cholera/diarrhea).

. Guides users on when to urgently seek hospital care (red-flag triage).

. Actively steers people away from seeking advice from unqualified friends, neighbours, or unregistered local pharmacies.

Crucially, it is strictly scoped as a complement to professionals; it never claims to replace clinical diagnosis.

A lightweight, resource-optimized runtime
I chose the llama.cpp runtime and carefully picked the Q4_K_M quantization to hit the sweet spot: coherent, medically-contextual responses while fitting comfortably inside an 8GB RAM budget (peaks at just 1.7 GB during my tests). The model runs at 16.76 tokens/second on integrated graphics; no discrete GPU required.
A zero-dependency deployment pipeline
I set up a download_model.sh script to fetch the model via Hugging Face, but after that initial download, the entire inference pipeline runs completely offline. I ensured the model file itself isn't committed to git, so evaluators can pull it fresh and run it immediately on the ADTC standard laptop (10th–12th gen i5, 8GB DDR4, Ubuntu 22.04).

In short: I built a portable, cost-free, offline health-information kiosk-in-a-laptop that gives populations accurate, actionable guidance—bridging the gap between community pharmacies and overstretched hospitals, without ever pretending to be a doctor.

Demo

https://youtu.be/PI3vzJGBDSA

Code

https://github.com/Vicarioy/adtc-2026-submission-template.git

How I Built It

Here is the technical walkthrough of how I built HealthBridge; covering the architecture, the tough trade-offs, and the exact steps I took to wrestle it onto resource-constrained hardware.

Model Selection: The "Goldilocks" Search The first decision was choosing the brain. I couldn't just pick the biggest model.

What I tested: I ran local benchmarks with Phi-3-mini-4k (3.8B—too much RAM, risked exceeding the 8GB target), Qwen2.5-0.5B (fit easily but gave vague, sometimes incorrect health advice), and Llama-3.2-1B (decent, but weaker at following multi-turn health Q&A prompts).

The decision: I chose Qwen2.5-1.5B-Instruct. It sits perfectly in the middle strong enough to reason through symptom-checking and triage logic, yet small enough to leave headroom for the OS and interface. Its multilingual tokenizer also helps handle local language nuances when users describe symptoms in pidgin or their native tongue.

Quantization Strategy (The Accuracy-vs-Memory Trade-off) A raw 1.5B FP16 model wouldn't fit in 8GB comfortably, so I needed compression.

I rejected Q4_0 (too much precision loss—critical for medical context) and Q5/Q8 (too heavy).

I settled on GGUF Q4_K_M. This uses K-means grouping on the most important weight matrices. It applies 4-bit quantization but allocates bits more intelligently across layers. This preserves the model's "reasoning" capability far better than flat quantization, giving me medically coherent responses while keeping peak RAM at just 1.7GB, well under the 8GB ceiling.

Runtime & Inference Engine (llama.cpp) The competition rules strictly mandated llama.cpp, but honestly, it was the perfect choice anyway.

I compiled llama.cpp with standard x86-64 CPU optimizations (leveraging AVX2 instructions on Intel chips).

I wrote a Python wrapper that uses llama-cpp-python to interface with the .gguf file. The key tweak here was setting n_ctx=32768 (to utilize Qwen's full context window for long patient histories) but carefully tuning n_batch to 512 to maximize throughput without blowing up the CPU cache.

This setup gave me 16.76 tokens/second on an integrated UHD 620; fast enough for real-time conversation.

The Interface Nightmare (Where I Almost Broke) You noted in the report that building the interface was a massive problem, and you're right. Streamlit kept timing out or failing silently.

The issue: Loading a 1.5B GGUF model inline during Streamlit's startup sequence blocks the main thread. Streamlit's watchdog timer would kill the process if the model loaded slower than ~10 seconds.

How I fixed it: I decoupled the model lifecycle. Instead of loading the model globally, I wrapped it inside a st.cache_resource decorator with lazy initialization meaning the model only loads on the first user prompt, not when the page boots. I also spawned the llama.cpp inference in a separate background thread with a queue, so the UI remains responsive while the model warms up. This single architectural shift made the app stable and prevented timeouts.

Deployment & Zero-Connectivity Pipeline Since internet is the problem, I had to make internet the non-requirement.

I wrote a download_model.sh script to fetch the .gguf from Hugging Face only once during setup.

I strictly excluded the ~900MB model file from git using .gitignore. The evaluator runs the script, grabs the model, and from that point forward, every inference call hits the local filesystem. No API keys, no network retries, no cloud latency.

The entire app runs on localhost; I pinned the environment dependencies (llama-cpp-python, streamlit) to specific versions to guarantee reproducible builds on the ADTC Ubuntu 22.04 target.

Domain-Specific Prompt Engineering Because I'm legally and ethically scoped to education/triage, not diagnosis, I hardcoded a system prompt that:

Explicitly states: "I am an educational assistant, not a doctor. Do not take my advice as a clinical diagnosis."

Guides the model to always include a "Red Flag" section in responses (e.g., "Go to a hospital immediately if you see blood in stool or have a high fever over 39°C").

Instructs the model to actively deflect dangerous queries if a user asks for a prescription, it responds with safety advice and urges them to visit an MDCN-registered practitioner.

Prize Technology Highlight: This project squarely falls into Edge AI / On-Device AI categories. By combining gguf quantization with llama.cpp, I built a production-ready, CPU-only LLM that democratizes health information in bandwidth-scarce regions; proving that impactful AI doesn't require the cloud.

5 ML Math Concepts I Stopped Memorizing (And Started Actually Using)

Victor — Sun, 06 Sep 2026 15:16:26 +0000

I've been deep in linear algebra and calculus for the better part of a year. But up until this week, I realized i was still treating core ML math like the chain rule and eigenvalue; as a syllabus checklist. Week 3 of my structured deep dive is when i finally forced myself to stop, derive them by hand and connect them to actual code using numpy. Here are some of the concepts that finally clicked:
1. The Chain Rule Is the Engine Behind Backpropagation
Every neural network learns by adjusting its weights, and it knows which direction to adjust them in because of the chain rule. A network is really just a stack of functions: input goes through a layer, then an activation function, then another layer, and so on. To know how much a single weight buried deep in that stack contributed to the final error, you need to differentiate through every function it passed through. That's exactly what the chain rule lets you do. I traced this end to end: starting from gradient descent (which says "move the weights a little in the
direction that reduces error"), through the activation function's own derivative, back to the weight itself. Once I could follow that chain by hand instead of trusting a library to do it, backpropagation stopped feeling like magic and started feeling like bookkeeping.
2. The Gaussian (Normal) Distribution Is Everywhere in ML
The bell curve isn't just a statistics-class cliché; it shows up constantly in machine learning: in how we initialize weights, in the noise assumptions behind linear regression, in probabilistic models, and in how we reason about errors clustering around a mean. Understanding its two parameters; mean and variance, is really understanding "where is the data centered" and "how spread out isit," which turns out to be one of the most reused ideas in the entire field.
3. Eigenvalues Are the Secret Behind Face Recognition
This was the concept that surprised me most. Eigenvalues and eigenvectors which sound purely academic are the foundation of techniques like PCA (Principal Component Analysis) and the classic "Eigenfaces" approach to face recognition. The idea: a face image has thousands of pixels, but most of the meaningful variation between different faces can be captured by a much smaller set of directions, the eigenvectors with the largest eigenvalues. Compress along those directions, and you keep what matters while discarding noise. Linear algebra, doing real work.
4. The Laplace Rule of Succession: Smoothing for Probability
What's the probability of an event you've never seen happen? Naively, zero but that's usually wrong and dangerous in ML, especially in text and classification models where an unseen word or category shouldn't automatically get a probability of zero. The Laplace rule of succession solves this by adding a small smoothing count to every outcome, so nothing is ever assigned a flat-out impossible probability. It's a simple idea with a name that sounds far more intimidating than it is.
5. Sampling and Confidence Intervals: Choosing Data You Can
Trust
You can't train on an entire population, so you sample from it; but that raises an obvious question: how do you know your sample actually represents the population? This is where confidence intervals come in. They give you a principled range around an estimate, along with a stated level of certainty, so that when you pick a dataset you know how much to trust conclusions drawn from it. This felt directly relevant to my own work, since a lot of real-world data (like the regional health data in my thesis project) is sampled, imperfect, and needs exactly this kind of scrutiny.
Why This Matters
None of these five ideas are new inventions, they're decades old. But sitting with the math instead of skipping to the library call changed how I read papers. When something behaves strangely, I now have a better instinct for whether the culprit is a distributional assumption, a smoothing issue, or a bad sample, instead of just tuning hyperparameters and hoping.
Next up: going deeper into differential calculus and connecting it more directly to optimizers beyond
plain gradient descent.

DEV Community: Victor

HealthBridge: Engineering an Offline, CPU-Only LLM to Bridge Nigeria’s 1:4,000 Doctor-to-Patient Gap

What I Built

Demo

Code

How I Built It

5 ML Math Concepts I Stopped Memorizing (And Started Actually Using)