HealthBridge: Engineering an Offline, CPU-Only LLM to Bridge Nigeria’s 1:4,000 Doctor-to-Patient Gap

This is a submission for Weekend Challenge: Generosity Edition

What I Built

I built HealthBridge; an offline, CPU-only AI assistant designed to tackle Nigeria's severe doctor shortage and connectivity barriers.

Specifically, here is exactly what I delivered:

1. A fully offline LLM-powered health education tool
   I packaged a quantized large language model (Qwen2.5-1.5B-Instruct in GGUF Q4_K_M) so it runs entirely on a standard laptop's CPU. No internet, no GPU, and no recurring cloud costs making it viable for rural and peri-urban Nigerian communities where connectivity is spotty or unaffordable.

2. A triage and patient education interface
   I built an interactive assistant (working through the Streamlit interface challenges you saw in the report) that does four specific jobs for community health workers, patients, and caregivers:

. Explains common symptoms in plain language.

. Describes basic treatments (e.g., oral rehydration therapy for cholera/diarrhea).

. Guides users on when to urgently seek hospital care (red-flag triage).

. Actively steers people away from seeking advice from unqualified friends, neighbours, or unregistered local pharmacies.

Crucially, it is strictly scoped as a complement to professionals; it never claims to replace clinical diagnosis.

1. A lightweight, resource-optimized runtime
   I chose the llama.cpp runtime and carefully picked the Q4_K_M quantization to hit the sweet spot: coherent, medically-contextual responses while fitting comfortably inside an 8GB RAM budget (peaks at just 1.7 GB during my tests). The model runs at 16.76 tokens/second on integrated graphics; no discrete GPU required.

2. A zero-dependency deployment pipeline
   I set up a download_model.sh script to fetch the model via Hugging Face, but after that initial download, the entire inference pipeline runs completely offline. I ensured the model file itself isn't committed to git, so evaluators can pull it fresh and run it immediately on the ADTC standard laptop (10th–12th gen i5, 8GB DDR4, Ubuntu 22.04).

In short: I built a portable, cost-free, offline health-information kiosk-in-a-laptop that gives populations accurate, actionable guidance—bridging the gap between community pharmacies and overstretched hospitals, without ever pretending to be a doctor.

Demo

https://youtu.be/PI3vzJGBDSA

Code

https://github.com/Vicarioy/adtc-2026-submission-template.git

How I Built It

Here is the technical walkthrough of how I built HealthBridge; covering the architecture, the tough trade-offs, and the exact steps I took to wrestle it onto resource-constrained hardware.

• Model Selection: The "Goldilocks" Search The first decision was choosing the brain. I couldn't just pick the biggest model.

What I tested: I ran local benchmarks with Phi-3-mini-4k (3.8B—too much RAM, risked exceeding the 8GB target), Qwen2.5-0.5B (fit easily but gave vague, sometimes incorrect health advice), and Llama-3.2-1B (decent, but weaker at following multi-turn health Q&A prompts).

The decision: I chose Qwen2.5-1.5B-Instruct. It sits perfectly in the middle strong enough to reason through symptom-checking and triage logic, yet small enough to leave headroom for the OS and interface. Its multilingual tokenizer also helps handle local language nuances when users describe symptoms in pidgin or their native tongue.

• Quantization Strategy (The Accuracy-vs-Memory Trade-off) A raw 1.5B FP16 model wouldn't fit in 8GB comfortably, so I needed compression.

I rejected Q4_0 (too much precision loss—critical for medical context) and Q5/Q8 (too heavy).

I settled on GGUF Q4_K_M. This uses K-means grouping on the most important weight matrices. It applies 4-bit quantization but allocates bits more intelligently across layers. This preserves the model's "reasoning" capability far better than flat quantization, giving me medically coherent responses while keeping peak RAM at just 1.7GB, well under the 8GB ceiling.

• Runtime & Inference Engine (llama.cpp) The competition rules strictly mandated llama.cpp, but honestly, it was the perfect choice anyway.

I compiled llama.cpp with standard x86-64 CPU optimizations (leveraging AVX2 instructions on Intel chips).

I wrote a Python wrapper that uses llama-cpp-python to interface with the .gguf file. The key tweak here was setting n_ctx=32768 (to utilize Qwen's full context window for long patient histories) but carefully tuning n_batch to 512 to maximize throughput without blowing up the CPU cache.

This setup gave me 16.76 tokens/second on an integrated UHD 620; fast enough for real-time conversation.

• The Interface Nightmare (Where I Almost Broke) You noted in the report that building the interface was a massive problem, and you're right. Streamlit kept timing out or failing silently.

The issue: Loading a 1.5B GGUF model inline during Streamlit's startup sequence blocks the main thread. Streamlit's watchdog timer would kill the process if the model loaded slower than ~10 seconds.

How I fixed it: I decoupled the model lifecycle. Instead of loading the model globally, I wrapped it inside a st.cache_resource decorator with lazy initialization meaning the model only loads on the first user prompt, not when the page boots. I also spawned the llama.cpp inference in a separate background thread with a queue, so the UI remains responsive while the model warms up. This single architectural shift made the app stable and prevented timeouts.

• Deployment & Zero-Connectivity Pipeline Since internet is the problem, I had to make internet the non-requirement.

I wrote a download_model.sh script to fetch the .gguf from Hugging Face only once during setup.

I strictly excluded the ~900MB model file from git using .gitignore. The evaluator runs the script, grabs the model, and from that point forward, every inference call hits the local filesystem. No API keys, no network retries, no cloud latency.

The entire app runs on localhost; I pinned the environment dependencies (llama-cpp-python, streamlit) to specific versions to guarantee reproducible builds on the ADTC Ubuntu 22.04 target.

• Domain-Specific Prompt Engineering Because I'm legally and ethically scoped to education/triage, not diagnosis, I hardcoded a system prompt that:

Explicitly states: "I am an educational assistant, not a doctor. Do not take my advice as a clinical diagnosis."

Guides the model to always include a "Red Flag" section in responses (e.g., "Go to a hospital immediately if you see blood in stool or have a high fever over 39°C").

Instructs the model to actively deflect dangerous queries if a user asks for a prescription, it responds with safety advice and urges them to visit an MDCN-registered practitioner.

Prize Technology Highlight: This project squarely falls into Edge AI / On-Device AI categories. By combining gguf quantization with llama.cpp, I built a production-ready, CPU-only LLM that democratizes health information in bandwidth-scarce regions; proving that impactful AI doesn't require the cloud.