Lipwad is my entry in the Meta Global AI Developer Hackathon. One of the prizes goes to the project whose X post gets the most engagement by Sunday Nov 8, 11:59 PM ET. If this build is useful to you, a like, repost or reply on the post above helps a lot.
The problem I kept seeing
I'm a clinical officer in Kenya. In Kenya, patients pay hospitals by cash, private insurance or SHA, the Social Health Authority. With SHA, the hospital treats first and claims later. That means the hospital's money depends on paperwork that is filled in during a busy shift.
SHA figures reported by The Star in February and July 2026 show the cost of that.
- About 1 in 5 county hospital claims go unpaid.
- 91,935 claims were rejected for missing or incomplete documents.
- 23,000 claims were rejected because they were not resubmitted in time.
Claims are checked carefully on the way in. I wanted hospitals to have a tool just as careful on the way out.
What Lipwad does
Lipwad is an AI agent that prepares and checks SHA claims before they leave the hospital.
- It reads the claim file. Staff photograph the claim form and invoice. A blurred page gets a retake request, not a guess.
- It hears the voice note. Staff can record what happened. A spoken date becomes a field cited to the seconds it was said.
- It verifies. It checks the practitioner and facility against the registry, compares what was said with what was written, and runs rule checks.
- It decides and explains. Each claim is green (ready), amber (one named fix) or red (blocked). The explanation comes in English and Kiswahili.
- A person approves. A green claim goes out only after a named clerk confirms every field.
- It rescues returned claims. It reads the corrected page, shows the one field that changed with its source, and resubmits only that correction after a person approves. The deadline countdown stays visible.
Lipwad is not medical AI. It gives no clinical advice, never adds services and never raises amounts. All demo data is synthetic.
Here's the full walkthrough.
The Meta models and what each one does
| Role | Model | Served by |
|---|---|---|
| Reading photographed documents, with the region each value came from | Muse Spark 1.3 (vision) | Meta Model API |
| Transcribing voice notes with timestamps | Muse Voice Transcribe 1.0 | Meta Model API |
| Agent reasoning and the bilingual explanation | Meta's open-weights Muse Spark model (muse-spark-open-1026) |
Baseten |
| Writing the model client layer | Muse Code | My terminal |
All model calls go through one client layer in backend/app/models/. It maps each role to a provider, so any role can fall back to the Meta Model API if a partner endpoint is down.
Design decision 1. Every value carries its source
A claim value with no source is just a guess that looks official. So every field in Lipwad holds both the value and where it came from.
- A field read from a photo cites the document and the region of the image. Clicking it in the review screen highlights that region.
- A field heard in a voice note cites the timestamp. The review screen plays that exact moment.
The clerk never has to trust the model. They can check any value in one click.
Design decision 2. Flag conflicts, never resolve them silently
In one demo case, the nurse's voice note says the patient was admitted on 11 October. The form says 12 October.
A model that "helpfully" picks one would put a wrong date on a legal document. Lipwad marks the claim amber with "admission date mismatch" and cites both sources, the photo region and the seconds of audio. A person decides which is right.
Design decision 3. The model explains, but never sets the status
Green, amber and red come from deterministic checks against the registry and the rules. The model writes the explanation but cannot change the status. That means no prompt, however clever, can turn a blocked claim green.
Design decision 4. Rescue means one change, with its source
The case I care most about is a returned claim near its deadline. In the demo, a claim comes back unsigned on day 13 of its 14-day resubmission window. Lipwad reads the newly signed page, proposes exactly one change, cites where it came from and resubmits after a person approves. It does not rewrite the claim or touch anything else.
The stack
- Backend. Python 3.12, FastAPI, Pydantic v2, SQLAlchemy and SQLite. Every agent tool is a typed function with Pydantic input and output.
- Frontend. React, Vite and TypeScript, as an installable PWA.
- Submission. A mock HIE that returns FHIR-shaped responses, standing in for the national sandbox, which wasn't available to me.
- Hosting. Modal, with state on a Modal Volume. The live demo resets itself every 6 hours.
How I tested it
The demo runs on 12 synthetic cases. Three clean claims (outpatient, inpatient, maternity) should come out green. Nine failure cases include an unsigned form, a non-itemised invoice, a practitioner not in the registry, a blurred photo, the voice-note date conflict and the returned claim near its deadline.
make test runs 224 backend tests with no network. CI replays recorded model responses, so tests never call a paid model. A claim takes 22 to 37 seconds from photos to a cited status.
What's not built yet
Here is what's still missing.
- Rule IDs are placeholders until the official SHA documents are registered in the repo. Lipwad never invents rule text.
- Submissions go to the mock HIE, not the national system.
- Voice notes are English only for now, because Kiswahili isn't yet on Muse Voice Transcribe's language list.
- The in-app camera capture screen, SAM 3.1 segmentation and terminology coding are next.
Try it
- Live demo (synthetic data) https://stellaacharoiro--lipwad-web.us-east.modal.run
- Code (Apache 2.0) https://github.com/EvarestTechnologies/lipwad
Hospitals deserve tools as good as the ones checking their claims. If you agree, please like, repost or reply on the post below before Sunday night. That's what counts for the prize.
Top comments (0)