In our latest CodingCat.dev podcast recording, Shivay Lamba (@howdevelop) walked through a practical real-world use case for 100% offline, on-device AI agents in healthcare.
The Problem: Privacy and Latency in Healthcare Prescriptions
When a doctor consults with a patient and writes a prescription, traditional cloud LLM pipelines require sending sensitive patient audio and medical records over the network, or handing the patient a physical slip to wait in line at a pharmacy.
The On-Device Agentic Architecture
Instead of routing sensitive patient conversations to external cloud APIs, Shivay demonstrates a two-stage local workflow running directly on a doctor's laptop or smartphone:
| Stage | Local Model / Component | Execution Environment | Role in the Workflow |
|---|---|---|---|
| 1. Consultation Transcription | Local Speech-to-Text (STT) | On-Device (Laptop / Phone) | Transcribes the doctor-patient consultation and prescription notes completely offline. |
| 2. Structured Tool Calling | Gemma (12B Parameter Model) | On-Device Local Runtime | Parses the completed transcript, extracts the medication order, and triggers an automated pharmacy tool call. |
| 3. Pharmacy Fulfillment | Internal Order API | Local / Hospital Network | Queues the prescription so the order is ready for pickup by order number as soon as the patient arrives. |
Watch the Clip
- Video Short: Watch the 2-minute breakdown
- Interactive Edge Inference Walkthrough: Inside an Inference Request
- More Episodes: CodingCat.dev Podcasts
Top comments (0)