Every clinic day starts the same way for me. A patient is due in a few minutes, and their chart is years deep. Old discharge summaries, a medication list that's been rewritten four times, labs that drift a little each visit, a scanned referral someone typed up in a hurry. You read fast, because that's the job. And somewhere in all of that there is usually one line that matters more than the rest. A potassium quietly climbing. An interaction sitting two pages back. What I always wanted was a second set of eyes on that part, something that reads the whole record properly while I stay focused on the person in front of me. That's why I built ChartPilot.
Now, people are quick to tell me a large language model can already read all of this. They're not wrong. Dump the files in, write a decent prompt, and you'll get a clean summary in seconds. I've done it. It's genuinely useful. But a summary is not the thing I actually need at 9am with a full clinic. What I need is something that makes sure I don't miss the finding that changes what I do next, shows me exactly where it got that from so I can check it in a second, and then lets me act on it or deliberately decide it doesn't matter. That last part matters more than it sounds. A tool that just flags everything is noise. A tool that lets me confirm, override, or correct, and remembers, is a colleague.
That combination is the bit I couldn't find anywhere. Plenty of things summarise. Very few help a doctor not miss, and let the doctor stay firmly in charge of what to do about it, and quietly get better while doing it. So I made one.
Before I get into how it works, here's a short walkthrough of the actual thing.
Here's how it actually works, without the marketing gloss. ChartPilot reads a patient's record and produces a one-page brief, findings ranked by how much they should worry me. Underneath, there's a rule I refused to bend: every fact on that page comes from plain deterministic code reading the chart, never from the model's imagination. The AI is allowed to be an editor, phrasing what the code already found, and nothing more. On top of that there's a layer that throws out any claim without a citation, and then a second AI model, kept blind to the first, whose only job is to try to prove each claim wrong. If a claim can't survive that, it gets dropped before I ever see it. And if something breaks in the pipeline, the page says so loudly and flags itself for review. It never gives me a calm, empty "nothing found," because in medicine that is the most dangerous output there is.
Here's the whole thing on one page, if you'd rather see it than read it.
The part I'm genuinely excited about is that it improves itself, and it does that in a way I think is underrated. I did not train a model from scratch. I don't have the data or the compute to build a clinical language model from the ground up, and honestly, most people who want to help doctors don't either. What I found is that you can move real outcomes just by changing and testing the prompts, the instructions the models run under, guided by what doctors actually confirm and reject. When I work through my patients and mark findings as right or wrong, those small signals feed a loop that drafts a new candidate prompt, tests it against held-out cases, and only keeps it if the claims survive review at a higher rate than before. In our runs it promoted a better prompt every round. Nobody sat down to annotate a dataset. The improvement came out of a doctor just doing their normal work.
Something about that process caught me off guard. Going in, I assumed better output would need a better model, or a smarter harness wrapped around it. That's the instinct you absorb from watching AI tools level up month after month. It wasn't really where my gains came from. Most of the improvement came from the prompt itself getting sharper, rewritten and re-tested against what my actual cases exposed. Self-improving prompt engineering, if that's a fair name for it, turned out to be a genuine skill, and a quietly underrated one. You can shift the outcome a fair amount without touching a single model weight, just by being strict about how you instruct the thing and what evidence you force it to show. I didn't expect that when I started.
I keep coming back to that, because I think it's the honest path for healthcare AI. The signal we need to make these tools better is already being generated every day, by doctors making judgement calls. If a tool can capture that judgement without adding friction, then every doctor using it is quietly making the next version better for the next doctor, and those prompt changes become a stream of training data for future medical AI without anyone stopping their day to produce it. That's a very different bet from building a giant model first and hoping doctors adapt to it.
I should be clear about what this is not. It is not a replacement for a doctor taking a proper history, and I don't think it should be, at least not yet. I am not trying to take the clinician out of the loop. I'm trying to take the misses out of the clinician's day. I believe, pretty strongly, that AI in medicine won't replace healthcare workers. It will reduce their burden and make them more effective, and the second-order effect of that is more important than it first appears: a doctor who spends less time drowning in chart review can see more people, and more carefully. That's not a productivity slogan. In a lot of the world, access to a careful doctor is the actual bottleneck in someone's health. Tools that give doctors their attention back make care more available, full stop.
And there's something about this work that no amount of automation changes. We don't treat diseases in the abstract. We treat people, and people are gloriously, frustratingly individual. Every single one is different, a bit unpredictable, and carrying a history that never quite fits the textbook. A model can help me hold all of that in view at once. It cannot be the one who sits with the person and decides what to do. I don't want it to be. I want to be a better doctor because I had help not missing things, not a slower one because I was busy fighting a tool.
The caveats I'm not going to pretend away
Anything that touches real patient data has to clear a high bar first. Strict privacy testing, a proper safety profile, and full compliance with the law, before it goes anywhere near a real clinic. I take that seriously, and everything I've built so far runs on synthetic data only, for exactly that reason. This is a prototype, not a medical device, and I'd rather say that plainly than dress it up.
But here's the tension I sit with. If we wait until everything is perfect before we let these tools touch a doctor's day at all, we will move at a crawl, while AI for writing code and drafting emails races ahead. Part of why code assistants got good so fast is that the field let them into the real workflow early and learned from millions of tiny corrections out in the open. Healthcare can't copy that recipe as is. The stakes and the rules are not the same. But it can borrow the shape of it, and what that really asks for is a change in how we think, not a cleverer algorithm. We'd have to be willing to let careful tools do real work next to doctors, and then treat the things doctors correct as the fuel that makes the next version better, all inside privacy and safety rules a code assistant never had to worry about. Healthcare is the backbone of a functioning society. It deserves to move quickly too, and the only responsible way I can see to do that is to start now, in small careful pilots, with tools that reduce a doctor's workload instead of adding to it, and then improve on the signal that produces, all inside the rules. Reduce the friction first and earn some trust. Then get better on what that produces. That order matters more than it sounds.
Where this could go, if someone runs with it
I'll be honest about the scope. This is a demo, built for a hackathon, and I don't have concrete plans to grow it into a product myself right now. What I'd really love is for the idea to get picked up and built properly, by people who can carry it through the parts that matter most before anything touches a real patient. The privacy and safety work would come first, and then small, careful pilots with real doctors who can tell me where it helps and where it gets in the way. If that resonates, or you're already building something in this space, I'd be happy to compare notes.
I built ChartPilot for a hackathon, but the feeling behind it isn't a hackathon feeling. It's the one I have every time I open a chart that's too long to read properly and hope I don't miss the thing that matters. I'd like to stop hoping and start knowing. That's the whole idea.
If you want to poke at it yourself, the live demo is here and the full source is on GitHub. The walkthrough video is near the top of this post.
I built ChartPilot for the All Things Agentic Hackathon, so this write-up was created for the purposes of entering it. If you share it anywhere, the tag is #AllThingsAgenticHackathon.
One necessary reminder: everything here runs on synthetic data only. ChartPilot is a prototype, not a medical device, and it isn't clinically validated.
Top comments (0)