I'm a critical care physician in Colombia. For months I've been
building OMAXI, an LLM pipeline that reads Spanish clinical
documents — nursing records, resident notes, lab reports — and
turns them into structured data plus a note a physician can sign.
The rule is narrow: structure what the documents say, never infer
what they don't. In critical care, the failure that matters isn't
a missed nuance. It's a confident fabrication.
Last week I ran an experiment: map the pipeline's output to FHIR
R4 and validate it against a real server (a Medplum project) —
actual POSTs, every resource read back to confirm what persisted.
Of 38 resources in the first run, the server rejected 2.
One of them was norepinephrine — the drug that defines the
severity of septic shock. The reason: R4 carries an invariant on
MedicationAdministration (mad-1) requiring a dose or a rate.
The source documented "0.35 mcg/kg/min", which can't become a
UCUM quantity without the patient's weight — and the pipeline
doesn't extract weight. Text alone isn't enough. The standard
said no.
There was an easy fix: write dose: 0.35 mcg and pass validation
instantly — producing a perfectly valid resource that lies about
a vasopressor dose. I did the opposite: declared the absence
explicitly with the standard data-absent-reason extension
(as-text). It satisfies the invariant without asserting any
quantity, and preserves the original text and route.
The goal isn't to produce valid resources. It's to avoid
producing valid resources that lie.
That rejection turned out to be the most useful moment of the
whole experiment. The rest of the friction log is just as
instructive:
- The schema kills data the model already read. Nursing documented exact administration times; the pipeline's type schema had no temporal field, so they vanished. The patient's weight appeared in two documents and reached the output in none. A negation ("PE ruled out") was extracted — into the wrong field — and a sanitizer flattened it into an asserted finding. None of this is fixable with a better prompt.
- Two blood draws became four Observations. The same creatinine value, restated in two physicians' notes, produced phantom data points — enough to fake a KDIGO staging change that never happened.
- My own layer was no cleaner. I assigned LOINC codes with clinical judgment; checked against NLM's official table, 15% were wrong. And a genuine end-to-end run caught a factor-of-ten unit bug my hand-written test fixture structurally couldn't — because I had written the fixture to match my own assumptions.
Both runs used entirely fictitious data — including a fabricated
day-2 ICU case (septic shock, four documents, Spanish) that's now
in the repo as a reusable test case for anyone working on
Spanish-language clinical NLP.
Full write-up — what survives the trip to FHIR, what breaks, and
in which layer:
https://github.com/JesusPantojaP/fhir-spanish-icu-notes
If you work on FHIR in Latin America, Spanish-language clinical
NLP, or LLM extraction pipelines with validation layers, I'd
genuinely like to compare notes — especially on terminology
assignment with a human in the loop, which I don't think is
safely automatable yet.
Top comments (0)