Nnamdi Felix Ibe

Posted on Oct 14 AI-assisted

Part 1: Why NHS Trusts Need an HL7 v2 to FHIR Layer, and How This One Is Built

#discuss #ai #community #architecture

Hi I'm Nnamdi.

This series walks through a project I built around one question: how does a patient's data get from a hospital system that speaks HL7 v2 to services that expect FHIR? Part 1 covers the problem and the architecture. The next six parts go deeper, one area at a time.

The problem

Two standards meet here.

HL7 version 2 is the older one. HL7 International calls it "the workhorse of electronic data exchange in the clinical domain". A v2 message is plain text. Each segment is a line of fields separated by pipes, and the standard says "the segment terminator is always a carriage return". An admission is sent as an ADT^A01 message, which the standard describes as "sent as a result of a patient undergoing the admission process". Its segments include MSH for the message header, PID for patient identification and PV1 for the patient visit.

FHIR is the newer one. It models data as resources, such as Patient and Encounter, and exchanges them as JSON or XML over a REST API.

National NHS services are moving to FHIR. The NHS service manual says the PDS FHIR API "meets the latest standards". It also says that if you want to start using a PDS API, "this is the version to use". The older HL7 V3 version is under review.

So a hospital system that sends HL7 v2 needs something in between. That layer has to translate each message, check the patient against national records, and store the result somewhere that speaks FHIR. This project builds that layer.

What I built

The NHS Interop Platform is a small, production-shaped reference implementation. It isn't running in a hospital. It runs on a laptop with one command, make demo, using example data only.

Five parts do the work:

ingest is a FastAPI service that receives HL7 v2 over HTTP. It checks the MSH header, gives the message a trace ID and passes it on.
transform parses the message and maps it to a FHIR R4 transaction Bundle.
pds-client looks the patient up in the Personal Demographics Service (PDS), the national database of NHS patient details. In the local demo it runs in stub mode by default and returns a canned record, so nothing leaves the laptop. It can call NHS England's PDS FHIR sandbox instead.
A HAPI FHIR server stores the result in Postgres.
Prometheus scrapes the services and Grafana shows the pipeline. Five alert rules cover traffic, latency, errors, dead letters and the PDS circuit breaker.

The repo also holds Terraform for AWS (VPC, EKS, RDS, S3, KMS and IAM), Helm charts and Argo CD definitions. They describe where the platform would run, and I've checked them with terraform plan. Everything in this article runs locally. Part 2 covers the AWS side.

Following one admission through

This is the sample admission from the repo's test fixtures:

MSH|^~\&|CERNER|GENERAL_HOSP|INTEROP|GENERAL_HOSP|20260101120000||ADT^A01|MSG00001|P|2.5
EVN|A01|20260101120000
PID|1||9000000009^^^NHS^NH~L12345^^^GENERAL_HOSP^MR||DOE^JOHN^A||19800101|M|||1 NHS WAY^^LONDON^^SW1A 1AA^GBR
PV1|1|I|WARD1^BED1^ROOM1||||12345^SMITH^JANE|||GEN

PID carries the patient. PID-3 holds two identifiers, separated by ~: the NHS number, with NHS as its assigning authority, and the hospital's own number. PID-5 is the name, PID-7 the date of birth and PID-8 the sex.

This is the FHIR Patient the mapper builds from that segment. I ran the mapper on the fixture to get it, so it's the real output:

{
  "resourceType": "Patient",
  "identifier": [
    { "system": "https://fhir.nhs.uk/Id/nhs-number", "value": "9000000009" },
    { "system": "https://fhir.nhs.uk/Id/local-patient-identifier", "value": "L12345" }
  ],
  "name": [{ "use": "official", "family": "DOE", "given": ["JOHN"] }],
  "gender": "male",
  "birthDate": "1980-01-01"
}

The middle initial, A, isn't mapped. PID-5 has a component for it and FHIR takes more than one entry in given, so that is a real gap rather than a limitation of either format. Part 3 is about the mapping, and it's on the list there.

PV1 becomes an Encounter, and the I in PV1-2 becomes the FHIR class code IMP, an inpatient encounter.

Three things happen next:

Enrichment. transform asks pds-client for the patient by NHS number. The PDS values for name, gender, date of birth, address and telephone replace the ones from the message, and the hospital's own identifier is kept. In the local demo the stub's canned record wins, which is why the stored patient is called Paul Smith rather than John Doe.
The write. transform sends the Patient, the Encounter and a MessageHeader to HAPI FHIR as one transaction Bundle.
The failure path. If mapping or the write fails, the raw message is saved to a dead-letter folder and a counter feeds the DLQRateHigh alert. Replaying it is still a manual step.

Three design decisions

Fail open on enrichment. If PDS is slow or down, the pipeline carries on with the demographics from the message. A circuit breaker stops calling PDS after five failures in a row, and lets one call through again after 30 seconds. A slow lookup shouldn't stop an admission being recorded.

Fail closed on writes. A failed FHIR write is never quietly dropped. The raw message is saved to the dead-letter folder so it can be replayed. More on this one below, because writing this article is how I found out it wasn't true.

One transaction per message. The FHIR specification says that in a transaction, "all interactions or operations either succeed or fail together". An admission is saved whole or not at all, never as a Patient without its Encounter.

The bug I found writing this article

I wanted a screenshot of a dead-lettered message for the section above, so I pointed transform at a FHIR server that wasn't there and sent the fixture in. Nothing arrived in the dead-letter folder, and the service answered 500 rather than 502.

The write path looked fine:

@retry(stop=stop_after_attempt(3), wait=wait_exponential(multiplier=0.2, max=2))
async def _persist_bundle(http, bundle, trace_id):
    resp = await http.post(FHIR_BASE_URL, json=bundle, ...)
    resp.raise_for_status()
    return resp.json()

and the caller catches the error the way you would expect:

try:
    result = await _persist_bundle(...)
except httpx.HTTPError as exc:
    _dlq(raw, trace_id, reason="fhir_write_error")
    raise HTTPException(status_code=502, ...)

The retry decorator is the problem. Tenacity's documentation says that when a function fails its final time, "a RetryError is raised", and that the original exception shows up "somewhere in the middle of the stack trace". RetryError is not an httpx.HTTPError, so the except never matched. Every retried write failure escaped the handler, skipped the dead-letter call and came back as an unhandled 500.

The fix is one argument. Tenacity's reraise=True raises the underlying exception instead:

@retry(stop=stop_after_attempt(3), wait=wait_exponential(multiplier=0.2, max=2), reraise=True)

The test is the more useful half. It mocks HAPI FHIR with respx, so it needs no running stack, and it asserts the three things the rule actually promises: the retries ran, the raw message is on disk, and the counter went up.

with respx.mock(assert_all_called=True) as hapi:
    route = hapi.post(FHIR_URL).mock(return_value=httpx.Response(503))
    with TestClient(main.app) as client:
        response = client.post("/transform", content=ADT_A01,
                               headers={"X-Trace-Id": "write-fails"})

assert route.call_count == 3
assert response.status_code == 502
assert _dlq_count() == before + 1

written = (tmp_path / "write-fails.hl7").read_bytes()
assert written == ADT_A01.encode("utf-8")
assert b"\n" not in written

That last pair of lines is a second lesson. My first version of the test read the file back with read_text() and failed on a message that was written correctly, because Python's universal newline handling turns the HL7 carriage returns into line feeds on the way in. Reading the bytes is the only way to see what is really on disk.

Both changes are in the code this article is tagged against.

I'd had the "fail closed on writes" rule written down in docs/architecture.md since the first week of the project. It read like a property of the system. It was a property of the documentation.

Worth stealing

Normalise line endings before parsing. HL7 v2 ends segments with a carriage return, but files and editors add line feeds. The parser turns both into carriage returns first, and the repo's .gitattributes stops Git converting the HL7 test fixtures.
Give every message a trace ID at the front door. ingest creates one and passes it on in an X-Trace-Id header, so one admission can be followed through every service's logs.
Test the failure path, not just the happy one. A retry wrapper, a circuit breaker and a dead-letter folder all look right in a diagram. The test is what tells you whether they are wired together.
Make the demo one command. make demo builds and starts the whole stack and waits until the FHIR server answers, so anyone can check the work quickly.

Where AI helps here, and where it doesn't

I built this project with Claude Code, Anthropic's coding assistant. It drafted the scaffolding for all three services, the Helm charts, the Prometheus rules and the Grafana dashboard, and the first pass of most of the docs. I made the mapping and architecture decisions, and I reviewed everything that went in.

It was quickest where the shape was clear. The Helm charts show that best: three services, seven templates each, and the only real differences between them are the name, the port, the environment variables and the resource requests. Once the first chart is right, writing the third one by hand is just typing.

It didn't replace checking, and the failure above is the clearest case. The retry decorator, the except clause and the dead-letter helper are each individually what you would write. Nothing in the code looks wrong until something runs it. Two other passes over the repo turned up the same kind of thing: two AWS statements in the docs that no longer matched the Terraform, a description of an in-memory queue that the code doesn't have, and an alert rule, PDSCircuitOpen, that watches a metric called pybreaker_state which none of the services export. That last one can't fire at all. Part 4 is about observability, and fixing it belongs there.

The rule I'm keeping: AI drafts, and the specification, the tests and the running system decide.

What I'd do differently

I'd use SQS from the start, not a synchronous HTTP hand-off. Right now, if transform is down, ingest logs the error and still replies 200, marked forwarded: false, and nothing retries the message. That is the same failure as the one above, one service earlier, and the dead-letter folder doesn't help because the message never reached the service that owns it. A queue makes the hand-off durable instead of hopeful. It's the first item on the roadmap, and Part 6 shows the fix.

Acronyms used in this article

Acronym	Stands for	What it means here
ADT	Admission, Discharge and Transfer	The family of HL7 v2 messages a hospital sends when a patient is admitted, moved or sent home
A01	Admit/Visit Notification	The ADT event for a patient being admitted
AI	Artificial intelligence	Here, the coding assistant used to draft parts of the project
API	Application programming interface	A defined way for one piece of software to ask another for data
AWS	Amazon Web Services	Amazon's cloud platform, where the project is designed to run
CD	Continuous delivery	Shipping changes automatically once they pass checks (as in Argo CD)
DLQ	Dead-letter queue	A holding place for messages that couldn't be processed, so they aren't lost. This project uses a folder
EKS	Elastic Kubernetes Service	Amazon's managed service for running containers with Kubernetes
EVN	Event Type segment	The line of an HL7 v2 message that says what happened and when
FHIR	Fast Healthcare Interoperability Resources	The modern health data standard, built on web technology
HAPI	HL7 application programming interface	An open-source project; HAPI FHIR is the FHIR server used here
HL7	Health Level Seven	The organisation behind both standards. The "Seven" refers to the top layer of a standard networking model, where applications talk to each other
HTTP	Hypertext Transfer Protocol	The protocol web browsers and web services use to talk
IAM	Identity and Access Management	AWS's system for who and what is allowed to do what
IMP	Inpatient	The FHIR code for a patient admitted to a ward
JSON	JavaScript Object Notation	A common text format for structured data
KMS	Key Management Service	AWS's service for managing encryption keys
MSH	Message Header segment	The first line of every HL7 v2 message: who sent it, to whom, and what type it is
NHS	National Health Service	The UK's public health service
PDS	Personal Demographics Service	The national database of NHS patient details, such as name, address and NHS number
PID	Patient Identification segment	The line of an HL7 v2 message that describes the patient
PV1	Patient Visit segment	The line of an HL7 v2 message that describes the hospital stay
R4	Release 4	The version of FHIR this project uses
RDS	Relational Database Service	Amazon's managed database service
REST	Representational State Transfer	A common style for building web APIs
S3	Simple Storage Service	Amazon's file storage service
SQS	Simple Queue Service	Amazon's message queue service, which holds messages until they're processed
v2, V3	Version 2, Version 3	Versions of the HL7 standard. v2 is the older, widely used one
VPC	Virtual Private Cloud	A private, isolated network inside AWS
XML	Extensible Markup Language	An older text format for structured data

Sources

Part 2, on 21 October, goes from docker compose up to Terraform on AWS.

What does the integration engine in your organisation still speak: HL7 v2, FHIR, or both?

This is a personal project. It is not affiliated with or endorsed by NHS England or any NHS organisation. It uses the public PDS FHIR sandbox and synthetic test data only. Views are my own.

Top comments (11)

Mustafa ERBAY • Oct 14

Great write-up, Nnamdi! 😄

What stood out to me wasn’t just the HL7-to-FHIR architecture, but how documenting the system helped you uncover a failure that normal testing had missed.

The Tenacity RetryError issue is a perfect example of why individually correct components don’t necessarily produce a reliable system.

One thought for the SQS migration: idempotency and duplicate handling will be just as important as durable delivery, especially when retries can replay patient admission events.

I also appreciated your honesty about the Prometheus alert that couldn’t actually fire. A control existing in configuration doesn’t mean it works in practice.

Looking forward to Part 2. Keep it up! 👏

Nnamdi Felix Ibe • Oct 15

Thank you, Mustafa! 😄

The idempotency point is the one I needed, and it's sharper than durable delivery because the queue can't solve it for me.

Half of it FHIR already answers. A transaction Bundle entry takes ifNoneExist, so the Patient can be a conditional create keyed on the NHS number: create only if no patient carrying that identifier exists. That half is easy, because NHS number is a stable natural key.

The Encounter is the hard half. I can't put it on patient identity, since two admissions for the same patient are legitimately two Encounters. So the question becomes what identifies this admission event, and the answer isn't in the FHIR layer at all.

Which is where your point bites. I went and read the SQS documentation properly before replying. FIFO deduplication runs on a five-minute minimum interval, and AWS states that if a message is resent with the same deduplication ID after that interval, SQS can't detect it. So the queue handles transport duplicates and nothing beyond them.

That leaves two different problems with the same name. Duplicate delivery is the queue redelivering something I already processed, and MSH-10, the message control ID, catches that. Duplicate event is the trust engine resending the same admission with a fresh control ID after an interface restart or a manual resend, and MSH-10 doesn't catch it at all, because at every layer it looks like a new message.

The second one is what would put two admissions on a patient record, and I don't have a settled answer yet. Probably a business key on the Encounter identifier built from NHS number, admission timestamp and the point of care in PV1. I'd want to test that against real engine behaviour before claiming it works.

And yes on the alert. PDSCircuitOpen is the same class as the dead-letter rule. It reads like a property of the system, and it's a property of the configuration. Part 4 fixes it, and this time I'm writing the test that proves it fires before I write the rule. 🙏

Mustafa ERBAY • Oct 15

That’s a great distinction, Nnamdi! 😄

You’re absolutely right that message deduplication and clinical event deduplication are two different problems.

One small thought: I’d be careful about including PV1 location in the business key, since patients can move between wards during the same admission.

If the source system provides a stable visit identifier (such as PV1-19), that might be a stronger starting point, scoped to the assigning authority.

And I really like your approach of testing against actual engine behaviour before declaring the design reliable.

Looking forward to seeing how this evolves! 👏

Nnamdi Felix Ibe • Oct 15

You're right, and the failure is worse than drift.

A transfer arrives as its own event. An A02 carries a different location for the same admission so the key would change mid-stay. The control I'd built to prevent duplicate Encounters would have manufactured one on every ward move.

PV1-19 is the better answer for a reason worth naming, because it's the thread running through everything I've been writing. A composite key built from demographics and location is me inferring an identity. PV1-19 is the sending system stating one. I've spent sixty-odd posts learning to prefer the stated thing over the derived thing, and I still reached for the derived one here.

One limitation I found while checking the spec. PV1-19 is optional in v2.5.1, CX, cardinality 0..1, so I can't assume it's populated. Some implementations carry visit identity in PID-18 Patient Account Number instead, and PV1-50 Alternate Visit ID exists as well. So the real design question isn't which field to use. It's what the pipeline does when the field it expects is empty, and whether it fails closed or quietly falls back to something weaker.

That's not answerable from the standard. It needs real engine traffic, which is the same conclusion I keep arriving at.

Thank you for this one. This is the second time in a week you've stopped me from building something that looked correct. 🙏

Mustafa ERBAY • Oct 15

That’s exactly the kind of architectural thinking I enjoy, Nnamdi! 😄

And I think you’ve reached the most important question: not just which identifier to trust, but what happens when that identifier is missing. Sometimes the safest design decision is to admit that we don’t have enough information rather than invent an identity that looks reliable. I’ve really enjoyed this exchange. We’re all learning from each other’s perspectives, and that’s what makes these discussions valuable.

Looking forward to the next part, my friend! 👏

𝐓𝐡𝐞 𝐋𝐚𝐳𝐲 𝐆𝐢𝐫𝐥 • Oct 14

Nnamdi, good luck with your new series! By the way, @xulingfeng , what about you? Which series are you planning to start?

Nnamdi Felix Ibe • Oct 14

Thank you! This has been sitting in my GitHub repository for a long time, so I thought it was time to share it with anyone interested in healthcare systems, particularly interoperability, career switchers, and healthcare professionals looking to explore the technology behind the systems they use.

I often receive questions from healthcare professionals about how these systems work behind the scenes and how AI can help improve them. I created this series specifically to answer these questions, break down the underlying concepts, and make healthcare technology and interoperability more accessible to everyone interested in the field.

I hope this will be a useful resource for anyone curious about the intersection of healthcare, technology, interoperability, and AI.