DEV Community: Anthony Master

Event-Based Systems vs. State-Based Systems

Anthony Master — Thu, 02 Apr 2026 21:03:05 +0000

Why this distinction matters far beyond healthcare

One of the most helpful ways to understand modern software architecture is to distinguish between state-based systems and event-based systems.

This is a concept I have had to explain many times in healthcare interoperability, especially to people trying to understand why certain systems behave so differently from others. It also helps explain why technologies built around live updates and server-driven changes make so much sense once you see the architectural difference underneath them.

To make it simple, imagine two fictional healthcare platforms: ChartStone Clinical and PulseTrail Health.

ChartStone Clinical: a state-based system

ChartStone Clinical is a records-focused application. A user opens a patient chart, the server sends the current data to the client, and from that point forward the user interacts with a local representation of that state until they submit their changes back to the server.

This is how a large portion of modern applications work, whether they are SPAs, traditional server-rendered apps, thick clients, or older LAMP systems. The client receives a snapshot, works against that snapshot, and eventually sends back an updated version of state for the server to merge into the source of truth.

That approach is practical and familiar, but it comes with a number of challenges.

What happens if another user changes the same record before the first user submits their update? What happens if authorization expires mid-edit? What happens if the client loses power, restarts unexpectedly, or drops connection before its local changes are persisted? Suddenly the system has to deal with file locks, merge conflicts, local drafts, recovery mechanisms, and complicated rules about reconciling client-side changes with the authoritative data on the server.

If the system is mostly read-only, some of this complexity fades into the background. But most business software is not read-only. Most systems support active collaboration against mutable shared data. That means the moment the client owns a working copy of state, the burden of consistency becomes much heavier.

PulseTrail Health: an event-based system

Now imagine PulseTrail Health, a monitoring platform used in a critical care environment.

In this world, devices connected to patients continuously emit events: heart rate updates, oxygen changes, alarm thresholds, patient assignment changes, and other vital signals. Those events flow to a central hub, which aggregates and distributes them to the places that need them, such as bedside displays, nurse stations, and mobile monitors.

The key difference is that the system is not waiting for someone to ask for the latest snapshot. It is reacting to what is happening as it happens.

If a patient’s heart rate drops suddenly, the system cannot afford to let that information sit idle until the next poll cycle or page refresh. The event itself is what matters. The architecture is built to push meaningful changes outward immediately.

This is why event-based systems are often a better fit for highly reactive environments. They reduce the delay between change and response. They also reduce waste. Instead of constant polling across the network just to ask whether anything has changed, a push-based system can remain relatively quiet until something meaningful actually occurs.

That is a major architectural advantage, especially when timing, visibility, and coordination matter.

The difference in plain terms

A state-based system is centered on the question:

What does the record look like right now?

An event-based system is centered on the question:

What just happened, and who needs to know?

The state gives you the latest snapshot whereas events give you the flow of change. Both of these matter, most applications still need some current state view. Users need dashboards, forms, lists, reports, and screens that show the latest known truth. But when an application is built only around snapshots, it can become blind to the significance of the transitions that produced them. The events capture those transitions directly.

Why this matters beyond healthcare

Healthcare makes the distinction easy to see because the stakes are high. But the principle applies almost everywhere. Most applications are not merely storing data; they are coordinating change over time.

Orders are placed. Payments clear. Inventory moves. Messages are sent. Users sign in. Documents are edited. Approvals happen. Sensors report. Notifications fire. Background jobs complete. External systems need to be informed. Audit trails need to be preserved. Failures need to be recoverable.

A state-based model can tell you where the system ended up while an event-based model can tell you how it got there, what happened along the way, and what other parts of the system should do in response.

That is why event-based architecture is useful far beyond medical monitoring. It fits CRMs, ERPs, commerce systems, logistics platforms, financial applications, content systems, communications tools, and operational dashboards. Nearly every serious application deals with meaningful change, and meaningful change is where events shine.

Why almost every application can benefit

This does not mean every application should be purely event-sourced or that state no longer matters. It means that thinking in events often leads to better systems. Event-based architecture can improve real-time responsiveness, integration with other systems, auditability, decoupled workflows, automation, resilience, and visibility into what actually happened. In other words, event-based thinking helps software behave more like the real world it is modeling. Real businesses do not operate as a sequence of static snapshots, they actually operate as a stream of things happening.

The more an application needs to react, coordinate, notify, synchronize, or explain itself, the more value there is in treating events as first-class citizens.

Final thoughts

State still matters, but events often tell the full story. In practice, that also shapes how I think about the web stack itself. If the goal is to model systems as a flow of meaningful change rather than a sequence of disconnected snapshots, then the transport and UI model should reflect that.

That is one reason I tend to favor DataStar over HTMX for this kind of architecture. HTMX is excellent at request-driven hypermedia and remains one of the most approachable ways to build server-rendered applications with minimal client complexity. But when an application starts leaning into truly live, server-driven behavior, DataStar’s model feels more aligned with the problem itself. Its philosophy centers the backend as the source of truth and treats the UI as something continuously updated through a unified SSE stream, with only small, intentional client-side signals and event pushes where needed. That creates a cleaner mental model than treating realtime communication as an add-on and reaching for WebSockets as the default transport for everything. WebSockets certainly have their place, but in many business applications they can bring extra operational weight around scaling, connection lifecycle, message handling, and recovery after failure. Even automatic reconnect becomes more complicated once distributed infrastructure, missed messages, retries, and partial client state enter the picture. By contrast, a server-led approach built around downstream SSE and very small upstream pushes can stay simpler, quieter, and easier to reason about both architecturally and operationally.

Bringing it back to healthcare, this is also part of why many FHIR-based architectures still feel incomplete when compared to the practical event movement that made HL7-style workflows so effective in the real world. FHIR has brought major improvements in structure, readability, and interoperability, but many implementations still lean too heavily on request-response thinking, polling, or subscription models that do not yet fully embrace true event-first communication. Until more FHIR-based ecosystems grow toward genuinely event-based exchange, where systems publish and react to meaningful changes as they occur, they will continue to struggle to replace the practical strengths that older HL7 workflows delivered in high-volume operational environments.

The future is not just better resource models. It is better system-to-system communication built around events as first-class citizens.

🚨 This isn’t a hit piece. It’s a reality check for anyone building serious systems with GraphQL.

Anthony Master — Sat, 30 Aug 2025 04:38:00 +0000

Anthony Master

Aug 30 '25

When the Graph Snaps: A Hard Look at GraphQL's Pain Points

#graphql #webdev #softwareengineering #softwaredevelopment

Comments

24 min read

When the Graph Snaps: A Hard Look at GraphQL's Pain Points

Anthony Master — Sat, 30 Aug 2025 04:31:53 +0000

TL;DR – GraphQL: Beautiful Nightmare

GraphQL looks great on the surface—flexible queries, typed schemas, and the promise of fewer endpoints. But once you move past demo apps and into real-world systems, it reveals serious flaws:

Versioning is a lie – No clean way to retire old fields without breaking clients.
Relational mapping breaks down – N+1 queries everywhere unless you hand-optimize.
Pagination is inconsistent – Multi-dimensional trees, different needs at different levels, no standard pattern.
Deep queries go unchecked – Clients can crater performance without guardrails.
Filtering gets messy – Complex filters require awkward nested input types, often forcing you to restructure your whole query.
Repeated nodes + inconsistent shapes – No normalization, tons of duplication, and brittle client logic.
Backend logic is hidden – Seemingly “cheap” fields might hit expensive services or timeouts.
Federation = Fragile – Stitching systems across domains is complex, slow, and hard to secure.
Rigid structures – Can’t return associative data, groupings, or CTE-style responses without workarounds.
Schema generators trap you – You inherit someone else's assumptions and can’t escape easily.
Solutions fade – Tools disappear, hype dies, and you’re left with a brittle graph.

Been there. Done that. Got the T-shirt.

GraphQL is elegant—but only if you're ready to fight for every inch of usability and scalability. Otherwise, fixing it later is nearly impossible.

GraphQL: Hard to do right. Brutal to fix. Cataclysmic if done wrong.

It's elegant. Until it isn't.

GraphQL is a beautifully marketed idea. Its promises are alluring: flexible queries, single-endpoint APIs, tight client control, and introspective schemas. For many developers, it feels like the answer to the endless bloat and over-fetching problems of REST. And in some use cases, especially small-to-medium internal tools or greenfield apps, GraphQL can indeed shine.

But in large-scale, real-world production environments—especially those with relational data, legacy systems, multi-tenant platforms, or federated architectures—GraphQL quickly reveals its cracks. The elegance of the syntax is often a facade hiding a minefield of architectural gotchas. What seems flexible on the surface can become rigid underneath. What appears powerful often comes with steep trade-offs. And what feels like control for the front-end can be chaos for the back-end.

The challenge isn’t just doing GraphQL. The challenge is doing GraphQL right—which means anticipating edge cases, managing performance at scale, implementing deep access controls, standardizing pagination, optimizing query resolution, and maintaining schema health over time. Without careful upfront planning, what starts as a clean schema quickly turns into a brittle interface that’s hard to evolve, slow to resolve, and costly to maintain.

This isn’t a hit piece. It’s a reality check. GraphQL has a role—but it’s not the magic wand it’s often sold as. Let’s walk through the real friction points developers and architects face when they try to use GraphQL seriously—not in tutorials, but in production.

🔁 Versioning: The Invisible Wall

GraphQL claims to solve versioning by encouraging schema evolution over time. In theory, instead of creating /v1/ or /v2/ API endpoints, you simply deprecate old fields and add new ones. But in reality, this approach only works in environments with tight governance, strict client discipline, and a limited audience. Once third-party consumers or mobile clients start hitting your schema, backward compatibility becomes non-negotiable—and deprecation is just a toothless warning.

The lack of true versioning means deprecated fields must remain indefinitely unless you’re willing to break clients or force updates across all consumers. This can cause a buildup of technical debt that clutters your schema and confuses newer developers trying to understand which fields are safe to use. Over time, your "clean" graph becomes an archaeological dig site of legacy data paths.

Without clear version boundaries, teams often resort to naming hacks like email_v2 or getUserUpdated just to introduce functional improvements. These hacks defeat the elegance of GraphQL's self-documenting nature and signal the same kind of decay we see in REST APIs that lack versioning standards. Worse, when fields are duplicated instead of evolved properly, bugs re-emerge due to misunderstood behavior or partial migration.

Versioning isn't just about code—it's about contract

The reality is, versioning isn't just about code—it's about contract. GraphQL obscures the natural contract boundary that REST made explicit. If you treat your schema as eternal and unchangeable, you lose agility. If you treat it as mutable and ephemeral, you lose stability. You're trapped between the two, with no clean way to reset without massive rewrites. That’s why many GraphQL projects eventually abandon schema purity and revert to namespaced APIs—or just deal with the mess until it's too late.

🔗 Relational Data ≠ GraphQL Data

One of the most common implementation strategies for GraphQL—especially in CRUD-style applications—is to place a GraphQL API directly over a relational database. It makes sense at first glance: your data already lives in tables with relationships, and GraphQL seems like a natural way to expose those relationships in a flexible, client-friendly structure. But this surface alignment is deceptive. GraphQL’s conceptual model and the relational model may overlap in terminology, but they diverge sharply in behavior and performance.

In SQL, relationships are navigated with joins. These joins are optimized by decades of research into query planning, indexing, and cardinality estimation. But GraphQL doesn’t come with an implicit query planner or optimizer. It delegates that responsibility to your resolvers. So when a query requests a list of users and their associated comments, you may accidentally execute one query to get users and then N additional queries for their comments—known as the N+1 problem. Without proper batching (e.g., via Dataloader), GraphQL becomes a multiplicative query nightmare over otherwise efficient relational structures.

Moreover, SQL is a declarative language, while GraphQL resolvers are imperative. That means logic that would be handled in a single elegant SQL query—such as filtering users based on the count of related orders, or performing a recursive CTE to get a hierarchy—is often impossible or impractical to express through GraphQL without writing deeply custom resolver logic or pushing it into suboptimal app-layer code.

Then comes the issue of granular access control. In a relational system, you might use row-level security, views, or carefully scoped SQL to control what data is exposed to which user. But in GraphQL, those access patterns must be reimplemented manually at the resolver layer, often leading to inconsistencies or logic duplication across multiple node types. This creates both a maintenance burden and a security liability if not carefully audited.

In short, mapping GraphQL directly over a relational schema seems like a shortcut, but it often leads to performance bottlenecks, poor data modeling compromises, and a leaky abstraction between what your database can do and what your API is forced to support. A well-structured SQL schema should remain expressive and performant—but shoehorning it into a resolver-per-field model can erode all those gains and leave your team wrestling with custom workarounds that SQL could have handled in a single line.

🧨 The N+1 Problem (Yes, Again)

If there’s a single GraphQL pitfall that every developer learns the hard way, it’s the infamous N+1 problem. It’s not a theoretical issue—it’s a practical performance trap that turns what should be a handful of database calls into hundreds or thousands. And worse, it’s often invisible until your app is under real user load or running in production.

The N+1 issue typically arises when a query asks for a list of entities, along with some nested or related data. For example, a query might ask for a list of posts, and for each post, include the author’s details. Unless you’ve optimized, this will result in one query to fetch all posts, and then one additional query for each post’s author—hence N+1 queries total. If you're fetching 100 posts, that’s 101 queries. Add more nesting, and the problem compounds exponentially.

While solutions like Facebook's Dataloader or batching resolvers can help, they require discipline, architecture, and explicit implementation. There’s no magical setting to fix N+1 globally. You must design your data-fetching strategy with this in mind from the beginning, or face major rewrites later. For every nested field or related list, you have to ask: “Is this being batched?” And if it’s not, your API is on a ticking time bomb.

It’s also not just a database issue. And while graph databases claim this problem is resolved (pun intended), it will quickly reappear when you have to add custom lambda resolvers to fill in the gaps of missing functionality and business logic. The N+1 problem happens across any boundary—database, service calls, file systems, or federated endpoints. If your resolvers trigger a microservice call, an external HTTP request, or an internal caching layer, the problem scales across network latency, rate limits, and third-party bottlenecks. What looked like an elegant tree of fields can suddenly become a forest of RPC swamp.

In REST or RPC-style APIs, you have clear control over what data is returned per request. In GraphQL, the client decides—meaning you must handle every possible shape of query efficiently, or be vulnerable to accidental (or intentional) query abuse. The N+1 problem isn’t just a quirk—it’s a systemic architectural challenge, and if ignored, it will quietly consume your performance budget one nested field at a time.

📦 Pagination: Choose Your Pain

In GraphQL, pagination becomes multi-dimensional

In most APIs, pagination is straightforward: you request a list of items, specify how many you want, and maybe where to start. But in GraphQL, pagination becomes multi-dimensional—because you're not paginating a single list, you're paginating potentially many lists, across multiple levels of a query tree, each with different sizes, shapes, and expectations. And it’s here that the elegant facade starts to crack.

Say you're querying a list of 1000 users, and for each user, you want to show their 5 most recent orders. In a REST world, you’d probably design this as two endpoints or add an explicit constraint on the nested list. In GraphQL, the client has the freedom to request all 1000 users, and for each of them, all of their orders—unless you enforce limits at every resolver layer. The result? You’ve just built a pagination-unaware N+1 problem… again.

But it gets more complicated. What if one field in your query—say, friends—needs deep pagination (500+ results), while another nested field—like roles or tags—only ever has 3 or 4 items? The ideology of one page = one GraphQL query no longer plays nice. To paginate through your friends you might send the whole request again to paginate just on the friends level. And to add even more complication, if there is a nested field on friends that also needs paginated like their shared interests, now you are paginating over every friend's interests instead of just one of them. That’s fine in theory, but GraphQL doesn’t give you a simple way to handle mixed granularity pagination. You’re forced to manage independent pagination logic within multiple queries, often duplicating logic across resolvers. Worse, on the client side, merging paginated data from nested fields into a clean UI becomes maddening.

And because GraphQL has no built-in pagination behavior, every field can—and often does—implement it differently. Some use limit/offset, others use cursors, some wrap data in Relay-style edges/nodes, and some don’t paginate at all. This inconsistency is painful for consumers, who must learn not just how to paginate, but how to paginate differently depending on the field they’re querying.

Ultimately, GraphQL pagination is hard not because pagination itself is hard, but because the GraphQL model amplifies the complexity of variable nested list sizes, unbounded queries, and client-side flexibility. You're not just paginating a dataset—you're paginating the entire shape of a request tree, one list at a time. And if you skip the upfront work of pagination rules, field limits, and documentation, you'll soon find your server bogged down by bloated queries and confused consumers trying to figure out why some nested item lists return 100 items, while others silently now show none.

🧬 Deep Queries, No Limits

One of GraphQL’s most powerful features is that it allows clients to define exactly what data they need—across deeply nested relationships. But that’s also one of its most dangerous features. In REST, each endpoint is fixed—you know what you’re going to get, and how big the response will be. In GraphQL, a client can request not just one entity, but every relationship beneath it, recursively, with no built-in limit on depth or breadth. The result? A query that looks elegant in the IDE but crushes your back-end under the weight of recursion, joins, and memory usage.

Unless you explicitly control for this, it’s possible for a single GraphQL query to pull data across dozens of nested entities. A user can query all customers, their orders, each order’s items, each item’s manufacturer, every manufacturer’s shipping history, and so on—in one call. While this may sound empowering for front-end developers, it poses massive threats to stability and performance. Left unchecked, it’s a self-denial-of-service attack vector waiting to happen.

To make matters worse, GraphQL’s introspective and self-documenting nature encourages exploration. It invites users—especially curious internal teams—to try bigger and deeper queries just to "see what comes back." That’s great in dev tools like GraphiQL or Postman, but in production? Every deep query hits your resolvers, triggers back-end logic, and pulls potentially huge volumes of data across multiple domains. What should have been a few milliseconds of data access becomes a cascade of latency, memory strain, and serialization bloat.

You can try to mitigate this with query depth limits, query complexity scoring, or third-party libraries like graphql-depth-limit. But these add configuration overhead, and they often need to be fine-tuned per use case to avoid blocking legitimate queries. And if you use federation or third-party resolvers, it's even harder to know where the back-end work is happening—and how costly that query really is.

The dream of “ask for exactly what you need” turns into a nightmare when clients start asking for everything. GraphQL empowers deep queries, but without enforced limits, guardrails, or architectural policies, it’s all too easy to build a system where the most enthusiastic users are the ones causing the most performance degradation.

🌀 Repeated Children, Inconsistent Structures

One of GraphQL’s hallmark selling points is that it lets clients shape responses to match their exact needs—fetching only the fields they want, in the structure they prefer. It sounds clean, efficient, and liberating compared to bloated REST payloads. But here’s the reality: most clients need most of the data. Whether you’re building tables, forms, or dashboards, your UI will almost always require a complete view of the record—not a pick-and-choose subset.

In practice, developers end up querying every field anyway—not because they want to over-fetch, but because the UI demands it. And if they don't as soon as they show that one hidden column, then the whole query runs again with the added field. Talk about over-fetching everything. The flexibility to pick your fields ends up becoming just another layer of ceremony. What was once sold as a lean, client-customizable API becomes a fragile dance: every component defines its own ad hoc query shape, even when 90% of those queries are identical across the app. Instead of encouraging reuse and consistency, GraphQL enables a kind of fragmented chaos, where the same resource is requested in a dozen ever so slightly different structures by different consumers.

And it gets worse in recursive or parent-child relationships. Fetch a list of items with nested children, and each child might show up again under a different parent or branch. For instance, querying for a person (yourself—1 record), then your posts (example 40 posts), and then the author of each post, now you have your same person record duplicated 41 times in a single response. GraphQL doesn’t deduplicate these—every instance is returned in full. The same object could appear verbatim multiple times in a single query response. On the backend, you’re re-resolving and re-serializing redundant data. On the frontend, you’re re-rendering and reconciling inconsistent payload shapes. Caching can help somewhat to resolve this, but then you have worry about one of the hardest parts of programming—cache invalidation.

Previously, APIs offered a hardened, predictable view into your data. REST endpoints were often crafted with specific use cases in mind. A GET /users endpoint might return a flat, consistent object; a GET /users/:id/details would return more—but you knew what to expect. In GraphQL, that consistency vanishes. The structure of the data is now determined by whoever writes the query, resulting in a tangled mess of optional fields, repeated nodes, alias collisions, and partial types that defy predictability.

So instead of saving effort, GraphQL often forces developers to reinvent the schema at the point of use—adding mental overhead, duplication, and inconsistency across teams. And when it comes time to refactor? Good luck—because every query is shaped differently, there’s no unified contract to update. What started as elegance has, in many cases, devolved into a structural free-for-all.

🧭 Filtering Gets Weird

Filtering in GraphQL feels like it should be simple—after all, you're just asking for a subset of data, right? But the moment you step beyond shallow, single-field filters like status: "active" or name: "Bob", things get awkward fast. GraphQL leaves filtering entirely up to the schema designer, which sounds like freedom… until you're the one designing it. There’s no native, standardized way to do filtering, so every API ends up reinventing its own filter input types—often inconsistently across entities and domains.

Want to filter across a parent-child relationship? Like: “Give me all users who have at least one order over $100”? That’s trivial in SQL—a simple join with a WHERE EXISTS clause. In GraphQL, it turns into deeply nested input objects, custom logic in your resolvers, and often brittle abstractions that require bespoke code for each case. You might define an orders_some: { total_gt: 100 } field, or worse, expose a raw JSON filter blob and let the back-end interpret it. Either way, you’re either duplicating SQL semantics manually or leaking internal logic into your schema.

But here’s where it really falls apart: sometimes you can’t express the filter at all from your current root. You might be querying users, but the condition you need only exists three levels deep—say, users → organizations → subscriptions → status. If the schema doesn’t support filtering by those deep relationships (and most don’t), your only option is to change your root to something like subscriptions just so you can apply the filter at the top. Now your response shape is completely different. The structure your UI was expecting is gone. You’ve lost your pagination context. You’re no longer building a page of users with subscription data—you’re building a page of subscriptions with user data bolted on. You’ve inverted your entire query just to make the back-end work.

That means every edge case filter becomes not just a filter problem—it’s a query design and layout problem. You start duplicating front-end logic to rebuild the expected structure, or you fragment your UI into separate fetches to work around shape mismatch. GraphQL was supposed to unify data access. Instead, you’re contorting your queries just to satisfy back-end limitations, and losing consistency in how you build and consume data.

Combine this with the fact that AND/OR logic, range filters, fuzzy matches, and custom operators all require bespoke design, and your filter inputs balloon in complexity. You end up duplicating logic across UserFilterInput, OrderFilterInput, GroupFilterInput, etc., with no cross-schema reuse unless you manually abstract it. It’s verbose, inconsistent, and hard to test. And unless you’re building your schema on top of a query builder library (like Prisma or Hasura), you’re hand-authoring most of it anyway—and debugging it when it goes wrong.

In short, filtering in GraphQL feels like trying to replicate the expressive power of SQL… but without joins, without context, and without a clear place to start. You either contort your shape, sacrifice your response consistency, or write multiple disjointed queries to get at what should have been one clean answer.

🧱 Federation Across Disparate Systems

At first glance, GraphQL federation seems like the holy grail: multiple teams, services, or domains exposing parts of a unified graph—stitched together seamlessly so clients can query across organizational and infrastructure boundaries. You imagine pulling user data from an internal service, product data from an external vendor, and analytics from a third-party platform—all in one clean query. But the promise of federation quickly fades when it meets the messiness of real-world systems.

GraphQL becomes a choreographed dance across networks

The first challenge is latency and reliability. In a federated setup, each field might hit a different upstream source—one field might query a local Postgres database, another might resolve from a legacy SOAP API, and another might pull from a cloud service over HTTP. A single GraphQL query now becomes a choreographed dance across networks, each leg introducing its own delays, timeouts, retries, and failure modes. One slow service degrades the entire query. One bad link breaks the chain.

Then there's the issue of orchestration logic. Who handles the joins? Who decides how one service maps a key to another? GraphQL doesn't solve this for you—it simply delegates. You're responsible for writing custom resolvers that know how to resolve keys across services, cache what needs caching, and reconcile differences in naming, data types, or even fundamental models of the world. You’re not just building a unified graph—you’re building a fragile abstraction layer on top of a dozen incompatible systems.

And don’t forget access control. Each system might have different permission models, audit requirements, or PII sensitivity. When you stitch them into a single graph, you can’t rely on the consumer to “just know” which fields are safe to access. You now need cross-system authorization rules, audit logging, and policy enforcement at every edge. Otherwise, a user with access to a harmless internal entity may gain visibility into sensitive external data, simply by crafting the right GraphQL query.

Worse still, federation often doesn’t stop at your infrastructure boundary. Teams try to federate GraphQL APIs across disconnected systems—between departments, partner organizations, or even public APIs. But GraphQL wasn’t designed for distributed trust models. There’s no native support for inter-service authentication, query shaping guarantees, rate limits, or cost negotiation across boundaries. You wind up building a bespoke gateway or proxy, re-implementing REST-like patterns just to maintain safety and performance.

Even within your own org, federation comes at a steep operational cost. Schema stitching, entity ownership boundaries, resolver orchestration, shared standards—it’s all manual, and fragile without rigorous governance. And all the existing tooling (Apollo Federation, GraphQL Mesh, etc.) works well only within certain guardrails. Step outside those, and you’re left duct-taping together services that were never meant to be siblings in a unified graph.

Federation sounds like distributed elegance. In reality, it’s a tightrope walk over a pit of legacy systems, contract mismatches, and latency traps. It works—but only with discipline, buy-in, and deep architectural investment. And once you're federating across systems you don’t fully control—cloud vendors, external APIs, or legacy black boxes—it becomes nearly impossible to debug, evolve, or optimize the graph holistically.

🕳️ Hidden Back-end Design Decisions

One of GraphQL’s promises is transparency: clients can explore the schema, understand what’s available, and query exactly what they need. But that visibility is surface-level. Beneath the schema lies a complex web of back-end logic, micro-services, database calls, and third-party APIs that aren’t exposed through introspection—and that’s where the real problems start. Because in GraphQL, you never really know what you’re triggering with a query.

On the surface, a field like user.lastLogin might look harmless—but it could call a logging micro-service, query a slow analytics database, or even make an API call to a vendor’s platform. Meanwhile, a seemingly heavy field like user.permissions might be fully cached and lightning fast. There’s no way for the client to know. GraphQL treats every field as equal—but performance, stability, and cost are not. As a result, clients write queries based on what they see, but the back-end is reacting to what they don’t know they’re asking for.

This disconnect leads to a serious architectural blind spot. Query A and Query B might look identical in shape, but one causes a massive spike in compute or network calls. And because the schema itself hides this complexity, there’s no way to proactively prevent misuse. You can introduce tooling to estimate query cost or trace resolvers—but these are afterthoughts. GraphQL gives consumers all the power, with none of the context to use it responsibly.

It also causes problems for collaboration. Back-end teams can update the logic behind a resolver—switching a fast in-memory lookup to a slow database join—and suddenly, a harmless-looking query becomes a bottleneck. There’s no contract, no warning, and no separation of concerns. Every field in the graph is a tightly coupled handshake between the consumer’s expectations and the back-end's implementation, which can (and will) change over time.

To compensate, teams start layering in rules, limits, and complexity guards—query depth checks, cost analyzers, rate limiting, and even field-level access controls. But these often feel bolted on. The elegance of GraphQL’s query model erodes as you’re forced to defensively wrap it in back-end logic just to make sure clients don’t trigger something catastrophic. It’s like giving someone a beautiful interface to a rocket engine—without warning them about the fuel system, heat shielding, or launch sequence.

So while GraphQL exposes the schema, it obscures the consequences of querying that schema. And in systems where performance, cost, and reliability matter, that’s not just an inconvenience—it’s a liability.

🧰 Lack of Structural Flexibility

GraphQL is often praised for its flexibility—but that flexibility only goes one way: the client gets to shape the query, but the back-end must rigidly conform to the types, fields, and structures defined in the schema. And as soon as your data doesn’t neatly fit into GraphQL’s strict trees of objects and lists, you start running into walls that are surprisingly hard to get around.

Take a common scenario: You want to return a collection of results, but you want them keyed by a meaningful identifier—something like an object in PHP or a dictionary in Python or JavaScript. GraphQL doesn’t support that. If you try to return an associative array or a map-like object keyed by IDs, GraphQL’s schema validation will reject it unless you wrap it in a custom scalar or convert it into an array of key/value objects—adding unnecessary complexity to both the schema and the client logic. You can alias fields, but you can’t change the structure of a list to suit your data modeling needs.

You also can’t express “dynamic keys” cleanly. If your data comes in keyed by dynamic values—like locales, timestamps, user IDs, or anything non-static—you’re forced to hack around it with custom types, nested lists, or pre-transformed responses. The end result is awkward and repetitive. Instead of letting the structure adapt to your data, you’re stuck bending your data to fit the rigid schema. And once you do that, you’ve already sacrificed both readability and usability for the sake of staying schema-compliant.

It becomes especially painful when trying to aggregate or group results. In SQL, it’s trivial to group by a field and return a structured result—say, a map of users grouped by role. In GraphQL, there’s no native mechanism to express that structure. You have to define new custom types and fields for every aggregation you want to support. What would’ve been a one-line SQL CTE now requires five schema types, a specialized resolver, and extra client logic to unroll the array into a usable keyed structure.

And again, this isn’t just a back-end annoyance—it bleeds into the client. Developers often expect the shape of the data to match their component needs. But since GraphQL only deals in nested objects and flat arrays, they frequently have to reshape the response manually, writing post-processing logic just to turn lists into dictionaries, flatten hierarchies, or collapse duplicates. You’re duplicating work that a database or ORM would have done for you—except now you’re doing it on the client, every time.

In short: GraphQL pretends to be flexible, but only if you stay within its object/list worldview. The moment your data has even a hint of structure that deviates from that model—associative maps, grouped records, conditional structures—you’re fighting the graph rather than flowing with it.

🧮 The Limitations of Deep Filters and Joins

One of the most frustrating aspects of GraphQL—especially for anyone coming from a strong relational or SQL background—is how limited it is when it comes to deep filtering and expressive data queries. In SQL, it’s common to chain filters, perform conditional joins, and use Common Table Expressions (CTEs) to build powerful, readable queries across complex relationships. In GraphQL? Good luck.

Want to get all users who belong to an organization that has at least one paid invoice in the last 30 days and who have never logged in? That’s a single, elegant SQL query with a few joins and conditions. In GraphQL, it becomes a nested monstrosity of filter input types, conditional wrappers, and deeply coupled resolvers. Worse, if your schema wasn't designed to allow filtering across those relationships (and most aren't by default), you can't even express the query—you’d need to restructure your schema or start from a different root entirely.

This is where the illusion of GraphQL’s power starts to break. You can ask for anything—yes—but you can’t necessarily filter or join on what matters. Relationships in GraphQL are typically resolved one level at a time. There’s no built-in concept of a “join”—you fake it with nested resolvers. And every time you nest, you lose the ability to apply filters or constraints to the overall result set. This leads to a situation where what you need can’t be described from the root you’re on… so you change roots, reshape your query, and break your UI expectations in the process.

Even in systems where complex filtering is supported via custom resolver logic or tools like Prisma or Hasura, that expressiveness is usually limited to one entity at a time. Want to join conditions across siblings or cousins in the graph? You're out of luck. You either write an entirely new API entry point for that special case, or you stitch together partial responses in the client. You’re not querying a true graph of data—you’re querying a tree of fragments and hoping you can merge them later.

This architectural mismatch forces back-end developers to rebuild SQL-like semantics from scratch—AND exposes front-end developers to the limitations of those partial abstractions. Filtering by deep relationships, chaining conditions, or expressing negative logic (e.g., “users who don’t have X”) is possible… but painful. And as your data model grows more complex, the friction grows with it.

So while GraphQL claims to be “the graph of your data,” it’s more like a projection of a graph—flattened, rigid, and unwilling to let you dig deeper than one resolver at a time. The deeper and more expressive your queries need to be, the more it becomes clear: you’re not designing the graph—you’re wrestling with it.

🔒 Trapped by Schema Generators

Many teams adopt GraphQL through auto-generators—tools that build a GraphQL schema automatically from an ORM, a database, or a set of models. It feels efficient at first: you get a full-featured API in minutes, complete with types, inputs, queries, and mutations. But what starts as a time-saver often ends in a rigid trap, because you’re now operating entirely within the assumptions of someone else’s system.

When a tool generates your schema, it decides what relationships look like, how filters behave, how mutations are structured, and what types are exposed. You inherit their data modeling philosophy, which might be close to what you want—but rarely perfect. Want to filter nested children using an advanced condition? You might find it’s unsupported. Need to paginate a list that wasn’t built with pagination in mind? Too bad. Want to restructure a return type for better performance or UI consistency? Not without writing a ton of overrides or custom resolvers.

The more sophisticated your use case becomes, the more you find yourself fighting the tool, not just extending it. And because the generated schema is often tightly coupled to the internal data model, even small changes—like adding a calculated field or excluding a sensitive column—require deep hacks or break downstream contracts. You lose control over the structure and behavior of your graph, which defeats the entire purpose of adopting GraphQL in the first place.

Eventually, your only options are to fork the schema, layer on wrappers, or start building custom resolvers beside the generated ones—none of which play nicely together. Worse, if the tool’s underlying assumptions don’t align with your front-end's needs, you may be forced to write multiple queries or compose data client-side, just to reshape what should’ve been a single efficient query. The dream of clean data orchestration becomes a sprawl of fragmented queries and brittle workarounds.

The tragedy here is that the tool was supposed to help you move faster. But instead, it put you in a box—and that box is made of someone else's opinions, someone else’s edge cases, and someone else's constraints. You didn’t build a custom graph API. You inherited a prefabricated lattice, and now you’re stuck bolting on your actual use cases with duct tape.

🧨 Looks Great Until It’s Real

GraphQL is undeniably beautiful on paper. It looks like the elegant middle ground between REST and RPC—flexible, introspective, strongly typed, and customizable. For early-stage apps, internal tools, or prototyping environments, it can even feel magical. You build your schema, light up the playground, and queries just work. But once you scale beyond trivial use cases, the cracks start to show. And if you’re not careful, those cracks become chasms.

The biggest issue is the false sense of completeness

The biggest issue isn’t even the technical limitations—it’s the false sense of completeness. GraphQL solves the surface problems. Over-fetching, under-fetching, rigid endpoints? Fixed. But behind that are deeper concerns: inconsistent pagination, broken filtering, leaky abstractions, performance bottlenecks, deeply nested N+1 bombs, unpredictable back-end behavior, rigid schema structures, federation chaos, and client-side cartwheels just to shape the data how you actually need it.

And here's the kicker—many of the tools that promised to fill those gaps? They're either abandoned, deprecated, or lost momentum after the hype wore off. Schema stitching tools gave way to federation frameworks, which gave way to server-less middle layers, which gave way to cloud providers who offered managed GraphQL back-ends with just enough functionality to impress... until you hit a wall and realized you couldn’t fix what they abstracted away. They sold solutions for problems GraphQL created, and many of those tools left behind unmaintainable complexity when the maintainers moved on.

Security? Also an afterthought. Without rate limits, complexity scoring, or access guards on every resolver, you’re one creative query away from exposing too much, doing too much, or costing too much. The shape of a query hides the danger it contains. By the time you realize it, you're chasing down 15 nested fields, 5 unbounded lists, and 12 micro-service calls—all from a single GraphQL request.

Been there. Done that. Got the T-shirt. Literally. I presented at Dgraph Day 2021 thinking our journey from SQL to GraphQL was almost complete with Dgraph's generative GraphQL API. I hyped myself on the promise, before most of this surfaced. And while I still believe GraphQL has its place, I now know this: if you’re not prepared to do it right from the start, don’t do it at all. Fixing it later is almost impossible—because the cost of change rises exponentially with adoption. Once clients depend on your schema, once tooling gets built around it, once your data model calcifies into your graph... every limitation becomes a liability.

So yes, GraphQL is elegant. But in real-world systems with messy data, competing priorities, multiple consumers, federated ownership, and deep access needs, elegance fades fast. What’s left is a schema-shaped prison—easy to enter, hard to escape, and nearly impossible to remodel without tearing down the walls.

Use it wisely. Or don’t.

SOLVING PROBLEMS THAT WE CREATED

Anthony Master — Sun, 16 Mar 2025 06:33:41 +0000

TL;DR; Rethinking Web Dev...

Everybody has tools that they use, and some tools they acquired to fix problems that someone else caused. I remember when I was the Service Manager for a GM dealership. Every month GM would send us new "special tools." These tools would eventually migrate from the corner of my desk to the "tool room." This tool room was not the place where all mechanics placed the tools they bought with their own money, but these were company tools. It didn't contain any "common" tools as you might call them for the average mechanic like a socket set, pullers, pliers, etc. No, this room contained the sockets, pullers, pliers and tools that didn't even resemble a typical tool that were specially designed for a very specific application. Then would come in the cars of that year and the mechanic would look up the repair guide and it would mention special tool GM-####, in order to complete the repair, the technician would need to locate the special tool to diagnose or repair the vehicle. More than once we would have the parts necessary to repair the vehicle, but could not for lacking of a special tool and we had to order or borrow it. If you are interested, the special tool website is actually public web: gmglobaltools.com. What I want to call out though is that many of these tools only existed because the engineers designed something that could only be diagnosed or repaired with a specially designed tool. I want to use this story to step into a deeper subject of web development, and our use of "special tools".

I've been doing research recently into some new, yet old, tech. If you want to peek into my world and get deep into web development (there is a reason why I call it "development" and not "design"), then check out a few of the following:

NATS.io
data-star.dev
protobuf.dev
htmx.org
go.dev
html
css
js

I say new, because some of these are really new like D* (data-star) isn't even out of beta v1 yet (really close though), but I say really old, because it takes roots back to the fundamental of how the web works, like html that was used on the first website (info.cern.ch) in 1993.

I have been "burned" so many times lately, well in my whole IT career of 18+ years, because I have chosen elements in my tech stacks because they were popular, maintained, recommended, or just to sum it up, the fad. Many times it even looks like the solved the problem I was facing, when sometimes years later I resolved that they only solved a problem I created and yielded to creating more problems.

It's the same truth, "if you don't learn from the past you are doomed to repeat it." It makes me look at what we have been doing with the recent fads of Web Dev, and makes me ask the question...

ARE WE SOLVING PROBLEMS THAT WE CREATED?

Ok, so I use the "we" very generally to include all the master minds of web development and not just the few of us that might ever read this post.

While doing some research I was watching a few tech talks while washing the dishes tonight and since my hands were wet the player continued to a video I had not anticipated watching: "Building the Hundred-Year Web Service with html" by Alexander Petros [https://youtu.be/lASLZ9TgXyc?si=2QA7NT_50-wtprEw]

Alex's opening introduction really caught my attention because I studied bridge architecture in high school in my first research project (thank you Mrs. Debbie Taylor. Readers please don't blame her for my bad grammar and run on sentences, she tried harder than I listened, haha!) Anyways, the intro brought to light some poor decision patterns I held as I try to be a web developer that builds something that lasts for generations. I usually think about problems as they exist today without taking into consideration what the problem might look like in 100 years. Now if I do solve the problem of today and do it well enough, it very well may stand the test of time. However, if I think fundamentally at what problem I am trying to solve and if that problem will exist and how it might exist in 100 years then I might solve it differently.

I use to do quite a bit of what I call "traditional web work" and that involves things like basic template driven website development. These usually always end up with the same problems and similar solutions but they don't usually include much in terms of future proofing. "How easily can a different template or theme be changed out?" while that is usually considered "easy" when you actually attempt it at a deeper level, you quickly find all the oddities you did to "fix" the old theme that now have to be refactored to work for the new theme or template. And that doesn't even include staying up to date with the changing SaaS solutions that most web sites deeply depend upon whether that be WordPress, WIX, Laravel, React, or whatever. I look back now and see much of this work (while being beneficial and producing results) to have been done in a way that caused more problems in the future than what it solved for the current state.

I look at a different project IFBMT, and I see that from a deeper level I approached some things wrong, but on a higher level I did some things well and that leads to its seeming success. The attempt to recevelop IFBMT into MissionBase with a modern framework and a state-of-the-art database platform should have worked by comparison of the fads of the trendy web. But that is just the problem, we set out to fix problems that I created, and while fixing those problems we only created additional problems up to the point that we had to make the hard decision to abandon the 2.0 project that visually was a success, but was a failure to launch. I have attempted a few times since to restart a new version of IFBMT, but each time seem to run into similar problems. The success of IFBMT was not just because it solved a niche problem that was unresolved and closed the three-way gap between donor management systems, contact relationship management systems, and independent baptists, but because it did it in such a way that it has been practically hands off running for the most part since 2018. Compared to other web apps, IFBMT (v1) is close to end of life. I say that not because it is planning to shut down any time soon, just because LTS usually ends for a version after 5 years, with most version 1 being replaced in only 2 years. So what is it that has helped it stand up so long with so relatively little maintenance and support? (a standing rhetorical question, I am continuing to answer - thoughts on using very LTS versioned tech stack?)

I now venture down a different brain pathway of where my professional career has taken me and I found another diamond (howbeit dull) in the rough with HL7 v2. It amazes me that HL7 v2 is almost the same age as I am, released in 1989! Literally the reasoning for my job is because hospitals and 3rd party applications are still utilizing a technology standard that is widely supported and hailed as the superior (arguably, it is not) mechanism for exchanging healthcare data. HL7 is a strange animal because there have been attempts and even whole new architectures released and pushed to replace v2, yet it persists forward nonetheless. Version 3 was not widely adopted due to it's complexity, lack of backwards compatibility with the widely used v2, and the steep learning curve associated with its implementation. The latest version of HL7 (commonly referred to as FHIR) is not even meant as a replacement for HL7 v2, but rather as a way to compliment it and offer a more modern approach to healthcare data exchange using RESTful web services. Just some interesting thoughts here...

REST is seen all across web dev today, but I wonder how many actually know what problems it solved and how well it still does at solving that problem. This takes us back from healthcare data exchange back into Web Dev. REST stands for REpresentational State Transfer, and is a software architectural style first defined in 2000 (a quarter century ago now). in my honest opinion, modern frameworks don't rely on REST as much as they should. The standard's purpose was to establish a standard for how servers could communicate and exchange data, leading to a simpler and more flexible approach to API design. REST fundamentally solves many of the SOLID principals and approach to development if the rest (no pun intended) would just be done correctly as well. REST if coupled with the correct architecture has been proven to solve scalability issues as well that has plagued many a startup (mine included). Now let's rethink the FE of the FullStack web developer...

Web Dev is divided into two lands, the Front End (FE) development, and the Back End (BE) development. The FE is the end user's device and what they see, do, and control. The BE is the server (think organization's controlled computer providing the web page.) What modern reactive frameworks have mostly done is pulled most of the load off from the BE and offload it onto the FE with tools like React. The naming of React with Reactive development is almost as amusing to me as the term Relate in correspondence to Relational Databases, but that is a tangent for a different post. While reactive websites handle asynchronous data streams and event-driven scenarios, they usually fault in the part of making the FE do all of the logic to make that happen. Let the FE do best what the FE does, and let the BE do best what the BE does. That is a novel concept, but one that we (web developers) keep trying to break and misconstrue. And why do we keep trying to break this concept, because we are attempting to solve problems that we created. The fads say to go "server-less" for scalability, but is that really the best approach? Maybe for static web content, but why attempt to completely absolve something that isn't the answer to every problem. And by using this solution, we simply create a world of others.

If you are a Web Dev, or anyone still reading this post: first, thank you; secondly, we probably disagree on many technical things and that is ok, but more importantly; thirdly, can I challenge you to rethink how you develop for the web? Why do you use the web stack you are using? How long can your project stand without being touched and it still solve the problem. Will the problem you are solving be a long-lasting problem (what might it look like in 100 years)? What would you do if you had to drastically simplify your project to just the bare necessities to solve the problem?

More thoughts later, but it is getting late for tonight/this morning.

[graphic from churchcanvas.ai which I HIGHLY recommend!]

Since When Did APIs Become Databases?

Anthony Master — Tue, 16 Jan 2024 05:15:00 +0000

While researching a database to use in a project, the team I am working with came upon yet another graph database. Every time I turn around there is a new graph/NoSQL database. How can anyone possibly keep up with these is beyond me, but I try. (Insert shameless plug for my GraphDev Discord server).

Maybe we should make our own database, lol.

Interestingly while theorizing testing and implementation to even begin comparing performance, we decided that we might take an un-traditional approach and fork the open source database written over yet another triple-store, and insert our business logic directly into the database. This was after I jokingly said that maybe we should make our own database.

But we don't want to make a database, we only need an API layer for our application. We already have an API being used in production. We just need to replace the database this API is using to store our data.

So, we went back to discussing how this new database handles API connection and end-user authentication. The API needs a connection to the database to be able to authenticate users, but an authenticated user needs to be able to use an individual connection through the API to the database. But why? One of the deciding factors in using this database is its advanced auditing which is directly attached to the client. We don't need to just audit at our API database connection level but at the end-user/client level.

So what if we instead send additional context through our API of the end-user/client making the mutation and query audit trails based upon this additional information? That might work, but would require again forking the database to change how the audit function runs. If we are then going to fork the database already, then why not just move the other logic from our API into the database too? But would that mean we have built (or forked and extended) a database and no longer have an API?

What is it exactly that distinguishes databases from APIs? According to ChatGPT:

The term "database" generally refers to a structured collection of data that is organized and can be easily accessed, managed, and updated. Qualifications for a project to be termed a database often include the ability to store, retrieve, and manipulate data in a systematic way.

If your project uses a datastore underneath and meets the criteria of a structured collection of data with organized access and management capabilities, you can likely refer to it as a database. However, it's important to consider specific features and functionalities associated with traditional databases, such as support for querying, indexing, transaction management, and data integrity.

Wait, is our API already classified as a database? Our API contains a structured collection of data with:

✅ Organized access and
✅ Management Capabilities.

Our API also has support on some levels for:

✅ Querying,
✅ Indexing,
✅ Transaction Management, and
✅ Data Integrity

So, as any intelligent person would do, I asked ChatGPT for help again, "Can an API layer be called a database?"

No, an API layer typically cannot be called a database. An API (Application Programming Interface) is a set of rules and protocols that allows different software applications to communicate with each other. It facilitates the exchange of data and functionality between different systems.

On the other hand, a database is a structured collection of data that is organized for easy retrieval, update, and management. It provides a mechanism for storing, querying, and manipulating data. While an API can be used to interact with a database, the API itself is not a database.

Umm, so then if this is correct, an API is only a set of rules and protocols to communicate between the layers. Some of these rules entail layer 7 authorization presumably. Reading into this a little deeper, the API exchanges data, while the database stores, queries, and manipulates the data. Ok, then by what right, is the database we are using hold its claim that it is a "database"? The same could be said of most of the "databases" released over the last decade.

The API exchanges data, while the database stores, queries, and manipulates the data.

This was enough circular reasoning to hurt my brain, so after I finished working for the evening and signed off, I checked LinkedIn and for sure they overheard our conversation because there it was again...

...FHIR servers can act like "databases", even though they are not... While normal databases had tools like Database Visualizers, FHIR had nothing but Postman. So Darren built the Vanya FHIR utility... Are FHIR servers acting more and more like databases? Is there a need to build tooling similar to what we have with SQL databases, but for the FHIR API?

So I must ask, and hence the reason for this short article,

When did APIs become databases?

Some added context on what is FHIR: for my day job, I am the Interface Specialist for a Hospital. Interoperability in a Hospital setting is connecting independent systems with application programming interfaces (APIs). The most widely adopted interfacing format is HL7 version 2 the precursor to FHIR, aka HL7 version 4. For a very brief explanation besides the change in message formats, version 2 is 80's technology with TCP event emitters while v4 is trying to bring healthcare to the somewhat closer present with RESTful APIs and even a proposal for a GraphQL API. This is shaking up healthcare interoperability by providing means to query health data when you need it from remote systems instead of filtering, storing, and aggregating data received that may never be needed. And already before it is even widely implemented, there is this conversation comparing an FHIR API to that of an FHIR "database". 🤦‍♂️

How to Apply SOLID with Testing JS/TS Class Methods

Anthony Master — Sat, 13 Jan 2024 07:55:08 +0000

Are your class test files growing out of control? Are you struggling with writing tests for class methods because there are too many variables to control? I was in the same situation recently and found a way out using SOLID principles.

// ...
export const MyClass implements IMyClass {
  constructor(
    @multiInject(IDENTIFIERS.FOO)
    protected foos: IFoo[]
    @inject(IDENTIFIERS.BAR) protected bar: IBar
  ) {}
  someMethod = (): Baz | undefined => {
    const baz: Baz | undefined = undefined;
    for (const foo of this.foos) {
      if ("baz" in foo) {
        baz = foo.baz();
      }
    }
    return baz;
  };
  // ...
}

Take a class for which we are tasked to write a unit test. This class may have a dozen methods and a dozen more attributes. In my environment we were already using inversify to dependency inject into this class, and using container snapshot and restore as setup and teardown operators, in our jest test file. But it began getting out of control even after refactoring into test cases and test runners.


container.bind<IBar>(IDENTIFIERS.BAR).to(BarMock);
container.bind<IMyClass>(IDENTIFIER.MY_CLASS).to(MyClass);

beforeEach(() => {
  container.snapshot();
})
afterEach(() => {
  container.restore();
});

const someMethodTests: {
  description: string;
  foo: IFoo[];
  expected: Baz | undefined
}[] = [
  {
    description: 'without foo',
    foo: [],
    expected: undefined,
  },
  // ...
];

describe('MyClass', () => {
  // ...
  describe('someMethod', () => {
    it.each(someMethodTests)('$description returns: $expected', ({ foos, expected }) => {
      foos.forEach(foo => container.bind(IDENTIFIERS.FOO).to(foo));
      const myClass = container.get(IDENTIFIER.MY_CLASS);
      expect(myClass.someMethod()).toEqual(expected)
    });
  });
});

Our unit test files became more complex than the actual classes we were testing. The comment was even made that we need tests for our test files. As funny as that was, it was partially correct because of the complexity of the unit test files.

we need tests for our test files

While researching to find a better method to jest test JavaScript classes, I came upon How to test classes with jest which provided insight into using spies with jest.spyOn methods. While this did provide a little bit of help, it just added again even more complexity, and as we were already using Inversify dependency injection spies offered little to no help.

While learning from Uncle Bob's design principles, I feel like I have strengthened myself in writing quality function tests. I wasn't however applying the SOLID principle one layer deeper into our class methods. What would it look like if we:

Singled the responsibility of the method into an external function?
Converted methods into Open-closed functions?
Made it possible to later Liskov substitute class methods?
Applied Interface segregation?
And took another step in Dependency inversion?

I took a step back and looked at another project I had been working on that also used classes and jest. I had written a parsing method that became so convoluted I had little choice but to separate it into an external function. My unit tests then covered this function separately from the rest of the class. That was it! I could separate class methods into external functions and thereby easily do unit testing on the external functions irrelevant to the class as a whole.

// types.ts
export type SomeMethod = (foos: IFoo) => Baz | undefined;

// someMethod.ts
import { SomeMethod } from './types';

export const someMethod: SomeMethod = (foos) => {
  const baz: Baz | undefined = undefined;
  for (const foo of foos) {
    if ("baz" in foo) {
      baz = foo.baz();
    }
  }
  return baz;
};

// MyClass.ts
import { SomeMethod } from './types';
import { someMethod } from './someMethod';

export const MyClass implements IMyClass {
  constructor(
    @multiInject(IDENTIFIERS.FOO)
    protected foos: IFoo[]
    @inject(IDENTIFIERS.BAR) protected bar: IBar
  ) {}
  someMethod: (() => SomeMethod) = () => someMethod(this.foos);
  // ...
}

The unit tests for the method then become much easier to write as well.

// someMethod.spec.ts
import { someMethod } from './someMethod';

const someMethodTests: {
  description: string;
  foo: IFoo[];
  expected: Baz | undefined
}[] = [
  {
    description: 'without foo',
    foo: [],
    expected: undefined,
  },
  // ...
];

describe('someMethod', () => {
  it.each(someMethodTests)(
    '$description returns: $expected',
    ({ foos, expected }) => {
      expect(someMethod(foos)).toEqual(expected)
    }
  );
});

We end up with:

A unit test that has the Single responsibility of testing a single function.
A class that is still Open for extension, while the function is closed for modification.
A function that can follow the Liskov substitution principle by using an argument referencing the property of the class without knowing the class itself.
Another implementation of the class does not have to use the same function if not needed, and can still make their implementation of the Interface.
Also the class is Dependent upon the type of the function and not the inline function itself. This allows for easier replacements when a better function may be found later that follows the same type or similar abstraction of such.

I would love to hear your feedback on this SOLID principle application and learn how you might better jest test classes in your JavaScript/TypeScript application. (If you have feedback on my horrible example code, yes I know... I wrote almost all of it without the help of an IDE to cross-check/correct my work).

What is HL7

Anthony Master — Sun, 21 May 2023 02:30:39 +0000

I posted recently on LinkedIn starting a topic on how to enable integration in your healthcare application using HL7 and becoming an HL7 interoperability expert. The basis of using HL7 is first understanding exactly what HL7 is and how it works. By the end of this post, you will be able to look at an HL7 v2 message and be able to parse it without any fancy encoding tools.

If you are interested in my referenced LinkedIn post, you can find it here:

Anthony Master on LinkedIn: So you have an healthcare application, and are trying to integrate with…

So you have an healthcare application, and are trying to integrate with healthcare facility data. You could request data extract files and reports from the…

linkedin.com

Most explanations of HL7 start with the big picture and then dive into the smaller pieces. I like to work backward when I explain new technologies. This is the same way I put puzzles together. I will study a small piece individually to figure out where it goes in the bigger picture.

If you can understand the smallest pieces, then you will be able to compile them into the big picture.

HL7 v.2 is comprised of 5 concepts:

Subcomponents
Components
Fields
Segments
Repetitions

By the way, if you want to read the official standards, you can find them at: www.hl7.org

Each of these concepts is associated with an encoding character. You can override these encoding characters, but for now, we'll keep with the defaults.

Subcomponents are separated by the ampersand symbol &
Components are separated by the caret symbol ^
Fields are separated by the pipe symbol |
Segments are separated by the return sequence \r seen in most text editors as a line break
Repeating Fields are separated by the tilde symbol ~

Subcomponents

The subcomponent is the smallest data concept of an HL7 message and in its most basic format is simply a string. An example might be something like Oklahoma. There is nothing in HL7 that signifies the difference of a subcomponent out of context as anything other than a string. The location of the subcomponent within the message is what assigns it a more specific data type. For example, given the subcomponent value of 20230527, we have no idea by itself what it is other than a string. We might want to assume it is a date, but without knowing its position in the message, it can only be known to be a string. If we assume it was a date, we might later be proven incorrect when it was a unit of measurement. Later in this post, we will discuss how to identify data types given the position, or path, to the data.

Given the default subcomponent separator, we can join multiple subcomponents together which might look like OK&Oklahoma. Indexing in HL7 usually uses a 1-based index. This means that the first item in the separated list of items is assigned the path 1, the second item 2, etc. So given this list of subcomponents OK&Oklahoma we can separate them as the list:

OK
Oklahoma

We might assume that this is a list of a US state abbreviation in position 1, and a state name in position 2, but we don't know at this point exactly what this data represents.

Components

Components are simply defined as groups of subcomponents. You have already seen two subcomponents, without really knowing it. 20230527 and OK^Oklahoma are both components. The first one, is only a single subcomponent, while the second one is a list of two subcomponents.

With the default component separator of the caret (^), we can start to join together subcomponent groups into lists of components:

T123456789^^^OK^DL^^20230527^20280531^OK&Oklohoma^DMV

Components follow similar indexing patterns as subcomponents using a 1-based index scale. We can represent this list of components above into the following structure:

T123456789
OK
DL
20230527
20280531
OK&Oklohoma
1. OK
2. Oklahoma
DMV

Notice how positions 2, 3, and 6 are empty. With just being empty, we cannot be confident whether the values were empty, or if the message sender just didn't include values in those positions. To signify that the value is indeed empty and not just not included, the syntax is to use an empty quoted string (""), which could look something like:

...^""^^...

With this, we know that position two was included and that it was indeed an empty value.

Let's start putting together paths for the component in the ordered list above. We can combine a component and subcomponent path with either a dash (-) or a period (.) like 9-2 or 9.2. So 9.2 equals the data Oklahoma. And the data OK is found in two paths, 9.1 and 4. When a component only has a single subcomponent, then it is common practice to leave off the subcomponent path. In our example though, both 4.1 and 4 would reference the same value OK. We can also reference things that don't exist in our component such as 9.3 or 12. If something is referenced but does not exist in the component, then depending upon the handling of your environment, it may be returned as either null, undefined, an empty string (""), or even a reference error.

Note: You will find other resources mix and match these separators of a dash - and a period . interchangeably. It is my practice to use a period . when writing the path between components and subcomponents.

Fields

You might have already come to the logical conclusion, that a field, is the combination of multiple component groups, and if so, then you are correct. So to say it similarly again, you have already seen a field, even though you didn't know it. The component list above forms a field.

Knowing from above that the field separating character is the pipe symbol (|), you might have already figured out what a field set would look like. But to make it clear, here is an example:

STF|amaster507^^L|T123456789^^^OK^DL^^20230527^20280531^OK&Oklohoma^DMV|

Now let me throw a curveball at you, this example, is a list of 3 fields. You might be wondering why that is a curveball. And that is for a few reasons. First of all, when a list of items ends with a separator, it is declaring an additional item in the list. Sort of like how some programming languages for an array, set, or map. If an array was presented as [1, 2, 3, ] we would say that there are 4 elements in the array, the 4th element was undefined. Because the comma was given, we knew that there was something else there, even though it was not defined. Seeing then that this list of fields ends with the field separator |, you might think that this list has 4 fields. But that is not correct either, because a list of fields together form the next higher concept of a segment, and Segments can be considered 0-based indexed. See Segments below for additional explanation. For now, just know that if we reference field 2, component 9, and subcomponent 2, such as 2.9.2 then we would have the value Oklahoma. This is because the 2nd field starts with the value T123.... If we put together a list of this data so far, we get:

amaster507^^L
1. amaster507
3. L
T123456789^^^OK^DL^^20230527^20280531^OK&Oklohoma^DMV
1. T123456789
4. OK
5. DL
7. 20230527
8. 20280531
9. OK&Oklohoma
10. OK
11. Oklahoma
12. DMV

Segments

A Segment is a named list of fields. We hinted earlier that a segment can be considered 0-based indexed meaning that the first element's index path is 0, which we said we would explain later, so here we are now. While you cannot usually reference field 0 in a segment, you do reference a segment by name directly. Given our field example, which we will show again below, you can see that this segment has the name STF. The name of a segment must be 3 uppercase alphanumeric characters.

STF|amaster507^^L|T123456789^^^OK^DL^^20230527^20280531^OK&Oklohoma^DMV|

Building upon our path to data, we can put together the segment name STF combined with our field-component-subcomponent path 2.9.2 and get STF-2.9.2.

Note: Again, just like with path joins between fields, components, and subcomponents, the separator can be either a dash (-) or a period (.). I find it common practice to use a dash after the segment name, and then a period for the other path separators.

The name of the segment begins to give us context to the data we have. With just a single segment, and assuming a few things such as the message version, and default encoding characters, we can derive data types.

Note: Because no one is expected to remember everything, I still use a reference to look up data types often. My go-to reference of choice is Caristix.com.

If we look up the segment STF we will get the "Staff Identification" from the "Personnel Management" from chapter 15 of the HL7 v2.5 standard. We furthermore can see that field 2 is the "Staff Identifier List" conforming to data type "CX". If we continue to look up the datatype CX (aka "Extended Composite ID with Check Digit") we will find position 9 to be the "Assigning Jurisdiction" conforming to datatype CWE. I know it seems tedious, but keep going down the trail, and you will find that data type CWE (aka "Coded with Exceptions") 2nd position is the "Text" and conforms to just the ST (aka "String") data type. So this means that STF-2.9.2 is the Staff Identification > Staff Identifier List > Assigning Jurisdiction > Text.

You might think that doesn't make much sense, but let's decode one more path, STF-2.5. This will lead you to table 0203 which tells you the value DL maps to the Driver's License Number. Now for security's sake, I would not post my actual driver's license number, but as you now can put together, my driver's license was assigned under the jurisdiction of Oklahoma.

IMPORTANT: Since Segment names must be 3 characters followed by a field separating character, it can be determined that field 1 of a segment begins with the 5th character. However, there is one exception to the rule! The MSH (aka "Message Header") segment defines the encoding characters with fields MSH-1 and MSH-2. And MSH-1 (aka "Field Separator") is the literal first field separator found at character 4 of the MSH segment.

Note: As you peruse HL7 dictionary resource specifications, you will find a thing called "OPTIONALITY". Some fields are always Required, but many others are Optional. When you start using HL7 with vendors, you will find vendor specifications. These specifications usually can override the standard specification in terms of what they require and allow it to be optional. This can help alleviate situations with seemingly overlapping data. For instance, STF-2.4 and STF-2.9 might be overlapping data depending on the exact use case specifics, and one or even both of these fields might be completely omitted if the vendor does not find it necessary data to send and/or receive.

Repetitions

In HL7 v2.x, fields and segments can repeat depending upon the allowable datatype. In Caristix, the repeatability of fields are denoted with - meaning no repeats allowed, a number indicating how many repetitions are allowed, or the infinity symbol ∞ indicating unlimited repetitions are permitted. Segments are repeated without any special repeating indicators. For example, the NK1 (Next of Kin/Associated Parties) segment, can usually depending upon the vendor, allow for repeating segments. This could be exemplified to show both my wife and my mother:

NK1|1|Master^Amanda|SPO
NK1|2|Master^^^^Mrs|MTH

Depending upon your environment and configuration, you may now get an error when trying to reference my "Next of Kin"s family name with NK1-2.1. This would be because there is ambiguity as to which NK1 segment you are requesting. To clear this up, you would add an iterator index to the repeating segment's name. The syntax and indexing base vary by your environment, but it is commonly 1-based indexed and syntax with the iteration surrounded by brackets. So to reference my first next of kin's relationship, we would use STF[1]-3 which finds us SPO (aka "Spouse").

Field repetitions take a slightly different form. When a field repeats, each iteration is separated with the field separating encoding character (MSH.2 2nd character) which is by default the tilde (~). Let's show an example and then explain it:

STF|amaster507|T123456789|MASTER^ANTHONY^LEE^^^^L^A~^TONY^^^^^N~MASTER^ANTHONY^^^BRO^BATh^REL|C^CANDIDATE|M

Looking at STF-3 (aka "Staff Name") you will see a repetition character followed by another field, and then once again. This shows that there are three names given for this person. Looking a little deeper at STF-3[*].7 you will find the values L, N, and REL. Using Table "0200" with version v2.7, you can map these to the Official Registry Name, Nickname, and Religous name types. You were also just introduced to a new syntax, the field follows by brackets ([]) wrapping an iteration index. From my experience, field iterations are always 1-based indexed. If we know that the second name is the nickname, then we can extract my nickname with STF-3[2].2 which would be TONY.

Note: Most of the time, repeating fields are not typed in the same order. Meaning that given two separate messages with different staff, one could have an STF-3[2] referencing a nickname, while another is either missing that iteration altogether or it is a completely different name type such as maiden name. It is important then to check for contextual indicators such as the STF-3[*].7 to ensure the correct field is being referenced.

Messages

An HL7 message is the composition of one or more segments. The first segment in a message will be the MSH (ak "Message Header") segment.

This wraps up the main concepts of the syntax of an HL7 v2.x message. Before wrapping up this article though, I want to briefly cover three more important factoids and a fourth for your own, personal research:

Escaping Encoding Characters
Message Trigger Events
Message Versions
Message Batching—Handling a batch of HL7 messages in a single stream or file.

Escaping Encoding Characters.

You might be wondering, what if one of the special encoding characters is used inside of a textual data element? For instance if "Little Bobby Tables" shows up in your message?

To cover these scenarios, there is another encoding character called the "Escape Character". The default escape character is the backslash (\) which can undoubtedly be cumbersome to work with in some editors and environments. This encoding character can be customized using the MSH-2 field's 3rd character. With this escape character, you can now safely include "Little Bobby Tables" and all other correctly escaped free text input into your messages as well.

Message Trigger Events

Without getting into all of the specifics and differences between HL7 v2.x and FHIR (v4), I will at least introduce the idea that v2 varies from FHIR in that v2 uses the concept of a messaging event system whereas FHIR uses the equivalence of a RESTful API. If you are not familiar with the terms "messaging event system" and "RESTful API", I highly encourage some additional research into those terms.

In HL7 2.x it is common for "Interfaces" to contain similarly typed messages. A common interface is the ADT interface that transmits and/or receives messages concerning Admits, Discharges, and Transfers of patients within a healthcare organization. Each message indicates its code in field MSH-9.1. And then another layer into these coded interface types are the trigger events. A trigger event describes what event happened that triggered the message to send. For a quick example, here are just 4 of the many ADT trigger events:

A01: Admit/visit notification
A02: Transfer a patient
A03: Discharge/end visit
A08: Update patient information

Message Versions

The message version can be found for each message in the MSH-12 field. Depending upon this message version, data types, optionality, repeatability, tables, and dictionaries will vary. It is common practice to conform an interface to a specific message version. Some vendors may mix versions depending upon supported specifications within each version. Consult your vendor's HL7 specification guides for more details regarding which versions are supported within which interfaces/events.

Summary

Thank you for taking the time for me to explain to you the main concepts and fundamentals of understanding HL7 v2.x messages. You should now be able to put your skills to use. I'd love to hear your feedback and continue the discussions.