I've spent the better part of the last decade watching the travel technology landscape undergo one of its most significant transformations. The shift from legacy GDS systems to IATA's New Distribution Capability (NDC) isn't just another industry buzzword—it's a fundamental reimagining of how airline content flows through the ecosystem. And for those of us building the data infrastructure that powers modern travel platforms, it represents both an enormous challenge and an unprecedented opportunity.
When I first encountered NDC specifications in earnest around 2015, I'll admit I was sceptical. The travel industry has seen countless "revolutionary" standards come and go. But NDC is different. It's not trying to replace the existing infrastructure overnight; it's creating an entirely new paradigm that forces us to rethink how we architect data systems, model complex offers, and maintain consistency across increasingly fragmented distribution channels.
Understanding the Offer-Order Paradigm Shift
The most fundamental change NDC introduces is the move from a shopping-centric model to an offer-order lifecycle. In the legacy world, we dealt with relatively static fare structures. A passenger information record (PNR) was king, and data engineers could rely on predictable schemas that had evolved over decades. Flight segments, fare basis codes, booking classes—these were well-understood entities with established relationships. And that matters.
NDC turns this on its head. An offer is now a dynamic, contextual entity that may only be valid for minutes or even seconds. It's personalised, it's perishable, and it's fundamentally different from the static fare displays we've built careers around. I've had to completely rethink data retention strategies because storing every offer variant for analytics purposes can generate terabytes of data in days for a medium-sized online travel agency.
The offer-order model means that what a traveller sees at search time (the offer) and what gets converted into a booking (the order) are now explicitly separate concerns with their own lifecycle management requirements. From a data engineering perspective, this demands event-driven architectures that can capture state transitions, maintain offer-to-order lineage, and reconstruct the customer journey across multiple touchpoints.
I've found that tools like Apache Kafka become essential in this new world. The ability to stream offer events, maintain ordered logs of state changes, and replay sequences for debugging or analytics isn't optional—it's foundational. Traditional batch processing approaches simply can't keep pace with the real-time nature of modern offer management.
API-First Architecture and the Death of Screen Scraping
One of the most refreshing aspects of NDC, from my perspective, is its API-first design philosophy. For years, travel technology has been plagued by screen scraping, format parsing, and brittle integrations built on protocols that predate the internet. EDIFACT messages, cryptic terminal commands, and fixed-width file formats were the norm. I've personally maintained parsers for booking confirmations that arrived as poorly formatted email attachments.
NDC brings us into the modern era with RESTful and SOAP-based APIs that use XML schemas with clear versioning and documentation. While the specifications themselves are verbose—sometimes maddeningly so—they represent a massive improvement over what came before. The shift to structured API responses means we can build more robust data pipelines with better error handling and clearer contract testing, however the API-first approach also introduces new complexities. NDC isn't a single API; it's a collection of message pairs covering shopping, order creation, order management, and service delivery. Each airline implements these messages differently, often supporting only a subset of the full specification. I've worked with airline APIs that claim NDC compliance but deviate quite significantly in their interpretation of optional fields and error handling.
This fragmentation means data engineers can't simply build one integration layer and call it done. We need sophisticated adapter patterns, extensive mapping logic, and comprehensive monitoring to handle the reality of multi-supplier NDC implementations. Tools like Postman and OpenAPI specifications become invaluable for documentation, but the real work happens in building flexible transformation layers that can accommodate airline-specific quirks while presenting a consistent interface to downstream systems.
Schema Design for Dynamic Content
The schema design challenges posed by NDC are unlike anything I've encountered in traditional travel data engineering. In the GDS world, we could get away with relatively normalised relational schemas. A flight was a flight, a fare was a fare, and the relationships were predictable.
NDC offers are composite entities that bundle flights, ancillaries, seat selections, baggage allowances, and dynamic pricing into a single, often hierarchical structure. The same flight might be offered at different prices with different restrictions depending on who's searching, when they're searching, and what other services they're bundling. Capturing this complexity in a way that supports both operational needs and analytical queries requires careful thought.
I've gravitated toward hybrid approaches that combine the strengths of different database paradigms. Document stores like MongoDB work well for preserving the full fidelity of NDC offer responses, maintaining the nested structure without forcing artificial normalisation. But for querying and analytics, I typically transform these documents into more structured formats—often using tools like dbt to build transformation pipelines that extract key dimensions and metrics into a star schema suitable for business intelligence tools.
The key insight I've developed is that with NDC, you often need multiple representations of the same data optimised for different use cases. The raw offer document serves as the system of record. Flattened views support operational queries. Aggregated metrics feed dashboards. And machine learning features require yet another transformation pipeline. This multi-model approach adds complexity, but it's the only way I've found to balance flexibility with performance.
Real-Time Pricing and Inventory Management
One of NDC's most significant implications for data engineering is the shift toward real-time pricing and inventory management. In the legacy world, fare rules and availability were relatively static. Systems could cache aggressively, and batch updates overnight were often sufficient. NDC's dynamic offers invalidate these assumptions.
I've had to rethink caching strategies entirely. Traditional time-based cache expiration doesn't work when offer validity is measured in minutes and pricing can change based on real-time demand signals. Event-driven cache invalidation becomes essential, but implementing it correctly requires careful coordination between offer retrieval, order creation, and inventory update systems.
The challenge is compounded when you're aggregating content from multiple airlines, each with their own NDC implementation and pricing logic. Ensuring consistency in search results when different suppliers have different latency profiles and cache invalidation semantics is a genuine technical challenge. I've found that probabilistic data structures like Bloom filters can help with quick validity checks, but there's no substitute for well-designed fallback strategies and clear timeout policies.
Stream processing frameworks like Apache Flink have proven valuable here. The ability to process offer events in real-time, maintain stateful computations for pricing analytics, and trigger downstream actions based on complex event patterns aligns well with NDC's operational model. But the learning curve is steep, and the operational overhead of running these systems reliably shouldn't be underestimated.
Data Quality and Reconciliation Challenges
Here's something I wish more people talked about: NDC makes data quality and reconciliation significantly harder. In the GDS world, a PNR was the single source of truth. If there was a discrepancy, you could retrieve the PNR and resolve it. NDC's distributed, offer-centric model means there's often no single authoritative record.
An offer exists in the airline's system, but it might be cached in multiple intermediaries. When it converts to an order, the order details live in the airline's order management system, but copies exist in the booking platform, the payment processor, and potentially multiple middleware layers. Keeping all these representations consistent, especially when changes occur, requires sophisticated reconciliation logic.
I've implemented reconciliation pipelines that regularly compare order states across systems, flagging discrepancies for investigation. These pipelines need to account for timing differences, eventual consistency, and legitimate variations in how different systems represent the same information. Data lineage tracking becomes crucial—tools like Apache Atlas or even custom metadata management solutions help maintain visibility into how data flows and transforms across the ecosystem.
The testing burden also increases dramatically. With legacy systems, you could often get away with a relatively small set of test cases covering common booking scenarios. NDC's flexibility means the combinatorial explosion of possible offer configurations, ancillary combinations, and order modifications is enormous. Property-based testing approaches, where you generate random valid inputs and verify invariants hold, have become part of my standard toolkit.
The Skills Gap and What It Means for Career Development
I'll be honest: the shift to NDC has exposed a significant skills gap in the travel technology sector. Many data engineers came up through the ranks working with legacy systems, and the transition to modern API-driven architectures, event streaming, and real-time data processing requires a substantial mindset shift.
Why does this matter? Because the alternative is worse. The engineers who will thrive in the NDC era are those who combine deep domain knowledge of travel distribution with modern data engineering skills. Understanding what a fare basis code is matters, but so does knowing how to design event-driven architectures, implement circuit breakers for API resilience, and build machine learning pipelines for dynamic pricing optimisation.
I've found that investing time in understanding the business context—why airlines want to move to NDC, what they're trying to achieve with dynamic offers, how ancillary revenue optimisation works—makes me a far more effective data engineer. The technical skills are table stakes, but the ability to translate business requirements into scalable data architectures is what creates real value.
For those looking to build expertise in this space, I'd recommend hands-on experience with the actual NDC specifications. IATA provides extensive documentation, and many airlines now offer sandbox environments for testing. Building a simple proof-of-concept integration, even if it's just for learning purposes, will teach you more than any amount of theoretical study.
Looking Ahead: What I Believe the Future Holds
As I look at where NDC is heading, I see both consolidation and continued fragmentation. The standard itself will mature—version 23.3 is already significantly more refined than the early releases—but airline implementations will remain diverse. This means the data engineering challenge isn't going away; it's evolving.
My view is that the winners in this space will be those who build flexible, composable data platforms that can adapt to airline-specific variations without requiring complete rewrites. Microservices architectures, where each airline integration is its own bounded context with clear interfaces, seem like the natural fit. But the operational complexity of managing dozens of microservices shouldn't be underestimated.
I believe we'll also see the emergence of specialised middleware layers that abstract away some of the NDC complexity. Companies are already building offer aggregation platforms and order management systems designed specifically for the NDC world. For data engineers, this means less time spent on integration plumbing and more time on higher-value activities like pricing optimisation, fraud detection, and personalisation.
The most exciting opportunity, from my perspective, is the richness of data that NDC enables. With dynamic offers and detailed order lifecycle tracking, we have visibility into customer behaviour and pricing elasticity that was simply impossible in the legacy world. The data engineers who can harness this information to drive business value—through better forecasting, smarter pricing algorithms, or more effective personalisation—will find themselves in high demand.
Ultimately, NDC represents a once-in-a-generation opportunity to rebuild travel technology infrastructure on modern foundations. It's challenging, sometimes frustrating, and definitely not a quick fix. But for those of us who love the intersection of complex domain problems and cutting-edge data engineering, it's exactly the kind of challenge that makes this work endlessly fascinating.
About Martin Tuncaydin
Martin Tuncaydin is an AI and Data executive in the travel industry, with deep expertise spanning machine learning, data engineering, and the application of emerging AI technologies across travel platforms. Follow Martin Tuncaydin for more insights on travel-tech, ndc.
Top comments (0)