There's a strange pattern in insurance right now. Every carrier has an AI roadmap. Boards are asking for it, vendors are demoing it, and the headline numbers are genuinely staggering: underwriting timelines compressing from three days to three minutes, claims resolved up to 75% faster, fraud detection improving by more than 30%.
And yet, when you talk to the people actually building these systems, you hear the same quiet admission over and over: most of these projects stall before they ever reach production.
The instinct is to blame the model. Maybe it wasn't accurate enough. Maybe the vendor over-promised. It's almost never the model.
The 80/20 nobody puts on the slide
When an insurance AI project succeeds, roughly 20% of the effort goes into the model and 80% goes into getting data into a state where the model can use it. When a project fails, it's usually because someone budgeted those numbers backwards.
Modern models — for claims triage, risk scoring, fraud detection — are largely commoditized. You can stand one up in an afternoon. What you cannot stand up in an afternoon is a clean, unified, governed, real-time view of your policyholders, their histories, their claims, and the hundreds of external signals a good model wants to consume.
That's the part that quietly eats the timeline. And it has nothing to do with AI at all. It's data engineering.
Where insurance data actually lives (and why that's the problem)
A typical mid-size carrier's data estate looks like this:
- A policy administration system on a mainframe or Oracle/Teradata stack that predates most of the team.
- A separate claims system that was bought, not built, and speaks its own schema.
- A billing platform, a CRM, and a document store, each integrated by a different vendor in a different decade.
- A growing pile of third-party feeds — credit, telematics, IoT, weather — that everyone wants to use and no one has unified.
Each system is individually fine. Collectively, they make a simple question — "show me everything we know about this customer" — surprisingly hard to answer. And an AI model is, at heart, a very demanding version of that question.
The four failure modes
1. Fragmentation. The same policyholder exists as four records in four systems with no reliable key to stitch them together. The model never sees the whole person.
2. Quality. Missing fields, inconsistent codes, free-text where there should be structure. Underwriting models trained on this quietly mis-price risk — far more expensive than a model that obviously doesn't work.
3. Latency. Fraud detection and straight-through processing need signals at the moment of the claim or quote, not in last night's batch. If your data arrives on a 24-hour delay, your "real-time" AI is a day late by design.
4. Governance. With the NAIC Model Bulletin on AI now adopted across 23 U.S. jurisdictions and the EU AI Act classifying most insurance AI as "high-risk," you increasingly have to prove how a decision was made. You cannot retrofit that lineage onto a pipeline that wasn't built for it.
Not one of these is solved by a better algorithm.
What "fixing the foundation" actually means
- Consolidate the policy, claims, billing, and document silos into a single cloud data platform — typically a Snowflake or Databricks lakehouse.
- Migrate off the legacy core in phases, with a parallel run, so the business never goes dark during cutover.
- Build real pipelines with transformation, deduplication, and a stable customer key, so "customer 360" is a table you can query, not a slide.
- Stream the signals that need to be live — FNOL, telematics, fraud signals — alongside the batch data that doesn't.
- Wire governance in from the start: lineage, access controls, PII handling, audit trails.
Do this, and the AI part gets almost boring. The same models that were "stalling" suddenly work, because for the first time they're being fed something they can actually learn from.
The reframe that saves the project
The question on most AI roadmaps is "which model do we deploy?" The question that actually determines success is "is our data ready for any model at all?" Carriers that ask the second question first ship. Carriers that skip it spend two years discovering, expensively, that they should have asked it.
The model was never the hard part. The data was. It always was.
We're a data-engineering team that has delivered 200+ cloud migrations — including the data foundations insurance AI runs on. More at IntelliBooks.
Top comments (0)