DEV Community: Iris Zarecki

The real bottleneck in enterprise GenAI isn’t the model - it’s the data architecture

Iris Zarecki — Wed, 04 Mar 2026 13:58:56 +0000

GenAI adoption is accelerating — but production deployments remain difficult.

A recent survey of 300 senior IT and data leaders shows why: the biggest obstacle isn’t the model.

It’s enterprise data architecture.

Let’s break down what the research revealed — and what it means for building GenAI systems on AWS.

The Shift: From Experiments to Production

Enterprise GenAI is moving out of experimentation.

Survey results show:

38% of companies plan production deployments in 2026
7% plan to scale existing GenAI use cases
Only 18% remain in exploration

Two years ago, only 2% had production deployments.

But as companies move toward production, the biggest issues appear.

The Top Obstacles Are Data-Related

Four of the top five deployment challenges are related to data, including:

Guardrails for responsible AI (76%)
Lack of skills (66%)
Enterprise data readiness (62%)
LLM reliability (52%)

Cost is no longer the primary concern.

The real issue is that GenAI stresses enterprise data architecture in ways BI systems never did.

The Current Enterprise GenAI Stack

The survey shows the most widely used technologies in GenAI deployments today are:

RAG — 85%
Prompt engineering — 72%
Vector databases — 64%

On AWS, a typical RAG pipeline looks like this:

This works well for unstructured data.

The survey found that 89% of organizations use unstructured data (documents, knowledge bases, support articles) for GenAI use cases.

But this only solves half the enterprise problem.

The Hard Problem: Operational Data

Enterprises still rely heavily on operational systems:

CRM
ERP
financial systems
billing platforms

The survey shows 66% of organizations use operational data in GenAI initiatives.

Unlike documents, operational data requires:

real-time access
cross-system joins
strict governance
privacy controls

This is where most GenAI architectures struggle.

What a Real Enterprise GenAI Architecture Looks Like

Production GenAI systems typically require multiple data layers, not just a vector DB.

Example AWS architecture:

This architecture combines:

vector search
real-time operational data
governed enterprise data

The Core Data Challenges

The survey identified the biggest concerns around enterprise data for GenAI:

Data quality and consistency — 59%
Fragmented data across systems — 50%
Security and privacy — 50%
Real-time access — 33%

None of these problems are solved by better prompts or bigger models.

They require better data architecture.

The Real Lesson

The biggest insight from the research is simple:

Enterprise GenAI success depends less on models and more on data architecture.

Models are improving rapidly.

But enterprise data is still:

fragmented
slow
hard to govern

Until that changes, many GenAI projects will remain stuck between:

PoC → Pilot → Production failure

📊 The full findings are available here

Why your GenAI pilot succeeds, but your production deployment fails - and what to do about it

Iris Zarecki — Tue, 20 Jan 2026 16:45:30 +0000

If you’ve built a GenAI pilot that works, you’re already ahead of most teams.

The demo answers questions correctly.
Latency feels acceptable.
The model seems “smart enough.”

Then you try to ship it to production - and everything starts to fall apart.

Responses slow down.
Answers become inconsistent or flat-out wrong.
Costs spike.
Security and governance suddenly matter.
And the trust you had during the pilot evaporates.

This isn’t because your LLM suddenly got worse.

It’s because operational GenAI changes the rules of how data is accessed, combined, and governed - and most production architectures weren’t built for that.

Pilots succeed because they cheat (and that’s okay)

Most GenAI pilots are intentionally constrained:

A small, static dataset
Pre-curated documents
One or two well-defined use cases
Friendly users
No real SLA
Minimal security constraints

In other words, pilots operate in a controlled environment.

You can:

Precompute embeddings
Cache aggressively
Manually clean the data
Ignore edge cases
Accept partial or stale answers

And that’s fine — pilots are supposed to prove possibility, not viability.

The problem starts when teams assume that scaling a GenAI pilot is just an infrastructure problem.

Production GenAI breaks because the data assumptions change

In production, GenAI systems behave very differently from traditional applications.

1. You no longer control the query shape

In a classic app, you know:

Which API is called
Which tables are accessed
Which joins are executed

With GenAI, the user prompt defines the query.

A single question can implicitly require:

Multiple domains
Multiple entities
Real-time and historical data
Structured + unstructured sources

Context that wasn’t anticipated at design time

Your data architecture now has to handle unpredictable access patterns - something most data platforms were never designed for.

Data freshness becomes non-negotiable

In a pilot, yesterday’s data is often “good enough.”

In production:

“Why did my order get delayed?”
“What’s the current status of this customer?”
“Is this user eligible right now?”

Stale context doesn’t just degrade quality — it destroys trust.

Batch pipelines, nightly syncs, and pre-materialized views simply can’t keep up with conversational, real-time inference.

Latency compounds fast

A GenAI request is rarely a single call.

It’s usually:

Retrieve context
Enrich with entity data
Apply business logic
Inject governance rules
Call the model
Post-process the result

Each additional hop adds latency.

Architectures that rely on:

Fan-out API calls
Remote joins
Warehouse queries per request

quickly exceed acceptable response times - especially under load.

What felt “fine” in a pilot becomes unusable at scale.

4. Governance can’t be bolted on later

In pilots, governance is often manual or ignored:

Everyone sees everything
No fine-grained access control
No auditability
No data residency constraints

In production, that’s a non-starter.

GenAI systems must:

Enforce row-level and entity-level access
Mask or exclude sensitive attributes dynamically
Adapt responses based on who is asking, not just what they’re asking Traditional data governance models assume static queries. GenAI produces dynamic, context-driven queries.

That mismatch is one of the biggest reasons GenAI apps fail security reviews.

The uncomfortable truth

Here it is:

If your GenAI app fails in production, it’s probably exposing architectural debt that already existed.

Teams that succeed don’t “optimize the pilot.”
They change how data is delivered.

Instead of asking:

“Where does this data live?”

They ask:

“What defines this customer, order, or user - right now?”

GenAI works best when context is built around business entities, not tables or files.

The takeaway

GenAI pilots succeed because the world is simple.

Production GenAI fails when systems can’t handle:

Unpredictable questions
Real-time context
Entity-level governance
Low-latency composition at scale

This isn’t an LLM problem.
It’s not even an AI problem.

It’s a data architecture problem — and GenAI is just the first workload that makes it impossible to ignore.

If you want GenAI to work in production, don’t start by swapping models.

Start by fixing how your systems deliver context, trust, and timeliness - because that’s what GenAI actually runs on.

Enterprise data is at the crux of the biggest challenges for GenAI deployment

Iris Zarecki — Mon, 11 Nov 2024 07:48:50 +0000

A recent survey from K2view, the generative data product company, examines the top challenges enterprises are facing as they work toward generative AI (GenAI) implementation. The report unveils the most significant roadblocks to realizing GenAI’s full potential lie in organizations’ existing data infrastructure, particularly in the areas of data accessibility and latency, data privacy, and security.

The responses came from 300 senior professionals who are directly involved in the planning, building, or delivery of GenAI applications.

Let's drill in to understand the details.

Assessing GenAI adoption in enterprises

GenAI adoption is accelerating as enterprises recognize its transformative potential. Organizations are strategically moving from experimentation to pilot projects, focusing on areas like marketing, sales, and customer operations where GenAI can deliver substantial benefits. Techniques like Retrieval Augmented Generation (RAG) are becoming essential tools for customizing AI models to meet specific business requirements.

The RAG revolution

While generic Large Language Models (LLMs) offer impressive capabilities, they often fall short of meeting specific business needs without customization. The survey shows that only 14% of organizations utilize off-the-shelf LLMs in their AI projects. The vast majority – 86% – are enhancing their LLMs using techniques like Retrieval Augmented Generation (RAG), fine-tuning, and embedding.

RAG is rapidly gaining traction as a leading method for enhancing GenAI models with specific organizational data, with 60% of respondents currently piloting RAG. Industry-specific adoption rates provide further insights.

These figures demonstrate strong momentum in adopting RAG, particularly in sectors where data privacy and precise responses are critical. However, the limited number of organizations that have transitioned to full production reflects the complexities involved in scaling these advanced technologies effectively.

Data is critical to realizing GenAI’s full potential

Enterprise data presents one of the biggest challenges for GenAI deployment, and organizations are still struggling with it as they move their GenAI projects to production.

48% of respondents cite data security and privacy concerns, and 33% cite enterprise data readiness as roadblocks to deployment. The difficulty lies in the fragmented nature of enterprise data, which is often spread across multiple systems and analytical data stores, making it hard to integrate, govern, and make accessible in near-real time, under stringent guardrails, to GenAI applications.

Growing focus on leveraging the technology for customer operations

According to our survey, Marketing and sales (63%) followed by Customer operations (54%) are the primary areas where companies plan to implement generative AI solutions in the next 12 months.

While marketing and sales have some of the strongest and most mature use cases for GenAI, there’s a growing focus on leveraging the technology for customer operations across various industries.

Customer service departments across industries are now deploying GenAI-powered solutions that transform traditional support models. These implementations are yielding measurable improvements in key performance metrics – from faster resolution times to enhanced personalization of customer interactions.

Download your copy here

As cloud complexity continues to escalate, visibility becomes a top challenge

Iris Zarecki — Wed, 28 Feb 2024 18:42:27 +0000

A survey of over 350 cloud users conducted by Firefly, a provider of a platform for automating cloud provisioning, found that as cloud complexity continues to escalate, the use of Infrastructure-as-Code (IaC) has become a de facto standard; as it is boosted by AI and GenAI for policy creation and remediation advice.
Let's drill in to understand the drivers, common approaches,
trends, and outcomes revealed in this survey.

Setting some common ground
We noticed some common trends and challenges that companies are seeing in their use of multi-cloud and IaC.

Multi-cloud remains the norm with a growing number of accounts

Companies face common issues in their use of IaC:

Manual labor
Shortage in qualified talent
Configuration drifts
Fragmented tools

Average respondent is using many more IaC frameworks this year than last year. 57% use 2 or more frameworks, the ones using 5 or 6 different IaC frameworks is much higher than last year.

Nearly 60% of the current Terraform users surveyed plan to abandon it. This is no surprise with 56% of respondents noting that the Terraform license change was disruptive to them.

How much is codified as IaC?
The majority of respondents have already made the shift to codifying their clouds, with more than 64% of respondents noting that they have codified more than half of their cloud assets.

When it comes to codifying SaaS application configurations like Okta, DataDog, and the like, more than ¾ of respondents are managing or planning to manage SaaS configurations using IaC.

This continued trend to manage everything as code provides greater security, disaster recovery, and more powerful automation capabilities

As cloud complexity escalates, visibility becomes a critical challenge
Security displaced governance/change management this year as a top challenge while visibility/multi-cloud remains a key concern. It makes sense that you can’t manage what you can’t see and manage.

40% of respondents said they cannot detect configuration drift at all, yet this is a marked improvement over last year’s 61%.
40% are spending days to weeks to remediate the drift. This includes everything from misconfigurations that can put systems at risk, to patching application software. While drift detection has improved year-over-year, remediation has not. At least 20% do not detect drift at all, and 13% do not fix it.

Progress remains difficult but AI holds promise
As cloud complexity continues to escalate, challenges of visibility and governance will continue to compound.

Platforms that can help cloud operations manage across cloud providers and across multiple IaC frameworks are becoming mission critical in order to provide visibility and consistency for organizations embracing cloud-native infrastructure.

AI can be used to manage your multi-cloud infrastructure - from code generation to policy enforcement, anomaly detection, remediation and even CloudOps. For example, Firedly's customers are using our built-in Policy as Code generator to write custom policies like asset tagging and more.

Download your copy here