DEV Community: Vishnu Viswambharan

RAG Doesn’t Fail Loudly — It Fails Quietly

Vishnu Viswambharan — Thu, 09 Apr 2026 22:12:14 +0000

RAG doesn’t fail loudly. It fails quietly.

That, to me, is one of the more interesting things you notice after using it beyond demos.

Most of the time, the answer looks correct. But it is slightly outdated, slightly mixed, or slightly off. And that is much harder to detect than a clear hallucination.

In early experiments, RAG feels impressive for good reason. You connect a knowledge source, ask a question, and the model responds with something that appears grounded and relevant. It feels like a practical way to make AI useful with real information.

But with more use, the cracks start to show.

Where It Starts to Feel Off

You ask the same question twice with slightly different wording and get two different answers. Both sound reasonable.

Or you get an answer that feels almost right, but not enough to trust fully. Not wrong enough to reject. Not right enough to rely on.

That is a different kind of failure. Not obvious failure, but erosion of trust.

The interesting part is that retrieval is often not the thing failing.

The system does retrieve relevant information. The problem is what happens next.

In real-world knowledge systems, information is rarely clean and consistent. It evolves. Newer versions replace older ones. Different sources describe the same thing in slightly different ways. Some sources contradict others. Retrieval brings back what is relevant, but relevance alone is not enough.

Given multiple similar or conflicting sources, the model does not determine which one is current, authoritative, or outdated. It produces a coherent answer. It blends the inputs into something that sounds right.

It does not resolve conflicts. It smooths them.

That is not really a bug. It is a consequence of how the system works. LLMs do not choose the right answer in a strict sense. They generate the most plausible one from the context they are given.

Why This Happens

At a system level, the pattern makes sense.

Retrieval gives you relevance. The model gives you probabilistic synthesis. Neither one is designed to track knowledge evolution, enforce authority, or resolve contradictions.

Put together, you get relevant inputs and a plausible answer, but not necessarily a reliable one.

RAG retrieves relevance, not truth.

This is also why the retrieval method itself is not really the point. RAG is often associated with vector databases, but the limitation is broader than that. Whether the system uses semantic search, keyword search, or a hybrid approach, retrieval still surfaces relevant information. It does not decide which information is correct.

There are a few other patterns that show up once you start noticing them.

Chunking helps retrieval, but often removes relationships between ideas. Important qualifiers get separated from the main statement.

Small changes in phrasing can lead to different retrieved context and therefore different answers.

And even when information is incomplete or inconsistent, the system still tends to produce an answer instead of clearly acknowledging uncertainty.

Why Common Fixes Only Go So Far

When answers feel off, the instinct is to tune the system: improve chunking, add recency filters, tweak prompts, adjust temperature, rerank results.

Some of that absolutely improves behavior. But none of it fully solves the underlying issue.

Because this is not just a retrieval problem. It is a knowledge problem.

The system has no built-in concept of which knowledge should be trusted.

Reranking can improve which pieces get shown, but it still does not introduce understanding of authority. Prompts can make the model more disciplined by telling it to cite sources or prefer recent content, but prompts are still a soft control layer. They do not solve missing versioning, unclear ownership, or weak conflict resolution in the knowledge itself.

Teams also add timestamps, ownership metadata, curated sources, and filters. These are all useful. But they are still ways of guiding the system around a deeper limitation rather than removing it.

They improve selection. They do not introduce understanding.

What This Suggests

To me, this suggests that the next step is not just better retrieval.

It is better knowledge modeling.

RAG in its current form tends to assume that knowledge is consistent, independent, and equally valid. Real-world knowledge is none of those things. It changes. It gets replaced. It contradicts itself. It has ownership, lifecycle, and varying levels of authority.

If RAG-based systems are going to become more reliable, they need to move beyond similarity-based retrieval and probabilistic synthesis alone. They need stronger ways to represent authority, versioning, trust, and knowledge lifecycle.

In other words, not just retrieving context, but curating it.

RAG is still a meaningful step forward. It makes knowledge more accessible and usable than before. But using it in practice highlights an important gap.

Retrieving information and understanding it are not the same problem.

Right now, RAG bridges only the first.

AI Can Write Code. It Still Doesn’t Know Your Company.

Vishnu Viswambharan — Sun, 05 Apr 2026 16:59:43 +0000

The gap that shows up only in real systems

You can now ask an AI to build a feature and get a reasonably clean implementation within seconds. The structure looks right, it follows familiar patterns, and in many cases it is good enough as a starting point.

What becomes visible only when you try to use that code in a real system is something else. It doesn’t fully fit.

A familiar problem in a new form

This is not very different from hiring a strong developer and asking them to start contributing without proper onboarding. They understand the language and the frameworks, and they can solve problems. But they don’t yet understand how things are done in your system.

They don’t know which internal libraries are expected, which patterns are actually followed in practice, or which earlier decisions still shape the system. So their code is technically correct, but it needs adjustment. Not because it is wrong, but because it does not align.

AI is currently in a very similar position. It has strong general knowledge of software development, but it does not have access to the context that defines how software is actually built inside a company.

The instinct to provide more data

The natural reaction to this gap is to provide more context. Teams start connecting documentation, repositories, tickets, and internal tools, assuming that better access will lead to better alignment.

In practice, this often introduces a different problem. Internal knowledge is rarely clean or consistent. Some documents describe systems that have evolved. Some decisions were never documented. Some patterns exist in reality but not in writing. In many cases, there are multiple valid ways to do the same thing, without a clearly preferred one.

When all of this is exposed to AI, it does not automatically become clearer. It simply becomes available.

When more information reduces clarity

This is similar to what happens when a new developer is given access to all onboarding material without guidance. They now have more information, but it becomes harder to understand what actually matters.

It is not obvious what is current, what is deprecated, or what is actually followed in practice. Without that clarity, they rely on judgment and guesswork.

AI behaves in the same way. It produces answers that are informed and structured, but still slightly misaligned with the system.

Why the output still feels “off”

This is why AI-generated code often looks correct while still requiring adjustment. It follows general best practices, but may bypass internal abstractions, duplicate existing platform capabilities, or introduce patterns that are inconsistent with the rest of the codebase.

A common example is service-to-service communication. AI might suggest a standard HTTP client setup, while many companies already provide an internal client that handles retries, authentication, and observability. The generated solution works, but it bypasses important parts of the platform.

Another example shows up in event-driven systems. AI may generate a clean producer or consumer using a generic library, while the company expects a specific abstraction that enforces schema validation, tracing, or error handling. Again, nothing is technically wrong, but it does not align with how the system is designed to operate.

These are not obvious errors. They appear as small inconsistencies that accumulate over time and show up during reviews and integration.

Why tooling alone does not solve it

Most companies are currently trying to address this through tooling. Wrapper CLIs, predefined workflows, skills, and integrations with internal systems are becoming common. These approaches improve usability and reduce friction when interacting with AI.

They are useful, but they do not fully address the underlying issue.

Because the problem is not just access to internal knowledge. It is the usability and clarity of that knowledge.

If a system has multiple competing patterns, AI will reflect that. If documentation is outdated, AI will use it. If conventions are implicit, AI will not infer them reliably.

Providing more data does not resolve these issues. It often amplifies them.

The shift from data to clarity

The shift that seems necessary is not about making AI better at coding. It is about making internal systems easier to understand.

In practice, AI-generated code often needs adjustment before it can be merged, not because it is incorrect, but because it does not align with internal expectations. That gap becomes smaller in systems where defaults are clear and patterns are consistent.

Organizations that are seeing better outcomes with AI tend to have clearer defaults, more consistent patterns, and a smaller gap between what is documented and what is actually practiced.

In those environments, AI outputs require less correction. Not because the model behaves differently, but because the system itself is easier to reason about.

What this means for platform and DevEx teams

This has implications for platform and DevEx teams. The focus is no longer only on enabling developers through tools and automation. It also includes reducing ambiguity, defining clear defaults, and ensuring that internal knowledge remains usable and current.

It is less about documenting everything, and more about making the important things clear.

A second consumer of internal systems

One way to think about this is that companies now have two consumers of their internal systems.

Humans, who can ask questions, resolve ambiguity, and build context over time.

And AI, which depends on the structure and clarity of the information it is given.

Most systems today are designed for the first. Very few are ready for the second.

Where this is heading

The companies that will get more value from AI are not necessarily the ones adopting more tools or building more integrations. They are the ones that make their internal knowledge easier to consume, more consistent, and closer to how things are actually done.

At that point, AI stops behaving like a capable but untrained developer and starts producing output that fits more naturally into the system.

That difference is subtle, but it changes how useful AI becomes in day-to-day engineering work.

Disclaimer: The views and ideas expressed in this article are my own and do not represent those of my current or previous employers.

How the Scope of Software Development Work Has Changed Over Time

Vishnu Viswambharan — Sat, 28 Mar 2026 19:19:28 +0000

How agile, DevOps, cloud platforms, and AI have gradually changed the day-to-day expectations around software development work.

Over the last fifteen years, the day-to-day scope of software development work has changed quite a bit.

In many teams, the role used to be narrower and more clearly separated from adjacent functions. Developers primarily wrote code. Testers focused on testing. DBAs handled database changes and performance concerns. Sysadmins or operations teams managed servers and deployments. Product managers shaped requirements. Designers handled UX. Work moved through a chain of specialists.

That model had its own strengths and weaknesses, but the boundaries were easier to see.

This will not describe every company or team equally, but it is a pattern that has become increasingly visible across the broader software industry.

To be clear, strong developers have always cared about users, quality, and business outcomes. What seems to have changed is not the existence of that mindset, but how much day-to-day lifecycle responsibility is now often expected within development teams by default.

Across the 2010s and into the 2020s, developers in many organizations were pulled closer to testing, then closer to operations and production concerns, and more recently closer to product context and faster decision-making. The role is no longer changing only across the stack. In many cases, it is changing across the lifecycle of delivery.

That is the pattern that stands out to me.

From one side, more of the surrounding execution work has become part of day-to-day developer work through automation, cloud, CI/CD, and self-service platforms.

From the other side, developers are increasingly expected to understand user problems, work with less complete requirements, and help move ideas toward usable results, not just implementation.

So this is not only a story about full-stack development. It is also a story about the changing scope of software development work.

The earlier model: software delivery through specialist handoffs

In many teams in the early 2010s, software delivery depended heavily on handoffs between specialist roles.

A requirement moved from product to design. Design handed off to development. Development handed off to QA. Infrastructure teams handled deployment. Production support often sat elsewhere.

That did not mean good developers were indifferent to users or outcomes. It meant those concerns were spread across more roles and more steps in the delivery process than they often are today.

There were valid reasons for this setup. It allowed deep specialization and made responsibilities easier to separate. In some environments, it also reduced risk by putting more checks into the process.

At the same time, every handoff added waiting, translation loss, and coordination overhead. Even when each team did its part well, the system as a whole could still move slowly.

A lot of the role changes in software development can be understood as responses to that problem.

The first shift: testing moved closer to development

One of the clearest shifts in the mid-2010s was how testing moved earlier and closer to day-to-day development.

As agile ways of working became more common, the model of “developers finish, then QA begins” became harder to sustain. Bugs found late were expensive. Feedback loops were slower than teams wanted. Quality needed to show up earlier in the cycle.

So more quality-related work moved into development teams.

Unit tests, integration tests, test automation, and earlier acceptance discussions became a normal part of development work in many organizations. Quality was no longer treated only as a downstream activity.

That did not mean testing disappeared as a discipline. In many places, QA evolved into quality engineering, automation enablement, exploratory testing, or test strategy. In others, specialist quality roles remained important. But developers were more often expected to take on a larger share of quality responsibility as part of their daily work.

That was one clear way in which the scope of the developer role widened.

The second shift: operations moved closer to development teams

In the late 2010s and early 2020s, cloud, DevOps, CI/CD, containers, and infrastructure automation accelerated another change.

Older delivery models often relied on infrastructure teams as gatekeepers. If a team needed environments, deployment help, or configuration changes, they typically depended on another team’s queue and process.

Modern software organizations increasingly tried to reduce that waiting.

As a result, development teams in many companies moved closer to operational concerns. Not necessarily to become full operations specialists, but to understand enough to build, ship, monitor, and support software more directly. Teams were more often expected to know how their systems were deployed, how they behaved in production, and what to do when something failed.

This is where ideas like “you build it, you run it” became more common. The meaning was not that every developer had to become an SRE. It was more that software delivery and software operation were becoming more tightly connected.

A feature was less likely to be treated as finished just because code had been merged. There was more attention on whether it ran safely and reliably in production.

What actually changed: not role replacement, but thinner boundaries

There is an easy but misleading version of this story that says developers simply absorbed every other role. That is not really accurate.

DBAs did not disappear. SREs did not disappear. Product managers and designers did not disappear. In many companies, specialist depth remains essential.

What seems to have changed is something more practical: the amount of specialist intervention needed for ordinary delivery work often went down.

Platform teams created paved roads. Managed cloud services reduced raw infrastructure effort. CI/CD standardized release workflows. Design systems made UI implementation more reusable. Observability tooling became easier to adopt. Security controls moved earlier into delivery paths.

So developers did not necessarily become experts in every adjacent domain. They gained the ability to move further without as many handoffs for common cases.

That is a more useful way to understand the shift. The role widened partly because the environment around it increased individual and team leverage.

AI may be the next shift, but in a different direction

More recently, AI has started changing the shape of development work in a different way.

The earlier changes pulled developers closer to quality and operations. AI may be different because it pushes the role upward toward judgment and framing.

AI can reduce the cost of many implementation tasks: boilerplate, tests, scaffolding, migrations, documentation drafts, and first-pass code. That does not remove the developer, but it may change where human value sits.

When implementation gets cheaper, judgment tends to matter more.

The bottleneck can shift toward deciding what should be built, how it should work, what tradeoffs are acceptable, whether it is safe to operate, and whether it actually solves a user problem.

In that environment, the valuable developer is not only someone who can produce code quickly. It is also someone who can move from ambiguity to something useful, safe, and testable.

None of this assumes that AI will simply replace developers outright. It may automate more implementation work over time, including through increasingly capable agents. But even in that scenario, the important question is likely to shift toward who frames the problem, validates the result, manages tradeoffs, and integrates change safely into real systems.

That is why AI may change more than coding speed. It may change what strong development work looks like day to day.

A more product-shaped developer role

This is where the role may evolve further.

Not every company will create a new title for it, and not every team will change in the same way. But in many places, developers seem to be operating across a wider stretch of the path from problem to production.

That can include understanding more user context, working with less polished requirements, using design systems and AI tools to accelerate delivery, shipping safely, observing results, and iterating.

That does not mean product, design, or platform functions become less important. If anything, their role may become even more important as providers of context, systems, guardrails, and reusable patterns. The point is not that every developer does every role. It is that everyday development work may now sit closer to a broader part of the delivery process than before.

That is a meaningful change, even if the exact shape differs from one company to another.

The risk: broader scope without enough support

This shift also has a clear failure mode.

A broader developer scope only works when companies invest in the systems around it: strong platforms, clear role boundaries, realistic expectations, good documentation, sensible delivery paths, and continued access to specialist expertise.

Without that, a wider scope can quickly turn into overload rather than empowerment.

That is an important distinction. Observing that the scope of development work has expanded is not the same as arguing that companies should endlessly add responsibilities to developers. Those are different things.

In practice, the healthiest setups seem to be the ones where development teams can move further by default, but still have access to specialist support when depth is genuinely needed.

Where this may be heading

Looking back, the last fifteen years are probably not best described only as a shift from backend to full stack, or from specialist to generalist.

The bigger pattern may be this:

development teams were pulled closer to quality
then closer to operations and production responsibility
and now AI may be pulling them closer to judgment, framing, and product context

The role is being compressed in one sense because fewer handoffs are tolerated.

It is being expanded in another because developers are often expected to cover more distance in day-to-day delivery.

So this seems less about inventing a completely new kind of developer and more about broadening what software development work often includes.

The next few years will likely reward developers who can connect product understanding, technical execution, operational awareness, and fast learning. Not because they replace every specialist around them, but because the structure of software delivery keeps pushing more of those concerns closer together.

That feels like a meaningful change in what it means to work as a software developer today.

Disclaimer: The views and ideas expressed in this article are my own and do not represent those of my current or previous employers.