DEV Community: Siddharth Pandey

Why “More Context” Still Doesn’t Fix Infrastructure

Siddharth Pandey — Fri, 15 May 2026 17:11:03 +0000

A common reaction to AI hallucinating infrastructure is:

“Just give it more context.”

More code.
More docs.
More Terraform.
More logs.
More everything.

It sounds reasonable.

But it quietly assumes something important:

infrastructure understanding is a “volume problem”.

It is not.

More Context Doesn’t Fix Wrong Structure

Let’s say you give an AI agent access to:

Terraform
CloudFormation
CDK
schema definitions
deployment configs
API code
docs

It can now “see more”.

But infrastructure problems are rarely about missing visibility.

They are about missing relationships.

For example:

Which service consumes this queue?
Which Lambda depends on this table?
Which schema version is actually deployed?
Which environment is stale?
Which index is required by this access pattern?

These are not text retrieval problems.

These are graph problems.

RAG Works on Similarity, Not Truth

RAG retrieves things that are:

semantically similar
textually relevant
contextually close

But infrastructure requires:

exact state
exact relationships
exact deployment reality

Similarity is not enough when:

a resource exists in dev but not prod
a schema changed but code didn’t
an index was removed but code still assumes it exists
two services are loosely coupled but heavily dependent

The system becomes inconsistent very quickly.

And AI happily reasons over that inconsistency as if it is unified truth.

The Real Issue: No Deterministic Infrastructure Graph

What AI actually needs is not just more text.

It needs a deterministic representation of the system itself:

services
dependencies
data flows
infrastructure topology
runtime relationships

In simple terms:
a graph where nodes are real infrastructure resources and edges are verified relationships between them — not assumptions inferred from code similarity.

Without that, the model is still guessing.

Even if it has access to your entire repository.

Because repos do not fully represent production systems.

Production systems are behavior.

This Is Where Things Start Breaking Quietly

Most infra failures don’t come from obvious mistakes.

They come from:

outdated assumptions
invisible dependencies
forgotten services
implicit coupling
undocumented flows

AI systems are especially vulnerable here because they treat code as the system.

But production systems are not code.

They are behavior.

This Is the Gap I Keep Thinking About

This is the gap that pushed me toward building Infrawise.

The goal is not just:

“give AI more context.”

It is to make infrastructure relationships structurally understandable for AI systems.

Things like:

real schema relationships
actual index existence
service dependencies
infrastructure topology

Derived from infrastructure state and system analysis — not guessed purely from source code patterns.

Closing

A lot of current AI tooling assumes that if the model sees enough files, eventually it will understand the system.

I don’t think that is true.

Because infrastructure problems are fundamentally relationship problems.

Not autocomplete problems.

Not retrieval problems.

And definitely not “just increase the context window” problems.

Current AI coding assistants are already very good at understanding source code.

The next generation will need to understand:

infrastructure topology
operational relationships
deployment reality
system behavior

Because repos do not fully represent production systems.

Production systems are behavior.

AI Generated a DynamoDB Query That Could Never Work

Siddharth Pandey — Wed, 13 May 2026 06:34:02 +0000

I asked an AI coding assistant to generate a DynamoDB query.

A few seconds later it confidently gave me this:

const result = await dynamo.query({
  TableName: "Orders",
  IndexName: "customerId-createdAt-index",
  KeyConditionExpression: "customerId = :customerId",
});

Looks clean.

Looks professional.

Looks like something a senior backend engineer would casually approve during code review while fighting for survival in their 14th Slack thread of the day.

Tiny issue though.

The index did not exist.

And honestly, this is becoming one of the biggest problems with AI coding tools:
they are extremely good at generating things that look correct.

The AI Wasn't Being Stupid

That’s the interesting part.

The assistant actually made a pretty reasonable guess.

It saw:

DynamoDB usage
query patterns
naming conventions
nearby code

…and predicted what probably existed.

Which is exactly what LLMs are designed to do.

The problem is:
production infrastructure is not based on probability.

Either the GSI exists or it does not.

AWS is unfortunately not very emotionally supportive about this distinction.

Infrastructure Is Not Just "More Context"

A lot of AI tooling conversations eventually turn into:

“We just need better RAG.”

Or:

“We just need larger context windows.”

That helps.

But I think there’s a deeper issue here.

Because infrastructure is not just information.

It’s relationships.

The assistant might read:

Terraform
schema files
docs
migrations
configs

…and still not reliably understand:

which indexes are actually deployed
which environments differ
which access patterns are safe
which infrastructure assumptions are outdated

That’s not just a retrieval problem.

That’s a system understanding problem.

DynamoDB Makes This Very Obvious

DynamoDB is actually a great example here because it is brutally honest about access patterns.

With SQL databases, developers sometimes get away with questionable decisions for a while.

DynamoDB basically says:

“No index? That sounds like a you problem.”

And honestly, fair enough.

The entire database is designed around:

partition strategy
access patterns
deliberate schema design
predictable query paths

Which means an AI assistant cannot just generate random “probably correct” queries and hope reality cooperates.

The query either aligns with infrastructure design or it does not.

There’s not much middle ground.

The Dangerous Part Is Confidence

What makes these failures tricky is that the generated code often looks completely believable.

No syntax errors.
No obvious red flags.
No broken TypeScript.

Just confident infrastructure hallucinations.

And that’s much harder for engineers to detect quickly.

Especially in large systems where:

nobody remembers every index
environments drift over time
schemas evolve constantly
infrastructure knowledge is fragmented across teams

Every company has at least one cloud resource held together entirely by:

historical accidents
undocumented assumptions
and collective organizational fear

AI assistants walk directly into these systems with the confidence of somebody who read half the README and decided they understand the architecture.

This Is The Gap I Keep Thinking About

Most AI coding assistants today are optimized for:

syntax
implementation patterns
framework familiarity
autocomplete quality

But production systems need something else:

operational awareness

The assistant needs deterministic understanding of:

infrastructure topology
indexes
schema relationships
runtime dependencies
deployment reality

Otherwise it is just making educated guesses about systems that may cost thousands of dollars per mistake.

That feels slightly important.

This Is One of the Reasons I Started Building Infrawise

This problem is one of the reasons I started working on opensource project Infrawise.

The idea is simple:

Instead of forcing AI assistants to infer infrastructure from scattered code and configs, provide deterministic infrastructure context directly.

Things like:

DynamoDB index awareness
schema relationships
infrastructure mapping
static analysis
AI-consumable infrastructure context

Not “AI magic.”

Just explicit system understanding.

Because hallucinated code is annoying.

Hallucinated infrastructure is how people accidentally discover entirely new AWS billing experiences.

AI Needs To Understand Systems, Not Just Code

Current AI coding assistants are already very good at understanding source code.

The next generation will need to understand:

architecture
topology
infrastructure relationships
operational constraints
deployed reality

Because software does not actually run inside VS Code.

It runs inside infrastructure.

And infrastructure is where “probably correct” stops being good enough.

Why RAG Alone Cannot Understand Infrastructure

Siddharth Pandey — Mon, 11 May 2026 12:04:13 +0000

Every AI tooling discussion eventually reaches the same sentence:

“Just use RAG.”

At this point, RAG is basically the duct tape of AI architecture.

Missing context?
RAG.

Bad answers?
RAG.

AI hallucinating production infrastructure?
Apparently also RAG.

To be clear: Retrieval-Augmented Generation is genuinely useful.

It improves grounding.
It reduces hallucinations.
It helps AI systems access external knowledge.

But there’s a growing misconception in AI engineering right now:

More retrieval does not automatically create system understanding.

And infrastructure problems are usually system understanding problems.

The Problem Is Not Missing Text

Most infrastructure failures in AI coding assistants are not happening because documentation is missing.

The problem is that infrastructure knowledge is relational, fragmented, and operational.

For example, imagine asking an AI assistant:

“Can this DynamoDB access pattern use an existing index?”

That answer depends on:

actual GSIs
partition keys
sort keys
deployed schema state
access patterns
environment-specific infrastructure
workload constraints

That is not just a “retrieve the right paragraph” problem.

A vector database cannot magically infer operational relationships from semantically similar chunks of text.

Infrastructure is not a Wikipedia article.

It is a topology.

The “Looks Correct” Problem

This is where AI gets dangerous.

The generated output often looks completely reasonable.

Example:

await ordersTable.query({
  IndexName: "customerId-createdAt-index",
});

Looks valid.

Compiles.

Very professional-looking.

Tiny problem: the index does not exist.

The AI retrieved enough surrounding context to generate something statistically plausible.

But plausible infrastructure is not the same thing as real infrastructure.

This is the core limitation of relying entirely on semantic retrieval.

RAG helps models retrieve information.

Infrastructure awareness requires understanding relationships.

Those are different problems.

Infrastructure Is a Graph Problem

Modern systems are deeply interconnected.

A single API endpoint might depend on:

DynamoDB tables
GSIs
SQS queues
Lambda functions
deployment environments
feature flags
event consumers
schema versions

Now imagine asking:

“What breaks if I change this schema?”

That is not a chunk retrieval problem.

That is dependency analysis.

The AI needs to understand:

who consumes the schema
where it is used
which systems depend on it
whether environments differ
whether migrations already exist

Most RAG systems fundamentally operate on semantic similarity.

Infrastructure problems often require deterministic relationship mapping.

Very different category of problem.

Bigger Context Windows Do Not Solve This Either

A lot of people assume larger context windows will eventually solve infrastructure awareness.

I do not think that is true.

A 2 million token context window filled with:

Terraform
CloudFormation
CDK
schemas
docs
configs
deployment files

still does not automatically produce reliable topology understanding.

Because the issue is not just visibility.

It is interpretation.

Even humans struggle with this.

Every engineering organization has:

infrastructure nobody wants to touch
queues nobody fully understands
“temporary” resources that became permanent
databases held together by historical accidents and organizational fear

Giving AI more tokens to read does not automatically create operational intelligence.

Otherwise every engineer who read all the docs would already fully understand production.

Which is obviously not how reality works.

Semantic Similarity vs Deterministic Context

This distinction matters a lot.

RAG systems answer questions like:

“What information is semantically related to this query?”

Infrastructure-aware systems need to answer questions like:

Which indexes actually exist?
Which services consume this queue?
Which schema version is deployed?
Which environment contains this resource?
Which APIs depend on this table?
Which resources are connected?

Those answers should not come from probabilistic guessing.

They should come from deterministic infrastructure context.

That is the missing layer most AI coding systems still lack.

This Is One of the Reasons I Started Building Infrawise

One of the reasons I started working on Infrawise was this exact gap.

The goal is not:

“put infrastructure docs into embeddings.”

The goal is to make infrastructure relationships explicit enough that AI systems stop guessing.

Things like:

DynamoDB index awareness
schema relationships
infrastructure mapping
static analysis
AI-consumable infrastructure context
deterministic extraction pipelines

Not just “more retrieval.”

Because honestly, some infrastructure mistakes are too expensive to leave to semantic similarity.

The Future Is Not Just Better Retrieval

RAG is useful.

It will absolutely remain part of AI systems.

But infrastructure-aware AI requires more than retrieval.

It requires:

topology understanding
dependency mapping
deterministic infrastructure state
operational context
relationship awareness

The next generation of AI coding assistants will not just retrieve infrastructure knowledge.

They will need to understand infrastructure reality.

And that is a much harder problem than embedding documents into a vector database.

Your AI Assistant Knows Your Code But Not Your Architecture

Siddharth Pandey — Sun, 10 May 2026 11:03:22 +0000

AI coding assistants are getting dangerously good.

They can refactor code, generate APIs, write SQL queries, explain regex you wrote at 2 AM during a production incident, and confidently suggest changes that look correct.

And that’s exactly the problem.

Because your AI assistant knows your code.

It does not know your architecture.

There’s a huge difference between:

“I read your files”

and

“I understand your system.”

Right now, most AI tools are basically that one engineer who joined last week, skimmed the repo for 20 minutes, and immediately started suggesting database changes in production.

The Confidence Problem

Let’s say you ask your AI assistant:

“Generate a DynamoDB query for customer orders.”

It happily gives you this:

const result = await dynamo.query({
  TableName: "Orders",
  IndexName: "customerId-createdAt-index",
  KeyConditionExpression: "customerId = :customerId",
});

Looks clean.

Very professional.

Tiny issue though.

That GSI does not exist.

The assistant saw patterns in code and statistically guessed what probably exists.

Which is honestly terrifying when you think about it.

Because infrastructure is not autocomplete-friendly.

Your Architecture Lives Outside the Code

Modern systems are scattered across:

Terraform
CloudFormation
CDK
databases
queues
topics
IAM policies
deployment pipelines
cloud consoles
“temporary” scripts from 2022 that somehow became production-critical

Meanwhile the AI assistant is sitting there reading:

await ordersTable.query(...)

and internally going:

“I have absolutely no idea what this system actually looks like, but statistically speaking… vibes.”

That’s how you end up with:

PostgreSQL joins on non-indexed columns
Event flow assumptions based entirely on naming conventions
DynamoDB scans quietly disguised as “optimizations”
APIs generated against schemas that were deleted three migrations ago
Infrastructure suggestions that look plausible but do not match deployed reality

The code compiles.

The architecture cries silently in the background.

Inference Is Not Infrastructure Awareness

This is the core issue.

Today’s AI coding assistants are really good at inference.

But infrastructure requires deterministic understanding.

There’s a massive difference between:

“This probably exists” and
“This definitely exists in production”

Humans already struggle with this.

Every company has at least one production resource that:

nobody fully understands
nobody wants to touch
somehow costs thousands per month
and is protected by pure organizational fear

Now imagine an AI confidently modifying systems like that.

Why Bigger Context Windows Won’t Solve This

A lot of tooling right now is focused on:

larger context windows
better embeddings
repository RAG
more docs ingestion

Helpful? Yes.

Sufficient? No.

Because infrastructure problems are relationship problems.

Questions like:

Which services consume this queue?
Which indexes support this access pattern?
Which schemas are actually deployed?
Which Lambda depends on this topic?
Which environment even has this resource?

These are topology questions.

Not autocomplete questions.

A 2 million token context window still cannot magically understand infrastructure relationships if the information itself is fragmented or ambiguous.

More tokens do not automatically create system awareness.

This Is Why I Started Building Infrawise

This problem is one of the reasons I started working on Infrawise.

The idea is simple:

Instead of forcing AI assistants to guess infrastructure,
give them deterministic infrastructure context.

Things like:

DynamoDB index awareness
schema relationships
infrastructure mapping
static analysis
infrastructure-aware context for AI systems
MCP integration paths

Not “AI magic.”

Just explicit system understanding.

Because honestly, hallucinated code is annoying.

Hallucinated infrastructure is how people accidentally discover new billing tiers on AWS.

The Next Generation of AI Coding Tools

The current generation of AI assistants understands syntax.

The next generation will need to understand systems.

Not just:

functions
files
classes
frameworks

But:

architecture
runtime dependencies
infrastructure topology
deployment reality
operational constraints

Because software does not run inside VS Code.

It runs inside infrastructure.

And infrastructure is where bad assumptions become expensive.