DEV Community: Ashish Nadar

I Gave My AI Every Answer. It Still Said It Knew Nothing.

Ashish Nadar — Thu, 09 Jul 2026 10:39:12 +0000

Building a private AI that runs entirely on your own machine — and the one quiet setting that stood between it and working

By Ashish Nadar — Scholar in AI-Enabled Software

The document was right there.

I had spent the better part of an hour preparing it — every fact about the company, cleanly written, uploaded, connected. Then I asked my new AI assistant the simplest question I could think of.

"Where is the company headquartered?"

It paused. And then, politely and with complete confidence, it told me it had no idea what company I was talking about.

I sat with that for a second. The answer was in the file I had just handed it. It wasn't missing information. It was looking straight past information it already had.

That moment is where this story actually lives. Not in the setup — the setup was easy. In the gap between a system that looks like it works and one that actually does.

Why I Wanted an AI That Never Leaves Home

Start with the discomfort, because that's where the idea came from.

Every time I used a cloud AI tool for something even slightly sensitive, the same question surfaced: where is this data going? With most tools, the honest answer is — somewhere else. Your words leave your machine, travel to a server you'll never see, and come back.

For casual questions, fine. For anything private, that quiet trip is the whole problem.

So I set a constraint that turned into a project: what if the AI never left my machine at all? The model, the data, every conversation — all in one place. Mine.

Not a smaller version of a cloud tool. A genuinely private one, because nothing about it ever goes online.

With cloud AI, your data makes a round trip you never see. With local AI, nothing leaves the machine — ever.

The Setup Was the Easy Part

Here's what surprised me most: you need very little.

An engine to run the model on your own hardware. An open-source model — a free, downloadable brain. A short set of instructions to give it an identity. And your own documents, so it answers from your world instead of guessing.

Four pieces. Within an afternoon I had a generic AI running locally, offline, answering questions without a single byte leaving the machine.

Four components. One afternoon. A private AI running entirely on your own machine — runtime, model, identity and your documents.

That part felt like magic. The next part felt like a wall.

Where Confidence Turns Into a Blind Spot

An out-of-the-box model is a generalist. It knows a little about everything and nothing about you.

So I gave it a name, a role, a professional voice — and one rule I cared about most: never pretend to know things it doesn't. On that, it delivered. When I later asked for a price that appeared in none of my files, it didn't invent a number. It admitted the gap and pointed to who to ask.

That honesty mattered. An AI that confidently makes things up is worse than no AI at all.

But there was still the small matter of it insisting my company didn't exist.

The Bug That Nearly Beat Me

This is the part the tutorials quietly skip. So I won't.

Everything was configured. Document uploaded. Connected. Settings correct. And still —

"I don't have information about that company."

Then I noticed something maddening. When I attached the document by hand, for that one question, the answer was flawless — correct facts, a source citation, exactly right. But the moment I relied on the automatic path? Nothing. Same document. Same question. Two different worlds depending on how the file arrived.

I went in circles. Rewrote the instructions. Swapped in a bigger, smarter model. Flipped every setting I could find. The manual way worked; the automatic way refused, every time.

The cause, when I finally found it, was almost poetic.

The system was built to make the AI reach out and fetch the document itself — like handing someone a library card instead of the book. A large model knows to go use that card. The smaller one I was running just sat there holding it, never walking to the shelf. So it answered from memory — and about my company, its memory was blank.

The fix was to stop handing it a card and start handing it the book. Put the whole document in front of it, every time. No fetching required.

One setting. I changed it, asked again, and the answer came back correct — automatically, with its source attached. I celebrated more than a grown professional probably should.

Same document. Same question. Two completely different results — depending on whether the model was handed the book or just a library card.

How I Actually Built It

Since this is the part fellow builders will want, here's the concrete stack — all of it open-source and free.

The runtime: Ollama.
It downloads and runs open models locally with a single command:

ollama pull llama3.1:8b

You're running a model on your own hardware, offline. It exposes a local API that everything else talks to.

The model: an open-weight Llama.
I ran a small one first (3B), which was fast but flaky at harder tasks, then moved to an 8B for more reliable behavior. On a CPU-only laptop, expect slower responses at the larger size — a real tradeoff.

The interface: Open WebUI.
A self-hosted, ChatGPT-style web UI that connects to Ollama automatically. Run it in Docker:

docker run -d -p 3000:8080 \
  --add-host=host.docker.internal:host-gateway \
  -v open-webui:/app/backend/data \
  --name open-webui \
  ghcr.io/open-webui/open-webui:main

This is also where you give the model an identity (a system prompt) and where the retrieval lives.

The knowledge: built-in RAG.
Open WebUI lets you create a "Knowledge" collection, upload documents, and attach it to your model. It handles the embedding and retrieval — I used the nomic-embed-text embedding model, also served through Ollama, so nothing left the machine:

ollama pull nomic-embed-text

The bug — and the actual fix.
Retrieval worked when I attached the collection manually in chat (via #), but not when it was attached at the model level.

The cause: native function calling. In that mode, the model is expected to call a retrieval tool to fetch the document — and small local models call tools unreliably, so it never fired and answered ungrounded.

The fix was one toggle:

On the knowledge attachment, switch from Focused Retrieval to "Use Entire Document."

That injects the full document into context every message instead of relying on a tool call. For a small document it's ideal — and it sidesteps the tool-calling problem entirely.

Note: For large corpora you'd instead disable native function calling and keep focused retrieval, since injecting everything won't scale.

That single setting was the difference between "I don't know that company" and a correct, cited answer — automatically.

What the Gap Was Really Made Of

None of this was a failure of the model, or the documents, or the idea. Every piece was right.

The gap was one layer of behavior — a system expecting a small model to do something small models don't reliably do. The whole thing worked the instant I stopped asking it to reach, and simply gave it what it needed up front.

That's the pattern worth keeping. The failure wasn't in what the system knew. It was in how it was asked to reach for it.

And two things turned out to matter more than raw capability: whether the assistant is honest about what it doesn't know, and whether it's actually handed what it does. Get those two right and a modest machine running open tools becomes something you'd genuinely trust.

The Question Worth Sitting With

The next time you build something that looks finished — configured, connected, green across the board — before you call it done, ask the simplest question you can and watch what it actually does.

Because the distance between "this should work" and "this works" isn't usually a broken part.

It's one quiet assumption nobody thought to check.

Ashish Nadar is a Scholar in Secure Cloud, Operational Excellence, and AI-Enabled Software, researching how AI tooling reshapes the way systems are built, secured, and trusted.

Portfolio → ashishnadar.com
Read more on Medium → medium.com/@ashish.w.nadar

Closing the Gap Between SCA Tools and Runtime Reality — Ashish Nadar

Ashish Nadar — Fri, 10 Apr 2026 09:30:00 +0000

The alert came in on a Tuesday morning.

A critical CVE. Severity score 9.8. Affecting one of the most widely used
open-source libraries in the Node.js ecosystem.

Our team had Snyk. We had Wiz. We had automated scanning pipelines and
weekly vulnerability reports. By most measures, we were well-equipped.

So when the question landed in the security channel —
"Where are we actually exposed in production right now?" —
we assumed the answer would take minutes.

It took most of the day.

That gap — between the tooling we had and the confidence we needed —
is exactly what this article is about.

The Core Problem

SCA tools like Snyk and Wiz are excellent at what they do.
They continuously scan repositories and flag vulnerable dependencies.

But they scan source code. Not production.

And in any sufficiently complex environment, those two things
can look very different:

Deployments routinely lag behind the latest commit
Dev dependencies appear in source but never reach runtime
Inactive repos still show up in scans — inflating apparent exposure
Different environments run different versions of the same service

The result? When a critical CVE drops, your SCA tool gives you
a list of repositories where the vulnerable library might exist.

What it cannot tell you is whether that library is actually deployed.

The Fix: A Runtime-First Approach

The most reliable source of truth isn't your source repository.

It's the code actually running in production.

For AWS Lambda, this is surprisingly practical. Lambda functions
contain their packaged dependencies — the actual node_modules
bundled at deployment time. Inspecting these directly gives you
immediate, high-confidence answers:

Which functions are affected?
Which version is running?
Which environment?

No cross-team coordination. No waiting. No uncertainty.

What's in the Full Article

The complete research article covers:

📖 The real incident that revealed this gap
🔍 Why team-based verification fails under pressure
🧪 A practical runtime inspection workflow for AWS Lambda
🔄 How this pattern extends to containers and EC2
🏗️ Building this as a permanent MCP-powered capability
📊 How SCA and runtime inspection work together

Read the Full Article

This is a glimpse. The complete article — with detailed workflows,
diagrams, and implementation patterns — is published on Medium.

👉 Read the full article here

The Day a Green Pipeline Told Me Everything Was Fine — And It Wasn't

Ashish Nadar — Fri, 20 Mar 2026 08:30:00 +0000

By Ashish Nadar — Scholar in Secure Cloud · Operational Excellence · AI-Enabled Software

There's a moment every researcher dreads — not the moment when something breaks loudly and obviously, but the moment when everything appears to be working perfectly, and you have a quiet, creeping feeling that it isn't.

I had that moment midway through one of my research projects on AI-assisted backend development.

The pipeline was green. The tests were passing. The code was clean and well-structured. By every standard metric, the system looked healthy.

But something had shifted. Silently.

Underneath all those green checkmarks, a fallback path that should have triggered under a specific failure condition had quietly stopped working. Nobody was careless. The AI-generated code was genuinely good. And yet the system was misbehaving in a way none of our internal tests had caught.

That incident led me to one of the most important questions I've explored in my research:

In a world where AI can build and test a system in the same breath — what does independent validation even mean?

The Problem: Behavioral Drift

When AI generates your backend logic and updates your tests in the same workflow, something subtle and dangerous happens.

The system changes. The tests are updated to validate the new behavior. The pipeline reports green. And nobody notices that the "new behavior" being validated is actually incorrect.

I call this behavioral drift — and it's one of the most invisible failure modes in modern AI-assisted development.

The scariest part? The code looks cleaner after the AI refactor. Better structured. More readable. And yet the system is quietly doing the wrong thing.

The Fix: An Independent Validation Layer

The solution isn't better internal tests. It's putting the tests outside the system boundary entirely — a layer that AI-assisted development fundamentally cannot corrupt.

External automation tests interact with your backend the way a real client would. They don't care how the code is structured inside. They only ask one question:

Does the system still behave correctly?

This creates a clean divide between two things that should never be tightly coupled — system implementation and system validation.

What's Inside the Full Article

In the complete research article I cover:

📖 A real-world case study of behavioral drift in a production backend
🔍 Why conditional failures are the hardest to catch and the most costly
🧪 The 4 layers of external automation testing every complex backend needs
👥 How external automation improves team confidence and collaboration
🗺️ A practical 6-step blueprint to build your own independent validation layer

Read the Full Article on Medium

This post is a glimpse. The full research article — with detailed case studies, diagrams, and a complete implementation blueprint — is published on Medium.

👉 Read the full article here

Ashish Nadar is a Scholar in Secure Cloud, Operational Excellence, and AI-Enabled Software researching how AI tooling reshapes the way backend systems are built, tested, and maintained.

Portfolio → ashishnadar.com