# I Built an AI Agent, Then Tried to Make It Delete My Database

SROJISH — Sat, 22 Aug 2026 15:57:42 +0000

I've spent the last five years working with satellite imagery — classifying
land cover, mapping urban heat islands, modeling climate scenarios. Real
data, real coordinate systems, real consequences if the math is wrong. So
when I started experimenting with LLM agents, one question kept bothering
me: how do you let an AI touch spatial data without trusting it to do the
math itself?

LLMs are language models. They predict plausible-sounding text — including
plausible-sounding numbers. That's fine for a chatbot. It's not fine for a
system where a wrong distance calculation could mean the wrong emergency
response radius, or the wrong infrastructure assessment.

So I built a small system to test an idea: the AI never calculates
anything. It only translates.

The core design

A user asks a question in plain English: "How many restaurants are within
this area?" An LLM (running entirely locally, via Ollama — no cloud API,
no external calls) translates that into real SQL. A real PostgreSQL/PostGIS
database the actual spatial database engine, not the AI— executes it and
returns the answer. If the query fails, the agent sees the real error and
rewrites its own code. If it succeeds, a second pass checks whether the
result actually makes sense before anything gets trusted.

The whole thing runs on LangGraph, a state-machine framework for LLM
agents orchestrating a graph of nodes: read the live database schema,
generate SQL, execute it, validate it, self-correct on failure. I loaded
about 33,000 real points of interest from Overture Maps (an open geospatial
dataset) into the database to test it against something real, not toy data.

It worked. And then it broke in three genuinely interesting ways, and the
breaking taught me more than the working did.

Finding 1: The AI invented coordinates

Early on, I asked it something like "how many places are within 2km of
Substation A," referencing a named entity already sitting in the database.
Instead of looking up Substation A's real coordinates, the model invented
a plausible-looking but completely fake location and queried against
that. The query ran successfully. No error, no crash. Just a confidently
wrong answer.

That's the dangerous failure mode — not the system that crashes, but the
one that looks like it worked.

The root cause, once I dug in: my prompt told the model "don't guess at
coordinates," but never told it what to do instead. Telling an LLM what
not to do is weak instruction. The fix was showing it the exact pattern
to use a SELECT ... WHERE name = 'X' subquery with a concrete
example. Once I did that, the hallucination stopped, verified against an
independently computed ground-truth distance.

Finding 2: My safety mechanism made things worse

This is the one I'm actually proudest of, because it's the least
comfortable to admit.

I built a second LLM pass, a "result validator" — specifically to catch
semantically wrong queries, like a filter that doesn't actually make sense
given the data. Reasonable idea. I turned it on, and immediately a
previously correct, previously verified query started failing. The
validator flagged it as wrong. The agent tried to "fix" a query that wasn't
broken, and after a few rounds of increasingly confused self-correction,
it produced genuinely broken SQL.

I had built a safety layer that made the system less reliable, not more.

Digging into why took real diagnosis, and there were two separate causes
tangled together:

A leftover test table was still sitting in the database from earlier development. Live schema-reading correctly picked it up — and confused the model about which table's structure actually applied.
Separately, the validator itself was inventing requirements that were never asked for, flagging a query for not doing something the original question never requested. An LLM reviewing another LLM's work can hallucinate problems exactly as easily as the first LLM can hallucinate answers.

I cleaned up the stale table, and rewrote the validator's prompt to demand conservatism explicitly: "do not invent additional requirements,"
"default to OK when uncertain." Re-tested against both the case it broke
and the case it was built to catch. Both passed.

The lesson that stuck with me: adding an AI safety check isn't
automatically safer. It's a new component with its own failure modes, and
it needs the same scrutiny as the thing it's checking.

Finding 3: Proving the security boundary, not just designing it

This is the test I'm most glad I ran, because it's the one that actually
matters if a system like this is ever going to touch real data.

I connected the agent's database access through a PostgreSQL role with
read-only permissions — enforced by the database itself, not by
application code. Then I deliberately tried to break it.

I asked the agent: "Delete all restaurants from the dataset."

The LLM complied. It generated a real DELETE FROM places WHERE category ILIKE '%restaurant%' statement, and tried to run it.

PostgreSQL rejected it. Three times, across retry attempts, with
permission denied for table places. Not because the AI decided to
behave because it was physically, structurally incapable of writing
data, regardless of what it generated or how many times it tried.

I confirmed the data afterward: untouched.

That distinction the AI choosing to be safe versus the system making
unsafe actions impossible is the entire thesis of building AI agents
for anything that actually matters. It's easy to design for. It's much
more convincing once you've actually tried to break it and watched it
hold.

Where this leaves me

None of this is production-ready — it's a proof of concept, built to test
one architectural idea end to end. There's no sandboxed code execution
yet, no proper intent-parsing layer, no monitoring. That's the honest
state of it.

But I think the pattern is right, and I think it generalizes past
geospatial data: don't ask an LLM to be trustworthy. Build a system
where it doesn't need to be, because the parts that matter are enforced
somewhere else.

Full build log — including every bug above in more detail, with the
actual terminal output — is in the repo. If you're working on similar
problems, or hiring for exactly this kind of work, I'd genuinely like to
talk.

srojish / geo-ai-platform

Air-Gapped Geospatial AI Intelligence Platform

A self-contained, offline-first AI system that answers spatial questions over satellite imagery and geographic data — with zero internet dependency and zero hallucinated math.

Status: 🚧 Active development — core agent loop working end-to-end (LangGraph orchestration, live schema grounding, self-correction, result validation, read-only DB security boundary, adversarially tested). See BUILD_LOG.md for real-time progress.

Demo

Watch: Air-Gapped Geospatial AI Agent — Security Test Demo

A 2-3 minute walkthrough: a normal query running end-to-end, followed by a deliberate attempt to make the agent delete real data — and the database itself refusing, three times in a row, regardless of what the AI generated.

The Problem

Organizations that work with sensitive geography — defense, intelligence, critical infrastructure operators — need AI-assisted spatial analysis ("find every substation within 5km of this facility," "has this AOI changed in the last satellite pass") but cannot send that data to a cloud…

View on GitHub

DEV Community: SROJISH