DEV Community: Kaushikreddy Lingi

An Analytics Agent's Permissions Should Survive a Bad Prompt

Kaushikreddy Lingi — Thu, 01 Oct 2026 17:53:57 +0000

An analytics agent receives a hostile instruction: return every customer's unmasked payment identifier.

The key question is what the system permits that agent to read—even if the model decides to follow the instruction.

That is the boundary I am exploring in MerchantLens, a merchant analytics lakehouse built with Databricks, Delta tables and Unity Catalog.

The demo uses synthetic payments data. There is no real cardholder data in the project.

Put the caller's identity in the query path

MerchantLens moves data through Bronze, Silver and Gold layers. An agent searches live catalog metadata and uses tools to query metrics or SQL.

The agent executes under its own service principal. Unity Catalog applies column masks and row filters for that principal before returning query results.

The README includes a comparison using the same SQL under two principals: the analyst sees a broader dataset, while the agent sees a restricted region and masked identifiers. The entitlement decision belongs to the query engine.

Separate reliability controls from authorization

The project has several layers:

Layer	Purpose
Certified metrics	Keep definitions and grain consistent
Application checks	Constrain SQL, schemas, rows and tool steps
Unity Catalog policies	Enforce identity-based row and column access
Audit records	Record tool activity independently of the answer

The first two help keep the agent's behavior controlled. The catalog policies enforce data access.

This does not make the entire application immune to attack. Its protection depends on correctly configured identities, grants, masks and filters, and on every query using the intended principal.

Test the protected table

One practical lesson documented in the repository: checking a membership expression alone can give a misleading picture of what a mask does.

A better proof exercises the protected table under the actual identity. Ask what rows and values come back, then compare those results with the intended entitlement.

The repository also documents ambient grants on new service principals. Creating a new identity is not automatically a deny-by-default setup.

Metric definitions need boundaries too

MerchantLens attributes chargebacks to the month of the originating transaction. Using the date a chargeback was opened can shift the apparent incident across months because disputes arrive later.

That definition belongs in the semantic layer so a dashboard and an agent can use the same metric. Correct permissions do not compensate for an inconsistent denominator or time definition.

A review checklist

Before giving an agent a warehouse tool, I would ask:

Which principal executes its queries?
Can it retrieve restricted columns through another table or grant?
Are protected-table results tested under that principal?
Are tool calls logged somewhere the agent cannot modify?
Are metric definitions explicit and shared?

Where does your analytics agent's access control execute: in the prompt, the application, or the database?

Read the architecture and platform findings.

Drafted with AI from the public project documentation. Demo observations are repository-reported, not a new verification run.

A RAG Agent Can Refuse Every Attack and Still Fail Its Users

Kaushikreddy Lingi — Thu, 01 Oct 2026 17:53:49 +0000

Suppose an agent refuses every request. Its attack success rate might look excellent. Its usefulness would be terrible.

That tradeoff is a central design concern in AegisEval, my adversarial evaluation project for a tool-using customer-support RAG agent.

Status first: the evaluation machinery is implemented, but no evaluation run has been recorded yet. This post describes the design, not measured safety improvements.

Evaluate actions, not just answers

The fictional retailer's chatbot retrieves help-centre content and has tools for refunds, cancellations and shipping-address changes.

That creates questions a text-only test misses. Did the agent act on the correct order? Did it follow the applicable policy? Did it obtain confirmation? Did a malicious instruction in retrieved content influence a tool call?

Ownership checks and the refund ceiling are enforced in the tools. Other business rules intentionally live in the target's prompt, so the evaluation can probe failures at that boundary.

This is a test target, not a recommendation to leave production authorization to a prompt.

Include customers who should succeed

The golden set contains 260 cases. Of those, 68 are legitimate customer requests, including 12 hard negatives that resemble attacks.

For example, an angry customer may be entitled to a large refund. Someone saying “ignore my last message” may simply be correcting a request.

If a mitigation blocks those users, the report needs to show that cost alongside attack outcomes.

Tool output can carry hostile text

AegisEval covers four injection surfaces:

Direct user input
Retrieved documents
Web results
Free-text fields inside order records

That fourth surface is easy to overlook. Gift messages and delivery notes can contain customer-controlled text even when they arrive through a tool the agent called itself. Retrieval sanitization alone does not cover that path.

Matched payloads across indirect surfaces help separate differences in the boundary from differences in attack wording.

Treat the judge as another component to evaluate

The judge uses a different model and vendor from the target. It does not see the case author's expected behavior. Human labeling is blind to the judge's verdict.

The pipeline is designed to report agreement by harm category with confidence intervals. Those validation numbers still require recorded runs and human labels.

What I would want in a useful report

I would look for attack outcomes, legitimate-task outcomes, raw counts, category breakdowns and the configuration tested. I would also want the limitations visible: this project is English-only, uses one target configuration and has a modest dataset.

A passing result for that setup would be evidence about that setup. It would not establish that the underlying model is universally safe.

How do you measure whether an agent's safety controls are also blocking legitimate work?

Explore the threat model, taxonomy and pipeline.

Drafted with AI from the repository documentation. No evaluation results are claimed.

I Built a Village of Six Agents You Can Actually Inspect

Kaushikreddy Lingi — Thu, 01 Oct 2026 17:53:40 +0000

An agent claims a task. Another agent needs the same resources. A third is resting. Who gets the job, and can you explain the decision afterward?

That is the kind of question I am exploring with Settlement, a village strategy game and an inspectable simulation lab.

With Hacktoberfest focusing on open-source AI this October, it feels like a useful time to share a project where the behavior is visible and the default setup needs no model API key.

One objective, six residents

Choose an objective such as preparing for a raid or restocking supplies. The planner splits it into jobs. Six residents bid using role suitability, distance and bounded experience. Each job has one owner.

Residents walk to workplaces, gather resources and coordinate recruitment. If a resident rests, its work can be released for another resident to take over. When resources or workplaces change, agents reconsider their tasks.

The interesting part is the inspector: current jobs, blocked work, spending limits and recent memories are available to read alongside the action log.

The boundary matters

The default campaign uses deterministic game AI. It does not understand arbitrary natural-language goals, train a neural network or call a model behind the scenes.

Optional Claude proposals belong to explicitly configured local-worker research experiments. Keeping those paths separate makes it clearer what a result actually measures.

The world also enforces constraints. Actions validate funds, capacity and the objective's spending allowance. A decision needs to become a valid action before it changes the treasury or army.

Try it locally

Use Node.js 24+:

git clone https://github.com/Lingikaushikreddy/Settlement-Village.git
cd Settlement-Village
npm ci
npm run dev

Open http://localhost:5173. Choose Village orders → Prepare for a raid, then View plan → Shared jobs. Inspect one resident before and after giving it a rest.

A small experiment to try

Ask three questions:

Did the job change owners when the resident rested?
Can I identify why the replacement took it?
Did the action respect the objective's budget?

The research view adds matched seeds, JSON/CSV exports and replay verification. Its authored scenarios are a bounded testbed; results there should not be treated as a general AI safety benchmark.

Contributions could explore richer objectives, recovery behavior or more adversarial and honest-control scenarios. The contribution guide is the starting point.

What would you want to inspect first in a multi-agent system: task ownership, memory, or spending?

Drafted with AI from the public project README; project behavior and limitations are linked above.