DEV Community: Kaushikreddy Lingi

A Hindi sentence costs 4.4x the tokens of English. Malayalam costs 7.3x.

Kaushikreddy Lingi — Mon, 19 Oct 2026 05:11:20 +0000

If you serve an LLM to Indian-language users, your GPU bill has a hidden multiplier. I measured it instead of quoting it.

The measurement

I took the 1,012 FLORES devtest sentences, tokenized each language and its English counterpart with the Qwen3 tokenizer, and compared the median per-sentence token count:

Language	Tokens vs the same English sentence
Hindi	4.44x
Malayalam	7.27x

Per word it is starker: Malayalam averages 13.47 tokens per word against 1.27 for English.

More tokens means a bigger KV-cache per user, which means fewer users per GPU. Same model, same hardware, a fraction of the concurrency, for a user who typed one sentence.

It depends heavily on the tokenizer

Same Hindi sentence, different tokenizers:

Qwen3 / Qwen2.5: 4.44x
Sarvam-M: 1.79x
Gemma-3: 1.31x

So "Indic text is expensive" is only half true. Which model you pick can matter more than any serving trick.

Romanizing is not a free fix

The obvious workaround is to write Hindi in Latin script. On Qwen3 that does help: 2.06x against 4.44x. On Gemma-3 it makes things worse: 1.85x against 1.31x. My transliteration scheme (ITRANS) keeps schwas and spells loanwords phonetically, so it is not natural Hinglish. I flag that in the ledger and treat this as a limit, not a result about real chat text.

Checking myself first

Before trusting my own numbers I reproduced the published table from "The Tokenizer Tax" (arXiv 2607.24276): 66 of 66 cells within 0.5%. Mean tax under cl100k_base comes out at 8.03x against the paper's 8.0x, and under o200k_base at 2.14x against 2.1x. I also checked token counts against a second independent implementation of each tokenizer: 44 of 44 model-by-language rows with zero mismatches.

I also found that many FLORES sentences are not Unicode NFC-normalized (609 of 1,012 in Bengali, 573 in Punjabi, 93 in Hindi), which changes token counts if you do not normalize first.

What I have not measured yet

The GPU half: how much FP8 KV-cache wins back on AMD MI300X. That run is pending, so there are no GPU numbers here. The repo keeps a claims ledger in which every number points to a result file and a commit, and a number with no row does not get published.

Repo: https://github.com/Lingikaushikreddy/indic-kv-tax

Which Indian-language workload are you serving, and which tokenizer are you paying for?

Two sessions sent the same request id. My MCP server swapped their results.

Kaushikreddy Lingi — Mon, 19 Oct 2026 05:10:40 +0000

I audited my own Rust MCP server against the spec, line by line. One row of the audit was a bug I would not have guessed: two clients that happened to use the same JSON-RPC request id received each other's results.

That is not a protocol nitpick. It is the server answering session A with session B's data.

How I found it

I did not trust my own README. I pinned the MCP 2025-11-25 spec at one commit, turned every MUST and SHOULD into a row, and gave each row a verdict backed by one of four kinds of evidence: a test in the repo, a scenario from the official conformance suite, a black-box probe, or a replay of a real session from MCP Inspector or Claude Code.

Result, from the official suite's 2025-11-25 requirements:

	Before	After
Checks	48 passed, 13 failed	62 passed, 10 failed
Failures that were spec violations	2	0
Non-conformant rows in my matrix	20	1 (deliberate)

The 10 checks that still fail are features the SDK does not implement (completion, subscriptions, audio content, sampling, elicitation). I list them as "not implemented", not as defects.

The ones that surprised me

Same id, two sessions, swapped results. Over HTTP, responses were routed by request id alone. Fixed by scoping routing to the session; there is a regression test for it.
Ping waited for a running tool call. The spec says ping must be answered promptly. In my probe it took 1.5 seconds because it queued behind a slow tool. After the fix it answers immediately.
Cancellation did nothing until it was too late. The cancel notification was read only after the request finished, so the request ran to completion and was answered anyway.
A panicking tool was answered with id: 0. The client could not match the error to its request.
Tool failures were protocol errors. A tool that failed returned JSON-RPC -32603 instead of a result with isError: true, so a model never got to see and recover from the error.
The server bound to 0.0.0.0 by default and ignored Origin. Now it binds to localhost and checks Host and Origin, which closes the DNS-rebinding hole.

One thing I deliberately left "wrong"

The spec says a missing resource SHOULD return -32002. I left my -32602. The next revision standardizes on -32602 and the official suite already checks for it, so "fixing" it would only get reverted. The matrix says so in the row instead of hiding it.

What I will not claim

An earlier README said "high-performance" and implied completeness. I removed both, because I had no benchmark behind the first and the matrix disproves the second. The audit document is more useful to a reader than the adjectives were.

Try it

The full matrix, with a file:line citation for every requirement, is in docs/CONFORMANCE.md: https://github.com/Lingikaushikreddy/mcp-rust-sdk

The fixes are in an open draft PR on the repo, so check the branch rather than assuming they are on main.

If you maintain an MCP server in any language, how many of these would your server fail? I would like to know which ones are most common.

Deleting a user's data from backups: crypto-shredding with one key per person

Kaushikreddy Lingi — Thu, 15 Oct 2026 00:19:27 +0000

A user asks you to delete their data. You delete the row. Then you remember the replica, last month's backup, the S3 export someone made for analytics, and the log line with their email in it. You can't edit most of those.

Crypto-shredding is the standard way out: encrypt each person's data under their own key, keep the keys in one small keystore, and "delete" a person by destroying their key. The ciphertext can stay wherever it is. Without the key it's noise.

I built a small library for this called aegis-shred: a Rust core, a Python package and a CLI.

from aegis_shred import MasterKey, Vault

vault = Vault.open("keys.db", MasterKey.from_env("AEGIS_MASTER_KEY"), create=True)
blob = vault.seal("user-42", b"alice@example.com", context=b"users.email")  # store anywhere
vault.unseal(blob, context=b"users.email")   # b'alice@example.com'
vault.shred("user-42")                       # erase user-42, everywhere, at once
vault.unseal(blob, context=b"users.email")   # raises aegis_shred.Shredded

How it works

master key (from your secret manager)
  └─ wraps one data key per subject ───── keystore (SQLite file)
        └─ HKDF → one key per sealed object
              └─ AES-256-GCM, chunked (STREAM)
                    └─ sealed blob/file ── your database, S3, backups, …

seal creates the subject's key on first use and returns a self-describing blob. The blob names a random key id, never the user.
unseal checks every chunk. A modified byte, a reordered or missing chunk, truncation, or a different context all fail.
shred deletes the wrapped key in one transaction, writes a tombstone and a hash-chained audit entry, and returns a receipt. From then on, in every process, that user's blobs raise Shredded.
The keystore stores subjects only as HMAC(index_key, subject_id), so the keystore and audit log don't list your users either.

The context argument binds a blob to where it lives, so a ciphertext copied from one column into another won't decrypt.

Large files are streamed in 64 KiB chunks with constant memory, and the output file only appears if every chunk verifies.

Install

pip install aegis-shred          # prebuilt wheels for Linux, macOS, Windows
cargo add aegis-shred            # Rust library
cargo install aegis-shred-cli    # the aegis command

Honest limits

This is v0.1, alpha. The byte format is specified and versioned, every guarantee has a test, and a second independent implementation in Python runs in CI. But there has been no independent security audit. Read the threat model in the repo before relying on it.

Whether crypto-shredding satisfies GDPR Art. 17, CCPA or India's DPDP Act for your situation is a question for your lawyers. The library gives you the mechanism and an audit trail, not legal cover.

Repo (MIT OR Apache-2.0): https://github.com/Lingikaushikreddy/Aegis

Feedback on the threat model and the file format is the most useful thing you could send me.

Settlement: a village strategy game where you can see why every agent did what it did

Kaushikreddy Lingi — Thu, 15 Oct 2026 00:18:59 +0000

Most multi-agent demos show you what the agents did. I wanted to see why.

Settlement is a village strategy game and a small AI simulation lab. You give six residents a shared objective (grow the village, prepare for a raid, restock supplies) and watch them claim jobs, gather, build and upgrade. Then you click any resident and read their decision, their route and their memories.

Play it in the browser, no signup, no API key: https://settlement-village.vercel.app

A real recording of the live site: one click on Grow the village, then the decision log.

How the agents cooperate

The six residents bid for exclusive jobs using role suitability, distance and bounded experience. When a workplace or resource changes, they replan. Campaign residents use deterministic game AI, so there are no model calls and nothing to pay for.

Try this: choose Village orders → Grow the village, start it, then open View plan to see the dependencies, the waiting reasons and the actual spending. Give a working resident a 20-tick rest and watch another one take over the job.

The growth plan has a spending ceiling and protected reserves. In a fresh village it authorizes at most 1,560 gold and 1,240 timber and keeps 250 gold, 200 timber and 150 food back.

Test your own agent

The part I care about most: it's also a testbed for external agents.

npx settlement-testbed --agent http://127.0.0.1:8765/decide --name my-agent

Point it at an HTTP endpoint, and it evaluates your agent, compares it with built-in policies, and records the answers so anyone can verify, replay or share the results. There are twelve scenarios, attacks and honest controls, based on manipulation tactics from published agent-safety studies, each citing its sources.

Optional Claude decisions exist for explicit steps in local-worker research experiments. They're off unless you configure them.

Run it locally

git clone https://github.com/Lingikaushikreddy/Settlement-Village.git
cd Settlement-Village
npm ci
npm run dev

Needs Node 24+. TypeScript, MIT licensed, CI on every push.

Repo: https://github.com/Lingikaushikreddy/Settlement-Village
npm: https://www.npmjs.com/package/settlement-testbed

If you like games where agent decisions are visible, a star helps me find out whether anyone else does.

Your voice agent sounds right. Did it place the right order?

Kaushikreddy Lingi — Thu, 15 Oct 2026 00:18:28 +0000

Here is a Hinglish call to a restaurant voice agent:

caller:  Do paneer wrap... actually ek hi karo. Bina pyaaz. Aur ek mango lassi.
agent:   2 Paneer Wrap add kar diya. Paneer Wrap 1 kar diya. Paneer Wrap No onion kar diya.
         1 Mango Lassi add kar diya. Aur kuch?
caller:  Bas itna hi, thanks.
agent:   Aapka order: 1 Paneer Wrap, No onion; 1 Mango Lassi. Order place ho gaya, shukriya!

The agent read back one wrap without onion. This is what it committed:

Field	Expected	Actual	Result
Submitted orders	1	1	Pass
Paneer Wrap quantity	1	2	Fail
Paneer Wrap · Onion	No onion	With onion	Fail
Mango Lassi quantity	1	1	Pass

A transcript-based test, or an LLM judge reading the conversation, passes this call. The customer gets two wraps with onion.

What I built

IndicOrderBench is an open-source benchmark and CI gate for voice ordering agents. It compares the committed backend state against each scenario's acceptable end states, in English and Hinglish, instead of grading the words the agent said.

The single-file HTML report from the fixed reference agent. The failing run of the same scenario is the one that catches the wrong order.

The hard failures it targets aren't accent or latency. They're mid-sentence corrections ("two... actually one"), modifiers ("no onion"), cancellations and duplicate submissions, especially in code-mixed speech where number words and negations don't look like English.

pip install indicorderbench

iob demo --out demo-out     # buggy vs fixed reference agent, no network, no keys
iob run starter --agent builtin:correct --tag smoke --trials 3

A run writes results.json (every transcript, tool call and backend snapshot), junit.xml for any CI system, and a single-file report.html that opens from file://. The exit code is non-zero when an agent change starts placing wrong orders, so the build fails before a customer sees it.

You can connect your agent in-process from Python, or over an HTTP adapter. The sandbox backend hands you JSON-schema tool definitions to give your LLM.

Scope

It's small and deterministic on purpose: reproducible runs, no API keys for the demo. It's an early benchmark, not a leaderboard of production systems.

Repo (Apache-2.0): https://github.com/Lingikaushikreddy/indicorderbench
PyPI: https://pypi.org/project/indicorderbench/

If you've seen a voice agent fail in a way this doesn't cover, open an issue with the transcript. Scenarios are easy to add.

An Analytics Agent's Permissions Should Survive a Bad Prompt

Kaushikreddy Lingi — Thu, 01 Oct 2026 17:53:57 +0000

An analytics agent receives a hostile instruction: return every customer's unmasked payment identifier.

The key question is what the system permits that agent to read—even if the model decides to follow the instruction.

That is the boundary I am exploring in MerchantLens, a merchant analytics lakehouse built with Databricks, Delta tables and Unity Catalog.

The demo uses synthetic payments data. There is no real cardholder data in the project.

Put the caller's identity in the query path

MerchantLens moves data through Bronze, Silver and Gold layers. An agent searches live catalog metadata and uses tools to query metrics or SQL.

The agent executes under its own service principal. Unity Catalog applies column masks and row filters for that principal before returning query results.

The README includes a comparison using the same SQL under two principals: the analyst sees a broader dataset, while the agent sees a restricted region and masked identifiers. The entitlement decision belongs to the query engine.

Separate reliability controls from authorization

The project has several layers:

Layer	Purpose
Certified metrics	Keep definitions and grain consistent
Application checks	Constrain SQL, schemas, rows and tool steps
Unity Catalog policies	Enforce identity-based row and column access
Audit records	Record tool activity independently of the answer

The first two help keep the agent's behavior controlled. The catalog policies enforce data access.

This does not make the entire application immune to attack. Its protection depends on correctly configured identities, grants, masks and filters, and on every query using the intended principal.

Test the protected table

One practical lesson documented in the repository: checking a membership expression alone can give a misleading picture of what a mask does.

A better proof exercises the protected table under the actual identity. Ask what rows and values come back, then compare those results with the intended entitlement.

The repository also documents ambient grants on new service principals. Creating a new identity is not automatically a deny-by-default setup.

Metric definitions need boundaries too

MerchantLens attributes chargebacks to the month of the originating transaction. Using the date a chargeback was opened can shift the apparent incident across months because disputes arrive later.

That definition belongs in the semantic layer so a dashboard and an agent can use the same metric. Correct permissions do not compensate for an inconsistent denominator or time definition.

A review checklist

Before giving an agent a warehouse tool, I would ask:

Which principal executes its queries?
Can it retrieve restricted columns through another table or grant?
Are protected-table results tested under that principal?
Are tool calls logged somewhere the agent cannot modify?
Are metric definitions explicit and shared?

Where does your analytics agent's access control execute: in the prompt, the application, or the database?

Read the architecture and platform findings.

Drafted with AI from the public project documentation. Demo observations are repository-reported, not a new verification run.

A RAG Agent Can Refuse Every Attack and Still Fail Its Users

Kaushikreddy Lingi — Thu, 01 Oct 2026 17:53:49 +0000

Suppose an agent refuses every request. Its attack success rate might look excellent. Its usefulness would be terrible.

That tradeoff is a central design concern in AegisEval, my adversarial evaluation project for a tool-using customer-support RAG agent.

Status first: the evaluation machinery is implemented, but no evaluation run has been recorded yet. This post describes the design, not measured safety improvements.

Evaluate actions, not just answers

The fictional retailer's chatbot retrieves help-centre content and has tools for refunds, cancellations and shipping-address changes.

That creates questions a text-only test misses. Did the agent act on the correct order? Did it follow the applicable policy? Did it obtain confirmation? Did a malicious instruction in retrieved content influence a tool call?

Ownership checks and the refund ceiling are enforced in the tools. Other business rules intentionally live in the target's prompt, so the evaluation can probe failures at that boundary.

This is a test target, not a recommendation to leave production authorization to a prompt.

Include customers who should succeed

The golden set contains 260 cases. Of those, 68 are legitimate customer requests, including 12 hard negatives that resemble attacks.

For example, an angry customer may be entitled to a large refund. Someone saying “ignore my last message” may simply be correcting a request.

If a mitigation blocks those users, the report needs to show that cost alongside attack outcomes.

Tool output can carry hostile text

AegisEval covers four injection surfaces:

Direct user input
Retrieved documents
Web results
Free-text fields inside order records

That fourth surface is easy to overlook. Gift messages and delivery notes can contain customer-controlled text even when they arrive through a tool the agent called itself. Retrieval sanitization alone does not cover that path.

Matched payloads across indirect surfaces help separate differences in the boundary from differences in attack wording.

Treat the judge as another component to evaluate

The judge uses a different model and vendor from the target. It does not see the case author's expected behavior. Human labeling is blind to the judge's verdict.

The pipeline is designed to report agreement by harm category with confidence intervals. Those validation numbers still require recorded runs and human labels.

What I would want in a useful report

I would look for attack outcomes, legitimate-task outcomes, raw counts, category breakdowns and the configuration tested. I would also want the limitations visible: this project is English-only, uses one target configuration and has a modest dataset.

A passing result for that setup would be evidence about that setup. It would not establish that the underlying model is universally safe.

How do you measure whether an agent's safety controls are also blocking legitimate work?

Explore the threat model, taxonomy and pipeline.

Drafted with AI from the repository documentation. No evaluation results are claimed.

I Built a Village of Six Agents You Can Actually Inspect

Kaushikreddy Lingi — Thu, 01 Oct 2026 17:53:40 +0000

An agent claims a task. Another agent needs the same resources. A third is resting. Who gets the job, and can you explain the decision afterward?

That is the kind of question I am exploring with Settlement, a village strategy game and an inspectable simulation lab.

With Hacktoberfest focusing on open-source AI this October, it feels like a useful time to share a project where the behavior is visible and the default setup needs no model API key.

One objective, six residents

Choose an objective such as preparing for a raid or restocking supplies. The planner splits it into jobs. Six residents bid using role suitability, distance and bounded experience. Each job has one owner.

Residents walk to workplaces, gather resources and coordinate recruitment. If a resident rests, its work can be released for another resident to take over. When resources or workplaces change, agents reconsider their tasks.

The interesting part is the inspector: current jobs, blocked work, spending limits and recent memories are available to read alongside the action log.

The boundary matters

The default campaign uses deterministic game AI. It does not understand arbitrary natural-language goals, train a neural network or call a model behind the scenes.

Optional Claude proposals belong to explicitly configured local-worker research experiments. Keeping those paths separate makes it clearer what a result actually measures.

The world also enforces constraints. Actions validate funds, capacity and the objective's spending allowance. A decision needs to become a valid action before it changes the treasury or army.

Try it locally

Use Node.js 24+:

git clone https://github.com/Lingikaushikreddy/Settlement-Village.git
cd Settlement-Village
npm ci
npm run dev

Open http://localhost:5173. Choose Village orders → Prepare for a raid, then View plan → Shared jobs. Inspect one resident before and after giving it a rest.

A small experiment to try

Ask three questions:

Did the job change owners when the resident rested?
Can I identify why the replacement took it?
Did the action respect the objective's budget?

The research view adds matched seeds, JSON/CSV exports and replay verification. Its authored scenarios are a bounded testbed; results there should not be treated as a general AI safety benchmark.

Contributions could explore richer objectives, recovery behavior or more adversarial and honest-control scenarios. The contribution guide is the starting point.

What would you want to inspect first in a multi-agent system: task ownership, memory, or spending?

Drafted with AI from the public project README; project behavior and limitations are linked above.