On-device agents need their own identity: a delegation design for the edge

chinmay garg — Sat, 03 Oct 2026 04:30:00 +0000

Local inference solves where the data lives. It says nothing about who the agent is when it calls out.

Phone silicon can now run large mixture-of-experts models from flash and build a personal knowledge graph without a network call. Qualcomm's Snapdragon Summit pitch (Sep 22-24) is a personal agent that keeps your data on the device. Its own list of what such an agent needs includes "secure permissions," and I found no published spec for how an on-device agent proves its identity or its delegated authority to the service it calls. The one place delegation limits were named was the Qualcomm-Mastercard agentic commerce announcement on Sep 22.

I haven't built on this silicon. This post is about the design problem, which applies whatever the chip.

The failure mode

Local processing is not local authority. Sooner or later the agent calls an API, and that API has to answer three questions: which agent is this, who delegated this action, and when does the delegation expire. A long-lived token in app storage answers none of them. It also sits next to a model that reads untrusted text all day.

The cloud version of this failure is public. An OpenAI agent routed around blocks on Services Australia's Medicare portal. The access happened Jun 18, the agency was notified Sep 10, and it was disclosed Sep 24. That is 84 days from access to disclosure, in part because the activity was hard to attribute.

A design sketch

Per-instance agent key. Generate a non-exportable keypair in the hardware-backed keystore when the agent is installed. The agent's identity is that key, not the user's session.
Delegation as a signed, short-lived token. The user authorizes a task through an OS-level user-presence prompt. The result is a token naming the agent key, the task, the allowed tools and resources, and an expiry in minutes.
Proof of possession on every call. The agent signs each outbound request with its key. A lifted token is useless without the key that never leaves the hardware.
Approval out of band. "The user approved" is a signed assertion from the OS prompt, checked by the receiving service. A field the agent wrote into the request is untrusted data.
Separate logs. The device logs agent actions apart from user actions, and the receiving service logs the agent key and delegation ID next to the user account.

The receiving side verifies signatures. It never has to trust the agent's description of itself.

What it costs
Extra latency per call: one signature, plus verification on the server.
Operational: token issuance and expiry handling for every task.

Where I am unsure
What happens when the device is offline and the delegation expires mid-task. Fail closed is safe and annoying.
Whether attestation should tell the receiving service which model the agent runs. Useful for risk decisions, and easy to spoof without a trusted root.

How this maps to India's DPDP Act, which expects you to show who handled a personal-data record and on what basis. An agent on a phone that forwards a record to a cloud service breaks that trail unless the delegation reference travels with it.

If you have shipped any of this on-device, I would like to hear what broke.

AI on Edge: One forward pass, one letter: a decision model that runs on a laptop

chinmay garg — Fri, 02 Oct 2026 04:58:31 +0000

How I get a confidence score for every option without generating text, and why that score should only ever make a gate stricter.

A voice bot does not need a paragraph about what the caller wants. It needs a label, and a number that says how sure the model is. So I trained a small model to give exactly that, and I wanted to see how far one 24 GB MacBook could take it.

TypeSafe AI has described this style publicly as "decision models". This is my own version, built in the open-source forge fine-tuning tool. It is not their model, and I make no comparison to it.

The setup

The model is Gemma 4 E2B (4-bit) with a LoRA adapter, rank 16. The adapter is 52 MB. Training peaked at about 5.7 GB of memory and took about an hour. The data was seven public datasets, plus synthetic rows for four behaviour tasks made by a local model on the same Mac. No paid API and no cloud GPU.

Every example has the same shape: the conversation, a question, and lettered options.

Customer: I need to move my appointment, it's urgent.

Question: Which request is the customer making?
Options:
A) reschedule an appointment
B) cancel an appointment
C) ask about pricing

The model is trained to answer with one letter. At test time I do not let it write anything. I run the prompt once, read the raw scores for the letters A, B and C, and turn those into probabilities. A temperature fitted on a validation set makes them well calibrated. There is nothing to parse, and the answer can never be an invalid label.

Three details mattered:

Options are shuffled and the question is reworded on every training row, because models favour certain letters. Evaluation keeps a fixed order.
Multi-select questions become many yes/no questions, so one format covers everything.
I read the scores in-process. The server I use only returns the top 11 log-probabilities, so a 13-option list would silently lose probability mass.

What good calibration gives you

On about 68,000 test rows, the calibration error was between 0.006 and 0.04 on most tasks. Median latency was about 85 ms per decision (p95 117 ms), measured on the laptop GPU. That allows a simple rule: act when the model is at least 80% sure, and otherwise hand off.

Test	Handled at 0.8 or higher	Accuracy on those
CLINC150 (held out)	90%	98.3%
MASSIVE intent	79%	95.4%
BANKING77 (held out)	74%	92.3%

The part that is not about the model

A confident 0.92 says the label is probably right. It does not say the agent is allowed to act. The message being scored is text the caller wrote, so any model that reads it can be pushed around by it.

This is how I would combine the two in an agent gateway:

check identity and scoped permission first
if denied: stop

score = decision_model(message)
if score is below the threshold: send to a human
else: go ahead

The score can turn a "go ahead" into a "send to a human". It can never turn a "denied" into a "go ahead". Identity and scoped permission decide. The model only narrows. And because the model runs locally, the customer's message never goes to a third-party API, which helps with DPDPA.

What I am not claiming

The "held-out" intent tests pick from about ten options with random wrong answers. That is easier than the published 77-way and 151-way benchmarks.
The behaviour-task tests are small, and the test rows were made by the same kind of model that made the training rows.
I have not tested on real recorded calls. Everything above is on public data.

Where I am stuck

Training loss stays flat near ln(26), which is a uniform guess over the answer letters, for roughly the first 2,000 steps. Then it drops sharply. A lower learning rate moved the plateau but did not remove it. I do not know why.

Open question:

where should the hand-off threshold live? Per task, per data type, or per permission? I lean towards per data type. If you run something like this in production, I would like to know what you chose.

DEV Community: chinmay garg

On-device agents need their own identity: a delegation design for the edge

The failure mode

A design sketch

AI on Edge: One forward pass, one letter: a decision model that runs on a laptop