Authentication answers who. Authorization answers what they're allowed to do. Neither one answers whether this specific action was actually intende...
For further actions, you may consider blocking this person and/or reporting abuse
The
who/whatframing is right, and the intent gap survives even a perfect authorization model — because a database can enforce what a role may do, but it can never see whether this particular call was intended. The useful consequence is that you don’t fix this at the check, you fix it at the blast radius. Two moves that map cleanly onto tool calls: (1) give each dangerous tool its own least-privilege role — thefileExpenseReporttool should connect as a role that can insert one row into one table, not a role that can also send mail, so even a successful injection can only do the one small thing it was scoped to; (2) treat the tool call as an effect to be proven, not text to be trusted — record the input that triggered it next to the effect, because the same document that injects the agent is the thing your audit trail needs to exonerate it afterwards. The injection layer is doing real work, but the durable guarantee is that the action the model is allowed to take is too small to be interesting.Agreed, that's the other half of it. The check at the tool boundary and a small scope behind it do different jobs, and the scope is what caps the damage if a check misses something.
My demo doesn't cover either of your two suggestions. The database is a stub, so there's no database-level role or privilege layer to scope, and
fileExpenseReportis the tool being tested. I useguardTooland my own allow-list onaccountIdinsideexecute(), and I also tested a remote policy separately. I don't tie a call back to the document that triggered it either, so that kind of audit trail isn't part of the demo. Both are worth considering in a real setup.Agreed, and the split you describe is the useful part: the check at the tool boundary and the scope behind it do different jobs, and the scope is what caps the damage when a check misses. A stub database is a fair reason your demo can't show the DB-level version, but that is also why the point is easy to under-weight — in a real stack the same call runs on a role that can be scoped with RLS or a per-task read view, and that is the layer that turns "the tool was told to only read X" into "this connection cannot read Y". The allow-list on accountId inside execute() is the per-call version of that; the DB version removes the success rate entirely. Not something the demo has to carry, just the line I'd add for anyone copying it into production.
Placing the deterministic check immediately before the side effect is the key design choice here. I would pair that boundary with an idempotency key scoped to the authorized action, so a retry can be recognized without treating a second model-generated call as new intent. That also gives the audit trail a clean join between approval, attempt, and write.
Agreed, that's a useful addition. My demo checks the account against an allow-list immediately before the write, but it doesn't use an idempotency key. The allow-list only checks
accountId, so it doesn't distinguish a retry from a new call for an allowed account. I haven't tested repeated calls, but both could reach the write. A key scoped to the authorized action would help identify retries and link the attempt to the write in the audit trail.Testing repeated calls is the useful next boundary: send the same authorized action twice, then a second action with a different key, and assert exactly one write for the first pair and a separately linked write for the second. Persisting the action scope and idempotency key with the authorization decision would make that result auditable as well as safe.
Good framing — 'allowed' and 'asked for' are different questions. We hit the same gap building BugTraceAI: a step can pass every auth check and still fire an intrusive action nobody asked for. Treating declared intent as its own permission dimension, checked again at the tool boundary rather than only in the planner, is what finally helped.
Thanks for sharing that. "Allowed" versus "asked for" is what I was trying to show, and I agree the check belongs at the tool boundary, not only upstream. My demo does that with an allow-list on
accountIdinsideexecute(). I haven't tried declared intent as its own permission check.Making declared intent a boundary check suggests binding it to a concrete action tuple: tool, target, parameters, and expiry. The executor can then reject a request that is authenticated but differs from the planned tuple, while the audit log records whether the mismatch was scope, target, or parameter drift.
Nice security post! In Japan, unauthorized access has increased dramatically these days, and data leaks have become a serious social problem. I also need to learn more about security. 🤔
Thank you so much! 😊 I think hands-on testing is such a valuable way to understand security issues and see where additional safeguards are needed. There's always something new to explore in this area!
Authn/authz answering who/what still leaves intent open — agree that is the hard gap for agents. A smaller adjacent signal when the agent (or its host) phones home is the egress address: shared VPS and recycled residential exits often look noisy in abuse telemetry before the prompt itself looks wrong.
Not a permission system, just cheap hygiene before a sensitive path. Soft poke:
curl ip99.com/v1/ip(omit the IP to ask about the caller) — free, no key; look at freshness/evidence_state, not a permanent ban list.Thanks, that’s an interesting adjacent signal. In my demo, the paraphrased injection passed detectPromptInjection, but the allow-list check inside execute() stopped the write because the accountId wasn’t permitted. I also tested an Arcjet remote policy that allowed one account and denied another, with no write for the denied account.
I didn’t test egress IP reputation. My only network-level test was at the HTTP boundary, where a curl request and the same request with a browser User-Agent were both denied as bots, but I can’t say which signals drove that decision. I’d see egress IP as additional context, not a replacement for checking whether a specific tool call is permitted.
The two separate enforcement points are the detail that matters. Auth answers who and what is allowed, but intent is a different question entirely, and collapsing it into one check is the mistake this whole piece is about.
Exactly, Manoj! That distinction is what I wanted to highlight. Being allowed to perform an action doesn't necessarily mean the agent should perform it in that situation.
The remote-policy experiment leaves an interesting next test: what happens when a previously allowed account is revoked while a policy is cached? I'd include a policy version in the decision log and test cached-allow, expired-cache and policy-service-unavailable cases separately. Your distinction between an attempted call and an actual write makes that matrix much easier to evaluate without treating a successful HTTP response as proof of enforcement.
Thanks for pointing out these cases, Mahdi! I'm glad the distinction between an attempted call and an actual write is useful here. I tested the remote policy with allowed and denied accounts, but didn't test policy changes between runs. Per Arcjet's docs, the SDK caches policies and rechecks them periodically, so testing account revocation, cache expiration and policy-service unavailability separately would be a useful next step. Logging the policy version with each decision would also help explain why a call was allowed or denied.
The paraphrased attack is the part that sticks. Detection asking "does this look suspicious" and the tool boundary asking "should this call with these arguments run right now" are different questions, and folding them into one check is how the clean looking note gets a write.
One addition next to the allow list inside execute(): bind the call to a short lived intent minted before the model runs. Hash the document, name the user who submitted it, list which tools may fire, and put a ceiling on amount or count. When fileExpenseReport runs, check arguments against that intent, not only the user role. A second filing, or a payee that never appeared in the document, has nothing to bind to and dies before the stub write.
That also leaves a trail of which document hash authorized the call. The checklist item about verifying a denied call produces no side effect is the one I wish more stacks tested. Do you keep that as an integration test that asserts the stub never logged an insert, or only as a console glance?
Agreed, those are two different questions. Detection judges the content. The tool boundary checks whether this call with these arguments should run.
Binding the call to an intent defined before the model runs is a good idea, and it goes further than what I built. My own check is a plain allow-list on
accountIdinsideexecute(), with no amount or count limit, and nothing binds the call to the document that triggered it. I haven't tried that design, so I can't say how it would behave.On your question, it's closer to a console check than an integration test. The script runs each fixture, inspects the tool execution log written right before the stub insert, and prints whether a write happened. It has no assertions, so an unexpected result wouldn't fail the run. I'd add an assertion that a denied call never reaches the insert.
That honesty about console check versus assertion is useful, and the paraphrase result you described is the strongest argument I have seen for putting the check inside execute().
One follow up that made my own denial tests trustworthy: assert on the absence of the side effect, not on the presence of a denial message. Count rows in the stub table before and after each denied fixture and fail the run if the count moves. A denial that gets logged while the write still happens is exactly the bug a log check misses.
When you add the assertion, will you also fail the run if a fixture you expect to be allowed gets denied, or only guard the denied path?
Fair point, and it applies to my script. The runs showed no write for the denied fixtures, but I haven’t added an assertion that fails the run if one happens. The execution log is written by the tool code right before the insert, so it isn’t independent of the code under test. My stub only logs, so I’d change it to record writes and compare the count before and after each fixture.
I’d check both directions. The baseline fixture should produce a write, and a check that only guards the denied path would still pass if everything got denied. So the run should fail if the baseline doesn’t produce a write, or if any denied fixture does.
The split between authentication, authorization and intention is the useful part. A signed-in user who is allowed to file expenses is still not a signal that the agent should file one because a document happened to mention it.
On client builds we treat that as a scoping problem as much as a security one. Before an agent is allowed to call a side-effecting tool, the work is written as one problem with acceptance criteria: which actions are in scope, which require a human, and what evidence shows the action was the right one. Auth checks stay necessary, but they do not replace that list.
Did the expense tool end up requiring an explicit confirm for every call, or only when the document was ambiguous?
Agreed. Being allowed to file expenses doesn't mean the agent should file one just because a document mentions it. Defining which actions are in scope and which need human approval goes beyond what my allow-list covers.
To answer your question, neither. The demo has no confirmation step.
detectPromptInjectionscreens the document before the model sees it, andguardToolwraps the tool call. Insideexecute(), my own allow-list rejects accounts that aren't permitted, before the database write. In the ambiguous-document test, the action boundary denied the call. This works here because the allowed accounts are known up front. For costly actions where a mistake matters, I'd consider human confirmation, but I haven't tested that in this demo.How can we prevent unintended side‑effects after an auth check?
Put a check inside the tool itself, immediately before the side effect. Auth tells you who's calling and what they're allowed to do, not whether this specific action was intended. A model-generated tool call is only a proposal.
In my demo, an allow-list inside
execute()rejects disallowed accounts and returns a structured denial before the database write.detectPromptInjectionandguardToolare separate checks earlier in the flow. Test that a denied call never reaches the write, not just that a denial was logged.I also wouldn't rely on the model refusing. In an earlier run, an injection slipped past detection and the model declined to act. A paraphrased version also passed detection, the model attempted the tool call, and the allow-list stopped the write.
Thanks! 😀
the intent gap is the one most agent security discussions skip because it is harder to model than permissions. you can have RBAC that is airtight and still watch an agent act perfectly within its allowed scope on a completely hallucinated goal.
what I keep running into is that the verification step needs to be stateful — checking whether this action was on the plan requires knowing what the plan was, and agents are surprisingly bad at maintaining that context across a long session.
does Arcjet handle anything on the intent side, or is it mostly rate limits and anomaly detection at the request level? curious whether the session state is scoped or per call.
Checking an action against a plan does need state, and my demo doesn't keep one. The allow-list I wrote only checks
accountIdon each call.In my test, Arcjet covered more than request-level checks. I used
detectPromptInjectionon the incoming document andguardToolaround the tool call. I tested a remote policy separately, with the local allow-list widened, and it allowed one account and denied another, with no database write for the denied call. That's enforcement on the tool inputs, not verification against the agent's plan. I didn't test session state, so I can't say whether it's scoped or per call.Shubhra, that paraphrased injection getting past detection is scary 😅 Good thing the check before the database write caught it!
Thanks, Hema! 😊 That's exactly why I wanted to test what happens after the detection step.