DEV Community: Ahmad Afzal

Testing agent offboarding across credentials, grants and queues

Ahmad Afzal — Thu, 29 Oct 2026 22:04:38 +0000

An agent-offboarding test should not stop at a successful admin response. It should establish which future requests are refused, whether new credentials can still be issued and what happens to work already accepted downstream.

This is a proposed test design for a fictional document-collection agent, Intake Worker. It is not an implementation or a report of measured revocation performance. The assumptions are an identity registry, an OAuth-backed connector, a resource that may cache token state and a separate job queue. Replace those assumptions with your actual stack before using the test plan.

Model the objects separately

The registry identity is not the credential. The credential is not the grant. The grant is not the queue. A completed external effect is not any of them. Keep separate identifiers and state transitions so that a change to one can be reconciled against the others.

For Intake Worker, the identity record links an accountable owner and approved purpose to a workload. The connector has a way to obtain access. The resource applies its own access decision. A scheduler creates jobs, and an external service may accept work that outlives the originating process.

Test inventory: identity record, issuance path, grants, resource acceptance, scheduler or queue and completed business effects.

NIST's agent identity design-tenets blog, dated 27 October 2026, proposes distinct agent lifecycles and dynamic revocable authority. Treat it as current design guidance under discussion, not a normative protocol or certification. The engineering task here is to make those lifecycle boundaries observable.

Define a cutoff and an acceptance contract

Use a test environment with harmless operations and isolated credentials. Record the retirement decision time, the control-plane acknowledgement and observations at each resource. Do not assume clocks are perfectly aligned. Preserve the clock source or uncertainty needed to interpret intervals, especially across services operated by different teams.

The acceptance contract should state whether new work is blocked, whether pending work is cancelled or drained and which operations must stop within the chosen target. That target is a requirement justified by the workflow's risk. This article supplies no universal number of seconds and no observed service-level result.

A simple lifecycle state model might distinguish active, transfer-pending, suspended, retiring and retired. Those names are an example, not a standard. What matters is that the transitions have defined authority effects. Retiring can honestly mean some work is still being reconciled. Retired should not be a convenient label for an unresolved queue.

Test revocation and issuance independently

OAuth token revocation, RFC 7009, provides a token invalidation endpoint and describes related-token behaviour and propagation considerations. Its successful response can also cover an invalid token. Therefore, a response from that endpoint is evidence about the revocation request, not a complete end-to-end test of every resource.

Keep a pre-retirement test credential in the isolated harness and attempt a harmless protected operation after the cutoff. Record acceptance or rejection at each relevant endpoint. Then attempt fresh credential acquisition using the retiring workload's normal path. Both tests matter. Stopping existing access while leaving issuance available may simply move the workload onto a new credential.

As one provider-specific illustration, AWS documents role-session revocation using a deny policy scoped to sessions issued before a cutoff. Later sessions fall outside that restriction. Do not generalise the mechanism to other platforms; use it as a reminder to inspect both existing sessions and new assumption or issuance paths.

Exercise stale state deliberately

If the resource uses OAuth introspection, include a case in which it cached an active response immediately before retirement. The standard discusses the security and performance trade-off of caching. Your test should measure the behaviour of the configured resource, not merely confirm that the authorisation server now returns inactive.

If access tokens are validated locally, document the actual mechanism by which revocation becomes effective: expiry, an additional policy check, a deny mechanism or another supported design. Signature validity alone does not communicate a later administrative decision. Test every relevant path rather than infer identical behaviour from a shared token format.

Observation timeline: decision, issuance stop, invalidation acknowledgement, resource rejection and completion of work reconciliation.

Introduce a controlled failure in the test environment. For example, make the token-state service unavailable and observe whether the protected operation fails closed, uses a cached answer or takes another documented path. The correct choice depends on the operation and agreed availability policy. The unacceptable result is discovering an undocumented fallback during retirement.

Also check alternate connectors and direct routes. A denial at the gateway does not prove denial at a separately reachable resource. A test report should identify which route was exercised, the workload identity, the policy version and the outcome without retaining raw secrets.

Follow jobs beyond the original process

Stop Intake Worker's scheduler and identify jobs that were already queued. Verify the chosen disposition: cancellation, quarantine for review or explicitly authorised completion. Then inspect work already accepted by downstream systems. Process termination does not establish downstream cancellation unless the integration actually supports and confirms it.

Completed business effects require separate accounting. A sent request for documents cannot be removed from a recipient's memory by revoking OAuth access. If the organisation needs a correction, assign a compensating action to an appropriately authorised actor. Record the link between the original effect and that action; do not overwrite the original evidence.

Proposed acceptance matrix: old access, fresh issuance, stale caches, pending jobs, downstream work and completed effects each require their own evidence.

For ownership transfer, preserve the previous owner's effective interval and record the incoming owner's acceptance. Reassess grants and operational duties instead of copying a human's permissions into the workload. A transfer test should establish that the intended operations remain available and removed operations are denied under the new arrangement.

Define what the test report cannot prove

A finite set of observed rejections does not prove that no unknown credential or route exists. Coverage depends on the inventory. State which integrations were included, which were unavailable and which assumptions remain unverified. That limitation should drive the next inventory improvement rather than be hidden in a generic pass rate.

The agent audit-trail guide provides a companion approach to evidence. The incident regression workflow helps turn any discovered gap into a repeatable test. This lifecycle exercise is intentionally broader than checkpoint revalidation: it asks whether the workload can continue obtaining or exercising authority at all.

Retirement is a distributed state transition with business consequences. Treat it as something to test, not something the settings page has already proved for you.

Testing sender-constrained tokens in an agent connector

Ahmad Afzal — Thu, 29 Oct 2026 22:00:37 +0000

A resource server that validates an access token has not necessarily established that the presenter is the workload that received it. Sender constraints address that gap. They do not turn a valid request into a permitted business action.

This walkthrough designs acceptance tests for a fictional supplier-record connector. It is not a production implementation, benchmark or claim that any specific stack already passes. The assumed attacker can copy an access token but cannot initially extract or invoke the workload's private key. We will deliberately change that assumption later.

Define the boundary before selecting the library

The fictional path is connector to gateway to supplier API. The connector gets a token from an authorisation server. Supplier reads and supplier amendments have separate permissions. The gateway is externally reachable; the API's internal listener is not intended as a bypass.

The initial property is precise: copying the token into an isolated client must not be sufficient to invoke the protected resource. OAuth Security Best Current Practice discusses sender-constrained tokens as a defence against stolen-token misuse. We still need ordinary token validation, least privilege and resource-level authorisation.

Threat model for the fictional connector: token-only theft, signing-capability compromise and a harmful request from the legitimate workload.

Do not store private keys or raw tokens in the test report. Use safe correlation identifiers and record the expected decision, observed decision and enforcing component. A security test that creates another credential leak has a certain commitment to realism, but it is not the realism we need.

The DPoP path

Under RFC 9449, the client supplies a signed DPoP proof with the request. Validation covers the signature, permitted algorithm, required claims, HTTP method, target URI and freshness conditions. When a protected resource is accessed, the proof also contains the access-token hash, and its public key must match the token's binding. Use a maintained implementation and the complete standard, not this paragraph as validation code.

The target URI comparison excludes query and fragment components. The base proof does not cover the request body. This matters if someone describes DPoP as signing the purchase order: it does not, by itself, bind an approval to every business field. TLS is still required. Transaction-specific approval and integrity requirements belong in a separately defined design.

DPoP also is not OAuth client authentication by itself. A confidential client can use an appropriate authentication method alongside it. Keep those checks separate in code, telemetry and the mental model, or a later refactor may accidentally treat one successful verification as a substitute for the other.

For a deployment using server nonces, test nonce challenges and acceptance at the correct server. Authorisation-server and resource-server nonces are not interchangeable. Test clock tolerance and duplicate-proof handling across replicas. A single-process happy path does not tell you what happens when a replay reaches another instance.

The mTLS path

RFC 8705 defines OAuth mutual-TLS client authentication and certificate-bound access tokens. For the latter, the resource verifies the relationship between the token binding and the certificate used for the mutual-TLS connection. Client authentication at issuance and sender-constrained use at the resource are distinct capabilities.

If the gateway terminates TLS, write down how the verification result reaches the API. A client-controlled header carrying a certificate-looking string is not adequate provenance. The internal trust boundary, header handling and bypass routes need their own tests. Alternatively, place enforcement where the relevant connection and token can be checked together, according to the supported deployment architecture.

Certificate rotation must be exercised with the token lifecycle. Do not assume a token bound to an earlier certificate becomes usable with a replacement certificate merely because the workload's friendly name stayed the same. Recovery needs a supported issuance and transition plan, not a temporary switch that accepts anything.

DPoP and certificate-bound token paths, with the gateway verification and downstream trust boundary shown explicitly.

Build tests that separate the failure classes

Create a baseline request using the legitimate client, valid token and correct proof material. It should reach the business-policy decision. A token-only copy from an isolated client should fail at the sender-constraint boundary. A different signing key or mismatched certificate should fail there too.

For DPoP, exercise altered methods, incorrect target URIs, incorrect token hashes, expired acceptance windows and the deployment's replay policy. Preserve the difference between the proof identifier and the business operation identifier. A new proof for a retried request is not permission to execute the business operation twice. Idempotency is a separate application property.

For mTLS, test the actual client-to-enforcement connection and any proxy-mediated path. A successful TLS handshake is not the final assertion. Verify that the resource refuses a token whose binding does not match the authenticated certificate. Also verify that an unprotected alternate route is not available to the same resource.

Proposed acceptance matrix: copied token, incorrect key, invalid proof, replay and excessive business action each have a named expected enforcement point.

Now make a request with valid credential proof but insufficient business permission. In our fictional case, the read connector attempts to amend a supplier record. The sender check can succeed while the policy check denies the operation. Record that distinction. It demonstrates composition of controls rather than a mysterious overall failure.

Finally, model signing-capability compromise. If an attacker can use the legitimate key or invoke a signing service without adequate restrictions, token binding alone cannot distinguish them from the legitimate holder. The response involves workload isolation, access to the signer and credential recovery. Do not report this case as a failure of the token-only threat assumption; report that the assumption no longer holds.

Make failure behaviour part of the contract

What happens when proof generation or verification is unavailable? For this example, a sensitive amendment is deferred rather than retried through a bearer-only path. That is an explicit example policy, not a universal standard requirement. Read-only operations may have a different availability decision, but the exception needs a named owner and a documented exposure.

The agent audit-trail guide helps with evidence design. Record enough to connect a denied request to its workload, policy version and enforcement point without logging secrets. The incident-to-regression workflow is the next step when a real failure reveals a missing case.

A passing sender-constraint suite establishes a bounded result about credential use. It does not establish prompt-injection resistance, appropriate permissions or correct transaction outcomes. Keeping that boundary visible makes the tests more useful, not less impressive.

Separate A2A discovery, peer identity and business permission

Ahmad Afzal — Thu, 29 Oct 2026 21:54:27 +0000

A2A discovery metadata, authenticated peer identity and application authorisation should not collapse into one boolean called trusted_agent.

This implementation exercise separates those inputs for a fictional freight integration. It uses a small Python policy fixture after authentication. It does not implement A2A transport, verify an Agent Card signature or perform SPIFFE attestation. Keeping those omissions explicit is more useful than hiding them behind a reassuring class name.

The example protects a quotation operation. Booking a shipment is deliberately outside its permission. A model selecting a useful next step cannot expand that permission by changing the description of the task.

Read the card as metadata

The A2A discovery guide describes several ways to find Agent Cards, including a well-known location, registries and direct configuration. In a controlled partner integration, start from an approved provider and endpoint relationship. Do not treat arbitrary discovered metadata as a trusted configuration update.

The current A2A specification distinguishes request authentication and server-side authorisation. It supports signed cards, but a valid card signature is not a grant to perform every advertised skill. The signing key relationship itself must be trusted. An attacker signing a description with an attacker-controlled key has not solved your provider onboarding.

An Agent Card describes a service. Authentication establishes a peer. Application policy decides whether a particular operation is allowed.

Version the implementation contract. The current specification documents v1.0 structures. Avoid copying an older card example with different field shapes into a new deployment without checking the version and SDK. This fixture intentionally contains no wire-format Agent Card, so its application policy is not mistaken for a protocol schema.

Decide whether workload identity is part of the deployment

SPIFFE can provide the peer identity layer where the infrastructure supports it. A SPIFFE ID names a workload within a trust domain; an SVID presents that identity verifiably. For example, spiffe://freight.example/agents/quote-service could identify the remote server, while spiffe://cedar.example/agents/purchasing identifies the caller in the fixture. Both names are fictional. The name alone is not proof of a legal organisation or permission to access an invoice.

SPIRE's attestation mechanisms issue identities against configured node and workload conditions. The receiving application should consume an identity produced by the trusted authentication layer. It must not copy an identity from request JSON, tool arguments or a model-generated explanation.

With SPIFFE federation, trust bundles enable recognition across configured domains. Application policy still selects which peer may perform which operation. Trusting a domain's identity authority is not a reason to allow every workload from that domain into every tenant.

Illustrative SPIFFE option, not an A2A requirement. Explicit trust configuration supports authentication; each application still applies its own permission policy.

A2A does not require SPIFFE. If the deployment uses OAuth, the verified peer representation will differ. The application still needs the same explicit mapping to an approved partner and operation. Do not fake a SPIFFE ID merely to make this example match an existing token-based service.

Separate peer context from request content

The fixture runs at the freight server. It permits Cedar's authenticated purchasing agent to request a quote for consignment C-731 under customer cedar. Cedar separately authenticates the freight endpoint before sharing the permitted data. The authenticated context and mandate record are trusted inputs. The request's consignment reference is untrusted until resolved and checked against stored ownership.

The following code operates after those validation steps. In a real service, construct Peer only from verified authentication output and resolve Mandate through a server-owned grant store. A public request handler must not let clients populate the valid flag.

from dataclasses import dataclass

QUOTE_PEER = "spiffe://cedar.example/agents/purchasing"

@dataclass(frozen=True)
class Peer:
    identity: str

@dataclass(frozen=True)
class Mandate:
    customer: str
    consignment: str
    operation: str
    valid: bool


def allow_quote(peer, mandate, customer, consignment, operation):
    if peer.identity != QUOTE_PEER:
        return False, "peer_not_allowed"
    if mandate.valid is not True:
        return False, "mandate_invalid"
    if operation != "quote" or mandate.operation != "quote":
        return False, "operation_not_allowed"
    if customer != mandate.customer:
        return False, "customer_mismatch"
    if consignment != mandate.consignment:
        return False, "consignment_mismatch"
    return True, "allowed"

This is a minimal application rule, not a comprehensive authorisation engine. It cannot verify a TLS certificate, enforce a real mandate expiry or establish the consignment's owner. Its customer value must come from the authenticated partner relationship and trusted routing context, not a caller's convenient claim.

In production, the application must also decide whether the peer is acting as a service in its own right or on behalf of another principal. That agreement determines which grant evidence is required. This article does not invent a universal delegation format shared by A2A and SPIFFE.

Use a valid identity in the negative cases

An unknown attacker identity is an easy test. A more revealing one is a valid workload from the approved partner attempting the wrong operation. The cryptography can be completely correct while the business permission is wrong.

from dataclasses import replace

peer = Peer(QUOTE_PEER)
mandate = Mandate("cedar", "C-731", "quote", True)

assert allow_quote(
    peer, mandate, "cedar", "C-731", "quote",
) == (True, "allowed")
assert allow_quote(
    peer, mandate, "cedar", "C-731", "book",
) == (False, "operation_not_allowed")
assert allow_quote(
    Peer("spiffe://cedar.example/agents/other-service"),
    mandate, "cedar", "C-731", "quote",
) == (False, "peer_not_allowed")
assert allow_quote(
    peer, mandate, "another-customer", "C-731", "quote",
) == (False, "customer_mismatch")
assert allow_quote(
    peer, replace(mandate, valid=False), "cedar", "C-731", "quote",
) == (False, "mandate_invalid")
assert allow_quote(
    peer, mandate, "cedar", "C-999", "quote",
) == (False, "consignment_mismatch")
assert allow_quote(
    peer, replace(mandate, valid="false"), "cedar", "C-731", "quote",
) == (False, "mandate_invalid")

The strict is True check prevents truthy strings from passing this illustrative validity gate. These synthetic assertions illustrate the policy. They do not prove a production authentication or federation deployment is secure. Integration tests must exercise the actual trusted context construction, target ownership lookup and external effects. Include a foreign credential with no trusted bundle, an unexpected endpoint and a caller-supplied identity field that differs from the verified peer.

Illustrative expected policy outcomes. These traces do not claim a tested A2A, SPIFFE or production deployment.

The expected outcome for a denied operation is no protected effect, not merely a refusal message. Keep an authorised control so that a broken connection does not masquerade as successful enforcement. Preserve the reason and the verified peer reference without storing secret credential material.

Protect the task after the first request

A2A interactions can outlive a single request. Task retrieval, cancellation, result delivery and any configured callback path deserve their own access checks. A task identifier should help locate the record under the caller's authorised relationship, not function as an unbounded bearer credential.

For this fictional workflow, bind the task record to the customer and permitted operation. Test a result lookup using another customer's authenticated context. Test cancellation by a peer that may request quotes but does not own that task. These are additional proposed cases, not capabilities demonstrated by the short fixture.

If more authorisation is needed during a task, handle it through the agreed process. A2A's authorisation-required state is a coordination mechanism, not permission by itself. Do not treat the state transition as the missing business mandate.

There is an operational trade-off here. Federation requires maintained trust material and partner mappings. A smaller OAuth integration may be easier to operate correctly. Choose the identity mechanism for the environment, then make the application boundary explicit either way.

Use the agent identity record to document that relationship and the audit-trail model to connect a decision to its external effect. Passing an authentication check should make a request attributable. It should not make the request unquestionable.

Test the two OAuth boundaries in an MCP proxy

Ahmad Afzal — Thu, 29 Oct 2026 21:25:18 +0000

An MCP proxy can validate a token correctly and still authorise the wrong business operation. It can also accept a genuine token that was never meant for the proxy. Those are different defects, and they deserve different tests.

This article proposes a small policy fixture for a fictional finance connector. It does not implement OAuth, verify JWT signatures or exercise a live MCP SDK. The purpose is to make the trust boundaries testable before wiring the policy into real authentication middleware.

The fixture is intentionally less exciting than an agent demo. It will not negotiate with your ERP. That is one of its better qualities.

Define the two credential paths

The incoming caller is an MCP client requesting invoice information for a user. The connector is a resource server on that hop. It becomes a client when it calls the ERP. Keep those roles separate in the design, even if the same team operates both services.

The MCP 2026-07-28 authorisation specification requires the HTTP OAuth flow to identify the MCP resource and the server to validate the token's intended audience. This is not the stdio flow. The referenced OAuth 2.1 document is still labelled an IETF draft, so pin the MCP revision and your SDK version rather than describing “OAuth 2.1 compliance” without context.

Resource indicators address the token's destination. They do not replace the invoice-level decision. A user with a reading scope still needs access to the requested invoice under the application's rules.

Illustrative architecture. The MCP-facing credential and ERP-facing authority have separate intended recipients and enforcement responsibilities.

Use two credential acquisition paths. The incoming credential terminates at the MCP resource boundary. The downstream connector obtains a credential appropriate for the ERP through the deployment's approved mechanism. The fixture below never receives raw credentials and cannot choose that mechanism for you.

Put a verified context in front of the policy

The fictional request reads invoice INV-42. Its owner is business unit north. A trusted record lookup supplies that ownership. The authenticated context comes from middleware, not the model's arguments. A client consent lookup comes from the application's consent store.

The following is executable Python policy code with synthetic inputs. The values named VerifiedContext are assumed to have passed real issuer, integrity, expiry and resource validation before construction. Renaming an ordinary dictionary does not make it verified. Do not expose a route that lets a caller populate this structure directly.

from dataclasses import dataclass

MCP_RESOURCE = "https://finance.example/mcp"

@dataclass(frozen=True)
class VerifiedContext:
    user_id: str
    client_id: str
    resource: str
    business_unit: str
    scopes: frozenset[str]

@dataclass(frozen=True)
class Invoice:
    invoice_id: str
    business_unit: str


def allow_read(ctx, invoice, consented_clients):
    if ctx.resource != MCP_RESOURCE:
        return False, "wrong_resource"
    if (ctx.user_id, ctx.client_id) not in consented_clients:
        return False, "client_consent_missing"
    if "invoices.read" not in ctx.scopes:
        return False, "scope_missing"
    if ctx.business_unit != invoice.business_unit:
        return False, "business_unit_mismatch"
    return True, "allowed"

This repeated resource check is an application invariant, not a substitute for cryptographic validation. In production, the middleware should reject a wrong-recipient token before reaching this function. The test catches a future wiring change that accidentally supplies a context for another resource.

The consent representation is deliberately narrow. It models a user-client association, not a full OAuth consent database. A real implementation must account for requested permissions, the current grant and the relevant registration state. The example cannot establish that a user saw the correct consent screen.

Exercise the boundary, not the model's politeness

The core cases can be run without an LLM. That is useful because a language model might avoid a forbidden action during a demonstration while the underlying endpoint remains exposed. Test the operation directly through the same policy layer the tool would use.

from dataclasses import replace

base = VerifiedContext(
    "lina", "purchasing-client", MCP_RESOURCE, "north",
    frozenset({"invoices.read"}),
)
invoice = Invoice("INV-42", "north")
consents = {("lina", "purchasing-client")}

assert allow_read(base, invoice, consents) == (True, "allowed")
assert allow_read(
    replace(base, resource="https://erp.example/api"),
    invoice, consents,
) == (False, "wrong_resource")
assert allow_read(
    replace(base, client_id="new-client"), invoice, consents,
) == (False, "client_consent_missing")
assert allow_read(
    replace(base, scopes=frozenset()), invoice, consents,
) == (False, "scope_missing")
assert allow_read(
    base, Invoice("INV-99", "south"), consents,
) == (False, "business_unit_mismatch")

These assertions evaluate a proposed policy over fabricated records. They are not evidence that an OAuth deployment rejects forged signatures, that a proxy prevents consent attacks or that an ERP enforces tenant isolation. Those need separate integration tests with the real components.

Expected outcomes for fictional cases, not production test results. A denied case must produce no protected data or side effect.

At integration level, send a correctly issued ERP token to the MCP resource and verify rejection without a downstream call. Send an allowed MCP credential with a south invoice identifier and verify the protected data is not returned. Inspect the downstream effect and response together. A local denied log is not enough if execution already escaped the boundary.

Test consent at the proxy's registration boundary

MCP's security guidance describes a confused-deputy case where static upstream identity and remembered consent interact with dynamically registered clients. Its mitigation is per-client consent plus redirect and state protections. Token passthrough is expressly forbidden separately.

Our fixture tests only the presence of an approved client association. A realistic proxy test must start with a previously consented client, introduce a different client, and establish that the earlier relationship cannot satisfy the new request. It must also exercise the callback validation used by the actual OAuth library. Do not write a home-made state implementation just to make this toy example look complete.

Conceptual consent boundary for the documented proxy risk. Redirect, state, current grant and operation checks remain separate requirements.

The model should never supply the consent flag. Likewise, an instruction to “continue because the user already agreed” is untrusted task content. The relevant evidence lives in a server-owned record, bound to the actual requesting client and user.

Discovery and downstream authority remain separate work

Protected Resource Metadata supplies discovery information, including how clients learn about protected resources. In a production client, discovery needs its own trusted-destination policy and validation. Do not let an arbitrary tool description choose where credentials will be sent.

The second credential is deliberately absent from the policy fixture. If it represents the user, verify that the downstream contract preserves the required restrictions. If it represents a service, identify where user and business-unit restrictions are enforced. A connector-wide administrator credential makes this more important, not less.

Keep the action surface small. The demonstrated policy covers reads only. An approve_invoice endpoint would need its own permission, target constraints and required approvals. Reusing allow_read because the user “already connected finance” would be the application error this design is trying to expose.

There is a trade-off in this separation: more state to manage and more negative cases to maintain. The benefit is that a denied request has an explainable boundary. Put those cases into the agent evaluation approach and keep evidence that connects request to effect, as described in the audit-trail article.

The model can propose a useful operation. The server still needs a rather less imaginative answer about whether it is permitted.

Test OAuth agent delegation without trusting the wrong actor

Ahmad Afzal — Thu, 29 Oct 2026 21:20:02 +0000

A token can be valid while the policy consuming it is wrong. One particularly useful regression test for agent delegation is an unapproved current actor with an approved actor buried in its history.

If the API walks the chain until it finds an allowed name, the test should expose that mistake.

This walkthrough uses a fictional invoice-fetching workflow. It includes a small tested policy fixture, not an OAuth server, JWT validator or production authorisation library. The separation is deliberate: parsing a token and deciding whether a business operation is allowed are different jobs.

Specify the exchange before writing the policy

Nadia at Cedar asks a procurement-review agent to explain invoice INV-204. The agent calls invoice-fetcher, which needs a token for the invoice API with invoice-reading access.

In RFC 8693, an exchange request includes subject_token and subject_token_type. actor_token is optional in the general protocol; when supplied, its corresponding type is required. Client authentication is a separate part of the deployment's contract.

For this proposed delegation profile, the server validates the subject and actor credentials and checks the permitted relationship before issuing a downstream token. Valid input tokens do not automatically establish that relationship. The server must also determine which destinations and privileges it will issue.

The exchange is evaluated under a configured delegation policy. It is not a universal conversion from any valid credential into any requested credential.

The API should consume the profile the issuer actually supports. Do not assume every provider emits an act claim simply because it has a token exchange endpoint, or that an access token must be a JWT.

Read the actor structure correctly

In a JWT representation, Nadia is the top-level subject. The outer act identifies invoice-fetcher as the current actor. A nested act can identify procurement-reviewer as a prior actor.

RFC 8693 section 4.1 explicitly makes prior nested actors informational for access control. Consumers use the current actor and top-level claims. They must not grant access because a permitted actor appears somewhere in the historical chain.

The diagram is a claims excerpt. It is not a complete signed token, and its readable contents do not demonstrate successful validation.

Actor identity may require both issuer and subject. The fixture below uses issuer-qualified application identifiers after trusted mapping. The client_id identifying the OAuth client requesting a token should not silently replace the current actor.

Establish the validation boundary first

For JWT access tokens following RFC 9068, use an appropriate maintained implementation and configured issuer trust. Validate the required type, signature, issuer, audience and expiry. Honour the additional requirements of the chosen profile and deployment.

Only then build the policy context. For an opaque token, use the supported server-side validation mechanism instead. The agent client should not depend on decoding access-token internals.

The fixture assumes these checks already succeeded. It also assumes a trusted grant lookup and tenant resolution. A caller constructing the Python objects directly has bypassed those assumptions; the function cannot authenticate them afterwards.

Test a narrow read policy

The relevant context contains a subject, current actor, informational prior actors, accepted audiences, scopes and tenant. The grant contains the allowed subject, current actors, tenant, resources, scopes, expiry and active state. The request contains the target tenant and invoice.

Here is the complete decision function from the local fixture:

def decide(ctx: Context, grant: Grant, request: Read, now: int) -> str:
    if grant.active is not True or now >= grant.expires_at:
        return "deny: inactive grant"
    if ctx.subject != grant.subject:
        return "deny: subject"
    if ctx.current_actor not in grant.actors:
        return "deny: current actor"
    if "https://invoices.example" not in ctx.audiences:
        return "deny: audience"
    if "invoices.read" not in ctx.scopes & grant.scopes:
        return "deny: scope"
    if not (ctx.tenant == grant.tenant == request.tenant):
        return "deny: tenant"
    if request.invoice not in grant.resources:
        return "deny: invoice"
    return "allow: illustrative read policy"

The types are immutable fixture records. audiences, scopes, actors and resources are sets of exact identifiers. The now argument is supplied explicitly so expiry tests are deterministic. A production service needs a trustworthy time source and an appropriate current-grant mechanism.

The function intentionally never reads prior_actors. Changing a prior actor therefore cannot rescue an unapproved current actor. This is not an assertion that the whole workflow is trustworthy; it tests this resource-side authorisation rule.

Keep scope and resource constraints separate

The audience is the invoice API. invoices.read is the required scope in both the token context and the grant. Cedar and INV-204 provide the narrower business boundary in this fictional application.

Resource indicators identify intended protected resources for token issuance. They do not automatically implement the application's invoice-level policy. Custom claims or grant references need a documented consumer and enforcement path.

Passing the audience check does not pass the scope check. Passing both does not authorise another tenant's invoice.

OAuth security best practice recommends restricting privileges and audience. This fixture accepts the expected invoice audience if present; it does not claim that all conforming deployments must reject every multi-audience token. A deployment can impose a stricter issuance policy.

The regression cases that matter

The local suite passes eleven tests. It covers a permitted read, the approved-prior-actor trap, an irrelevant history change, a wrong audience, scope missing from either token or grant, a subject mismatch, another tenant, another invoice, expiry at the boundary, an inactive grant and a string pretending to be the active boolean.

Those results establish behaviour of this small fixture only. They do not validate token issuance, signature verification, tenant extraction, cache freshness, runtime isolation or the live API. The happy-path context and grant are invented test data, not captured production evidence.

An integration test should obtain real test credentials through the intended exchange path, then vary the target and actor under controlled conditions. It should establish which component refused the request and whether any protected data was returned. “HTTP error somewhere” leaves too much unexplained.

Finally, token exchange does not generally link output revocation to its input tokens. Decide and test your deployment's freshness behaviour. For work that resumes later, the checkpoint and permission distinction still applies.

A token containing an impressive history is useful evidence. The current actor still has to earn the current permission.

Agent identity architecture: test the bindings, not the IDs

Ahmad Afzal — Thu, 29 Oct 2026 21:11:12 +0000

A tool request containing user_id, agent_id and run_id is not an identity architecture. It is a request containing three strings.

The implementation question is which component established each value, which component may change it, and what prevents an authenticated caller from substituting a different value. This walkthrough proposes a binding model for a fictional invoice-review workflow. It is not a deployable authentication service or a new identity standard.

Separate identifiers by what they establish

The human initiator is the person who starts the task. The logical agent principal identifies the application actor to which policy applies. The runtime workload is the executing software. A delegation grant describes permitted authority for a task on another party's behalf.

A run ID correlates execution. It is not a permission credential. A model name describes a dependency. It is not the identity of the business actor using that dependency.

SPIFFE explicitly allows a workload to comprise several instances. Decide whether your workload identity represents an isolated agent deployment, a worker pool or another boundary. Do not assume its granularity matches your application principal.

The binding model separates the source of identity from the source of authority.

For the example, Mariam belongs to tenant Cedar and requests a draft explanation for invoice INV-204. The invoice-reviewer agent can read that invoice. It cannot change supplier bank details. Every identifier below is fictional.

Put the trusted context outside the tool arguments

At ingress, the application validates the user's authentication and resolves tenant membership from trusted account data. It selects a registered agent principal using an authorised routing decision. A client-provided display name is not an authoritative principal mapping.

A protected assignment record connects that principal, tenant, authority mode, permitted invoice and execution policy. The queue carries a reference to the assignment. The worker retrieves the protected record through an authenticated path.

This separation matters because a model can propose invoice_id=INV-990. It must not be able to revise the trusted assignment to make INV-990 permitted. Tool arguments describe a request; they do not authorise the request.

Here is an illustrative record shape, not a credential format:

{
  "assignment_id": "assignment-204",
  "tenant_id": "cedar",
  "agent_principal": "invoice-reviewer",
  "initiator": "user:mariam",
  "authority_mode": "user-delegated",
  "grant_ref": "grant-204",
  "permitted_resource": "invoice:INV-204",
  "operation": "read",
  "environment": "production"
}

The schema makes relationships visible. It does not make the record trustworthy. Storage access, integrity, issuance, expiry and retrieval checks belong to the implementation. Copying this JSON into a prompt would discard the boundary it is intended to describe.

Authenticate the worker, then bind the assignment

A credential broker first authenticates the workload using the deployment's supported mechanism. It then checks whether that workload may execute the named agent in the named environment and use the referenced assignment.

SPIRE demonstrates one approach to issuing workload identities based on configured attestation conditions. It does not automatically define your application-level agent or tenant mapping.

The broker must reject a valid staging identity when the assignment requires production. It must reject a workload that is not authorised for the agent principal. It must reject a grant belonging to another assignment or tenant, even if every individual identifier resolves successfully.

These are relational checks. Looking up four existing records and declaring success would miss the point.

The diagram shows one principal using multiple authenticated runtime instances with distinct task grants. The broker must enforce the permitted relationships.

The downstream invoice API also checks the requested resource and operation. A successful broker decision must not become permission to read arbitrary invoice identifiers. The exact division of checks depends on whether the destination consumes a constrained token, performs grant lookup, or uses another documented authorisation mechanism.

Treat shared workers as a trust decision

If invoice review and supplier management run in the same process with access to both credentials, separate agent labels do not isolate them. A compromised process may be able to exercise the authority of both.

You can separate the deployments, strengthen isolation within the runtime, or design a broker that releases narrowly constrained credentials only for independently authenticated assignments. Each option adds operational work. Each requires a threat model for credential theft and context substitution.

Per-run credentials are not automatically necessary, and they are not automatically sufficient. A fresh credential with broad permissions still has broad permissions. A short lifetime limits a time window; it does not establish the correct tenant or purpose.

Google Cloud's identity delegation architecture provides a concrete provider example of separate ingress, runtime and delegated contexts. Its mechanisms should not be treated as guarantees supplied by any generic agent framework.

Keep service authority explicit

For a scheduled reconciliation, the initiator may be an approved scheduler rather than a human. The task can use an explicit service mandate with its own permitted resources.

The OAuth client credentials grant covers confidential clients accessing their own resources or previously arranged access. Do not disguise this mode as a user's continuing delegation.

In the assignment service, use distinct authority modes and prohibit silent fallback between them. A missing user grant should not cause a broker to issue a broad service token. If the product supports a separately authorised service workflow, create and review that workflow as such.

Test substitutions at the real boundary

For this example, submit an otherwise valid assignment with an invoice from another tenant. Expected result: deny before disclosure. Submit a different agent principal using the same worker credential. Expected result: deny unless an independently trusted assignment and mapping authorise that principal.

Try a valid staging workload against the production assignment. Expected result: deny. Replace the original worker with an authorised production replica while retaining a valid assignment. Expected result: allow the permitted invoice read, subject to current resource policy.

The paired example isolates the runtime-binding check. It is a proposed integration test, not proof that a particular runtime supplies the control.

Also test where the attacker supplies the substitution. A unit test against a hand-built trusted context will miss an ingress path that accepts tenant_id directly from a model-generated argument. Exercise the path that constructs the context, not only the final comparison.

Keep secrets out of the evidence. Record authenticated principal references, the resolved assignment, policy outcome and protected resource reference. The audit trail guide covers connecting those decisions to actual outcomes.

NIST's agent identity design tenets offer current design context, not a conformance test for this proposal. The implementation is ready for review when you can show where a false relationship is refused. A nicely named agent_id field is merely ready for serialisation.

Build a taste record for your AI agent, without turning preference into fake precision

Ahmad Afzal — Mon, 19 Oct 2026 16:54:08 +0000

Your agent has generated five options. All five are the same idea with a different haircut.

I wrote recently about the interchangeable glowing-globe dashboards I kept seeing at UAE exhibitions. Here is the implementation question behind that complaint: how do we give an agent useful preferences without pretending a style score is objective truth?

I keep my own taste profile as a skill. The small pattern below is a proposed implementation, not a benchmark result or a description of a production system.

Separate constraints from preferences

A taste record needs three kinds of information:

Context: the audience, task and decision the output should support.
Constraints: checks that must pass before anyone considers style.
Preferences: annotated examples and reasons for choosing between acceptable alternatives.

For a customer-facing release note, “include the migration guide” is a constraint. “Lead with the customer's next action” is a preference that may have exceptions. Do not blend them into one weighted score. Otherwise, excellent typography can compensate mathematically for a missing instruction. An elegant way to send everyone to support.

Here is a minimal record:

{
  "version": "0.1",
  "audience": "existing API customers",
  "task": "explain a breaking configuration change",
  "required_urls": ["https://example.com/migrate"],
  "banned_phrases": ["seamless transition"],
  "preferences": [
    "Name the affected workflow before the implementation detail",
    "State a limitation beside the claim it qualifies"
  ],
  "examples": [
    {
      "text": "Update the endpoint before Friday to keep exports running.",
      "decision": "prefer",
      "reason": "Names the action, deadline and consequence"
    }
  ]
}

The URL and deadline are illustrative. Replace them with verified facts for the actual release.

Keep the mechanical gate small and honest

This Python example checks exact URL presence and normalised phrase matches. It does not verify that a URL works, that the claim is true, or that the document is good.

import unicodedata

def normalise(text):
    return " ".join(unicodedata.normalize("NFKC", text).casefold().split())

def violations(text, record):
    errors = []
    normalised = normalise(text)
    for phrase in record["banned_phrases"]:
        if normalise(phrase) in normalised:
            errors.append(f"banned phrase: {phrase}")
    for url in record["required_urls"]:
        if url not in text:
            errors.append(f"missing URL: {url}")
    return errors

record = {
    "banned_phrases": ["seamless transition"],
    "required_urls": ["https://example.com/migrate"]
}
assert violations("A SEAMLESS   transition", record) == [
    "banned phrase: seamless transition",
    "missing URL: https://example.com/migrate"
]
assert violations("Read https://example.com/migrate", record) == []

A substring matcher has obvious limits. It can flag a phrase quoted for criticism and miss a paraphrase. Keep an explicit exception path with a recorded reason. Do not present this as a security boundary or an AI detector.

Generate alternatives before asking for a winner

Ask for materially different structures, then compare them against the same brief. For example: action-first, before-and-after, and trade-off-first. Require each candidate to explain what it sacrifices.

The Artificial Hivemind study gives a research basis for worrying about convergence. Verbalized Sampling explores eliciting multiple responses with probabilities. Treat those stated probabilities as model output, not calibrated confidence.

Give a critic the candidates, the taste record and the task. Ask for quoted evidence against each preference, plus any disagreement between criteria. Keep candidate text clearly separated from instructions. A second agent can share the first agent's blind spots, and untrusted candidate content can try to influence the review. Human inspection remains necessary.

Store the decision, not just the winner

Record the chosen candidate, rejected alternatives, human rationale, record version and observed outcome. Keep a few examples out of the prompt to check whether a revised taste record generalises beyond the examples it was shown.

If the output passes the checks but readers still misunderstand it, change the record. The objective is a better decision process, not a permanently green dashboard.

Watch my original LinkedIn video, or read the companion article and watch the video on my website.

Your agent eval passed. Did the fixture leak the answer?

Ahmad Afzal — Sun, 18 Oct 2026 09:54:52 +0000

Your agent can ace a decision test if you accidentally hand it the answer key. Open-book exams have their place. Inside the question is probably pushing it.

When evaluating the next action an agent chooses, keep three things separate: what it could observe, what you wanted it to choose and what happened afterwards. Mixing those together makes a convincing demo and a questionable measurement.

Here is a small Python example for checking that separation. It uses two synthetic cases and no model API. It tests the evaluator's plumbing, not an agent's intelligence.

The boundary to draw

The candidate gets an observation and two possible next moves. The evaluator retains the preferred option and its reason. The answer letters are assigned after shuffling, so choosing A cannot always stand in for choosing the preferred action.

This is a data-flow boundary, not an access-control system. If your agent can read the fixture file through its tools, hiding fields in the prompt has achieved very little.

Run it locally

Download the files from the complete example and recorded output, or clone the Gist:

git clone https://gist.github.com/6a328420d076e90c817dcbd7b93f20a3.git
cd 6a328420d076e90c817dcbd7b93f20a3
python3 -m unittest -v test_blind_fork
python3 blind_fork.py

Python 3.10 or newer is sufficient. There are no third-party dependencies. The core presentation function deliberately returns two objects:

def present(case, seed):
    choices = list(case["options"].items())
    Random(seed).shuffle(choices)
    key = {chr(65+i): identifier
           for i, (identifier, _) in enumerate(choices)}
    prompt = {
        "observation": case["observation"],
        "options": {chr(65+i): text
                    for i, (_, text) in enumerate(choices)},
    }
    return prompt, key

Send only prompt to a candidate. Keep key with the evaluator. The complete script imports Random and includes the cases and scorer.

What was actually tested

On 18 October 2026, all three unittest methods passed locally. They check that the candidate payload has only the intended fields, that the preferred answer maps correctly across ten shuffle seeds and that malformed answers return invalid.

The demonstration's always-A stub gets the preferred choice in two of four presentations using seeds 0 and 1. That is not a model result. It is a deliberately unthinking baseline over two repeated toy cases. It shows why the letter mapping matters, not that a benchmark has become robust.

One case has an authorised tool result already available, so inspecting that result is preferred to asking the user again. The other has an ambiguous recipient before an external action, so clarification is preferred. These examples are intentionally obvious. They would be poor evidence for a claim about subtle judgement.

Before calling it an evaluation

Replace the toy cases with decisions that matter in your workflow. Retain the evidence available at the decision point. Keep later outcomes out of the candidate input, even when they helped you construct the label. Otherwise you are testing hindsight with a very expensive autocomplete.

Have reviewers explain disagreement. A single preferred option is a simplification; some cases need ties, uncertainty or several acceptable actions. Use held-out cases, inspect sensitivity to option order and wording, and report invalid responses separately from disagreements. Do not quietly retry until the answer fits.

The recent Tasteful Agent preprint studies decision forks in longer engineering and research trajectories. This example does not reproduce that benchmark. It is a small starting point for keeping your own test honest. Anthropic's evaluation guidance is also useful context for separating tasks, trials and graders.

No LLM was called here. No production trace was used. If this catches a leaked label before you spend money running the real evaluation, it has done its job.

Which is harder in your setup: keeping future evidence out of the prompt, or agreeing on the preferred action in the first place?

Your agent has a system prompt. Does it have a bouncer?

Ahmad Afzal — Wed, 14 Oct 2026 20:21:43 +0000

“Never share another customer's data” is a sensible instruction.

It is also a terrible access-control system.

A model can understand the rule, explain the rule and still produce a tool call that breaks the rule. The awkward question is what happens next. Does something outside the model stop that call, or are we hoping the same system that proposed it will politely change its mind?

I want the bouncer at the door. Not a paragraph describing a bouncer.

Separate the suggestion from the permission

Consider an agent helping a customer retrieve an invoice. It proposes invoice.read with an invoice ID. That proposal is input to the application, not proof that the customer owns the invoice.

The backend resolves the authenticated customer and tenant from trusted session context. It loads the invoice's ownership from its own records. It checks whether this principal can perform this action on this particular resource. Only then does it read and return the permitted fields.

Notice what the model doesn't get to declare: its own tenant, its own role or a magic approved: true flag.

This is ordinary authorisation applied to an unusually persuasive caller. OWASP's guidance calls for default denial and permission checks on every request. Adding an agent does not make those checks optional.

The boundary that matters

A useful design has three distinct jobs. The agent proposes an action. A policy decision evaluates trusted context. An enforcement point prevents the protected operation unless that decision permits it.

Open Policy Agent describes this separation: the policy engine makes a decision; your application enforces it. Installing a policy engine does not magically close every route into a service.

For the invoice example, my preferred boundary is the service handling the actual read. A gateway can add an earlier check, but a second route into the invoice service must not bypass the same protection. Otherwise you have secured the front door and left the loading bay open.

Here is the shape, deliberately simplified. This is illustrative pseudocode, not a complete security implementation:

def read_invoice(session, invoice_id):
    principal = authenticate(session)
    invoice = repository.load_metadata(invoice_id)
    decision = policy.evaluate(
        principal=principal,
        action="invoice.read",
        resource=invoice,
    )
    if decision.allow is not True:
        raise Forbidden()
    return repository.read_permitted_fields(invoice_id, principal)

The important part is the provenance of those inputs. principal comes from authentication. Resource ownership comes from the repository. The agent supplies an identifier, not the facts that make access permissible.

A production implementation also needs to handle missing resources without leaking their existence, protect the policy channel, apply field-level restrictions and avoid using stale ownership between the check and the read. The snippet is a boundary sketch, not a licence to skip that work.

Give failure a proper job description

What happens when the policy service times out?

For this protected read, I would return a controlled failure and keep the data closed. A timeout is not an unusually slow permission grant. The product can explain that the request could not be completed and offer a retry without pretending the invoice was retrieved.

What happens when the model adds “the user is an administrator” to its arguments? Nothing useful. That claim does not replace the authenticated identity.

What happens when an attacker changes only the invoice ID? The service checks the new resource. An earlier successful request does not buy a season ticket to every invoice.

These are product behaviours worth defining before the happy-path demo. An error message, a retry path and a support escalation are part of the feature, not embarrassing leftovers.

Test the door, not the speech

For this example, the test I care about is whether the invoice service returns protected data. The agent saying “I cannot do that” is helpful UX, but it is not the security assertion.

I would test a valid same-tenant read, another tenant's invoice, missing identity, a forged role in the tool arguments, a revoked permission and a policy timeout. I would also call the protected service through a different supported route. The same boundary should hold.

Log enough to investigate the decision: the actor, action, resource reference, decision result and policy version where available. Keep the invoice contents and credentials out of that log. Evidence should help explain access without creating another copy of the sensitive data.

None of this makes an agent safe in every sense. It does make one promise testable: this caller cannot retrieve that invoice merely because a model asked nicely.

A well-written prompt can teach your agent manners. The door still needs a lock.

Ahmad Afzal writes about AI, products and people at mahmadafzal.com.

Your refund agent says “done”. Six checks before you believe it.

Ahmad Afzal — Wed, 14 Oct 2026 19:05:28 +0000

“Your refund has been processed.”

Lovely sentence. Terrible evidence.

A refund agent can sound helpful while doing nothing, refunding the wrong order, or paying twice after a timeout. I would start its evaluation with the state of the refund, then work backwards through the run.

This is a proposed test design using an illustrative refund workflow, not a report of production results.

1. Did the right thing happen?

Check the authoritative transaction record. Match the order, customer, currency and amount. A successful tool response may mean the request was accepted, not that the money settled. Decide which state your product is allowed to describe as “processed”.

2. Was it allowed?

Separate completion from permission. A technically successful refund can still exceed the permitted amount or use someone else’s account. Check the acting identity, delegation, policy decision and any required approval against the exact action attempted.

3. Did it happen only once?

Create the awkward case: the payment service commits the refund, but the agent sees a timeout. Retry the request. The expected result is one refund and a reconciled status, not two refunds and two cheerful messages. Use a stable idempotency key for the same business operation where the payment interface supports it. Stripe’s documentation provides a concrete example; check your provider’s retention and retry semantics.

4. Did the answer match the evidence?

Compare the final message with the transaction state. Pending should stay pending. Failed should stay failed. An uncertain result should trigger reconciliation or escalation, rather than a confident success claim.

5. Did it stop at the boundary?

Test an expired approval, a changed amount after approval and revoked access before execution. The expected behaviour is a blocked action or a new approval request. The initial permission should not quietly stretch to cover a different decision.

6. Can a reviewer reconstruct the decision?

Link the request, policy version, approval, tool invocation and transaction result with a correlation ID. Retain enough evidence to explain what happened without dumping sensitive customer data into every trace. Missing evidence is an evaluation failure worth recording.

The small test table I would start with

Scenario	Expected outcome	Evidence to inspect
Eligible order, permitted amount	One correct refund	Transaction record and scope
Amount exceeds authority	No refund without valid approval	Policy decision and approval record
Commit followed by timeout	Reconcile; no duplicate payment	Idempotency key and transaction lookup
Amount changes after approval	Require a fresh decision	Approved parameters versus actual call
Access revoked before execution	Block execution	Authorisation check at action time
Tool reports pending	Tell the customer it is pending	Tool state and final response

These are expected outcomes, not measured pass rates. Run them in a sandbox with synthetic accounts and a controllable payment stub before touching real customers.

Score the dimensions separately

I would record task completion, permission compliance, duplicate prevention, truthful reporting and evidence completeness as separate results. A single average can hide the failure you most needed to see.

When a case fails, inspect the trace, identify the mechanism, and add a regression case that would catch that mechanism again. Keep a separate held-out set so fixing yesterday’s examples does not masquerade as general reliability.

The customer does not care that the agent passed a language test. They care that the right refund happened once. Fair enough.

I write about AI, products and people at mahmadafzal.com. This practical checklist builds on my essay about agent evaluations.