A short TTL does not make an agent's cached tool result safe.
A result can still be wrong for the current user, tenant, credential epoch, policy version, or side-effect state. Worse, a cache can turn an old success into a fresh-looking authorization decision.
For agents, cache correctness is not only about freshness. It is about whether the result is still trustworthy for this run.
This article builds a small cache contract, shows where to enforce it, and ends with failure-injection tests you can run before putting cached tool results in a long-lived runtime.
The failure mode: fresh enough, wrong authority
Imagine an agent that calls get_invoice and caches the result for five minutes. During those five minutes:
- the user loses access to the account;
- the tenant policy changes from allow to deny;
- the credential is revoked or rotated;
- the resource moves to another tenant;
- the tool contract changes its meaning;
- a previous request completed with an UNKNOWN external outcome.
A normal TTL check answers only “Is this entry old?” It does not answer:
- Was this entry produced for the same principal and tenant?
- Was it produced under a still-valid policy and credential epoch?
- Does the current tool contract interpret the result the same way?
- Is the result safe to reuse for this operation, or was it only an observation?
- Can the caller prove where the value came from?
Treating those as separate checks prevents a cache hit from silently becoming an authorization bypass.
Define the cache entry as evidence
Do not store only key -> value. Store the context that makes the value valid:
cache_key = hash(
tool_name,
normalized_arguments,
principal_id,
tenant_id,
resource_scope,
policy_version,
credential_epoch,
tool_contract_version
)
A useful entry also contains produced_at, expires_at, the producing run ID, a provider request ID, a sensitivity classification, a schema hash, and a revocation or invalidation sequence.
The key prevents accidental cross-context reuse. The metadata lets the read path reject entries that are technically fresh but no longer valid.
Separate observation from authority
The safest default is to cache observations, not permissions.
A cached “invoice total is 42” may be useful for display, but it should not be treated as proof that the current run may refund that invoice. The refund path should perform a current authorization check and, where necessary, a fresh resource read.
A practical classification is:
| Class | Example | Reuse rule |
|---|---|---|
| OBSERVATION | list of recent logs | Reuse only within scope and freshness bounds |
| ADVISORY | model-generated summary | Never use as authorization evidence |
| DECISION_INPUT | account state used for a plan | Revalidate policy and resource version |
| EFFECT_CONFIRMATION | provider request ID | Reconcile by stable effect key, not TTL |
This also protects against using a cache to “remember” that an external write succeeded. A timeout after dispatch is an UNKNOWN outcome. It needs provider reconciliation, not a cache hit that happens to contain a similar response.
Enforce the contract at read time
The read path should reconstruct the current context and reject mismatches before returning a hit.
def usable(entry, current, now):
if entry is None or entry.expires_at <= now:
return False
same_context = (
entry.principal_id == current.principal_id
and entry.tenant_id == current.tenant_id
and entry.resource_scope == current.resource_scope
)
same_versions = (
entry.policy_version == current.policy_version
and entry.credential_epoch == current.credential_epoch
and entry.tool_contract_version == current.tool_contract_version
)
if not same_context or not same_versions:
return False
if current.use == "authorization" and entry.classification != "DECISION_INPUT":
return False
if current.use == "effect_confirmation":
return False # reconcile by effect key instead
return True
The invariant is simple:
A cache hit is usable only when its validity context matches the current run and its classification permits the requested use.
Do not rely on the model to decide whether a cached value is safe. The model can request a cache read, but a deterministic layer should decide whether the read is allowed.
Make invalidation explicit
TTL is a fallback. Important changes should invalidate immediately:
- policy version increment;
- credential epoch increment;
- tenant membership or resource ownership change;
- tool schema or contract change;
- manual incident response;
- deletion or retention event;
- provider reconciliation that contradicts the cached result.
An append-only invalidation record makes the behavior inspectable:
CREATE TABLE cache_invalidations (
scope TEXT NOT NULL,
version INTEGER NOT NULL,
reason TEXT NOT NULL,
created_at TIMESTAMP NOT NULL,
PRIMARY KEY (scope, version)
);
Writers attach the current invalidation version to each entry. Readers reject entries written before the active version for the relevant scope.
That is safer than trying to find and delete every matching key during an incident. Deletion can be incomplete; a version boundary is easy to check and audit.
Prevent stampedes without widening authority
When many runs miss simultaneously, single-flight loading can help, but the lock must not blur security boundaries.
Use a lock key with the same principal, tenant, resource, and version dimensions as the cache key. Never let a privileged request populate a shared entry that an unprivileged request can read.
Bound the wait:
- attempt a scoped cache read;
- if missing, acquire a scoped single-flight lease;
- recheck the cache after acquiring the lease;
- load from the provider with current credentials;
- validate the response schema and resource scope;
- write the entry with its context and invalidation version;
- release the lease.
If the provider call times out after dispatch, do not populate a success entry. Record UNKNOWN and reconcile with the provider using a stable request key.
Failure-injection checklist
Add these tests to CI or a staging drill:
- change the tenant between write and read;
- revoke the credential while the entry is within TTL;
- increment policy version while the entry is fresh;
- change the tool contract or result schema;
- replay identical arguments under a different principal;
- return an observation on an authorization path;
- crash after provider dispatch but before cache write;
- make the provider return an UNKNOWN timeout;
- let two tenants contend for normalized arguments;
- restore stale cache entries with current policy versions;
- fill the cache with an oversized or sensitive result.
For every test, record the expected decision: HIT, MISS, REVALIDATE, DENY, or RECONCILE. A green suite should show not only that the request failed safely, but why.
What to monitor
Track cache behavior as a control-plane signal, not just a latency metric:
- hit and miss rate by classification;
- rejected hits by reason;
- cross-context mismatch attempts;
- policy and credential invalidations;
- stale-entry reads after restore;
- UNKNOWN outcomes awaiting reconciliation;
- cache fill failures and provider request IDs;
- sensitive-data eviction and retention events.
A rising hit rate can be a bug if rejection and revalidation data disappear. Keep enough evidence to answer which context produced a value and why the current run was allowed to use it.
If you need a managed place to run an always-on OpenClaw or browser-automation workload, managed OpenClaw hosting on Ampere is one option to evaluate. Hosting does not define cache authority, invalidation, credential scope, or reconciliation semantics; those contracts still belong in the application.
Final rule
A TTL answers “how old is this value?” It does not answer “who may trust it now?”
For agent systems, bind cache entries to identity, tenant, resource scope, policy version, credential epoch, and tool contract. Classify what the value is allowed to prove. Invalidate by version when authority changes. Reconcile external effects instead of treating cached responses as confirmation.
That turns caching from a speed optimization into an explicit, testable part of the agent's control plane.
Top comments (0)