DEV Community

Cover image for Cloudflare AI Gateway User Insights: Spend Checklist
Ahab
Ahab

Posted on • Originally published at indieseek.co

Cloudflare AI Gateway User Insights: Spend Checklist

#ai #security #cloudflare #devops

Cloudflare AI Gateway User Insights: attribute spend before chasing anomalies

Quick answer

Cloudflare AI Gateway User Insights can now show which identities are driving AI cost and flag sessions that deviate from a user's normal pattern. Do not treat the new dashboard as a complete cost-control system by itself.

Roll it out in this order:

  1. attach a stable, non-sensitive identity to gateway traffic;
  2. measure identity coverage before trusting per-user charts;
  3. use anomaly flags as an investigation queue, not proof of abuse;
  4. enforce separate global and per-user spend limits;
  5. test 429 and fallback behavior with a tiny sandbox budget; and
  6. reconcile Cloudflare's estimate with the model provider's bill.

That sequence separates attribution, detection, enforcement, and billing truth—four jobs that are easy to blur together.

Who this is for

This guide is for teams and indie developers routing product features, coding agents, support agents, or internal AI tools through Cloudflare AI Gateway. It is most useful when several users share one gateway and a retry loop, compromised credential, or unusually expensive workflow can create a surprise bill.

If you only need an OpenAI organization-level cap and 429 recovery, use the hard spend limit runbook. This article handles the different problem of attributing and controlling spend across users behind one AI gateway.

What changed on August 5, 2026

Cloudflare released two related AI Gateway capabilities:

  • User Insights reports active users, requests, tokens, median spend, identity coverage, model/provider usage, cache behavior, and users whose spending deserves review.
  • Cloudflare Access identity controls can attach the verified Access subject to request metadata as cf.user_id. That identity can be used in logs, analytics, routing, and spend controls without trusting a user ID sent by the client application.

User Insights works on existing gateway traffic and is available to AI Gateway customers at no additional cost. But unattributed traffic is grouped under one anonymous identity. The dashboard cannot recover a useful per-user view unless requests carry custom identity metadata or pass through an Access-protected custom domain.

Choose the identity source first

Path Use it when Trust boundary Main risk
Server-added custom metadata Your application already authenticates users Your server must derive the ID after authentication A client-supplied ID can be forged
Cloudflare Access An IdP already protects the gateway custom domain AI Gateway records the verified Access JWT subject as cf.user_id Service tokens and requests without a user subject do not receive it
No identity Only aggregate gateway totals matter None Every user is merged into one anonymous bucket

For custom metadata, send an opaque internal identifier—not an email address, access token, prompt, or customer secret. AI Gateway accepts at most five custom metadata entries, values must be strings, numbers, or booleans, and objects are not supported.

Keys beginning with cf. are reserved. Customer-supplied cf.* keys are removed, so do not try to create your own cf.user_id. When Access provides a valid user subject, Cloudflare saves the verified cf.user_id; if five custom entries are already present, the last custom entry may be removed to make room.

A practical rollout workflow

1. Define an attribution contract

Keep the first schema small:

{
  "user_id": "usr_7f3a",
  "team": "support",
  "workflow": "ticket_draft",
  "environment": "production"
}
Enter fullscreen mode Exit fullscreen mode

Generate user_id on the trusted server. Keep the same ID across model providers, and never let browser code choose another user's value. If Cloudflare Access is the identity source, treat cf.user_id as the user dimension and reserve custom metadata for team, workflow, and environment.

2. Gate decisions on identity coverage

User Insights exposes an identity-coverage metric. IndieSeek's recommendation is to avoid per-user enforcement decisions until coverage is at least 95% for the target workflow and the remaining anonymous traffic is explained. A perfect chart built on half the traffic is worse than an honest aggregate total.

Check coverage separately for production, staging, browser clients, server jobs, and service-token traffic. Do not silently assign background jobs to the last human user who touched them.

3. Interpret anomaly flags correctly

Cloudflare calculates a rolling baseline from each user's p95 session cost over the previous 30 days. A session is flagged only when it exceeds both twice that user's p95 and the organization's p99 session cost. The signal is session-based, not request-based, and User Insights does not block the request.

Treat a flag as a question:

Known high-value task? -> record the expected outcome and cost
Retry or tool loop? -> stop the workflow and inspect errors
Credential or identity mismatch? -> revoke and investigate
No supporting task evidence? -> escalate for review
Enter fullscreen mode Exit fullscreen mode

A new rollout may not yet have representative history. Do not convert an early flag into an automatic user suspension.

4. Add enforcement separately

Spend limits are the enforcement layer. Cloudflare can split a budget by metadata value, so one rule can create independent buckets for each user_id; it can also filter by model or provider. When a limit is exceeded, AI Gateway returns 429, or a Dynamic Route can send traffic to a cheaper fallback model.

Keep at least three controls:

Control Purpose
Global gateway limit Contains a system-wide loop or configuration error
Per-user or per-workflow limit Limits one identity or feature without stopping everyone
Application task budget Stops a single agent run before repeated calls consume the period budget

Cloudflare documents spend limits as eventually consistent: the current request is recorded after completion, so concurrent bursts can briefly exceed a limit. Cost is also a best-effort estimate based on tokens and known model pricing. Keep concurrency, retry, and output-token caps in the application even after enabling gateway limits.

5. Run eight low-cost acceptance checks

Use a development gateway, a cheap model, and a deliberately tiny budget:

  1. an authenticated request appears under the expected opaque identity;
  2. an unauthenticated path appears as anonymous rather than another user;
  3. a client attempt to set cf.user_id is stripped;
  4. Access traffic receives a stable verified cf.user_id;
  5. a sixth custom metadata entry is not silently relied upon;
  6. crossing a test spend limit returns the expected 429 or documented fallback;
  7. concurrent requests demonstrate the bounded overshoot your application can tolerate; and
  8. Cloudflare's observed cost is compared with the provider dashboard for the same time window.

Record request IDs, identity source, workflow, model, provider, result, estimated cost, and accepted user outcome. Cost per successful task is a better operating metric than tokens per request.

Common mistakes

  • Passing a browser-provided user ID straight into gateway metadata.
  • Storing emails or secrets when an opaque identifier is enough.
  • Treating “user to review” or an anomaly flag as confirmed abuse.
  • Expecting User Insights to block traffic automatically.
  • Assuming a spend limit is a transactionally exact hard cap during concurrency.
  • Creating one rule per person instead of splitting one rule by identity value.
  • Using all five custom fields without accounting for Access inserting cf.user_id.
  • Ignoring anonymous or service-token traffic when calculating coverage.

FAQ

Does User Insights block anomalous users?

No. It highlights identities and sessions for review. Spend limits, rate limits, application budgets, credential revocation, or a human decision provide enforcement.

Is custom metadata as trustworthy as Cloudflare Access?

Only if a trusted server derives and adds it after authenticating the user. Metadata chosen by an untrusted client is attribution, not verified identity. Access supplies a verified subject through cf.user_id.

Can a spend limit prevent every dollar of overshoot?

No. Cloudflare says spend accounting is eventually consistent, so concurrent requests can exceed the threshold briefly. Treat the gateway limit as one layer and keep request concurrency, retries, and task budgets bounded in your application.

What should trigger an incident?

Use multiple signals: a flagged session, an unexplained task, error or retry spikes, identity mismatch, unusual model/provider choice, or provider-bill divergence. A cost spike by itself may be a legitimate expensive task.

Sources

Top comments (0)