Azure API Management Policies: Scopes, Rate Limits and Caching

Why did adding a cache to one operation switch off the partner rate limit for that operation? Policies in API Management don't override each other by scope: the gateway runs the policy at the narrowest scope, and one <base /> element decides whether, and where, everything above it runs too. Leave it out and the gateway saves your XML without a warning and skips the product and API rules for that section.

A policy is a set of runtime rules the gateway applies to every call: check a limit, rewrite a header, pick a backend, cache a response. It has nothing to do with Azure Policy, which governs how resources are configured; APIM policies act on requests. Microsoft ships more than 75 of them. Part 1 already left one in the order API of the companion repo: it routes calls to the function backend and removes the subscription key header before the call leaves the gateway. This part adds a policy at each of the other scopes and breaks one of them on purpose, on a Basic v2 instance, to see what a missing <base /> costs.

Anatomy of a policy document

This is the API policy as Part 1 left it:

<policies>
  <inbound>
    <base />
    <set-backend-service backend-id="orders-func" />
    <set-header name="Ocp-Apim-Subscription-Key" exists-action="delete" />
  </inbound>
  <backend>
    <base />
  </backend>
  <outbound>
    <base />
  </outbound>
  <on-error>
    <base />
  </on-error>
</policies>

Every policy document has this shape: a <policies> root and four sections, and the statements inside a section run top to bottom.

• inbound runs on the request first: authentication, rate limits, cache lookups, routing, request rewrites. Here it picks the orders-func backend, which carries the function key, and deletes the subscription key header so the function never logs it.
• backend runs right before the gateway forwards the request. The forward itself is a policy, <forward-request />, and it sits in this section at global scope, which is why the API policy has nothing but <base /> there.
• outbound runs on the response: response headers, cache-store, body transformations.
• on-error runs only when something above fails. The gateway skips the rest of the pipeline and jumps here, with the failure in context.LastError.

Not every policy works in every section or at every scope: rate-limit is allowed in inbound only, and at product, API and operation scope but not globally. Each policy's reference page has a Usage block listing the sections, scopes and gateway types it supports; read it before you write the element.

In the portal, every scope has its own code editor, and it saves whatever parses. It doesn't tell you that a section lost its <base />. The sample keeps one XML file per scope in infra/policies/ and loads each into a policy resource in infra/apim.bicep. Every scope has its own resource type, and all of them are named policy:

resource globalPolicy 'Microsoft.ApiManagement/service/policies@2024-05-01' = {
  parent: apim
  name: 'policy'
  properties: {
    format: 'rawxml'
    value: loadTextContent('policies/global.xml')
  }
}

resource getOrder 'Microsoft.ApiManagement/service/apis/operations@2024-05-01' existing = {
  parent: ordersApi
  name: 'get-order'
}

resource getOrderPolicy 'Microsoft.ApiManagement/service/apis/operations/policies@2024-05-01' = {
  parent: getOrder
  name: 'policy'
  properties: {
    format: 'rawxml'
    value: loadTextContent('policies/operation-get-order.xml')
  }
}

Product and API policies are service/products/policies and service/apis/policies, same body. The operation is declared existing because the OpenAPI import created it. Use rawxml, not xml: with xml, the " and < inside policy expressions have to be XML-encoded, and the file stops looking like the policy the portal shows. A diff on a policy then stays a diff on XML, which is what Parts 5 and 6 build on.

Four scopes, one pipeline

The sample now has a policy at every scope. The product orders-partners limits each partner subscription to 10 calls a minute with <rate-limit calls="10" renewal-period="60" />. The operation GET /orders/{orderId} caches responses for 30 seconds, separately for each subscription (the x-partner header comes from the API policy; the caching section shows why the cache needs it):

<policies>
  <inbound>
    <base />
    <cache-lookup vary-by-developer="false" vary-by-developer-groups="false">
      <vary-by-header>x-partner</vary-by-header>
    </cache-lookup>
  </inbound>
  <backend>
    <base />
  </backend>
  <outbound>
    <base />
    <cache-store duration="30" />
  </outbound>
  <on-error>
    <base />
  </on-error>
</policies>

<base /> is where the parent runs

The scopes go from broadest to narrowest: global (all APIs), workspace (only if you use workspaces), product, API, operation. When Contoso calls GET /orders/{orderId}, the gateway doesn't run four policies and let the narrow ones win. It runs one: the operation's. Each <base /> in it is a placeholder for the same section one scope up, and that section's <base /> pulls in the next one, up to global.

Resolved, the inbound the gateway runs for that call looks like this:

<inbound>
  <!-- global -->
  <set-header name="x-correlation-id" exists-action="skip">
    <value>@(context.RequestId.ToString())</value>
  </set-header>
  <!-- product: orders-partners -->
  <rate-limit calls="10" renewal-period="60" />
  <!-- API: orders -->
  <ip-filter action="forbid">
    <address-range from="203.0.113.0" to="203.0.113.255" />
  </ip-filter>
  <set-backend-service backend-id="orders-func" />
  <set-header name="Ocp-Apim-Subscription-Key" exists-action="delete" />
  <set-header name="x-partner" exists-action="override">
    <value>@(context.Subscription?.Id ?? "none")</value>
  </set-header>
  <choose>
    <when condition="@(context.Request.Method == "POST")">
      <rate-limit-by-key calls="3" renewal-period="60"
          counter-key="@("create-order:" + context.Subscription.Id)" />
    </when>
  </choose>
  <!-- operation: get-order -->
  <cache-lookup vary-by-developer="false" vary-by-developer-groups="false">
    <vary-by-header>x-partner</vary-by-header>
  </cache-lookup>
</inbound>

The policy editor in the portal has a Calculate effective policy button that shows the merged result for the scope you're on. The management API returns the same thing if you add effective=true to a policy GET, which is handy in a pipeline:

az rest --method get --url ".../service/<apim>/apis/orders/operations/get-order/policies/policy?api-version=2024-05-01&effective=true"

On the sample it came back with <!--base: Begin Api scope--> and <!--base: Begin Global scope--> comments around each spliced section, and without the product's rate limit. Which product applies depends on the subscription key of the call, so a view from the operation can't know it.

Where <base /> sits decides the order. The sample puts it first in every section, as Microsoft recommends, so the rate limit runs before the cache lookup and a cached response still costs the partner a call. On the live instance, Contoso created one order and then read it back twelve times: the first read went to the function, the next nine came from the cache (all with the first response's correlation ID), and calls 11 and 12 got 429 Too Many Requests with Retry-After: 4. Counting the POST, that's 11 calls through a limit of 10; the token bucket below explains the extra one. Move <base /> below cache-lookup and the operation's own statement runs first: cache hits skip the rate limit, and the lookup builds its key before the API policy has set x-partner.

Remove it and the parent is gone

This is the operation policy someone writes when they add the cache by hand and start inbound from an empty section:

<policies>
  <inbound>
    <cache-lookup vary-by-developer="false" vary-by-developer-groups="false">
      <vary-by-header>x-partner</vary-by-header>
    </cache-lookup>
  </inbound>
  <backend>
    <base />
  </backend>
  <outbound>
    <base />
    <cache-store duration="30" />
  </outbound>
  <on-error>
    <base />
  </on-error>
</policies>

I pushed it to the Basic v2 instance with az rest --method put on the operation's policies/policy resource. The management API answered 200 and echoed the policy back. No warning, no validation error.

Twelve calls to GET /orders/{orderId} with Contoso's key then returned twelve 401 Unauthorized responses with an empty body, and no 429. The gateway didn't produce those 401s. With the API's inbound gone, set-backend-service never ran, so the gateway sent the call to the API's plain service URL, without the x-functions-key header the backend entity adds, and the function host rejected it. GET /orders, which has no operation policy, kept answering 200 at the same time.

A 401 is the lucky case, because someone notices. The likely fix is to copy <set-backend-service backend-id="orders-func" /> into the operation's inbound, see 200s again, and move on. I did that too: 20 calls in 8 seconds, 20 times 200 OK, against a product limit of 10 per minute. Nothing in the response, the portal or the deployment tells you the partner limit is off for this one operation, and it's the one partners poll.

Only the section without <base /> loses its parent. The operation above still has it in outbound and on-error, so those chains run as before.

Two checks catch this before production: the effective policy view shows an inbound without the API and global statements, and Azure Policy has a built-in definition, API Management policies should inherit parent scope policies using <base/>, that audits or denies policy documents without it. Part 5 puts that into the pipeline.

Which subscription you call with changes which scopes run

Product policies run only for calls made with a product-scoped subscription. Learn says it in a note on the subscriptions page: "If you're using an API-scoped subscription, an all-APIs subscription, or the built-in all-access subscription, policies configured at the product scope aren't applied to requests from that subscription."

On the sample, 20 calls to GET /orders with the built-in all-access key all returned 200. The same 20 calls with Contoso's product key: 11 times 200, then 429. The portal's test console uses the all-access key by default, so a product rate limit you hammer from there never fires, and it looks broken when it isn't. Test product policies with a product subscription key from outside the portal.

The count wasn't exactly 10 either; two separate bursts got 11 calls through. The v2 tiers rate-limit with a token bucket, and Learn warns that rate limiting "is never completely accurate". Write "about 10 a minute" into the partner contract, not "the 11th call fails".

Picking a scope

Global runs for every API on the instance, so keep it to what applies everywhere: the correlation ID, response headers that leak your stack, and the <forward-request /> that sends every call on. The product is where the consumer lives, so per-partner limits and quotas go there, and a second product for internal tools can carry different numbers for the same API. The API owns what's true for the backend regardless of caller: routing to orders-func, stripping the key header, later the managed identity from Part 3. Operations are for exceptions, and each one is another place where a missing <base /> switches the scopes above it off.

Policies the order API needs

Most of what the order API needs is true no matter who calls it, so it goes into api-orders.xml:

<policies>
  <inbound>
    <base />
    <ip-filter action="forbid">
      <address-range from="203.0.113.0" to="203.0.113.255" />
    </ip-filter>
    <set-backend-service backend-id="orders-func" />
    <set-header name="Ocp-Apim-Subscription-Key" exists-action="delete" />
    <set-header name="x-partner" exists-action="override">
      <value>@(context.Subscription?.Id ?? "none")</value>
    </set-header>
    <choose>
      <when condition="@(context.Request.Method == "POST")">
        <rate-limit-by-key calls="3" renewal-period="60"
            counter-key="@("create-order:" + context.Subscription.Id)" />
      </when>
    </choose>
  </inbound>
  <backend>
    <base />
  </backend>
  <outbound>
    <base />
  </outbound>
  <on-error>
    <base />
  </on-error>
</policies>

All of it was deployed through Bicep to the Basic v2 instance, which doubles as the schema check: the management API compiles every expression when it saves a policy. The status codes below are the ones that instance returned.

A second limit for placing orders

Reading an order is cheap; placing one writes to the store and, in a real system, starts fulfillment. So POST /orders gets its own limit of 3 a minute per partner, on top of the product's 10. That can't be a second rate-limit: Learn says the policy "can be used only once per policy definition", and the product's is already part of every partner call. rate-limit-by-key counts per key, and the key is whatever string you compute:

<rate-limit-by-key calls="3" renewal-period="60"
    counter-key="@("create-order:" + context.Subscription.Id)" />

For Contoso the key evaluates to create-order:contoso-shop. The prefix is there because counters belong to the key, not to the policy: every rate-limit-by-key that produces the same string, at any scope, draws from the same bucket. Key a second limit on the bare subscription ID somewhere else and the two quietly share one counter.

Contoso placed five orders in a row. Four came back 201 Created, the fifth got 429 with Retry-After: 19; the token bucket let one extra call through, as it did for the product limit. Two GET /orders right after the 429 returned 200, because the limit only runs for POST. Fabrikam, a second partner subscription on the same product, placed an order at the same moment and got 201: its key is create-order:fabrikam.

Plain rate-limit counts per subscription only, but it runs on every tier. rate-limit-by-key can count per anything (the subscription, context.Request.IpAddress for anonymous APIs, a claim, a tenant header) and also works at global scope, but not on Consumption, and you find out at save time: the management API refused it on a Consumption instance with Policy is not allowed in 'Consumption' sku. From this part on, the sample deploys Basic v2 by default.

Caching GET /orders/{orderId}

My first version of the operation policy cached with vary-by-developer="true", which reads like "one cache per partner". It isn't. Learn defines it as caching "per developer account that owns [the] subscription key". Contoso's subscription comes from Bicep, and I added Fabrikam's on the same product through the management API. Neither has an owner, so to the cache they are the same developer.

On the live instance Contoso read an order, a cache miss with a fresh correlation ID. Fabrikam read the same order a second later and got 200 with Contoso's cached response, correlation ID included. The sample returns the same order to everyone, so nothing leaked here. In an API that filters by caller, Fabrikam would have been reading Contoso's data, and nothing in the response would have told either of them. Partners who sign up through the developer portal own their subscriptions, so this stays hidden until the day subscriptions come from a pipeline, which is where Parts 5 and 6 are going.

The fix keys the cache on the subscription itself, through the x-partner header the API policy sets:

<cache-lookup vary-by-developer="false" vary-by-developer-groups="false">
  <vary-by-header>x-partner</vary-by-header>
</cache-lookup>

<base /> comes first in the operation's inbound, so the API's set-header has already written x-partner when the lookup builds its key. Varying on Ocp-Apim-Subscription-Key instead would put the secret into the cache key, and it wouldn't work here anyway: the API policy deletes that header before the lookup runs. Because x-partner is set with exists-action="override", a caller can't send its own value to borrow someone else's entry.

After the redeploy, Contoso's first read was a miss and the next three were hits (same correlation ID, 0.21 to 0.25 seconds instead of 0.52). Fabrikam's first read was a miss with its own correlation ID. Fabrikam sending x-partner: contoso-shop got Fabrikam's entry back.

Only GET requests are cached, and only 200 OK responses unless you set cache-response="true" on cache-store. A hit returns the stored headers too, so per-request tracing headers such as the correlation ID are stale on cached responses. On Consumption the cache policies save, but there's no built-in cache behind them.

Headers

The global outbound returns the correlation ID and removes a header that gives away the stack:

<outbound>
  <set-header name="x-correlation-id" exists-action="override">
    <value>@(context.Request.Headers.GetValueOrDefault("x-correlation-id", ""))</value>
  </set-header>
  <set-header name="X-Powered-By" exists-action="delete" />
</outbound>

There is no remove-header policy. Removing a header is set-header with exists-action="delete"; the other values are override (the default), skip and append. In inbound the statement changes the request the backend receives, in outbound the response the caller receives. The function doesn't send X-Powered-By, but ASP.NET on IIS does, which is why the line sits at global scope, ready for the next backend.

Called directly, the function answered with a Server header; through the gateway it didn't, with no policy involved. The Application Insights Request-Context header went the other way: it survived a delete, because the gateway adds it after outbound has run. A policy can only remove what's there when it runs.

Blocking callers with ip-filter

<ip-filter action="forbid">
  <address-range from="203.0.113.0" to="203.0.113.255" />
</ip-filter>

forbid blocks the listed addresses and lets everything else through. allow is the opposite: anything not listed gets 403 Forbidden. The sample forbids a documentation range (203.0.113.0/24 is reserved for examples) so it works for whoever deploys it; replace it with what you want to block, or switch to allow with your partners' ranges.

Switching the sample to allow with that range locked me out within 20 seconds. The reverse test, a forbid with my public address as a lookup service reported it, changed nothing for five minutes: the gateway saw a different address. Before you write an allow-list, echo context.Request.IpAddress into a response header from a test policy and use what the gateway reports. Behind Azure Front Door or any other proxy, the gateway sees the proxy, not the partner; What Each Hop Trusts covers which address each hop can rely on.

Policy expressions: C# inside the XML

These are the expressions in the sample:

<value>@(context.RequestId.ToString())</value>
<value>@(context.Subscription?.Id ?? "none")</value>
<when condition="@(context.Request.Method == "POST")">
counter-key="@("create-order:" + context.Subscription.Id)"

@(...) holds one C# expression, and its value becomes the attribute value or the element text. Most attributes accept one; the reference page says when an attribute doesn't (rate-limit takes none in calls or renewal-period). Everything the gateway knows about the call is on context: the request and response, the subscription, product and user, the API and operation, your own context.Variables, and context.LastError in on-error. Subscription is null on APIs that don't require a key, hence the ?..

@{ ... } holds a block of statements, and every path through it has to end in return. This one, tried on the live instance but not in the sample, forwards the customer ID from the order body as a header:

<set-header name="x-customer-id" exists-action="override">
  <value>@{
    if (context.Request.Method != "POST") { return ""; }
    var order = context.Request.Body.As<JObject>(preserveContent: true);
    return (string)order["customerId"] ?? "";
  }</value>
</set-header>

Reading the body consumes it. With preserveContent: false the header still came out right (C-2001), but the function received an empty body and answered 500. The XML saved without complaint; only the call failed.

The language is C# 7, not the C# 14 the function is written in, and only an allow-listed set of .NET types is available: System.Linq, System.Text.RegularExpressions, Newtonsoft.Json.Linq (where JObject comes from) and a few dozen more, listed on the policy expressions page.

What the save checks, and what it doesn't

The save compiles the expressions. A typo in the <choose> condition (context.Request.Methd) got a 400 ValidationError back: "'IRequest' does not contain a definition for 'Methd'".

An expression that compiles but throws gets through. @(context.Request.Headers["x-tenant"][0]) saved fine. Every call without an x-tenant header then returned 500 Internal server error with no correlation ID; calls with the header returned 200. Learn calls the save-time check "limited verification", and any exception at runtime goes to on-error. Use GetValueOrDefault on headers and ?. on anything that can be missing, and test the paths where the input is absent.

Conditions with <choose>

<choose>
  <when condition="@(context.Request.Method == "POST")">
    <rate-limit-by-key calls="3" renewal-period="60"
        counter-key="@("create-order:" + context.Subscription.Id)" />
  </when>
</choose>

<choose> takes one or more <when> elements and an optional <otherwise>. The gateway runs the first when whose condition is true and skips the rest, so put the narrow cases first. If nothing matches and there's no otherwise, nothing happens. The body can hold any policy allowed in that section, including another <choose>, and it works in all four sections, at every scope, on every tier.

The same limit could have gone into an operation policy on create-order. That adds one more operation-scope file, with its own <base /> in every section to get right. The <choose> keeps the rule in the API policy, next to the routing it belongs with.

Conclusion

The management API refused two things in this part: a misspelled property and a policy the tier doesn't support. Everything else saved, including an inbound without <base /> that switched off the partner limit, a vary-by-developer="true" that shared one cache entry between two partners, and a header lookup that turned every call without the header into a 500. The save checks syntax and compiles expressions; it doesn't know who owns a subscription, which address the gateway sees, or what the gateway adds after outbound. A call with each partner's key, read for status codes and headers, does. Part 3 builds on these files: validate-jwt in front of the order API and a managed identity to the function instead of the key in a named value.

Do your partner subscriptions have an owner in API Management, or does a pipeline create them?