Azure OpenAI Pricing: Reading the Deployment Rates

#ai #llm #machinelearning #programming

This page prints no rates. Azure OpenAI prices change on a schedule nobody outside Microsoft controls, and a table of them in an article is a wrong answer with a publication date attached. What is stable is the structure of the price list and the API that serves it, and those are what you need to read a rate correctly.

What you are billed on

Pay-per-token deployment types bill separately on input tokens and output tokens, and output is the dearer of the two on essentially every model. That asymmetry is not a margin decision — generation is sequential and memory-bandwidth-bound while prompt processing is a single parallel pass, so the two operations genuinely cost different amounts to serve.

Beyond the per-token rate, three documented mechanisms change what you pay:

Batch is half. Microsoft documents Global Batch and Data Zone Batch as processing asynchronous request files with a 24-hour target turnaround at 50% less cost than Global Standard, with a separate enqueued-token quota that does not disturb online workloads. If work tolerates a day, this is the single largest lever available. Microsoft, Understanding deployment types in Foundry Models.
Provisioned is not per token at all. Provisioned deployment types bill reserved capacity in provisioned throughput units rather than consumption, so the cost question becomes utilisation rather than volume. See provisioned throughput.
Priority processing is a paid tier of the same deployment. Microsoft documents Global Standard and Data Zone Standard as supporting priority processing for faster response times on a pay-as-you-go basis — a separate rate on the same deployment type.

Three axes, not one price

A rate is the intersection of three things, and quoting any one of them alone is meaningless:

The model. Obviously. Note that model families are priced independently of each other; a “mini” is not a fixed fraction of its larger sibling.
The deployment type. Global Standard, Data Zone Standard, Standard, the Batch variants and the Provisioned variants are different SKUs with different rates. Microsoft names Global Standard as having the lowest price of the standard family and the broadest region coverage.
The region. Rates are per armRegionName in the price data. Two deployments of one model in two regions are two meters.

The pricing page is organised by model, which makes the other two axes easy to skim past. If you are comparing a quote to an invoice and the numbers disagree, the deployment type is the first thing to check.

Getting today’s number

Microsoft publishes the Azure Retail Prices API, unauthenticated, at https://prices.azure.com/api/retail/prices. It is the primary source, it is the same data the pricing page renders, and it is scriptable. Filter on the service and the region:

curl -sG "https://prices.azure.com/api/retail/prices" \
  --data-urlencode "api-version=2023-01-01-preview" \
  --data-urlencode "\$filter=serviceName eq 'Cognitive Services' and armRegionName eq 'swedencentral'" \
  | jq '.Items[] | {meterName, skuName, retailPrice, unitOfMeasure, effectiveStartDate}'

Four documented properties of that API change how you use it:

Filter values are case sensitive from api-version 2023-01-01-preview onwards. Earlier versions accepted virtual machines as well as Virtual Machines; the current one does not, and a case error returns an empty result set rather than an error.
Pagination is 1,000 records per response, with a NextPageLink at the end. A filter broad enough to be useful is usually broad enough to paginate, and code that reads only the first page silently misses meters.
Prices are USD unless you ask otherwise. Microsoft states that the currency used to price all Azure services is USD, that USD prices are the Microsoft retail prices, and that other currencies returned by the API are “for your reference to help you estimate budget expenses”. Do not treat a currencyCode=EUR figure as the number that will appear on the invoice.
effectiveStartDate is your date stamp. It is the date the retail price became effective, which is exactly what you want to record next to any figure you quote internally.

Filterable fields include armRegionName, meterName, skuName, productName, serviceName, serviceFamily and priceType. Narrow with priceType eq 'Consumption' to drop reservation meters, or the reverse to see only them.

The unit of measure trap

retailPrice is meaningless without unitOfMeasure, and this is where cost models go wrong. The API returns the unit as a string — 1 Hour, 1M Tokens, 1K Tokens and so on — and the unit is not uniform across meters, across models, or necessarily stable over time for one meter. A spreadsheet that hardcodes “per million” is off by a factor of a thousand the first time it meets a meter priced per thousand.

The arithmetic, with its assumptions stated:

# Given one meter's retailPrice R and unitOfMeasure "<N> <UNIT>",
# the cost of T tokens against that meter is:
#
#     cost = (T / N) * R
#
# where N is parsed from the unitOfMeasure string, not assumed.
# Input and output are separate meters, so:
#
#     total = (T_in  / N_in ) * R_in
#           + (T_out / N_out) * R_out
#
# T_in and T_out come from the response's usage object
# (prompt_tokens and completion_tokens), not from your own count.

Take the token counts from the API response rather than tokenising yourself. Your tokeniser and the service’s billed count agree until they do not — a different tokeniser version, a system message the service adds, or a multimodal input all break the correspondence.

Billed tokens are not rate-limited tokens

This is the most useful non-obvious fact in the territory and it is documented in plain language. Microsoft states that TPM rate limits are based on the maximum number of tokens estimated to be processed at the time a request is received, and that it is not the same as the token count used for billing, which is computed after all processing completes.

The rate-limit estimate is built from the prompt text and count, the max_tokens setting and the best_of setting. So a request with max_tokens of 4,000 that returns 200 tokens is billed for roughly 200 output tokens and consumes roughly 4,000 tokens of your per-minute allowance. Microsoft also notes that requests rejected for input length can still count toward rate limiting while never appearing in billed token metrics.

Two operational consequences. Your Azure Monitor token metrics will not reconcile with your 429 rate, and they are not supposed to — use metrics for spend and response headers for throttling. And setting max_tokens tightly is a throughput optimisation before it is a cost one. The 429 page works through the rest of that accounting.

Reconciling a per-provider price list against per-request usage is irritating in one vendor and genuinely hard across several, because each returns its token counts in a differently named field and prices in a differently sized unit. That normalisation — one usage shape, one currency, per request rather than per monthly invoice — is a large part of what an LLM gateway is doing when it sits in the request path, and it is the layer at which per-feature or per-customer cost becomes answerable at all.

No rate is stated on this page on purpose. The batch discount, the deployment-type list and the API behaviours described here are Microsoft’s documented state at the time of writing; treat the Retail Prices API response, with its effectiveStartDate, as the only figure worth quoting.