The catalog presents hundreds of models behind one Deploy button, and that button leads to two entirely different products with different billing, different quota and different failure modes.
Two deployment paths, two bills
Microsoft documents the catalog as offering managed compute and serverless deployments, and describes serverless as the preferred and most capable path.
- Serverless API deployment. The model runs on Microsoft’s infrastructure and you consume it as an API without hosting anything. Billing is per token. Microsoft states these are regional deployments, and that this path carries the standard, provisioned throughput, batch and developer deployment types — the same family as an Azure OpenAI deployment.
- Managed compute. A managed GPU platform-as-a-service that hosts open-source and custom-weight models on dedicated GPU capacity. You need compute quota in your subscription, and Microsoft states billing is per compute uptime — not per token. An idle endpoint bills exactly as much as a busy one.
Which one a given model offers is chosen by the model provider, not by you. A model available only on managed compute is a model whose minimum monthly cost is set by a GPU SKU, and that is the number to work out before deploying it rather than after the first invoice.
Serverless API deployment
For models sold directly by Azure, this is the same control-plane object as an Azure OpenAI deployment — a Microsoft.CognitiveServices/accounts/deployments child resource on your Foundry resource, with a SKU naming the deployment type and a capacity.
az cognitiveservices account deployment create \
--resource-group rg-inference \
--name mg-foundry-weu \
--deployment-name llama-33-70b \
--model-name Llama-3.3-70B-Instruct \
--model-format Meta \
--model-version "1" \
--sku-name "GlobalStandard" \
--sku-capacity 50
--model-format is the trap. It is OpenAI for OpenAI models and the publisher’s name for others, and getting it wrong produces a model-not-found error that reads as a region-availability problem. The exact format, name and version triple for any given model is shown on its card in the catalog; copy it rather than typing it.
For models from partners and community rather than sold directly by Azure, the deployment may instead be a marketplace subscription plus a serverless endpoint, and Microsoft states that these models do not support quota increase requests. That is worth knowing before you build a production path onto one.
Calling it
A Foundry resource exposes a model-agnostic inference route, which is the point of deploying a non-OpenAI model here rather than to its own provider: one request shape across the catalog. The Azure AI Inference SDK targets it directly.
from azure.ai.inference import ChatCompletionsClient
from azure.ai.inference.models import SystemMessage, UserMessage
from azure.identity import DefaultAzureCredential
client = ChatCompletionsClient(
endpoint="https://mg-foundry-weu.services.ai.azure.com/models",
credential=DefaultAzureCredential(),
credential_scopes=["https://cognitiveservices.azure.com/.default"],
api_version="2024-05-01-preview",
)
response = client.complete(
model="llama-33-70b", # the deployment name
messages=[
SystemMessage("Answer in one sentence."),
UserMessage("What is a serverless API deployment?"),
],
max_tokens=200,
)
print(response.choices[0].message.content)
print(response.usage.prompt_tokens, response.usage.completion_tokens)
As with Azure OpenAI, model is the deployment name rather than the model name. The token counts come back in the same usage shape regardless of publisher, which is the practical benefit — cost accounting does not need a per-model branch.
Capability, however, is not normalised. A catalog model may not support tool calling, structured outputs or streaming even though the request shape accepts the parameters. Check the model card before assuming a feature transfers.
Managed compute
Managed compute exists for the models that cannot be served any other way: open-weight models with no hosted offering, and your own fine-tuned or custom-weight checkpoints. Microsoft documents it as a preview GPU platform-as-a-service on dedicated capacity.
Three things behave differently and all three catch people out. The endpoint takes minutes to come up rather than seconds, so it is not something to create on demand. It bills for uptime, so the cost model is a virtual machine’s and scaling to zero is the only way to stop paying. And it draws on your subscription’s GPU compute quota, which is a completely separate allocation from the token quota everything else on this page consumes.
The instance type is also a per-model constraint rather than a free choice. A model’s weights have to fit in the accelerator memory of whatever SKU you pick, so a large open-weight model rules out the smaller GPU families entirely and the catalog card, not the portal dropdown, is where the supported list lives. Pick the SKU first and check quota for that specific virtual machine family in that specific region, because GPU quota is granted per family — headroom on one series tells you nothing about another.
Two consequences follow for anything user-facing. Scaling to zero reintroduces a cold start measured in minutes, not the seconds a serverless function takes, which makes it a batch pattern rather than an interactive one. And a deleted endpoint stops the bill while a stopped one may not, so “we turned it off over the weekend” is worth verifying against Cost Management rather than assuming.
Two quota systems
That last point deserves stating on its own, because it is the most common way a catalog deployment fails for a reason the error does not explain.
- Serverless consumes tokens-per-minute quota, scoped per subscription, per region, per model, per deployment type — the same pool described on the quota page, and requestable through the same form.
- Managed compute consumes GPU vCPU quota for a virtual machine family in a region, requested through Azure’s standard compute quota process. Nothing about your token quota affects it.
A managed-compute deployment failing on quota is a compute-quota request, not a Foundry one, and sending it to the wrong queue costs days. Read which path the model took before you read the error.
The deployment name is the interface
Everything above converges on one operational habit. The model name, the version, the publisher and the deployment type all live in the deployment object; the only thing your application knows is the deployment name. Treat that name as the contract and the rest becomes cheap to change.
Concretely, that means model upgrades are a deployment exercise rather than a release. Create a second deployment of the new model under a new name, point a fraction of traffic at it by configuration, compare on your own evaluation set, then move the rest. Nothing is rebuilt and the rollback is the configuration value you changed. The alternative — upgrading in place — replaces a model under live traffic with no comparison and no way back that does not involve a redeployment.
It also means names should describe the role rather than the model. chat-default and classify-cheap survive three model generations; llama-33-70b is accurate today and misleading the moment the deployment behind it changes, which is exactly what you want to be able to do without touching code.
The reason this matters more on the catalog than on Azure OpenAI alone is retirement. Catalog models arrive and leave on their publishers’ schedules rather than Microsoft’s, and a partner model can stop being offered with far less notice than an Azure OpenAI model gets. The indirection is what makes that a configuration change instead of an incident.
Serving a Llama model from a Foundry resource and an OpenAI model from an Azure OpenAI resource means two endpoints, two SDK clients and two sets of capability caveats even though both are “Azure”. Whether that consolidation belongs in a Foundry resource, in a gateway, or in your own client wrapper is a real architectural choice — Multigrid is a gateway that takes that position across providers rather than within one cloud, which is the right comparison to make before adding either.
Top comments (0)