A local MCP server is a
dotnet run. A production one is a stateless HTTP service that also holds long-lived streams, gets hammered at 3,200 requests a second, deploys without dropping a call, and never leaks a connection string. That's an infrastructure problem, and on Azure it has a specific — and mostly pleasant — answer, plus one gotcha that will bite you exactly once.
This is Part 13 of a 15-part deep dive on Model Context Protocol (MCP). Part 12 built the server in .NET; now we run it. The three servers — mattrx-analytics, mattrx-reports, mattrx-admin — live on Azure Container Apps behind a gateway.
TL;DR
| Concern | Naive hosting (before) | Azure at scale (after) |
|---|---|---|
| Compute | fixed VM / always-on App Service | Container Apps (serverless, revisions) |
| Scaling | fixed replicas | KEDA HTTP autoscale (+ scale-to-zero dev) |
| Edge | servers exposed directly | Front Door / APIM gateway |
| Streaming | SSE dies behind the LB | affinity + keepalive + raised timeout |
| Deploys | in-place, drops calls | revisions + canary + readiness |
| Secrets / network | connection strings, public | managed identity, Key Vault, private |
The Azure topology
Agents / clients
|
[ Azure Front Door / APIM ] <- org MCP gateway
TLS · Entra pre-check · rate-limit · route by path
|
+-----------------+------------------+
| | |
v v v
Container Apps environment (managed identity, internal ingress)
mattrx-analytics mattrx-reports mattrx-admin
min 2 / max 30 (-> Service Bus) (locked)
| | |
+--------+--------+---------+--------+
| |
v v
Azure SQL (private) Key Vault · Service Bus · App Insights
1. Compute: Azure Container Apps
Each MCP server is an Azure Container App — a serverless container with built-in ingress, revisions, and KEDA autoscaling, in a shared environment.
resource analytics 'Microsoft.App/containerApps@2024-03-01' = {
name: 'mattrx-analytics'
identity: { type: 'SystemAssigned' } // managed identity (section 6)
properties: {
managedEnvironmentId: env.id
configuration: {
ingress: { external: false, targetPort: 8080, transport: 'http', allowInsecure: false }
}
template: {
containers: [ { name: 'server', image: 'mattrxacr.azurecr.io/mcp-analytics:2.4.0' } ]
scale: { minReplicas: 2, maxReplicas: 30 }
}
}
}
Three servers, three apps, one environment: mattrx-analytics scales on read load, mattrx-reports barely scales (it just enqueues), mattrx-admin stays tiny. One shared plan could never balance those three.
2. Autoscaling with KEDA
A KEDA HTTP scale rule adds replicas by concurrency, with a warm floor and a hard ceiling — and scale-to-zero for non-prod.
scale: {
minReplicas: 2 // keep warm; AOT (Part 12) makes even cold starts fast
maxReplicas: 30
rules: [ {
name: 'http-concurrency'
http: { metadata: { concurrentRequests: '100' } } // +1 replica per ~100 concurrent requests
} ]
}
// Dev / preview environment: minReplicas: 0 -> scale-to-zero, $0 when idle.
MCP load is bursty — campaigns end on the hour, agents fan out — so scale by concurrency, not CPU. Keep a warm floor so the first request of a burst isn't a cold start, cap maxReplicas so a runaway agent can't scale you into a surprise bill, and set minReplicas: 0 in dev to pay nothing when idle. The analytics server sits at 2 replicas overnight and scales toward ~30 at the ~3,200 rps peak, serving read-tool p95 120 ms throughout.
3. The gateway in front
The org gateway — Azure Front Door or API Management — sits in front of the environment as the one public, governed edge.
Agents ---> [ Azure Front Door / APIM ] (the org MCP gateway)
- TLS termination
- Entra token pre-validation
- rate-limit per tenant / connector
- route by path: /analytics -> mattrx-analytics
/reports -> mattrx-reports
/admin -> mattrx-admin
|
v
Container Apps environment (internal ingress — servers not public)
Because it fronts everything, the servers keep internal ingress and stay identical — no server re-implements the edge. Rate-limits at the edge are what stop a runaway agent from turning autoscaling into a cost spike.
4. The SSE gotcha — streaming behind the load balancer
A streaming tool works perfectly on localhost, ships to Azure, and then "randomly" dies in production — connections drop mid-stream, or a stream loses its state.
ingress: {
external: false
targetPort: 8080
transport: 'http' // match what your SSE client expects (HTTP/1.1 chunked)
stickySessions: { affinity: 'sticky' } // pin a stateful SSE session to one replica
}
// + server keepalive pings every ~15s so the ingress never sees the stream as "idle".
This is the number-one Azure MCP surprise. SSE holds a connection open and sends events sparsely — which the Container Apps ingress and Front Door read as idle and reap, and load-balance mid-stream to a replica that doesn't hold the session. It only fails in Azure, never locally, so it ambushes you in prod. The fix is three settings: raise the ingress idle timeout, turn on session affinity for stateful streams, and keepalive-ping from the server. Miss any one and streaming breaks intermittently.
5. Zero-downtime deploys with revisions
Container Apps revisions give you blue-green and canary — a new revision must pass its readiness probe before it takes any traffic, and you shift weight gradually.
# Roll out a new revision; canary 10%, watch, then shift to 100% — no dropped calls.
az containerapp update -n mattrx-analytics --image mattrxacr.azurecr.io/mcp-analytics:2.5.0
az containerapp ingress traffic set -n mattrx-analytics \
--revision-weight latest=10 mcp-analytics--2-4-0=90 # 10% canary on the new revision
// Readiness gates traffic: an unhealthy revision never receives a request (/readyz).
probes: [ { type: 'Readiness', httpGet: { path: '/readyz', port: 8080 } } ]
The new revision boots, its /readyz must pass, then you canary 10% and watch the per-tool metrics before going to 100%. The old revision keeps serving in-flight calls and streams until traffic drains — no restart, no dropped work.
6. Secretless security — managed identity, Key Vault, private
A managed identity authenticates to Entra, Azure SQL, and Service Bus — no connection strings. Any remaining secrets come from Key Vault by reference, and ingress is internal.
identity: { type: 'SystemAssigned' } // -> Entra, Azure SQL, Service Bus, no secrets
secrets: [ { name: 'sb-conn', keyVaultUrl: kv.properties.vaultUri, identity: 'system' } ]
// Azure SQL reached over a PRIVATE ENDPOINT; ingress external:false -> only the gateway can reach it.
The servers authenticate with a managed identity, so there are no connection strings to leak. Secrets that must exist come from Key Vault by reference, Azure SQL sits behind a private endpoint, and internal ingress means only the gateway is reachable from outside. Credential rotation becomes Azure's job, not yours.
The model to carry forward
An MCP server on Azure is a stateless HTTP service that also holds long-lived streams — host it as both. Container Apps gives you elastic, per-server scale, revisions for zero-downtime, and internal ingress; a Front Door / APIM gateway gives you one governed public edge; a managed identity gives you secretless security. And the single thing that will surprise you is SSE behind the load balancer — configure affinity, keepalive, and timeouts, and it disappears.
Three habits for hosting MCP on Azure:
- Scale by concurrency; keep prod warm. KEDA HTTP rules with a warm floor; scale-to-zero only for dev.
- One gateway in front; servers internal. A single public, rate-limited, secretless edge — nothing else reaches the servers.
- Prove streaming in Azure, not localhost. The SSE settings only fail behind the real load balancer, so test them there.
Originally published at prepstack.co.in. Part 14 widens the lens past .NET and Azure: wiring MCP into OpenAI and other agent frameworks.
Top comments (0)