karleeov

Posted on Aug 7 • Originally published at studio.resurrects.co

Building a Production AI Agent App on Azure in 2026

#azure #dotnet #ai #devtools

Building a Production AI Agent App on Azure in 2026: .NET Aspire, AI Foundry, and Container Apps

Azure shipped a major upgrade for AI app developers in 2026, and the pieces line up in a way that's worth a deep look. .NET Aspire 9.2 went GA, Azure AI Foundry's agent service became generally available, and Azure Container Apps gained native .NET Aspire support. Put together, those three releases mean you can now build, test, and ship a multi-agent AI application on Azure without leaving the .NET ecosystem. This post walks through how the pieces fit and what a real production stack looks like.

The mental model: three layers

Modern Azure AI apps break naturally into three layers:

Composition layer — .NET Aspire, which models your distributed app (agents, APIs, queues, storage) as a single declarative graph.
Intelligence layer — Azure AI Foundry, which hosts your models, fine-tuning, and now agent orchestration via the Microsoft Agent Framework.
Hosting layer — Azure Container Apps, which runs the composed graph with autoscaling, revisions, and built-in ingress.

Each layer has a clear job. When they ship in lockstep — as they did this year — you stop glue-coding and start shipping.

Layer 1: .NET Aspire 9.2

.NET Aspire is the opinionated stack for cloud-native .NET. The 9.2 release brought three things that matter for AI apps:

Production dashboard GA — what used to be a dev-time visualizer is now a hardened observability surface suitable for staging and limited production use.
New AppHost APIs — multi-resource composition got cleaner. You can now express an agent, its dependencies, and its replica strategy in one place.
AI workload telemetry — first-class spans for LLM calls, vector searches, and tool invocations flow into your existing OpenTelemetry pipeline.

A typical Aspire AppHost for an AI agent looks like this:

var builder = DistributedApplication.CreateBuilder(args);

var chatModel = builder.AddAzureAIFoundry("foundry")
    .WithEndpoint("https://your-foundry.services.ai.azure.com")
    .WithDeployment("gpt-4o");

var agentApi = builder.AddProject<Projects.AgentApi>("agent-api")
    .WithReference(chatModel)
    .WithEnvironment("OTEL_EXPORTER_OTLP_ENDPOINT", "http+http://aspire-dashboard");

builder.AddNpmApp("web-ui", "../web")
    .WithReference(agentApi);

builder.Build().Run();

That's the whole distributed app declaration. The dashboard, health checks, and tracing come for free.

Layer 2: Azure AI Foundry agent service

Foundry is where your models live. The 2026 update put the agent service at GA and shipped the Microsoft Agent Framework as the canonical multi-agent orchestrator. Four orchestration patterns are first-class:

Group chat — agents collaborate on a shared thread.
Sequential — output of one agent feeds the next.
Concurrent — fan-out then aggregate.
Handoff — one agent delegates to a specialist.

For production workloads, the most interesting piece is Workflows: durable, long-running execution with checkpointing. If your agent crashes mid-task, it resumes from the last checkpoint — not from scratch. That's the difference between a demo and a product.

A minimal Foundry agent definition in C#:

var agent = new ChatAgent(
    chatClient: foundryChatClient,
    instructions: "You triage incoming support tickets and route them to the right team.",
    tools: [new LookupCustomerTool(), new CreateTicketTool()])
    .WithMiddleware(new LoggingMiddleware(), new RetryMiddleware(3));

var response = await agent.RunAsync(
    new ThreadMessage("user", "My order #8842 hasn't arrived."),
    threadId: existingThreadId);

The RunAsync call hands off to the Foundry runtime. State persists across calls, tools execute in Azure (you can run them as Container Apps jobs), and every step is traced.

Layer 3: Azure Container Apps

Container Apps is the runtime that ties everything together. The 2026 update added native .NET Aspire deployment — azd up from inside an Aspire project deploys the entire graph as a set of Container Apps revisions.

What you get:

Auto-scaling on AI metrics — scale on queue depth, token throughput, or custom Foundry-emitted metrics.
Cold-start under 2 seconds — the new image-pull pipeline with cached layers matters a lot for agents that scale to zero between requests.
Integrated ingress — public endpoints, mTLS between services, no separate API gateway required.
Per-revision traffic splitting — canary a new agent version to 5% of traffic.

A deployment command:

azd auth login
azd up --environment prod

That command reads azure.yaml at the repo root, provisions the resource group, builds the container images, pushes them to ACR, and creates the Container Apps revisions. End-to-end in about 8 minutes from a fresh checkout.

Production wiring: a real layout

Here's the full stack for a production AI agent app on Azure:

                    ┌────────────────────────┐
   Web UI (SPA) ───▶│   Container Apps: web  │
                    └───────────┬────────────┘
                                │ HTTPS + mTLS
                                ▼
                    ┌────────────────────────┐
                    │ Container Apps: agent  │  ← Aspire-manifested
                    │ (multi-replica)       │
                    └────┬──────────────┬───┘
                         │              │
                Tool calls│              │ LLM calls
                         ▼              ▼
        ┌────────────────────┐ ┌────────────────────┐
        │ Tool jobs (ACA)    │ │ Azure AI Foundry   │
        │ + Cosmos + Service │ │ (Microsoft Agent   │
        │   Bus              │ │  Framework)         │
        └────────────────────┘ └────────────────────┘

Three things worth highlighting:

No API gateway. Container Apps ingress handles public traffic; internal mTLS handles the rest.
Tool calls are first-class Container Apps jobs. Anything your agent can do — query a database, send an email, hit an internal API — runs as a job, not a side-channel.
Foundry is a black box to the agent code. The agent just calls the chat client; Foundry handles model selection, fine-tuning, and evaluation behind a stable API.

Gotchas we hit shipping this

A few things that aren't in the docs yet:

Aspire's WithReference for Foundry requires the endpoint URL, not the resource ID. Don't pass the bicep output symbol; pass the literal https://...services.ai.azure.com URL or the env-var expansion won't resolve.
Workflow checkpointing needs a Cosmos DB or SQL backend. Default in-memory checkpointing loses state on restart. Wire one up before going to production.
Auto-scaling rules need at least 3 minutes of warmup. Don't expect instant scale-up under bursty load. Pre-warm during business hours if your traffic spikes predictably.
The Foundry agent service has a 2 MB tool-result limit per turn. If your agent returns large documents, paginate or summarize before returning.

When to use this stack

Use Aspire + Foundry + Container Apps when:

You're a .NET shop and want to stay in one language.
Your AI app has more than one model call (multi-agent, tool use, RAG).
You need durable, resumable workflows.
You want autoscaling and zero-downtime deploys without Kubernetes.

If you're a Python shop or your app is a single LLM call wrapped in a FastAPI endpoint, this stack is overkill — Azure Functions + AI Foundry is lighter.

DEV Community