Where Should Your Agent Run? Three Patterns with Microsoft Foundry

A practical comparison of who owns the code, state, deployment, and telemetry.

In my previous articles, I covered individual agents and multi-step workflows with Microsoft Agent Framework. Once the prompt works and the tools return the right data, the next question sounds simple:

Where should this agent actually run?

That decision changes more than deployment. It determines who owns the process, conversation state, scaling, identity, and telemetry. It also determines which problems your team will debug when the agent fails at 2 a.m.

The accompanying demo repository contains three deliberately different patterns:

1. A declarative prompt agent managed by Foundry.
2. Custom Agent Framework code deployed to a Foundry-hosted runtime.
3. The same kind of custom code running on your own infrastructure and exporting telemetry to Foundry.

These are not three copies of the same agent. They are three different ownership choices.

To keep the comparison practical, demos 2 and 3 use the same two C# tools: a weather function and an office-climate function. The user can ask the same question and receive the same deterministic result. What changes is everything around the tool call—how the process starts, where state lives, how it is deployed, and who must make its trace visible. Demo 1 intentionally stays simpler: it has instructions and managed conversation state, but no custom tools.

The runtime spectrum

Think of the options as a spectrum. Moving from left to right gives you more control, but also more operational responsibility.

From a prompt agent to self-hosted code: more infrastructure control also means more to operate. Open in full size

The useful question is not “Which option is best?” It is “Which responsibilities do I actually need to own?”

Pattern 1: let Foundry run a prompt agent

A Foundry prompt agent is a declarative, configuration-driven agent—the closest of these three patterns to a no-code option. In the Foundry portal, you can select a model, write instructions, attach supported tools, and test the agent in the Playground without writing application code or using Microsoft Agent Framework.

That does not make it a toy. You can improve its instructions with the Prompt Optimizer (currently in preview), ground responses in your own documents with file search, and attach tools for search, code execution, APIs, MCP servers, and other actions. Custom function tools are supported too, but they are not fully no-code: your application or another service must execute the function code.

The trade-off is control. You can configure the model, instructions, knowledge, and supported tools, but Foundry owns the orchestration loop and runtime. If you need arbitrary application logic or custom orchestration, a code-based agent is the better fit.

The portal is only one authoring path. The same agent can also be created and versioned through the Foundry SDK or REST API. This demo uses C# so the definition is reproducible in source control, but that code creates a managed Foundry resource—it is not a runtime you host yourself.

The demo creates a small pirate assistant. The persona is not important; the definition is:

ProjectsAgentDefinition definition = new DeclarativeAgentDefinition(modelName)
{
    Instructions = agentInstructions,
};

ProjectsAgentVersion version =
    await aiProjectClient.AgentAdministrationClient.CreateAgentVersionAsync(
        agentName: agentName,
        options: new(definition));

Every update creates a new version under the same agent name. Whether you author the definition in the portal or in code, Foundry runs the agent; your application only connects to it.

The repository does use Microsoft Agent Framework on the client side. AsAIAgent(agentRecord) adapts the managed prompt agent to the common AIAgent API, which the console uses to invoke the agent and resume an AgentSession. It does not turn Demo 1 into a code-hosted agent: Azure.AI.Projects still creates the Foundry resource, and Foundry still owns the runtime.

There is a small detail in this demo that explains the ownership model well. The local file stores only a Foundry conversation ID. The message history remains in the cloud. On restart, the console app reads that ID and reconnects to the same conversation.

You can see both pieces in the repository: the prompt-agent setup and the small conversation ID store.

The prompt agent stays intentionally simple: Foundry runs it, keeps the conversation state, and the agent answers without custom code tools. Open in full size

With project monitoring enabled, Foundry records the prompt agent's server-side trace automatically—no instrumentation code required.

The trace shows the agent invocation, model call, duration, and token usage. Open in full size

Choose this pattern when instructions and service-supported tools describe most of the agent, and you want the smallest runtime surface to operate.

Pattern 2: bring your code, let Foundry host it

The second demo needs custom C# functions. It exposes deterministic weather data and mock office-climate data, then packages that logic as a Foundry hosted agent.

Here, you own the agent code and tools:

AIAgent agent = new AIProjectClient(projectEndpoint, new DefaultAzureCredential())
    .AsAIAgent(
        model: modelName,
        instructions: "Use the appropriate demo tool and keep answers concise.",
        tools:
        [
            AIFunctionFactory.Create(WeatherTools.GetWeather),
            AIFunctionFactory.Create(WeatherTools.GetOfficeClimateStatus)
        ]);

var builder = AgentHost.CreateBuilder(args);
builder.Services.AddFoundryResponses(agent);
builder.RegisterProtocol("responses", endpoints => endpoints.MapFoundryResponses());

AddFoundryResponses connects the agent to the Responses protocol handler, while MapFoundryResponses exposes the /responses endpoint. According to the Agent Framework hosting documentation, Responses is the recommended starting protocol for most conversational agents. Foundry manages conversation history, streaming, and session lifecycle around that endpoint.

What is happening here? The AIAgent still contains normal application logic. AgentHost supplies the hosting boundary, and the Responses adapter supplies the HTTP contract. This separation is useful: the tool methods do not need to know whether the process is running locally or inside Foundry.

Local execution has one extra concern. A request sent directly to localhost does not contain the user and chat isolation context injected by Foundry, so the demo registers a local fallback session-isolation provider. That fallback exists to make direct development calls work; the deployed platform provides the real request context.

The source manifest declares the hosted-agent kind, Responses protocol, CPU and memory, and environment variables. During initialization, azd turns it into deployment configuration that also specifies the .NET runtime, entry point, and remote build. From the demo directory, the intended deployment flow is short:

azd ai agent init -m .\agent.manifest.yaml --deploy-mode code
azd deploy
azd ai agent invoke "Is the AC working in Venya's home office?"

If you prefer a UI, the Microsoft Foundry Toolkit for Visual Studio Code provides a Foundry Toolkit: Deploy Hosted Agent workflow. It supports both code and container deployment and lets you review the settings before deploying, so the azd commands are not the only path.

A useful mental model is serverless containers: you provide code or an image, while Foundry manages the infrastructure and scaling. This demo uses the code path, so azd or the Toolkit uploads the project as a ZIP and Foundry performs the .NET build remotely.

Under the hood, the result still runs in a platform-provided container on Azure Container Apps, inside a per-session VM-isolated sandbox. Idle compute can scale to zero, while session files are restored when the session resumes. The Hosted agents documentation and networking deep dive cover the Container Apps and Micro VM layers in more detail.

The agent code is still yours. The running environment is not. Foundry gives the deployed agent an identity and endpoint and supplies platform configuration such as the project endpoint and Application Insights connection string. Microsoft describes the managed Hosted Agents service as generally available, while this sample still pins preview versions of some .NET integration packages.

The complete hosted-agent program, deployment manifest, and demo commands are kept together in the repository.

The Foundry-hosted Agent Framework agent executes the custom get_office_climate_status C# function and returns the deterministic FAC-1042 result. Open in full size

With project monitoring enabled, the Foundry hosting integration exports OpenTelemetry traces automatically, so this demo needs no separate tracing setup.

The hosted trace also reveals protocol, identity, model, and platform storage spans. Open in full size

Choose this pattern when you need custom code and tools but do not want to operate the agent runtime yourself.

Pattern 3: keep the runtime, export the telemetry

The third demo uses the same weather and office-climate scenario, but the process runs locally—or anywhere you choose to deploy it. That means your team owns process lifetime, scaling, secrets, networking, and state.

It also means telemetry is your responsibility. The demo creates an OpenTelemetry resource with a stable agent identity and exports traces to Azure Monitor:

const string agentId = "local-weather-agent";

var resource = ResourceBuilder.CreateDefault()
    .AddService(agentId)
    .AddAttributes([
        new("gen_ai.agent.id", agentId),
        new("gen_ai.agent.name", agentId),
    ]);

var tracerProvider = Sdk.CreateTracerProviderBuilder()
    .SetResourceBuilder(resource)
    .AddSource("Experimental.Microsoft.Agents.AI")
    .AddSource("Experimental.Microsoft.Extensions.AI")
    .AddAzureMonitorTraceExporter(options =>
        options.ConnectionString = applicationInsightsConnectionString)
    .Build();

Your process sends OpenTelemetry traces directly to Application Insights. Registering it as an external agent does not deploy or host the code; it simply tells Foundry which traces belong to that agent. The key link is a stable ID: the gen_ai.agent.id emitted by the runtime must match the ID used during registration.

The self-hosted demo shows the exporter, stable agent ID, two tools, and local conversation loop in one file.

The self-hosted trace separates the agent run, two model calls, and the office-climate tool execution. Open in full size

Here, execute_tool is a separate span. Foundry shows which function ran, how long it took, and—when sensitive-data capture is enabled—its input and output.

Choose this pattern when infrastructure control, private networking, portability, or a custom hosting platform matters more than managed operations.

State is not one thing

“Foundry manages state” is an easy phrase to overread. In practice, there are at least three different concerns:

• Conversation history: the messages exchanged with the agent.
• Agent session: the technical context used to continue that conversation.
• Business state: orders, approvals, user preferences, workflow checkpoints, and anything else your application must preserve.

In the prompt-agent demo, Foundry stores the conversation while the console application saves only its ID. With Foundry-hosted code, the Responses protocol manages conversational history and session lifecycle around the endpoint. Neither pattern turns Foundry into a database for your domain model: durable business state still belongs in your application and data store.

In the self-hosted demo, the current AgentSession exists inside the running process. If that process restarts, persistence and recovery are your responsibility. This leads to a more useful design question than simply asking whether a platform “has state”: what must survive a process restart, deployment, or regional failure?

The decision in one table

	Prompt agent	Foundry-hosted code	Self-hosted code
Best fit	Mostly declarative behavior	Custom code, managed runtime	Maximum infrastructure control
Runtime owner	Foundry	Foundry	Your team
Deployment unit	Agent definition	Code package/container	Your application
Conversation state	Foundry-managed	Protocol/platform-managed	Your implementation
Telemetry	Platform-managed	Hosting integration	You configure export and identity

The middle option is often the practical default for a code-based agent: you keep normal application code without immediately taking ownership of another production service. But the table is not a maturity ladder. Self-hosting is not “more advanced,” and a prompt agent is not “just a demo.” Each option optimizes for a different ownership boundary.

Demo notes

The weather and office-climate tools return deterministic fake data. The self-hosted demo also enables sensitive telemetry content so prompts and tool results are easier to inspect. Keep that setting disabled by default unless your privacy and retention requirements allow it.

Conclusion

Choosing an agent runtime is really choosing an ownership boundary.

• Use a prompt agent when the definition is the product.
• Use a Foundry-hosted agent when custom code is the product but operating the runtime is not.
• Self-host when you need to own the environment—and accept responsibility for state, deployment, and telemetry.

The durable idea is simple: choose who owns the runtime, then keep a consistent telemetry contract.