Code Interpreter Internals in Microsoft Foundry: What Actually Happens Inside That Sandbox

Day 8 of Microsoft Foundry: 100 Days / 100 Blogs — a daily deep dive into the Foundry ecosystem for developers building production AI systems.

Your agent just told a user "I calculated the standard deviation of your Q3 revenue and it's $42,318." How? It didn't call a math library you wrote. It didn't hallucinate a number and hope. Somewhere between the model's token stream and that answer, a real Python interpreter spun up in a container you don't manage, ran actual code, and returned a real result.

That's Code Interpreter — one of the most misunderstood tools in Microsoft Foundry's Agent Service. Most tutorials show you a five-line snippet that uploads a CSV and gets a bar chart back. What they don't show you is what's actually happening in between: container provisioning, session lifecycle, file staging through Azure Storage, execution isolation, and the failure modes that will bite you the first time you put this in front of real users at real scale.

This post goes under the hood.

Table of Contents

1. The Problem Code Interpreter Solves
2. Core Concepts: Tool, Container, Session
3. Architecture: How a Request Actually Flows
4. Auto Containers vs. Explicit Containers
5. The File Lifecycle: Upload → Execute → Citation → Download
6. Prompt Agents vs. Hosted Agents: Two Very Different Execution Models
7. Implementation Walkthrough (Python)
8. What Happens at Runtime, Step by Step
9. Security Considerations
10. Performance, Scalability, and Session Economics
11. Cost Considerations
12. Common Mistakes and Pitfalls
13. Alternatives and Trade-offs
14. Practical Recommendations
15. Conclusion
16. References

The Problem Code Interpreter Solves

LLMs are terrible at arithmetic, terrible at exact string manipulation over large datasets, and terrible at anything requiring deterministic, verifiable execution. Ask GPT-5-class models to compute the factorial of 100 by "reasoning it out" in tokens and you'll get a plausible-looking but wrong number more often than you'd like. Ask them to filter a 50,000-row CSV by three conditions and aggregate a column, and you're rolling dice on hallucinated row values.

The fix predates Foundry — OpenAI shipped it as "Code Interpreter" for ChatGPT, and the same pattern shows up as "the local sandbox tool" in Anthropic's Claude, in Gemini, and now natively wired into Microsoft Foundry's Agent Service. The idea is simple in principle and hard in implementation: give the model a real, isolated Python runtime it can write to, execute in, read output from, and iterate against — without giving it network access to your infrastructure or persistent state across unrelated conversations.

What makes the Foundry implementation worth understanding at a systems level is how it wires that sandbox into the rest of the agent runtime — the toolbox model, the container lifecycle, the annotation-based file citation mechanism, and the divergent paths for prompt agents versus hosted agents built on the Microsoft Agent Framework.

Core Concepts: Tool, Container, Session

Three primitives matter here, and conflating them is the source of most confusion:

• The Code Interpreter tool — a tool definition (CodeInterpreterTool in the Python SDK, CodeInterpreterToolboxTool when attached via a toolbox) that you attach to an agent definition. This is a declaration of capability, not a running process.
• The container — the actual sandboxed execution environment. Foundry provisions this lazily, on first use, and associates it with either an explicit ID you manage or an automatically-managed lifecycle tied to your conversation.
• The session — the billable, time-bounded lifetime of a container. A session is active for up to one hour by default, with a 30-minute idle timeout that tears it down early if nothing happens. If your agent calls Code Interpreter from two separate concurrent conversations, that's two separate container sessions — full isolation, no shared state, no shared filesystem.

This distinction matters because it dictates cost and correctness. If you're building a data-analysis agent that a user comes back to three separate times over an hour with follow-up questions ("now filter by region," "now compute the median"), you want those calls landing on the same container session so previously-loaded dataframes and generated intermediate files persist. If you're not managing the container reference deliberately, you will silently get a fresh, empty sandbox on every call, and the agent will look inexplicably forgetful about data it "just" analyzed.

Architecture: How a Request Actually Flows

At a high level, the request path looks like this:

1. Your client sends a message to the Foundry-hosted agent (via the Responses API or the Invocations protocol, depending on agent type — see Day 2 of this series for that distinction).
2. The model, given the code_interpreter tool in its tool list, decides code execution is the right move and emits a tool call containing Python source.
3. Foundry's agent runtime intercepts that tool call, resolves the associated container (creating one if none exists for this conversation/agent pair), and ships the code to the sandbox.
4. The sandbox executes the code in an isolated environment with no outbound network access to your VNet or the public internet by default, reads any input files that were staged into the container's file store, and writes stdout/stderr plus any generated artifacts (PNGs, CSVs, whatever the code produces) back to that store.
5. Execution results — return values, printed output, error tracebacks — are fed back into the model's context as a tool result.
6. The model incorporates that result into its next reasoning step, possibly issuing another code execution (this is the "iterative problem-solving" Microsoft's docs allude to — the model can see an error, fix its code, and retry, all within one turn).
7. The final response includes container_file_citation annotations pointing at any output files, which your client resolves via the containers API to download the actual bytes.

The important architectural detail: the model doesn't see raw bytes of generated files. It sees a citation — a container_id and file_id pair — embedded as an annotation on the output text. Your application code is responsible for walking the response's annotations and calling the containers/files retrieval endpoint to actually pull the PNG or CSV down. This indirection exists because file payloads (a rendered chart, a multi-megabyte CSV) don't belong inline in a token stream; they belong in blob-backed storage with a stable reference.

Auto Containers vs. Explicit Containers

Foundry gives you two container management strategies:

Automatic (AutoCodeInterpreterToolParam) — you hand Foundry a list of file_ids at agent-definition time (or per-request via structured inputs) and it manages container creation, file staging, and teardown for you. This is what almost every quickstart shows. It's the right default for stateless, single-shot analysis tasks: "here's a CSV, make me a chart," done.

Explicit container management — you create and reference a container ID directly, controlling exactly when it's provisioned and reused across multiple turns or multiple agent invocations. This is what you want for multi-turn analytical sessions where a user iterates on the same dataset ("now group by region," "now export that as JSON") and you need the dataframe state, intermediate variables, or previously-generated files to persist between calls without re-uploading everything each time.

The trade-off is exactly what you'd expect from any resource-lifecycle decision: automatic mode is simpler and harder to misuse, but you pay per-call container spin-up costs and lose continuity. Explicit mode gives you continuity and can be cheaper for chatty sessions, but now you own cleanup — an orphaned container that nobody deletes keeps its billable session alive until the one-hour ceiling regardless of whether anyone's using it.

The File Lifecycle: Upload → Execute → Citation → Download

This is the part that trips people up in production because it spans three different storage boundaries:

1. Upload: You call openai.files.create(purpose="assistants", file=...) against the project's OpenAI-compatible endpoint. This lands the file in Foundry-managed storage, independent of any container — it's a durable, reusable file object referenced by file_id.
2. Staging into the container: When you attach that file_id to a CodeInterpreterTool's container parameter, Foundry copies (or lazily mounts) that file into the sandboxed container's local filesystem so the executing Python code can open() it like a normal local path.
3. Generation: Code running inside the container writes new files — a chart, a transformed dataset — to the container's local working directory.
4. Citation: When the agent's response references that output, Foundry attaches a container_file_citation annotation with file_id, filename, and container_id.
5. Download: Your client calls the containers files-content endpoint with container_id + file_id to retrieve the actual bytes, entirely outside the model's token stream.

Two failure classes live here. First, forgetting step 5 — treating the citation as if it were the file, then wondering why your downstream pipeline received a JSON blob instead of PNG bytes. Second, container lifetime mismatches — if you try to retrieve a file after the container's session has expired (past the 30-minute idle window or the one-hour hard ceiling), the file is gone. There's no persistent, container-independent storage of generated outputs unless you explicitly copy them out during the active session — only uploaded inputs survive as durable file objects.

Prompt Agents vs. Hosted Agents: Two Very Different Execution Models

Foundry supports Code Interpreter through two structurally different agent shapes, and picking the wrong one for your use case creates unnecessary complexity.

Prompt agents are server-side declarative agents you define with PromptAgentDefinition and register via project.agents.create_version(...). You attach CodeInterpreterTool directly to the definition. Foundry owns the entire execution loop — you send a message, Foundry orchestrates model calls, tool calls, and sandbox execution server-side, and you get a finished response. This is the simpler path and the right default for most agentic data-analysis features.

Hosted agents, built with the Microsoft Agent Framework (Agent/FoundryChatClient), run your orchestration code in-process — you own the agent loop, the framework just gives you a chat client abstraction over the Foundry-hosted model. For these, Code Interpreter isn't attached directly to the agent; it's exposed through a toolbox — a versioned, reusable collection of tools published behind an MCP-compatible endpoint ({project_endpoint}/toolboxes/{name}/versions/{version}/mcp). Your hosted agent connects to that MCP endpoint via FoundryToolbox, and the code-execution capability is negotiated over MCP just like any other remote tool (see Day 6 of this series on the Toolbox/MCP pattern).

Why does this split exist? Toolboxes decouple tool curation from agent code. A platform team can define a code-interpreter-enabled toolbox once, version it, apply governance (allow-lists, credential scoping) at the toolbox layer, and let a dozen different hosted agents — written by different teams, in different languages, using different orchestration frameworks — consume the exact same governed capability without re-implementing container management logic themselves.

Implementation Walkthrough (Python)

Prompt agent: direct attachment

import os
from azure.identity import DefaultAzureCredential
from azure.ai.projects import AIProjectClient
from azure.ai.projects.models import (
    PromptAgentDefinition,
    CodeInterpreterTool,
    AutoCodeInterpreterToolParam,
)

PROJECT_ENDPOINT = os.environ["FOUNDRY_PROJECT_ENDPOINT"]

project = AIProjectClient(endpoint=PROJECT_ENDPOINT, credential=DefaultAzureCredential())
openai = project.get_openai_client()

# Step 1: upload the input file as a durable, container-independent file object
with open("quarterly_results.csv", "rb") as f:
    uploaded = openai.files.create(purpose="assistants", file=f)

# Step 2: declare the agent with Code Interpreter, auto-managed container,
# and the uploaded file pre-staged for the sandbox
agent = project.agents.create_version(
    agent_name="finance-analyst",
    definition=PromptAgentDefinition(
        model="gpt-5-mini",
        instructions=(
            "You are a financial analyst. Use Python to compute exact figures — "
            "never estimate arithmetic mentally. Show your work."
        ),
        tools=[
            CodeInterpreterTool(
                container=AutoCodeInterpreterToolParam(file_ids=[uploaded.id])
            )
        ],
    ),
    description="Analyst agent with sandboxed Python execution.",
)

conversation = openai.conversations.create()

response = openai.responses.create(
    conversation=conversation.id,
    input="What's the standard deviation of the operating_profit column?",
    extra_body={"agent_reference": {"name": agent.name, "type": "agent_reference"}},
)

print(response.output_text)

# Step 3: walk annotations for any generated artifacts (charts, exports)
for item in response.output:
    if item.type == "message":
        for part in item.content:
            for ann in getattr(part, "annotations", []) or []:
                if ann.type == "container_file_citation":
                    data = openai.containers.files.content.retrieve(
                        file_id=ann.file_id, container_id=ann.container_id
                    )
                    with open(ann.filename, "wb") as out:
                        out.write(data.read())
                    print(f"Downloaded artifact: {ann.filename}")

Note the instruction line: "never estimate arithmetic mentally." This isn't decoration — it's a real behavioral lever. Without an explicit nudge, models frequently answer numeric questions directly from context rather than routing through the tool, especially for "simple-looking" arithmetic. If correctness matters, say so in the system instructions.

Hosted agent: toolbox + MCP

import asyncio
import os
from agent_framework import Agent
from agent_framework.foundry import FoundryChatClient, FoundryToolbox
from azure.identity import AzureCliCredential
from azure.ai.projects import AIProjectClient
from azure.ai.projects.models import CodeInterpreterToolboxTool, AutoCodeInterpreterToolParam

PROJECT_ENDPOINT = os.environ["FOUNDRY_PROJECT_ENDPOINT"]

async def main() -> None:
    credential = AzureCliCredential()
    project = AIProjectClient(endpoint=PROJECT_ENDPOINT, credential=credential)
    openai = project.get_openai_client()

    with open("quarterly_results.csv", "rb") as f:
        uploaded = openai.files.create(purpose="assistants", file=f)

    # Curate the tool once, as a versioned, governable toolbox
    toolbox = project.toolboxes.create_version(
        name="analyst-toolbox",
        description="Sandboxed Python execution for the finance team's agents.",
        tools=[
            CodeInterpreterToolboxTool(
                container=AutoCodeInterpreterToolParam(file_ids=[uploaded.id])
            )
        ],
    )

    mcp_url = (
        f"{PROJECT_ENDPOINT}/toolboxes/{toolbox.name}"
        f"/versions/{toolbox.version}/mcp?api-version=v1"
    )
    toolbox_tool = FoundryToolbox(credential, url=mcp_url)

    agent = Agent(
        client=FoundryChatClient(credential=credential),
        instructions="You can write and execute Python to answer quantitative questions precisely.",
        tools=[toolbox_tool],
    )

    result = await agent.run(
        "Load the uploaded CSV and tell me the standard deviation of operating_profit."
    )
    print(result.text)

asyncio.run(main())

Same underlying sandbox, same billing model — different ownership boundary. The hosted-agent path is the one to reach for when your orchestration logic (retries, branching, multi-agent handoff) needs to live in your own process rather than inside a Foundry-managed prompt-agent definition.

What Happens at Runtime, Step by Step

Walking through the factorial of 100 example from Microsoft's own hosted-agent sample is instructive because the answer (a 158-digit integer) is unambiguously either correct or wrong — no room for a model to fudge it:

1. The model receives the prompt and recognizes this needs exact computation, not token-level reasoning.
2. It emits a tool call with source resembling import math; print(math.factorial(100)).
3. Foundry's runtime routes this to the container associated with the current conversation. If none exists yet, one is provisioned — this adds latency on the first call (container cold start), typically low hundreds of milliseconds to a few seconds depending on region and load.
4. The code runs inside the sandbox's Python process. No network egress, no access to your Azure resources, no shared filesystem with other conversations.
5. stdout is captured and returned as the tool result.
6. The model reads the tool result and composes a final natural-language answer around the exact returned value.

If the code throws an exception — a KeyError because the CSV column name doesn't match what the model assumed, for instance — the traceback comes back as the tool result too, and a capable model will often self-correct on the next turn by inspecting column names first (df.columns.tolist()) before retrying the original computation. This iterative repair loop is one of Code Interpreter's most valuable properties in practice, and it's also why you should budget for multiple tool-call round trips per user question, not just one, when estimating latency and token cost.

Security Considerations

The sandbox boundary is the whole point, so treat it as a real security control, not a formality:

• No inherent network egress. By default, code running in the container cannot reach your VNet, your databases, or arbitrary internet endpoints. Don't architect a workflow that assumes the sandbox can call back into your services — it can't, and shouldn't.
• File scope is explicit, not ambient. Only files you explicitly attach via file_ids are visible inside the container. The model cannot browse your Foundry project's other files, other conversations' uploads, or the host filesystem.
• Prompt injection via uploaded data is a real vector. If a user uploads a CSV and a cell contains something like "; import os; os.system(...), the data itself isn't executable — Code Interpreter runs code the model writes, not arbitrary content embedded in files, and the sandbox has no external commands to run anyway. The bigger risk is indirect: adversarial content in an uploaded file convincing the model to write and execute code that exfiltrates or corrupts other data within the same container/session (e.g., overwriting other uploaded files). Scope containers per user/session and don't co-mingle multiple users' files in one container.
• Session isolation is your multi-tenancy boundary. Each container session is isolated per your usage pattern — but it's on you to make sure you're not reusing one container across two different users' conversations to save on cold-start latency. That's a tenant-isolation bug waiting to happen.
• Generated files aren't automatically scanned. If your product pipes Code-Interpreter-generated files (say, an HTML report) directly to end users, put your normal content-safety and file-type validation in front of that path just as you would for any other user-facing file download.

Performance, Scalability, and Session Economics

Three numbers matter for capacity planning:

• 1-hour maximum session lifetime. Long-running analytical sessions get capped; design your UX so users understand a session that's gone quiet for an hour needs re-initialization (which, in auto-container mode, is transparent — you just pay for the re-provisioning).
• 30-minute idle timeout. A user who uploads a file, asks one question, then wanders off for 45 minutes will find their next question hits a cold container. This is a UX detail worth surfacing — "your session expired, re-analyzing your data" is a better message than silent failure.
• Concurrent conversations = concurrent containers. There is no session pooling or sharing across conversations. At scale, if you have thousands of concurrent users each triggering Code Interpreter, you have thousands of concurrent sandboxed containers being provisioned and torn down. This is fundamentally a per-tenant, per-conversation compute cost model, not a shared-pool one — closer to serverless function invocations than to a shared compute cluster.

Design implication: if your product's traffic pattern is bursty (e.g., a monthly reporting rush), expect provisioning latency variance during bursts, and don't assume container spin-up time is constant across load levels.

Cost Considerations

Code Interpreter is billed separately from token usage — it's a session-based charge on top of whatever Azure OpenAI/Foundry model tokens the surrounding conversation consumes (verify current per-session pricing in the Azure pricing calculator before budgeting, as this is a distinct SKU from model inference). The practical cost drivers are:

• Number of sessions provisioned, not just number of code executions — a single session can serve many executions if you keep the container alive and reuse it across a multi-turn conversation.
• Session duration relative to the 1-hour ceiling — you're billed for the session window, so a conversation that fires one code execution and then goes idle for 55 minutes before firing another still occupies (and pays for) that session the whole time, up to the idle-timeout cutoff.
• Auto vs. explicit container strategy — auto-mode, used naively (creating a fresh agent/container per single question instead of reusing one across a conversation), multiplies session counts unnecessarily. If your app pattern is "many short independent questions," explicit container reuse across turns is usually cheaper than defaulting to a new session per call.

Common Mistakes and Pitfalls

1. Treating file citations as inline file content. The annotation is a pointer, not a payload — always make the follow-up retrieval call.
2. Assuming state persists without explicit container management. If you're not deliberately reusing a container reference, don't expect the model to "remember" a dataframe it built two calls ago.
3. Not instructing the model to actually use the tool. Left to its own judgment, a model will sometimes answer numeric questions from token-level reasoning rather than invoking code execution, especially for numbers that "look easy." Be explicit in system instructions when precision matters.
4. Ignoring the idle timeout in multi-turn UX. A silently expired container producing a "fresh start" response confuses users who think they're continuing an ongoing analysis.
5. Co-mingling multiple users' uploaded files in a shared container to save provisioning overhead — a tenant-isolation anti-pattern that trades a small cost saving for a real security bug.
6. Forgetting Code Interpreter has its own charges. Teams that model cost purely on token counts get surprised by session-based billing showing up as a separate line item.
7. Using Code Interpreter for what should be a deterministic API call. If the task is "call this internal service and return JSON," that's a job for a proper tool/function definition or an MCP server — not for having the model write ad hoc HTTP-adjacent code in a network-isolated sandbox where it can't even reach your service.

Alternatives and Trade-offs

Code Interpreter isn't the only way to get code execution in front of a model, and it isn't always the right one:

• Azure Container Apps dynamic sessions give you a similar sandboxed-Python-execution primitive but as a standalone Azure service you call directly, independent of the Foundry Agent Service tool-calling loop. Reach for this if you need code execution outside an agent conversation context, or need finer control over the container image and installed packages than the managed Code Interpreter tool exposes.
• Third-party sandbox providers (E2B, Daytona, and similar) offer comparable isolated-execution primitives with different language/runtime support and different pricing models, useful if you're building a multi-cloud or non-Foundry-centric agent stack.
• A proper function-calling tool backed by your own service is the better choice whenever the "code" the model would write is really just "call this deterministic API with these parameters." Code Interpreter is for genuinely open-ended computation — data transformation, statistics, visualization, math — not as a general-purpose remote procedure call mechanism.
• Custom containers with pre-installed domain packages (via explicit container configuration rather than the fully automatic mode) are worth it if your use case needs heavyweight or unusual dependencies (e.g., geospatial libraries, specific scientific computing stacks) not present in the default sandbox image.

Practical Recommendations

• Default to auto-managed containers for single-shot analysis; move to explicit container management the moment your product has genuinely multi-turn analytical conversations over the same dataset.
• Write explicit instructions telling the model when to reach for code execution rather than reasoning in tokens — don't rely on default judgment for correctness-critical numeric tasks.
• Build your file-retrieval logic to always walk annotations, never assume text output contains the artifact.
• Treat the 30-minute idle / 1-hour hard cap as product-facing constraints, not just billing details — communicate session expiry to users where relevant.
• If you're building hosted agents with the Agent Framework, prefer the toolbox/MCP path so code-execution governance (allow-listing, credential scoping, versioning) lives in one place your platform team controls, not scattered across every agent's code.
• Keep containers scoped to a single user/session — never share one across tenants to save cold-start latency.
• Budget cost and latency for multiple tool-call round trips per question, since the model's ability to read errors and retry is a feature, not a rare edge case.

Conclusion

Code Interpreter is Foundry's answer to a problem every serious agent eventually hits: language models are fluent but not reliably correct at exact computation. The sandbox — with its explicit container lifecycle, annotation-based file citation model, and split path between prompt agents and MCP-exposed toolboxes for hosted agents — is a genuinely well-thought-out piece of infrastructure once you understand the primitives underneath the five-line quickstart. Get the container lifecycle, file lifecycle, and session economics right, and you get an agent that can be trusted with real arithmetic, real data transformations, and real charts — not just plausible-sounding ones.

If your agent currently answers numeric or data-heavy questions purely by "reasoning" in tokens, that's the tell it's time to wire this in.

References

• Microsoft Learn: Use Code Interpreter with Microsoft Foundry agents
• Microsoft Foundry samples repository: microsoft-foundry/foundry-samples
• Microsoft Agent Framework samples: foundry_chat_client_with_code_interpreter.py
• Related in this series: Day 2 (Responses vs. Invocations protocols), Day 6 (MCP Toolbox governance)
• (verify current Code Interpreter session pricing directly in the Azure pricing calculator before finalizing cost estimates for production workloads)

---

This is Day 8 of Microsoft Foundry: 100 Days / 100 Blogs — a daily series covering the breadth of Microsoft Foundry for developers building real production AI systems. Follow along for the next 92 days.

Comments

[Ahmet Özel]

The gap you are pointing at is real: the five-line CSV-to-bar-chart demo hides container provisioning, session lifecycle and file staging, and all three are where production behaviour differs from the demo.

Session lifecycle is the one I would put first for anyone building on this. Whether state persists between calls decides your whole prompting strategy, because an agent that assumes its variables survived will write code that silently recomputes nothing and returns a stale or empty result rather than erroring.

The other thing worth stating explicitly is that the interpreter changes the trust boundary. Model-generated code running against staged customer files means an injected instruction in a document can now reach a real execution environment, which is a different risk class from a wrong answer.