Executable Code Actions Elicit Better LLM Agents — Exploring a different way to build more capable AI agents.
Introduction: Are We Building AI Agents the Hard Way?
When building AI agents, we often focus on adding more tools, integrating more APIs, and connecting more MCP servers.
The assumption is straightforward: the more tools an agent has, the more capable it becomes.
But what if the real limitation isn't the number of tools available to an agent, but how the agent uses those tools?
This is the question explored by Executable Code Actions Elicit Better LLM Agents, a research paper presented at ICML 2024.
The paper introduces CodeAct, an approach in which an LLM can express actions through executable code instead of relying exclusively on conventional tool-call formats.
Rather than making isolated tool calls, an agent can write code that combines operations, processes intermediate results, controls execution flow, and interacts with an execution environment.
The idea is simple, but its implications for agentic AI are significant.
📄 Read the original research paper
1. What Is CodeAct?
Traditional LLM agents commonly interact with tools through predefined interfaces.
The model selects a tool, generates arguments, receives a result, and decides what to do next.
For example, an agent might need to retrieve customer records, filter them, and generate a report.
A tool-calling workflow could involve several separate calls:
{
"tool": "get_customers",
"arguments": {}
}
The model then receives the results and generates another tool call:
{
"tool": "filter_customers",
"arguments": {
"status": "active"
}
}
Finally, it might call another tool to generate the report.
This approach works well for many tasks. However, coordinating several dependent operations can require repeated model decisions, intermediate messages, and orchestration logic.
CodeAct offers another possibility.
The agent can express the workflow as executable code:
customers = get_customers()
active_customers = [
customer
for customer in customers
if customer["status"] == "active"
]
report = generate_report(active_customers)
print(report)
The functions above represent tools exposed to the agent by its environment.
The important distinction is that the agent can express the relationships between operations directly in code.
It can use variables, loops, conditions, functions, and exception handling to coordinate its actions.
CodeAct treats executable code as an action format, not merely as text to be displayed to the user.
2. From Individual Tool Calls to Executable Workflows
Consider a task that requires collecting information from three services.
A conventional orchestration layer might explicitly define the sequence:
accounts = call_tool("get_accounts")
cards = call_tool("get_cards")
transactions = call_tool("get_transactions")
result = call_tool(
"combine_financial_data",
{
"accounts": accounts,
"cards": cards,
"transactions": transactions
}
)
A code-based approach can combine the same operations and perform additional processing locally within the execution environment:
accounts = get_accounts()
cards = get_cards()
transactions = get_transactions()
summary = {
"account_count": len(accounts),
"card_count": len(cards),
"transaction_count": len(transactions),
"total_spending": sum(
transaction["amount"]
for transaction in transactions
if transaction["amount"] > 0
)
}
print(summary)
This example assumes that the tools return data in the expected format and that positive transaction amounts represent spending.
The advantage is not simply fewer lines of code. It is the ability to define data flow and control flow dynamically.
An agent can decide how to process results without requiring a separate tool for every possible combination of operations.
Visualizing the difference
flowchart TD
A[User Task] --> B[Traditional Agent]
B --> C[Tool Call 1]
C --> D[Observe Result]
D --> E[Tool Call 2]
E --> F[Observe Result]
F --> G[Tool Call 3]
G --> H[Final Answer]
A --> I[CodeAct-Inspired Agent]
I --> J[Generate Executable Code]
J --> K[Execute Multiple Operations]
K --> L[Observe Execution Result]
L --> M{Task Complete?}
M -- No --> J
M -- Yes --> N[Final Answer]The diagram illustrates the conceptual difference. Actual architectures vary, and conventional agents can also support batching, loops, and multi-step execution.
3. Why Executable Code Is a Powerful Action Space
3.1. Dynamic control flow
Not every task follows the same path.
Imagine an agent that retrieves customer data. If the request succeeds, it analyzes the result. If it fails, it investigates the error.
response = fetch_customer_data()
if response["status"] == "success":
customer = response["data"]
analysis = analyze_customer(customer)
print(analysis)
else:
error_details = response.get(
"error",
"Unknown error"
)
investigation = investigate_error(error_details)
print(investigation)
In a rigid workflow, this branching logic might need to be defined in the orchestration layer.
With executable code, the agent can express the branch directly in the action it generates.
3.2. Repeated operations
Suppose an agent must inspect several repositories or services.
repositories = [
"frontend",
"backend",
"shared-libraries"
]
results = {}
for repository in repositories:
results[repository] = search_repository(
repository,
query="payment"
)
for repository, matches in results.items():
print(repository, len(matches))
A loop can coordinate repeated operations while preserving their results for later analysis.
The tools still need to be available to the execution environment. CodeAct does not automatically create new external capabilities.
3.3. Reusing intermediate results
An agent may need to process data from one operation before passing it to another.
transactions = get_transactions()
high_value_transactions = [
transaction
for transaction in transactions
if transaction["amount"] >= 10000
]
customer_ids = {
transaction["customer_id"]
for transaction in high_value_transactions
}
customers = [
get_customer(customer_id)
for customer_id in customer_ids
]
print({
"high_value_transactions": len(
high_value_transactions
),
"related_customers": len(customers)
})
Here, the agent can filter records, extract identifiers, and retrieve related entities using a single executable workflow.
In a real financial application, operations would need appropriate authorization, privacy controls, and clearly defined amount semantics.
3.4. Composing operations into reusable functions
Code can also make an action easier to understand and reuse.
def investigate_customer(customer_id):
customer = get_customer(customer_id)
transactions = get_transactions(customer_id)
flagged_transactions = [
transaction
for transaction in transactions
if transaction.get("flagged", False)
]
return {
"customer": customer,
"transaction_count": len(transactions),
"flagged_transactions": flagged_transactions
}
result = investigate_customer("customer-123")
print(result)
This is useful when several operations form a logical unit of work.
It also introduces a new responsibility: the environment must control which functions the generated code is allowed to invoke.
4. The Execution Feedback Loop
One of the most interesting aspects of executable actions is the possibility of observing runtime feedback.
An agent does not have to stop after generating code. It can execute the code, inspect the result, and use that observation to decide what to do next.
The basic cycle is:
Generate → Execute → Observe → Revise
Consider this example.
The first attempt assumes a field exists:
response = fetch_customer()
print(response["customer_name"])
Suppose execution returns a KeyError.
The agent can inspect the returned structure:
response = fetch_customer()
print("Available fields:", list(response.keys()))
It might discover that the field is named name instead.
The next attempt can account for the actual structure:
response = fetch_customer()
customer_name = response.get(
"customer_name",
response.get("name", "Unknown")
)
print(customer_name)
This demonstrates how execution feedback can guide subsequent actions.
It does not mean that the model permanently learns from every error. Rather, the error becomes an observation that can inform the next step in the current task.
A simplified agent loop
task = "Retrieve and analyze customer data"
while not task_is_complete(task):
code = llm.generate_code(
task=task,
observations=get_observations()
)
result = sandbox.execute(code)
record_observation(result)
if result.error:
task = update_task_with_error(
task,
result.error
)
print(get_final_answer())
This is conceptual pseudocode, not the exact implementation described in the research paper.
A production system needs explicit termination conditions, execution limits, observation handling, and protection against endless retries.
5. What Did the Researchers Discover?
The paper introduces M³ToolEval, a benchmark designed to evaluate tool use in tasks involving multiple tools and multiple interaction turns.
The researchers evaluate 17 different LLMs to investigate how executable code actions affect agent performance.
The paper reports improvements of up to:
- 20% in task success rate
- 30% fewer actions in reported experiments
These are maximum reported improvements, not guaranteed gains across all models, benchmarks, or tasks.
The findings suggest that how an agent represents and executes actions can have a significant impact on its ability to complete complex tasks.
The lesson is not simply that Python is better than JSON.
The more important point is that executable code offers a flexible action space in which an agent can coordinate operations, reuse intermediate results, and express conditional behavior.
For tasks that require several dependent operations, this flexibility can reduce the amount of explicit orchestration required.
6. CodeActAgent and CodeActInstruct
The research also introduces resources to support the approach.
CodeActAgent
CodeActAgent demonstrates an agent architecture that uses executable code to interact with tools and an execution environment.
CodeActInstruct
CodeActInstruct is a dataset of approximately 7,000 multi-turn interactions designed to support training models to use code as an action format.
The research includes models based on Llama 2 and Mistral, exploring how models can be trained to generate executable actions instead of relying exclusively on conventional tool-call formats.
This matters because the work investigates both the architecture and the model behavior needed to use it.
Explore the official CodeAct implementation on GitHub.
7. CodeAct and MCP: A Natural Combination?
MCP provides a standardized way for AI applications to discover and invoke tools exposed by external servers.
But having access to many tools does not automatically mean an agent can combine them efficiently.
Imagine an agent with access to these tools:
search_codebase(query)
read_file(path)
find_references(symbol)
get_file_structure(path)
A conventional agent could call these tools one at a time.
A CodeAct-inspired agent could coordinate them in a program:
query = "GET_CREDIT_CARD_DETAIL_DOCUMENT"
matches = search_codebase(query)
investigation = []
for match in matches:
file_path = match["file"]
source = read_file(file_path)
investigation.append({
"file": file_path,
"match": match,
"source_excerpt": source
})
print(investigation)
The code above assumes the tools expose compatible return structures.
The distinction is important:
- MCP standardizes how tools are exposed and invoked.
- CodeAct explores using executable code as the format for agent actions.
- An orchestration framework manages execution, state, permissions, and the broader agent lifecycle.
These ideas can complement one another, but they are not interchangeable.
A CodeAct-inspired architecture could use MCP tools as building blocks and generate code to coordinate them.
8. A Practical Developer-Agent Example: Trace an Endpoint to a Frontend Screen
Consider a banking application built around a frontend, GraphQL, a backend-for-frontend (BFF), and several microservices.
The task is:
Given a service endpoint, identify which frontend screens depend on it.
A useful investigation might follow this chain:
REST endpoint
↓
Backend service
↓
BFF service or resolver
↓
GraphQL operation
↓
Frontend hook or data layer
↓
Component
↓
Screen
A CodeAct-inspired agent could combine repository search tools to trace these relationships.
endpoint = "/creditcards"
service_matches = search_codebase(endpoint)
graphql_operations = set()
for match in service_matches:
operations = find_related_graphql_operations(
match["file"],
match["line"]
)
graphql_operations.update(operations)
screens = set()
for operation in graphql_operations:
consumers = find_frontend_consumers(operation)
screens.update(consumers)
print({
"endpoint": endpoint,
"graphql_operations": sorted(graphql_operations),
"screens": sorted(screens)
})
The search and tracing functions are hypothetical tools. They are not built-in CodeAct functions.
The agent can extend the investigation when it encounters incomplete results.
For example, it might search for an operation's name, inspect the matching file, and then follow imports into the frontend.
operation = "GET_CREDIT_CARD_DETAIL_DOCUMENT"
matches = search_codebase(operation)
frontend_files = []
for match in matches:
if match["path"].startswith("apps/"):
frontend_files.append(match["path"])
elif match["path"].startswith("libs/"):
frontend_files.append(match["path"])
unique_files = sorted(set(frontend_files))
print(unique_files)
This simplified example only filters search results by path. A real implementation would need to understand the repository structure, distinguish declarations from usages, and follow imports and data dependencies.
A possible investigation workflow
flowchart TD
A[User provides endpoint] --> B[Search repository]
B --> C[Find service implementation]
C --> D[Trace BFF and resolver]
D --> E[Find GraphQL operation]
E --> F[Find frontend consumers]
F --> G{Evidence complete?}
G -- No --> H[Generate another search action]
H --> B
G -- Yes --> I[Return evidence-backed report]This illustrates a possible CodeAct-inspired architecture, not a workflow implemented or evaluated in the paper.
The value is that the agent can dynamically compose existing search capabilities instead of requiring a separate hardcoded workflow for every endpoint.
However, CodeAct does not replace code indexing, semantic search, or reliable repository navigation. Those capabilities remain essential for large codebases.
9. Parallel Operations and Data Aggregation
Some investigations contain independent operations that can run in parallel.
For example, an agent might search the frontend, BFF, and shared libraries for the same operation.
Conceptually:
import asyncio
async def investigate_repository(repository, query):
return await search_repository_async(
repository,
query
)
async def investigate_all(query):
repositories = [
"frontend",
"bff",
"shared-libraries"
]
tasks = [
investigate_repository(repository, query)
for repository in repositories
]
results = await asyncio.gather(*tasks)
return {
repository: result
for repository, result in zip(
repositories,
results
)
}
results = await investigate_all(
"GET_CREDIT_CARD_DETAIL_DOCUMENT"
)
print(results)
This is an illustrative example. It assumes asynchronous tool functions and an execution environment that supports top-level await.
CodeAct does not automatically make tool calls parallel. The generated program, runtime, and available interfaces must support concurrency.
Parallel execution can improve latency when operations are independent, but it requires sensible concurrency limits, error handling, and resource management.
10. Does CodeAct Replace Traditional Tool Calling?
Not necessarily.
Traditional tool calling remains useful when operations are simple, well-defined, and require strict argument validation.
Executable code becomes more attractive when tasks involve:
- Multiple dependent operations
- Conditional or repetitive execution
- Data transformation between tools
- Dynamic workflows
- Complex multi-step investigations
A hybrid design can use structured tool calls for well-defined operations and code-based orchestration when several operations need to be composed.
For example:
# A structured tool provides a trusted operation.
account = get_account(account_id)
# Generated code handles task-specific processing.
if account["status"] == "active":
transactions = get_transactions(account_id)
result = summarize_transactions(
transactions
)
else:
result = {
"message": "Account is not active"
}
print(result)
The tool layer can preserve access controls and input validation while the agent uses code to coordinate the task.
The best architecture depends on the task, the available tools, the model, and the required security guarantees.
11. The Security Challenge
Giving an agent the ability to execute code increases its flexibility, but it also introduces risks.
Generated code must not receive unrestricted access to a host machine, production database, or sensitive filesystem.
A production implementation should consider:
- Sandboxed execution
- Execution time and memory limits
- Restricted filesystem and network access
- Explicit tool permissions
- Validation of sensitive operations
- Logging and auditing
- Approval requirements for high-impact actions
A hypothetical sandbox interface might look like this:
result = sandbox.execute(
code=generated_code,
timeout_seconds=10,
network_access=False,
filesystem_access="restricted"
)
This is an illustrative API, not a real CodeAct function. Actual security controls depend on the execution platform.
The core principle is straightforward:
Give the agent flexibility over how it performs a task, but keep strict boundaries around what it is allowed to do.
Code execution should be isolated, observable, and governed by explicit permissions.
12. What CodeAct Does—and Does Not—Solve
CodeAct is not a complete solution to every agent problem.
It does not automatically provide:
- Accurate reasoning
- Reliable codebase indexing
- Correct tool implementations
- Persistent learning from previous failures
- Guaranteed task completion
- Safe execution by default
The approach changes the action space available to an agent. The quality of the outcome still depends on the model, the tools, the execution environment, and the way the task is evaluated.
This is an important distinction when deciding whether to adopt CodeAct in a real application.
Conclusion: Better Agents May Need Better Ways to Act
What makes CodeAct interesting is not simply that it allows an LLM to write Python. Models have been generating code for years.
The more significant idea is treating executable code as a general-purpose action format for AI agents.
Instead of designing every possible sequence of tool calls in advance, we can give agents a set of capabilities and let them compose those capabilities into executable workflows.
The research provides experimental evidence that this approach can improve performance on certain multi-tool tasks while reducing the number of actions required.
For developers building MCP integrations, agent orchestration platforms, and AI-powered developer tools, CodeAct offers a valuable architectural perspective.
Perhaps the next step in building more capable agents isn't simply adding more tools.
Perhaps it's giving agents better ways to use the tools they already have.
Top comments (1)
The part I keep coming back to is what happens to enforcement when actions become code. I spent today testing a guard at the tool-call seam: a pre_tool_call interceptor that denies a refund above a cap. One decision, one record, and the tool function is never entered — easy to audit. Move that logic into agent-written code and the audit point moves with it: you stop gating N tool calls and start gating an interpreter, and "this loop calls issue_refund 40 times" is not in any single call's arguments. The flexibility looks real; so does the argument that enforcement has to drop a layer with it.