Agent-based LLM workflows move beyond single-shot prompting by giving models access to tools, memory, and iterative reasoning loops. Instead of treating the LLM as a pure text generator, you treat it as a reasoning node that can observe, act, and reflect. This shift introduces new infrastructure requirements: reliable function calling, support for long multi-turn contexts, and predictable pricing as agent loops accumulate tokens across many steps.
What Are Agent-Based LLM Workflows?
An agent, in this context, is an LLM wrapped in a control loop. The loop typically follows a pattern of reasoning, action, and observation. The model receives a task, decides whether to use a tool, receives the tool output, and then decides whether to continue reasoning or return a final answer. This architecture is useful for tasks that require calculation, retrieval, or multi-step planning that exceeds the capacity of a single prompt.
The core components are:
- A model with strong instruction following and tool-use capabilities.
- A set of tools, usually exposed via function schemas.
- A memory or state mechanism that persists context across turns.
- An orchestration layer that handles the loop, parses outputs, and manages errors.
Core Patterns
Three patterns dominate production agent design.
ReAct (Reasoning + Acting) interleaves chain-of-thought reasoning with tool calls. The model explicitly states what it is doing before calling a tool, which improves interpretability and reduces error propagation.
Plan-and-Execute splits the workflow into two phases. A planning model decomposes the task into subtasks, and an execution model carries them out. This works well for complex coding or research workflows where the plan itself needs review before action.
Multi-Agent Orchestration assigns specialized models to different roles, such as a coder, a reviewer, and a planner. These agents communicate through a shared message bus or a structured handoff protocol. This pattern is effective for long-horizon tasks but adds latency and state complexity.
Tool Use and Function Calling
Tool use is the primary mechanism by which agents interact with external systems. For this to work, the inference backend must support robust function calling and JSON mode.
Oxlo.ai provides function calling, tool use, JSON mode, streaming responses, and multi-turn conversations through a fully OpenAI-compatible API. This means you can point the official OpenAI SDK at Oxlo.ai and use the same chat.completions.create patterns you already have in production.
Managing State and Memory
Agent workflows are stateful. Each turn appends new messages to the conversation history, and tool outputs can be large. A single agent loop can easily consume tens of thousands of tokens in context. This makes long context windows essential.
Oxlo.ai hosts models with extended context capabilities that are relevant here. DeepSeek V4 Flash supports a 1M context window, and Kimi K2.6 offers a 131K context window with advanced reasoning and agentic coding capabilities. When selecting a model, match the context budget to your expected tool output size plus your reasoning scratchpad.
Cost Considerations for Long-Context Agents
Token-based pricing creates a hidden tax on agent architectures. Every tool result, every reasoning step, and every system prompt is billed by the token. In a multi-turn agent loop, costs scale non-linearly with conversation length.
Oxlo.ai uses request-based pricing: one flat cost per API request regardless of prompt length. Unlike token-based providers such as Together AI, Fireworks AI, OpenRouter, Replicate, or Anyscale, Oxlo.ai does not charge more when you send long contexts or large tool outputs. For agentic and long-context workloads, this can be 10-100x cheaper because the cost is decoupled from token volume. See https://oxlo.ai/pricing for plan details.
A Minimal ReAct Agent with Oxlo.ai
Below is a complete, minimal ReAct agent using the OpenAI SDK against Oxlo.ai. It loops until the model returns a final answer or hits a step limit.
import os
import json
from openai import OpenAI
client = OpenAI(
base_url="https://api.oxlo.ai/v1",
api_key=os.environ["OXLO_API_KEY"]
)
# Define a simple calculator tool
tools = [
{
"type": "function",
"function": {
"name": "calculate",
"description": "Evaluate a basic math expression",
"parameters": {
"type": "object",
"properties": {
"expression": {"type": "string"}
},
"required": ["expression"]
}
}
}
]
messages = [
{
"role": "system",
"content": "You are a precise reasoning agent. Think step by step, use tools when necessary, and produce a final answer."
},
{
"role": "user",
"content": "A warehouse has 42 crates. Each crate holds 24 boxes. How many total boxes are there? Then add 150."
}
]
def run_agent(max_steps=5):
for step in range(max_steps):
response = client.chat.completions.create(
model="qwen-3-32b", # Qwen 3 32B agent model on Oxlo.ai
messages=messages,
tools=tools,
tool_choice="auto"
)
message = response.choices[0].message
messages.append(message)
if message.tool_calls:
for tool_call in message.tool_calls:
if tool_call.function.name == "calculate":
args = json.loads(tool_call.function.arguments)
# In production, run this in a sandbox, not with raw eval
result = eval(args["expression"])
messages.append({
"role": "tool",
"tool_call_id": tool_call.id,
"content": str(result)
})
else:
return message.content
return "Reached step limit without final answer."
answer = run_agent()
print(answer)
This example uses Qwen 3 32B, a model on Oxlo.ai built specifically for multilingual reasoning and agent workflows. Because Oxlo.ai charges per request, adding extra reasoning steps or large tool outputs does not increase the per-call cost.
Selecting Models for Agent Tasks
Oxlo.ai offers 45+ models across 7 categories. For agent workflows, the following are particularly relevant:
- Qwen 3 32B: Strong multilingual reasoning and agent workflow support.
- DeepSeek R1 671B MoE: Deep reasoning and complex coding tasks that require extended thought.
- Kimi K2.6: Advanced reasoning, agentic coding, and vision with a 131K context window.
- DeepSeek V4 Flash: Efficient MoE with a 1M context window for agents that ingest large documents or codebases.
- GLM 5: A 744B MoE designed for long-horizon agentic tasks.
- Llama 3.3 70B: A general-purpose flagship that works well as a default agent backbone.
For specialized needs, Oxlo.ai also provides code models such as Qwen 3 Coder 30B and Oxlo.ai Coder Fast, vision models such as Kimi VL A3B, and embedding models such as BGE-Large for retrieval-augmented agent memory.
Production Deployment Notes
Agents in production need more than a working loop. Consider the following.
Idempotency and retries: Tool calls may fail or time out. Wrap external actions in retry logic and store intermediate state so the agent can resume without repeating expensive steps.
Latency and cold starts: Agent workflows often require rapid back-to-back calls. Oxlo.ai has no cold starts on popular models, which keeps iteration latency consistent.
Observability: Log every turn, tool input, and tool output. Because Oxlo.ai uses request-based pricing, your cost per step is fixed, which simplifies budgeting and forecasting compared to token-based billing.
Security: Never execute tool outputs blindly. Run code in isolated sandboxes and validate all arguments before execution.
Conclusion
Agent-based workflows are becoming the default architecture for complex automation. They demand infrastructure that supports reliable tool use, long context windows, and transparent pricing. Oxlo.ai meets these requirements with an OpenAI-compatible API, a broad model catalog including agent-optimized options like Qwen 3 32B and Kimi K2.6, and request-based pricing that removes the penalty for long-context agent loops. If you are building agents today, Oxlo.ai is a relevant, cost-effective backend to evaluate.
Top comments (0)