Enterprise Resource Planning systems manage the core transactions and master data that run a business, but their rigid interfaces and complex schemas often create friction for end users. Large Language Models can bridge this gap by enabling natural language queries, automated report generation, and intelligent workflow orchestration directly on top of existing ERP investments. Integrating an LLM with a legacy ERP stack is not a rip-and-replace project. It is an incremental API and middleware exercise that preserves your existing database while adding a conversational or agentic layer on top.
Assess Your ERP Landscape and Integration Points
Before writing any code, audit your ERP environment for integration surfaces. Most legacy systems expose data through one or more of the following: REST or SOAP APIs, OData endpoints, database views, message queues, or flat-file exports. Identify the entities you want the LLM to access, such as purchase orders, inventory levels, or financial journals. Map read-only versus write-capable paths, and flag any tables that contain personally identifiable information or regulated financial data. The goal is to define a bounded context that the LLM can reason over without gaining unrestricted access to the underlying ERP database.
Document your API contracts or schema definitions in a machine-readable format. You will later inject these schemas into system prompts or retrieval pipelines so the model understands table relationships, field types, and business logic constraints.
Choose an LLM and Inference Provider
ERP workloads have a distinct cost profile. A single natural language request can require injecting thousands of tokens of schema context, transaction history, and multi-turn conversation state. Under token-based billing, these long prompts directly inflate costs. Oxlo.ai is a developer-first AI inference platform that uses request-based pricing: one flat cost per API request regardless of prompt length. Unlike token-based providers such as Together AI, Fireworks AI, OpenRouter, Replicate, or Anyscale, cost does not scale with input length, so Oxlo.ai is significantly cheaper for long-context and agentic ERP workloads.
Oxlo.ai offers 45+ open-source and proprietary models across 7 categories and is fully OpenAI SDK compatible. For ERP integrations, consider Llama 3.3 70B as a general-purpose flagship for routine queries, DeepSeek R1 671B MoE for deep reasoning over complex financial logic, or Kimi K2.6 for advanced reasoning and agentic coding with vision support. All popular models have no cold starts, which keeps ERP chat interfaces responsive. You can view the exact request-based rates at https://oxlo.ai/pricing.
Design the Integration Architecture
A safe ERP integration never exposes the database directly to the LLM. Instead, use a middleware layer that acts as a policy enforcement point. The typical flow is:
- The user submits a natural language query to your application backend.
- The backend authenticates the user and enforces role-based permissions.
- The backend retrieves relevant ERP context, either from a cache, a vector store, or a live API call.
- The backend constructs a system prompt containing the schema, business rules, and the user question.
- The prompt is sent to the LLM inference provider.
- The model returns structured data or a natural language response, which your backend validates before returning to the user.
If the LLM needs to trigger actions, such as creating a draft invoice, expose these through function definitions rather than free-form SQL. This keeps the ERP's authorization and validation layers in control.
Implement the API Layer with Oxlo.ai
Because Oxlo.ai is fully OpenAI SDK compatible, you can integrate it using the official Python or Node.js client with a single change to the base URL. The following example shows a Python service that accepts a user question, fetches sanitized ERP context, and calls Oxlo.ai with a tool definition for safe data retrieval.
import os
from openai import OpenAI
client = OpenAI(
base_url="https://api.oxlo.ai/v1",
api_key=os.environ["OXLO_API_KEY"]
)
def query_erp_assistant(user_question: str, schema_context: str):
tools = [
{
"type": "function",
"function": {
"name": "get_open_pos",
"description": "Retrieve open purchase orders by vendor",
"parameters": {
"type": "object",
"properties": {
"vendor_id": {"type": "string"}
},
"required": ["vendor_id"]
}
}
}
]
messages = [
{
"role": "system",
"content": (
"You are an ERP assistant. Use the provided schema context to answer questions. "
"If you need live data, call the available functions. "
"Never generate SQL directly.\n\nSchema context:\n" + schema_context
)
},
{"role": "user", "content": user_question}
]
response = client.chat.completions.create(
model="llama-3.3-70b",
messages=messages,
tools=tools,
tool_choice="auto"
)
return response.choices[0].message
Oxlo.ai supports streaming responses, function calling, JSON mode, and multi-turn conversations, so you can extend this snippet into a persistent chat session or force structured JSON output for downstream ERP automation pipelines.
Manage ERP Context and Prompt Engineering
ERP schemas are large, and sending an entire data dictionary on every request can overwhelm the model or degrade latency. Use a retrieval step to pull only the tables and fields relevant to the user intent. For example, if the user asks about accounts payable, inject only the GL accounts, vendor master, and payment term definitions.
Because Oxlo.ai charges per request rather than per token, you can afford to include generous context windows when they are necessary. A complex multi-step agentic workflow that passes lengthy transaction histories between turns will not trigger the cost explosions common on token-based platforms. You can also use DeepSeek V4 Flash for efficient processing with a 1M context window, or GLM 5 for long-horizon agentic tasks that require extended reasoning across ERP modules.
When you need deterministic output shapes, such as JSON arrays for dashboard widgets, enable JSON mode and provide a strict schema in the system prompt. This eliminates parsing ambiguity and makes it easier to feed LLM output back into your ERP's input APIs.
Secure and Govern the Integration
Treat the LLM layer as an external service. Store your Oxlo.ai API key in a secrets manager, rotate it regularly, and scope it to the minimum required models. Within your middleware, enforce the same row-level and field-level security rules that your ERP enforces. The LLM should never see data that the requesting user is not authorized to view.
Log every prompt, retrieved context chunk, and model response for audit purposes. If your ERP handles regulated industries, implement PII masking before context leaves your network. Oxlo.ai acts as the inference engine, but data governance remains your responsibility at the application boundary.
Monitor, Test, and Iterate
Deploy the integration behind a feature flag and measure accuracy against a golden set of ERP questions. Track latency end to end, from ERP data retrieval to final token streamed from Oxlo.ai. Because Oxlo.ai uses flat per-request pricing, your forecasting is simple: multiply expected daily requests by the plan rate. There are no surprise bills from a sudden increase in prompt length when users paste large spreadsheet ranges or multi-page invoice details.
Start with a narrow use case, such as a natural language purchase order lookup, then expand into agentic workflows that span multiple ERP modules. Oxlo.ai's broad catalog, including code-specialized models like Qwen 3 Coder 30B and vision models like Kimi VL A3B, allows you to evolve from text-only queries toward multimodal interfaces without switching providers.
For current plan details and to estimate costs for your request volume, see https://oxlo.ai/pricing.
Top comments (0)