title: Your AI Agent Should Not Be Locked to One LLM Provider
published: false
description: Why serious AI agents need a provider-agnostic architecture, model routing, fallback, and a unified API gateway.
tags: ai, llm, agents, architecture
Your AI Agent Should Not Be Locked to One LLM Provider
Most AI agent prototypes start the same way.
You pick one model provider.
You install one SDK.
You write a few prompts.
You add tool calling.
You build a demo.
It works.
Until it does not.
The moment you want to try another model, reduce cost, add fallback, improve latency, or support different task types, your simple agent starts turning into a messy collection of provider-specific logic.
That is when you realize something important:
A real AI agent should not be locked to one LLM provider.
If you are building a personal AI agent, coding assistant, research assistant, internal workflow agent, or AI-native product, the model should be replaceable infrastructure — not a hardcoded dependency.
The Problem with Single-Provider Agents
A simple agent architecture often looks like this:
CopyUser
↓
Agent
↓
One LLM Provider
↓
Response
This is fine for a proof of concept.
But real-world agent systems need more flexibility.
Different tasks often need different models:
Task Better Model Strategy
Quick summarization Fast, low-cost model
Complex coding Strong coding model
Long document analysis Long-context model
Reasoning-heavy planning Reasoning model
Multilingual writing Model strong in that language
Background automation Cheap and reliable model
Production fallback Backup provider
If your agent is deeply coupled to one provider, every optimization becomes harder.
You cannot easily answer questions like:
What happens if the provider is down?
What if latency spikes?
What if another model is cheaper for simple tasks?
What if a new model is better for coding?
What if a user wants Claude for writing but GPT for structured reasoning?
What if you want to route Chinese tasks to a different model than English tasks?
This is not just a model problem.
It is an infrastructure problem.
The Better Pattern: Provider-Agnostic Agents
A more scalable architecture looks like this:
CopyUser
↓
Agent Runtime
↓
Model Router
↓
AI API Gateway
↓
Multiple Model Providers
In this design, your agent does not talk directly to every model provider.
Instead, it talks to a unified gateway.
The gateway handles access to multiple models, while your agent focuses on:
user intent,
planning,
tool use,
memory,
task execution,
result evaluation.
This keeps your core agent logic clean.
Why OpenAI-Compatible APIs Matter
One of the easiest ways to build provider-agnostic agents is to use an OpenAI-compatible API format.
Many developers already understand this request shape:
Copy{
"model": "gpt-4o",
"messages": [
{
"role": "user",
"content": "Explain model routing for AI agents."
}
]
}
If your gateway supports this format, your agent code can stay mostly the same even when the underlying model changes.
That is the idea behind platforms like OpenRain.
OpenRain provides an AI API Gateway with OpenAI-compatible access to many model providers through a unified API layer.
Instead of wiring your agent directly to each provider, you can call one endpoint and manage model access behind the gateway.
A Simple Example
Here is a minimal Python example using an OpenAI-compatible gateway:
Copyimport os
import requests
API_KEY = os.getenv("OPENRAIN_API_KEY")
response = requests.post(
"https://openrain.ai/v1/chat/completions",
headers={
"Authorization": f"Bearer {API_KEY}",
"Content-Type": "application/json",
},
json={
"model": "gpt-4o",
"messages": [
{
"role": "user",
"content": "Explain why AI agents need model routing."
}
],
},
timeout=60,
)
print(response.json()["choices"][0]["message"]["content"])
The important part is not the specific model name.
The important part is that your application talks to a stable interface.
That gives you room to experiment, route, fallback, and optimize later.
Add a Simple Model Router
Once your agent uses a unified API, you can introduce routing logic.
For example:
Copydef choose_model(task_type: str) -> str:
if task_type == "coding":
return "code-strong-model"
if task_type == "summary":
return "fast-low-cost-model"
if task_type == "reasoning":
return "reasoning-model"
return "general-purpose-model"
Then your agent can do this:
Copytask_type = "summary"
model = choose_model(task_type)
Send request to the gateway with the selected model
This is a small change, but it unlocks a better architecture.
Your agent can now choose models based on task type instead of being hardcoded to one provider.
Add Fallback
Production agents need fallback.
LLM providers can fail.
Requests can timeout.
Rate limits happen.
Models can be temporarily unavailable.
A basic fallback function might look like this:
Copydef call_with_fallback(client, models, messages):
last_error = None
for model in models:
try:
return client.chat(model=model, messages=messages)
except Exception as error:
last_error = error
print(f"{model} failed. Trying next model...")
raise last_error
Now your agent has a recovery path.
Instead of failing immediately, it can try another model.
This is especially useful for personal agents that run background jobs, scheduled tasks, research workflows, or production automation.
The Real Shift: From Model Usage to Model Orchestration
The next generation of AI applications will not be built around a single model.
They will be built around model orchestration.
A serious agent should be able to decide:
which model to use,
when to use it,
how much budget to spend,
when to retry,
when to fallback,
when to use a faster model,
when to use a more capable model.
This is why a model gateway becomes a foundational layer.
It turns model access into infrastructure.
Final Thought
Hardcoding one LLM provider is fine for a weekend demo.
But if you are building a real AI agent, you probably want:
provider flexibility,
model routing,
fallback,
usage tracking,
cost control,
lower integration complexity.
That is why I believe every serious AI agent framework should start with a provider-agnostic model layer.
This is also the direction I am exploring with OpenRain: a unified AI API Gateway for developers building agents, tools, and AI-native applications.
If your agent can switch models without changing core application code, you are already designing it the right way.
You can check out OpenRain here:
Copy
Article 2
---
title: The Missing Layer in Most AI Agent Frameworks: A Model Gateway
published: false
description: AI agents need more than prompts, tools, and memory. They need reliable multi-model infrastructure.
tags: ai, agents, llm, devtools
---
The Missing Layer in Most AI Agent Frameworks: A Model Gateway
When developers talk about AI agent frameworks, they usually talk about:
planning,
tool calling,
memory,
RAG,
workflows,
multi-agent collaboration.
These are important.
But there is one layer that often gets ignored:
the model gateway.
Most agent frameworks assume model access is simple.
Call an LLM.
Get a response.
Continue the workflow.
In reality, model access is one of the most important infrastructure decisions in an agent system.
If your agent depends on a single provider, a single endpoint, or a single model, the rest of your architecture becomes fragile.
Agents Are Not Normal Chatbots
A chatbot usually answers one user message at a time.
An agent may need to perform a sequence of tasks:
CopyUnderstand goal
↓
Create plan
↓
Call tools
↓
Read files
↓
Search web
↓
Use model
↓
Validate result
↓
Retry if needed
↓
Return final answer
This process may involve many model calls.
Some calls are simple.
Some calls are expensive.
Some calls require long context.
Some calls need strong reasoning.
Some calls need fast latency.
Using the same model for every step is often wasteful.
A Personal Agent Needs Multiple Model Modes
Imagine a personal general-purpose agent.
It helps you:
summarize documents,
draft emails,
write code,
prepare reports,
compare products,
translate content,
search information,
organize tasks,
automate workflows.
Should all these tasks use the same model?
Probably not.
A better system uses different model strategies:
CopySimple classification → cheap model
Long document summary → long-context model
Code generation → coding model
Strategic planning → reasoning model
Polished writing → writing model
Fallback path → backup model
That means the agent needs a routing layer.
And the routing layer needs a stable way to access many models.
That is where a model gateway fits.
What Is a Model Gateway?
A model gateway sits between your application and model providers.
CopyYour Application
↓
Model Gateway
↓
OpenAI / Claude / Gemini / DeepSeek / Qwen / Mistral / Others
The gateway gives your application one consistent way to access multiple models.
A good gateway can help with:
unified API access,
provider abstraction,
model routing,
automatic failover,
usage tracking,
latency optimization,
cost visibility,
API key management.
OpenRain is an example of this type of infrastructure.
It provides an AI API Gateway with OpenAI-compatible APIs, access to many model providers, global routing, failover, and usage statistics.
Why This Matters for Agent Frameworks
An agent framework usually has several internal modules:
CopyIntent Classifier
Planner
Tool Caller
Memory Manager
Evaluator
Final Response Generator
Each module may have different model requirements.
For example:
Agent Module Model Requirement
Intent classifier fast and cheap
Planner strong reasoning
Tool-call formatter reliable structured output
Memory summarizer low-cost summarization
Evaluator accurate judgment
Final writer strong language quality
If your framework can route each module to a different model, your agent becomes more efficient and reliable.
Without a model gateway, this gets messy quickly.
A Simple Architecture
Here is a practical architecture:
CopyUser
↓
Agent API
↓
Task Classifier
↓
Planner
↓
Tool Runtime
↓
Memory Runtime
↓
Model Router
↓
OpenRain AI Gateway
↓
Model Providers
The key idea:
The agent should not care which provider serves the final model response.
The agent should only care about capability.
For example:
CopyNeed fast classification
Need strong code generation
Need long-context reasoning
Need low-cost summarization
Need high-quality writing
The model router maps capability to model.
The gateway handles the access layer.
Example: Capability-Based Routing
Instead of writing code like this:
Copymodel = "some-specific-provider-model"
You can design around capabilities:
CopyCAPABILITY_TO_MODEL = {
"fast_classification": "fast-low-cost-model",
"code_generation": "code-strong-model",
"long_context": "long-context-model",
"reasoning": "reasoning-model",
"writing": "general-writing-model",
}
Then:
Copydef select_model(capability: str) -> str:
return CAPABILITY_TO_MODEL.get(capability, "general-writing-model")
Your agent can say:
CopyI need reasoning.
The router decides which model to use.
This makes your framework more maintainable.
Add Failover at the Gateway Layer
Failover should not be an afterthought.
If your agent is executing a multi-step workflow, one failed model call can break the whole task.
A better design has fallback candidates:
CopyMODEL_FALLBACKS = {
"reasoning": [
"primary-reasoning-model",
"backup-reasoning-model",
"general-purpose-model",
],
"writing": [
"primary-writing-model",
"backup-writing-model",
],
}
Then your runtime can try the next model if the first one fails.
This is much easier when all models are accessed through a unified API.
Usage Tracking Is Part of Agent Design
Agents can be expensive.
A single user request may trigger:
planning call,
search query generation,
document summarization,
tool result analysis,
final answer generation,
evaluation call.
That means one visible user action may contain many hidden model calls.
Without usage tracking, you may not know:
which tasks are expensive,
which models are overused,
where latency comes from,
how often fallback happens,
which users or workflows consume the most tokens.
This is why a gateway with usage statistics is valuable.
It gives developers operational visibility.
The Gateway Is Not the Agent
It is important to separate responsibilities.
A model gateway does not replace your agent framework.
It supports it.
The agent framework handles:
goals,
tools,
memory,
planning,
execution,
evaluation.
The model gateway handles:
model access,
provider abstraction,
routing support,
reliability,
usage visibility.
Together, they form a much better foundation.
Final Thought
Most AI agent discussions focus on intelligence.
But production agents also need infrastructure.
A reliable agent needs:
the right model for each task,
fallback when models fail,
cost control,
usage tracking,
provider flexibility,
clean API abstraction.
That is why I think the model gateway is one of the missing layers in many AI agent frameworks.
If you are building agents, do not just ask:
Which model should I use?
Ask:
How will my agent access, route, monitor, and fallback across models?
That question leads to a much better architecture.
I am building OpenRain to make this layer easier for developers.
OpenRain provides an OpenAI-compatible AI API Gateway for accessing multiple model providers through one unified interface.
https://openrain.ai
Copy
---
# Article 3
markdown
title: A Practical Architecture for Building a Personal General-Purpose AI Agent
published: false
description: A simple layered architecture for building personal AI agents with tools, memory, planning, routing, and multi-model access.
tags: ai, agents, architecture, productivity
A Practical Architecture for Building a Personal General-Purpose AI Agent
A useful personal AI agent is not just a chatbot.
It is a system that can understand your goals, remember your preferences, use tools, choose the right model, and complete tasks across different contexts.
That sounds complex.
But the architecture can be simple if you break it into layers.
In this article, I will describe a practical architecture for building a personal general-purpose AI agent.
The goal is not to build a fully autonomous sci-fi assistant.
The goal is to build something useful, reliable, and extensible.
The Core Idea
A personal AI agent should work like this:
CopyUser gives a goal
↓
Agent understands intent
↓
Agent creates a plan
↓
Agent chooses tools
↓
Agent chooses models
↓
Agent executes steps
↓
Agent checks result
↓
Agent returns useful output
This requires more than one prompt.
It requires an agent runtime.
The 7-Layer Architecture
Here is the architecture I recommend:
Copy1. Interface Layer
- User Profile Layer
- Intent Layer
- Planning Layer
- Tool Layer
- Memory Layer
Model Gateway Layer
Let’s go through each one.Interface Layer
This is where the user interacts with the agent.
It could be:
web app,
CLI,
browser extension,
mobile app,
Slack bot,
Telegram bot,
desktop assistant.
The interface should be thin.
It should not contain your model routing, memory logic, or tool execution logic.
A good structure is:
CopyFrontend
↓
Agent API
↓
Agent Runtime
This keeps your agent portable across different interfaces.
- User Profile Layer A personal agent needs to know basic user preferences.
For example:
Copy{
"language": "English",
"tone": "clear and practical",
"timezone": "Asia/Shanghai",
"preferred_format": "markdown",
"technical_stack": ["Python", "TypeScript", "PostgreSQL"]
}
This helps the agent personalize output.
For example, if the user often publishes on DEV.to, the agent can format drafts in Markdown with front matter, tags, headings, and code blocks.
Personalization should be useful, not invasive.
The user should be able to inspect, edit, or delete stored preferences.
- Intent Layer The intent layer classifies what the user is asking for.
Examples:
User Request Intent
“Summarize this document” document_summary
“Fix this bug” coding_help
“Write a blog post” content_generation
“Compare these tools” research
“Plan my week” planning
“Extract action items” information_extraction
Intent classification helps the agent decide what workflow to run.
A coding task should not use the same workflow as a writing task.
A research task should not use the same model strategy as a quick classification task.
- Planning Layer The planning layer breaks a goal into steps.
Example:
CopyGoal:
Write a DEV.to article about AI model gateways.
Plan:
- Identify target audience.
- Explain the problem with single-provider agents.
- Introduce model gateway architecture.
- Add a simple code example.
- Explain routing and fallback.
- End with a practical takeaway. Not every task needs planning.
If the user asks:
CopyTranslate this sentence into French.
The agent can answer directly.
But if the user asks:
CopyCreate a launch content plan for my AI API gateway.
Planning becomes useful.
- Tool Layer Tools allow the agent to interact with the world.
Examples:
Copysearch_web(query)
read_file(path)
write_note(title, content)
query_database(sql)
send_email(to, subject, body)
create_calendar_event(data)
call_internal_api(endpoint, payload)
Tools should be explicit and permissioned.
The model can suggest an action, but the runtime should decide whether the action is allowed.
For personal agents, this is especially important.
You probably do not want an agent sending emails, deleting files, or charging a credit card without confirmation.
- Memory Layer Memory gives the agent continuity.
There are several types of memory:
Short-Term Memory
Current conversation context.
CopyWhat is the user asking now?
What files are being discussed?
What steps have already been completed?
Long-Term Memory
Persistent user preferences.
CopyThe user prefers Markdown.
The user writes for developers.
The user is building OpenRain.
Project Memory
Context for a specific project.
CopyProject: OpenRain content strategy
Audience: developers building AI agents
Positioning: AI API Gateway for multi-model access
Operational Memory
Execution traces.
CopyWhich model was used?
How many tokens were consumed?
Which tools were called?
Did fallback happen?
This last type is often ignored, but it is very important for improving agent reliability.
- Model Gateway Layer The model gateway layer handles model access.
This is where a platform like OpenRain fits naturally.
OpenRain provides an AI API Gateway with:
OpenAI-compatible API access,
access to many AI models,
unified model calling,
smart routing,
automatic failover,
usage statistics,
developer-friendly integration.
For an agent framework, this means you do not need to hardcode every provider directly into your agent runtime.
Instead of this:
CopyAgent → OpenAI SDK
Agent → Anthropic SDK
Agent → Gemini SDK
Agent → DeepSeek SDK
Agent → Qwen SDK
You can design this:
CopyAgent → OpenRain Gateway → Multiple Model Providers
That makes your agent easier to maintain.
Example Workflow
User request:
CopyWrite a technical article about why personal AI agents need model routing.
The agent runtime may process it like this:
CopyInterface:
Receive request
User Profile:
User prefers practical developer writing
Intent:
content_generation
Planning:
Create article outline
Tool Layer:
Optionally fetch product context
Memory:
Recall OpenRain positioning
Model Gateway:
Use a strong writing model
Final:
Return DEV.to-ready Markdown article
The result feels simple to the user.
But internally, the system is layered.
That is what makes it extensible.
Why the Model Gateway Should Be Designed Early
Many developers start with memory or tools first.
That is understandable.
But I think model access should be designed early.
Why?
Because everything depends on it.
Your planner needs a model.
Your summarizer needs a model.
Your evaluator needs a model.
Your tool-call generator needs a model.
Your final response writer needs a model.
If model access is messy, the whole agent becomes messy.
A gateway gives you a clean foundation.
Final Thought
A personal general-purpose AI agent is not one giant prompt.
It is a layered system.
The model is important, but the infrastructure around the model is just as important.
A practical agent needs:
interface,
user profile,
intent detection,
planning,
tools,
memory,
model routing,
fallback,
usage visibility.
If you build these layers cleanly, your agent becomes easier to extend over time.
That is the kind of architecture I believe more AI applications will move toward.
And it is the kind of infrastructure OpenRain is designed to support.
Copy
Article 4
---
title: From Chatbot to Agent: Why Tool Use Is Not Enough
published: false
description: Tool calling is only one part of building reliable AI agents. You also need routing, memory, budget control, fallback, and observability.
tags: ai, agents, llm, python
---
From Chatbot to Agent: Why Tool Use Is Not Enough
Tool calling is one of the most exciting features in modern AI applications.
It allows a model to interact with external systems:
search the web,
read files,
query databases,
call APIs,
create calendar events,
send messages,
run workflows.
Because of this, many developers think:
If my chatbot can call tools, it is now an agent.
Not quite.
Tool use is important, but it is not enough.
A real agent also needs routing, memory, budget control, fallback, and observability.
Chatbot vs Agent
A chatbot usually does this:
CopyUser message
↓
Model response
An agent does this:
CopyUser goal
↓
Understand intent
↓
Plan steps
↓
Select tools
↓
Select model
↓
Execute actions
↓
Validate result
↓
Retry or fallback
↓
Return final output
The difference is not just tool use.
The difference is workflow.
Tool Use Without Control Is Risky
Imagine an agent that can call tools but has no strong runtime control.
That can create problems.
For example:
It may call the wrong tool.
It may call a tool too many times.
It may spend too much money.
It may use an expensive model for simple tasks.
It may fail if one provider is unavailable.
It may produce inconsistent results.
It may be hard to debug.
This is why agent engineering is not just prompt engineering.
It is systems engineering.
The Five Capabilities Agents Need
A practical agent should have at least five capabilities:
Copy1. Tool use
2. Memory
3. Model routing
4. Budget control
5. Failover and observability
Let’s look at each.
1. Tool Use
Tools allow the agent to act.
A tool can be simple:
Copydef create_note(title: str, content: str):
return {
"status": "created",
"title": title,
"content": content
}
But the important part is not the function itself.
The important part is the control layer around it.
Before executing a tool, your runtime should know:
Is this tool allowed?
Does the user need to confirm?
Are the inputs valid?
Is there a rate limit?
Should the action be logged?
Can this action be undone?
The model should not be the only decision-maker.
2. Memory
Memory helps the agent become personal.
For example:
Copy{
"preferred_output": "markdown",
"writing_style": "technical and practical",
"current_project": "OpenRain",
"audience": "developers building AI agents"
}
Memory helps the agent avoid asking the same questions repeatedly.
But memory should be scoped.
A good memory system should support:
user preferences,
project context,
task state,
execution history,
deletion and editing.
Do not store everything blindly.
Store what improves future work.
3. Model Routing
Different tasks need different models.
A simple model router might look like this:
Copydef select_model(task_type: str, priority: str) -> str:
if priority == "high":
return "premium-reasoning-model"
if task_type == "classification":
return "fast-low-cost-model"
if task_type == "coding":
return "code-strong-model"
if task_type == "writing":
return "writing-model"
return "general-purpose-model"
This allows your agent to optimize for the task.
Using the same model for everything is simple, but often inefficient.
4. Budget Control
Agents can trigger multiple model calls for one user request.
For example:
CopyUser asks for a research summary
↓
Generate search queries
↓
Analyze search results
↓
Summarize sources
↓
Compare claims
↓
Write final answer
↓
Evaluate answer
That may be six or more model calls.
Without budget control, costs can grow quickly.
A practical agent should track:
input tokens,
output tokens,
model used,
task type,
cost estimate,
number of retries,
fallback events.
This is why usage visibility is important.
OpenRain includes usage statistics so developers can better understand how their AI applications consume model resources.
5. Failover and Observability
LLM providers are not perfect.
Requests fail.
Models timeout.
Rate limits happen.
Network issues happen.
A reliable agent needs fallback.
Example:
Copydef call_models_with_fallback(client, messages, candidates):
errors = []
for model in candidates:
try:
return client.chat(model=model, messages=messages)
except Exception as error:
errors.append({
"model": model,
"error": str(error)
})
raise RuntimeError(f"All models failed: {errors}")
This is simple, but powerful.
The agent can continue operating even if the primary model fails.
Observability then helps you understand what happened:
Copy{
"task_type": "writing",
"primary_model": "model-a",
"fallback_model": "model-b",
"fallback_used": true,
"latency_ms": 3200,
"tokens": 1800
}
Without logs, you are guessing.
With logs, you can improve the system.
Where OpenRain Fits
OpenRain is useful as the model infrastructure layer.
Instead of building direct integrations with many providers, your agent can call an OpenAI-compatible gateway.
CopyAgent Runtime
↓
OpenRain AI Gateway
↓
Multiple Model Providers
This helps with:
unified API access,
multi-model support,
smart routing,
automatic failover,
usage tracking,
simpler developer experience.
For agent builders, this means less time maintaining provider glue code and more time improving the actual agent experience.
Final Thought
Tool calling is important.
But tool calling alone does not make a reliable agent.
A real agent needs a runtime.
It needs memory.
It needs routing.
It needs budget awareness.
It needs fallback.
It needs observability.
It needs clean infrastructure.
That is the shift from chatbot building to agent engineering.
If you are building agents, do not only ask:
What tools can my model call?
Also ask:
How does my agent choose models, recover from failure, track cost, and improve over time?
That is where the real engineering begins.
I am exploring this infrastructure layer with OpenRain, an AI API Gateway for developers building multi-model AI applications and agents.
https://openrain.ai
Copy
---
# Article 5
markdown
title: Building AI Apps with One API Key Across Multiple Models
published: false
description: A practical introduction to using a unified AI API gateway to simplify model access for AI applications and agents.
tags: ai, api, llm, devtools
Building AI Apps with One API Key Across Multiple Models
One of the annoying parts of building AI applications is managing model access.
You may want to use:
OpenAI for general reasoning,
Claude for writing or long-context tasks,
Gemini for multimodal workflows,
DeepSeek for reasoning,
Qwen for Chinese and open-source model use cases,
Mistral for coding,
other providers for specialized needs.
Each provider may have different:
API keys,
SDKs,
request formats,
rate limits,
billing dashboards,
model names,
error formats,
availability patterns.
For small experiments, this is manageable.
For real applications, it becomes operational overhead.
That is why I like the idea of using a unified AI API gateway.
The Problem
A typical multi-model application can quickly become messy:
CopyApp
├── OpenAI client
├── Anthropic client
├── Gemini client
├── DeepSeek client
├── Qwen client
├── Mistral client
├── custom retry logic
├── custom usage tracking
└── custom fallback logic
This is not where most developers want to spend their time.
If you are building an AI product, you probably want to focus on:
user experience,
workflows,
prompts,
tools,
evaluation,
product logic.
Not provider glue code.
The Gateway Pattern
A cleaner architecture looks like this:
CopyYour App
↓
Unified AI API Gateway
↓
Multiple Model Providers
Your app calls one API.
The gateway handles the model access layer.
OpenRain is built around this idea.
It provides an AI API Gateway with OpenAI-compatible APIs, access to many models, smart routing, automatic failover, and usage statistics.
For developers, this means simpler integration.
Why OpenAI-Compatible Matters
OpenAI-compatible APIs are useful because many developers and tools already understand the request format.
A common chat request looks like this:
Copy{
"model": "gpt-4o",
"messages": [
{
"role": "user",
"content": "Write a product description for an AI API gateway."
}
]
}
If your gateway supports this style, you can keep your application code simple.
You can switch models by changing configuration instead of rewriting integrations.
Minimal Python Example
Copyimport os
import requests
api_key = os.getenv("OPENRAIN_API_KEY")
response = requests.post(
"https://openrain.ai/v1/chat/completions",
headers={
"Authorization": f"Bearer {api_key}",
"Content-Type": "application/json",
},
json={
"model": "gpt-4o",
"messages": [
{
"role": "user",
"content": "Give me 5 ideas for an AI agent app."
}
],
},
)
data = response.json()
print(data["choices"][0]["message"]["content"])
This is enough to start building.
Minimal JavaScript Example
Copyconst response = await fetch("https://openrain.ai/v1/chat/completions", {
method: "POST",
headers: {
"Authorization": Bearer ${process.env.OPENRAIN_API_KEY},
"Content-Type": "application/json"
},
body: JSON.stringify({
model: "gpt-4o",
messages: [
{
role: "user",
content: "Explain AI API gateways in one paragraph."
}
]
})
});
const data = await response.json();
console.log(data.choices[0].message.content);
Again, the point is not just the code.
The point is the abstraction.
Your app talks to one API layer.
Use Case: AI Agent Model Routing
Suppose you are building a personal AI agent.
You may define model roles like this:
Copy{
"fast": "fast-low-cost-model",
"reasoning": "reasoning-model",
"writing": "writing-model",
"coding": "coding-model",
"fallback": "backup-model"
}
Then your agent can choose based on task type:
Copydef model_for_task(task_type):
if task_type == "coding":
return "coding-model"
if task_type == "reasoning":
return "reasoning-model"
if task_type == "summary":
return "fast-low-cost-model"
return "writing-model"
This keeps your app flexible.
Use Case: Fallback
If a model fails, your app can try another one.
Copydef call_with_fallback(messages, models):
for model in models:
try:
return call_model(model, messages)
except Exception:
continue
raise Exception("All models failed")
With a unified gateway, fallback is easier because all candidates can share the same API interface.
Use Case: Cost Control
Not every task needs a premium model.
For example:
CopyTag generation → cheap model
Daily summary → cheap model
Blog post draft → balanced model
Legal analysis → premium model
Complex coding → premium model
A unified gateway plus usage statistics makes it easier to understand and optimize model spend.
Why This Matters
AI applications are moving from simple chat interfaces to complex workflows.
That means developers need better infrastructure.
A modern AI app may need:
multiple models,
multiple providers,
retries,
routing,
fallback,
usage tracking,
team-level API keys,
cost visibility.
Building all of that yourself takes time.
Using a gateway lets you start with a cleaner foundation.
Final Thought
The future of AI development is multi-model.
No single model will be the best choice for every task.
Developers need a simple way to access, route, monitor, and fallback across models.
That is why the gateway pattern is becoming increasingly important.
If you are building AI apps or agents, start by asking:
Can my application switch models without changing core code?
If yes, your architecture is already in a better place.
OpenRain is an AI API Gateway designed around this idea: one OpenAI-compatible API to access multiple AI models with routing, failover, and usage visibility.
Top comments (0)