RDAI: Building a Self-Healing Multi-Provider AI Orchestrator for Python
One Python SDK. Any AI Provider. Automatic Failover.
Modern AI applications are increasingly built on APIs from multiple LLM providers.
Gemini. OpenAI. Groq. Claude. DeepSeek. New models appear constantly, and each provider brings different capabilities, pricing, latency, limits, and failure modes.
But there is a problem that is easy to overlook:
What happens when the AI provider your application depends on suddenly stops responding?
That question became the starting point for RDAI.
RDAI is an open-source Python AI orchestration SDK and CLI designed around a simple idea:
An AI provider failure should not have to become an application failure.
🚨 The Problem: Your AI Provider Is a Dependency
A typical application starts simply:
Your Application
|
v
One AI Provider
|
v
Response
Then reality happens.
A provider can hit a rate limit, time out, become temporarily unavailable, reject a request, or experience an outage.
Suddenly:
Your Application
|
v
One AI Provider
|
Provider fails
|
v
Application fails
The natural response is to add retries and another provider:
try:
response = gemini.generate(prompt)
except:
try:
response = openai.generate(prompt)
except:
response = groq.generate(prompt)
That may be enough for a prototype.
But as the application grows, so does the surrounding infrastructure:
- retry logic
- timeout handling
- provider selection
- API-key discovery
- health checks
- fallback chains
- recovery behavior
- debugging and diagnostics
I wanted that complexity to live in one reusable layer instead of being rewritten inside every application.
That layer became RDAI.
🧠 From One Brain to Multiple Brains
I like thinking about LLM providers as different brains.
Each brain can have different strengths and availability.
Instead of an application being permanently coupled to one provider, RDAI places an orchestration layer in the middle:
YOUR APPLICATION
|
v
+---------------+
| RDAI |
| AI Orchestrator|
+-------+-------+
|
+-------------+-------------+
| | |
v v v
Gemini OpenAI Groq
| | |
+-------------+-------------+
|
FAILOVER
Your application talks to RDAI.
RDAI handles the provider layer.
That separation is the foundation of the project.
⚡ How RDAI Routes Requests
RDAI provides two routing strategies.
Smart Routing
Smart routing can use request traits and provider capabilities to prefer a better-fit available provider.
Conceptually:
Prompt
|
v
Intent / Trait Detection
|
v
Provider Matching
|
v
Best-Fit Available Brain
|
failure?
v
Next Available Brain
The current implementation keeps this intentionally lightweight: request signals such as coding-oriented terms or speed-oriented terms can influence provider ordering, while each provider exposes traits that the router can match.
The goal is not to make routing mysterious.
The goal is to make the default behavior useful while keeping the architecture extensible.
Manual Priority
Sometimes developers already know exactly how their provider chain should work.
For example:
strategy: manual
providers:
- openai
- groq
- gemini
This gives you a predictable preference order.
If the preferred provider fails, other available providers remain eligible for fallback.
OpenAI
|
| failure
v
Groq
|
| failure
v
Gemini
No provider-specific fallback code is required in the application.
🛡️ Automatic Failover
The most important part of RDAI is not the first response.
It is what happens when that response cannot be produced.
The core failover flow is:
REQUEST
|
v
Provider A
|
+------+------+
| |
SUCCESS FAILURE
| |
v v
RETURN Provider B
|
+------+------+
| |
SUCCESS FAILURE
| |
v v
RETURN Provider C
RDAI also maintains circuit-breaker state so repeatedly failing providers can temporarily be skipped before being probed again.
In practical terms:
Route
↓
Call provider
↓
Failure?
├── No → return response
└── Yes
↓
update failure state
↓
try next eligible brain
Reliability becomes part of the application architecture instead of an afterthought.
🩺 A CLI That Treats AI Infrastructure Like Infrastructure
I did not want RDAI to be just another Python wrapper.
The project also includes a CLI for setup, diagnostics, configuration, and benchmarking.
rdai init
Start with:
pip install rdai
Then:
rdai init
The interactive wizard lets developers choose providers and a routing strategy, then creates the local configuration files.
The intended workflow is:
Install
↓
Configure
↓
Verify
↓
Generate
rdai doctor
Before shipping, you need to know which brains are actually reachable.
Run:
rdai doctor
The diagnostic flow checks configured provider credentials and performs live checks, reporting status and measured latency.
Conceptually:
Provider Key Status Live Check Latency
------------------------------------------------------
Gemini ✔ DETECTED 🟢 ALIVE 420ms
Groq ✔ DETECTED 🟢 ALIVE 180ms
OpenAI ✔ DETECTED 🔴 ERROR -
That turns a vague question like:
"Why isn't my AI application working?"
into a much more actionable question:
"Which provider is failing, and is it authentication, connectivity, rate limiting, timeout, or another provider-side error?"
🔐 Configuration Without Hardcoding Secrets
RDAI separates routing configuration from credentials.
Provider credentials can be discovered from the process environment or a local .env file, while routing policy can live in rdai.yaml.
For example:
GEMINI_API_KEY=your_key_here
GROQ_API_KEY=your_key_here
OPENAI_API_KEY=your_key_here
And:
strategy: smart
providers:
- gemini
- openai
- groq
Environment values take precedence over .env values.
The result is a cleaner separation:
Secrets
↓
Environment / .env
Routing Policy
↓
rdai.yaml
Application Logic
↓
RDAI SDK
💻 One Python API Across Multiple Providers
Without an orchestration layer, applications often end up learning several provider SDKs.
With RDAI, application code can stay focused on the task:
from rdai import AI
ai = AI()
response = ai.generate(
"Explain dependency inversion in simple words."
)
print(response)
The public API stays small while provider-specific adapters live underneath it.
🎯 Overriding Default Models
Different applications may prefer different models.
RDAI supports model overrides without changing the overall orchestration API:
from rdai import AI
ai = AI(
models={
"gemini": "gemini-1.5-flash",
"groq": "llama3-8b-8192",
}
)
response = ai.generate("Hello world!")
The architecture keeps provider selection separate from the application-facing API.
🔌 Bring Your Own Model
The AI ecosystem moves quickly.
A useful orchestration layer should not have to know every provider that will ever exist.
That is why RDAI exposes a BaseProvider abstraction for custom adapters.
from rdai.providers.base import BaseProvider
from rdai import AI
class CustomProvider(BaseProvider):
def generate(self, prompt, **kwargs):
return "Private model response"
ai = AI(
providers=[CustomProvider(api_key="your_key")]
)
print(ai.generate("Hello custom engine!"))
This opens the door to private endpoints, internal models, and future providers without changing the application-level API.
🏗️ The Architecture
At a high level, RDAI separates responsibilities into distinct layers:
Environment
(.env / vars)
|
v
Credential Discovery
|
v
Provider Registry
|
v
Smart / Manual Router
|
v
Failover Engine
|
v
Provider Adapters
|
v
Response
That separation matters.
The application should not need to know how Gemini, OpenAI, Groq, Claude, or a custom provider implements its API.
The adapter handles that boundary.
The router decides what to try.
The failover layer decides what happens when a provider fails.
🌍 Why This Matters for AI Applications
There is a larger architectural shift happening in AI development.
We are moving from:
One Application
|
One Model
|
One Provider
toward:
One Application
|
Orchestration Layer
|
Multiple Models
|
Multiple Providers
That does not mean every application needs ten providers.
It means the application can be designed so that provider choice is not an irreversible architectural decision.
That is the space RDAI is designed to explore.
🧪 Building RDAI: From Prototype to Hardened Release
The first versions focused on proving the core idea:
- multi-provider adapters
- routing
- automatic failover
- configuration discovery
- an interactive CLI
- diagnostics
Then real testing exposed places where the implementation needed to become stronger.
The 1.0.2 release focused on hardening the foundation: unifying the routing/failover architecture, resolving a missing runtime dependency, adding explicit REST-provider timeouts, and improving diagnostic error classification.
That process reinforced an important lesson:
Resilience is defined by the failure path, not just the happy path.
A demo can prove that an AI response works.
A real developer tool must also explain what happens when it doesn't.
🧭 RDAI vs. Building the Fallback Layer Yourself
There are two broad approaches.
Build it inside every application
Application
|
+-- Provider A logic
+-- Provider B logic
+-- Retry logic
+-- Timeout logic
+-- Health checks
+-- Fallback logic
+-- Diagnostics
This can work, but the infrastructure becomes part of every application.
Put the orchestration behind an SDK
Application
|
v
RDAI
|
+-- Routing
+-- Failover
+-- Provider adapters
+-- Diagnostics
+-- CLI
The application stays smaller, while the resilience layer can evolve independently.
That is the problem RDAI is trying to solve.
✨ The Developer Experience Matters
A technically capable system can still be frustrating to use.
I wanted the RDAI workflow to feel simple:
pip install rdai
then:
rdai init
then:
rdai doctor
then:
from rdai import AI
ai = AI()
response = ai.generate("Build a resilient AI workflow.")
The goal is not to make developers think about orchestration every time they make an AI call.
The goal is to make the resilient path the easy path.
🔭 What's Next?
The next direction for RDAI is moving from resilience toward visibility.
The roadmap includes live streaming-oriented features such as:
Live Streaming
|
+-- Brain Activity
+-- Frontend Events
+-- Provider Timeline
Imagine seeing a request move through the system:
🧠 Gemini
|
| Timeout
v
⚡ Failover
|
v
🧠 Groq
|
v
✅ Streaming Response
The long-term idea is simple:
Don't just make AI resilient. Make the resilience visible.
📈 Why I Think the Problem Is Bigger Than One Package
AI providers are moving fast.
New models appear.
Old models are deprecated.
Pricing changes.
Rate limits change.
Capabilities change.
Application architecture should not have to change every time the underlying provider changes.
An orchestration layer can provide a stable interface while the infrastructure underneath evolves.
That is the long-term direction behind RDAI.
👨💻 Who Is RDAI For?
RDAI is particularly interesting for:
- Python developers building LLM applications
- AI engineers working with multiple providers
- Startups that need a resilient AI integration without building the routing layer from scratch
- Developers who want provider health and latency visibility from the CLI
- Teams experimenting with custom or private models
🚀 Get Started
Install RDAI:
pip install rdai
Initialize your setup:
rdai init
Check your providers:
rdai doctor
Then use the SDK:
from rdai import AI
ai = AI()
response = ai.generate(
"What happens when my primary AI provider goes down?"
)
print(response)
🌱 Open Source, Built in Public
One of the most useful parts of building RDAI has been seeing how an open-source project changes once other developers start using it.
A bug report becomes a fix.
A feature request becomes a design discussion.
A contribution becomes part of the roadmap.
That feedback loop is a core part of where I want RDAI to go.
The project is open source, and issues, ideas, and contributions are welcome.
Final Thought
RDAI started with a simple question:
What if your AI application had more than one brain?
The answer is not to pretend that providers never fail.
The answer is to build an architecture that can route, recover, diagnose, and evolve when they do.
That is what RDAI is trying to become:
A developer-first orchestration layer for resilient, multi-provider AI applications.
🔗 Explore RDAI
- GitHub: https://github.com/ranajitdharpersonal/rdai
- PyPI: https://pypi.org/project/rdai/
- Website: https://www.ranajitdhar.in
⭐ Built something with RDAI? I'd genuinely love to see it. Open an issue, share feedback, or contribute a new idea.
Built with ❤️ by Ranajit Dhar for the next generation of AI developers.
Top comments (0)