RelayRouter

Posted on Oct 12

One Gateway, Multiple LLMs: Building a Flexible Python Integration

#api #python #ai #tutorial

Building an application with a large language model often starts with a simple API request. But as your application grows, the integration layer can become harder to maintain.

You may want to experiment with different models, change the model used for a particular task, or monitor requests from one place.

A useful starting point is to separate your application logic from your model configuration.

In this tutorial, we'll explore that pattern using Python and RelayRouter as the multi-model API gateway.

1. Why a Model-Abstraction Layer Matters

Consider an application that uses AI for several tasks:

Summarizing customer feedback
Categorizing support requests
Drafting product descriptions
Extracting structured information

These tasks do not necessarily require the same model.

Hardcoding a provider, model name, and credentials throughout your application makes experimentation and maintenance more difficult.

Instead, define a small configuration layer.

2. Prepare Your Environment

Install the Python SDK:

pip install openai

Configure your environment variables:

export RELAYROUTER_API_KEY="your_api_key"
export RELAYROUTER_BASE_URL="https://api.relayrouter.ai/v1"
export RELAYROUTER_MODEL="your_available_model_id"

Use the Base URL and model identifier documented in your RelayRouter dashboard. The URL above illustrates an OpenAI-compatible configuration; confirm the currently supported endpoint before running the example.

Never commit actual API keys to your repository.

3. Create a Reusable Client

import os
from openai import OpenAI

client = OpenAI(
    api_key=os.environ["RELAYROUTER_API_KEY"],
    base_url=os.environ["RELAYROUTER_BASE_URL"],
)

def ask_model(prompt, model=None):
    selected_model = model or os.environ["RELAYROUTER_MODEL"]

    response = client.chat.completions.create(
        model=selected_model,
        messages=[
            {
                "role": "user",
                "content": prompt,
            }
        ],
    )

    return response.choices[0].message.content

print(ask_model("Explain model abstraction in two sentences."))

This function keeps the request logic in one place. Your application can pass a different model identifier without duplicating the entire integration.

4. Make Model Selection Configurable

Rather than scattering model identifiers across different modules, use environment variables or a dedicated configuration file.

For a production application, this allows you to:

Change model assignments without modifying unrelated code.
Maintain separate settings for development and production.
Evaluate models against the same task.
Centralize the request layer for future maintenance.

Model compatibility still matters. Different models can have different context limits, supported parameters, and tool-use capabilities.

A unified gateway simplifies integration, but it does not make all models interchangeable.

5. Handle Failures Properly

A production integration should distinguish between different error types.

Authentication failures generally require configuration changes. Rate limits may warrant controlled backoff. Timeouts require careful retry decisions.

For example:

from openai import (
    AuthenticationError,
    RateLimitError,
    APIConnectionError,
)

try:
    answer = ask_model("Summarize the benefits of modular design.")
    print(answer)
except AuthenticationError:
    print("Check your credentials and endpoint configuration.")
except RateLimitError:
    print("Rate limit reached. Retry according to your policy.")
except APIConnectionError:
    print("Connection failed. Check network and service status.")

For production use, add logging, bounded retries, request identifiers where supported, and clear error reporting.

Avoid blindly retrying operations that may have side effects.

6. What Should You Measure?

Before deciding which model is better for a task, evaluate more than the response itself.

Useful metrics include response quality, latency, input and output token usage, error rates, and total cost per successful task.

A low token price does not automatically mean a lower end-to-end application cost.

Final Thoughts

Flexible AI applications are built around replaceable components.

Separating model selection from business logic is a practical step toward cleaner code, easier experimentation, and more maintainable systems.

You can explore the RelayRouter developer setup and quickstart examples here:

RelayRouter: https://api.relayrouter.ai/register?aff=xc9N

GitHub Quickstart: https://github.com/whale0yep-ai/relayrouter-api-quickstart

Disclosure: This tutorial was prepared with AI assistance and should be reviewed against the current API documentation. The snippets are illustrative and are not presented as benchmark results or proof of a completed live test.

DEV Community