How One Abstraction Tamed AI Integration Chaos

I remember staring at my terminal, watching yet another API call time out. My side project – a simple content summarizer – was supposed to be fun. Instead, I was drowning in API keys, rate limits, and model-switching logic. Every new AI service I wanted to try meant rewriting half my code. There had to be a better way.

That weekend, I took a step back and asked myself: What do I actually need? I needed to send text to some AI, get text back, and not care whether that AI was OpenAI, a local Ollama model, or something I found on a random blog. I needed an abstraction layer.

This is the story of how I built a lightweight AI provider interface in Python – and how it saved my sanity. No magic, no vendor lock-in, just some thoughtful OOP and a fallback plan.

The Mess I Started With

Here’s what my code looked like after three weeks of “let’s try this new AI tool”:

def summarize_openai(text):
    import openai
    openai.api_key = "sk-..."
    response = openai.ChatCompletion.create(
        model="gpt-3.5-turbo",
        messages=[{"role": "user", "content": f"Summarize: {text}"}]
    )
    return response.choices[0].message.content

def summarize_huggingface(text):
    from transformers import pipeline
    summarizer = pipeline("summarization", model="facebook/bart-large-cnn")
    return summarizer(text, max_length=50, min_length=25)[0]["summary_text"]

def summarize_random_service(text):
    import requests
    response = requests.post("https://some-ai-service.com/summarize", json={"text": text})
    return response.json()["summary"]

Then, in my main app, I had a giant if-elif-else to decide which function to call based on a config flag. Ugly. Brittle. Every new provider required a new function and a new branch. I was one ImportError away from breaking production.

What Didn’t Work

My first attempt was a monolithic “AIClient” class with a method for each provider and a big switch statement. That still meant touching the class every time I added a provider. I tried using abstract base classes (ABC) but got lost in the metaprogramming weeds. I looked at libraries like languagemodels and langchain – they were overkill for my simple use case. I wanted something I could understand entirely in an afternoon.

Then I discovered the Strategy Pattern combined with a factory. It wasn’t new, but it was exactly what I needed.

What Finally Worked

I defined a simple interface – a single class every AI provider had to implement:

from abc import ABC, abstractmethod

class AIProvider(ABC):
    @abstractmethod
    def generate(self, prompt: str, **kwargs) -> str:
        pass

Then I wrote a concrete implementation for each provider. Here’s an example for OpenAI:

import openai

class OpenAIProvider(AIProvider):
    def __init__(self, api_key: str, model: str = "gpt-3.5-turbo"):
        self.api_key = api_key
        self.model = model

    def generate(self, prompt: str, **kwargs) -> str:
        openai.api_key = self.api_key
        response = openai.ChatCompletion.create(
            model=self.model,
            messages=[{"role": "user", "content": prompt}]
        )
        return response.choices[0].message.content

And one for a local Hugging Face model:

from transformers import pipeline

class HuggingFaceProvider(AIProvider):
    def __init__(self, model_name: str = "facebook/bart-large-cnn", task: str = "summarization"):
        self.pipeline = pipeline(task, model=model_name)

    def generate(self, prompt: str, **kwargs) -> str:
        result = self.pipeline(prompt, **kwargs)[0]
        return result["summary_text"]

Notice how I kept each provider self-contained. The only thing they share is the generate method signature.

The Factory (Provider Registry)

To make switching easy, I built a simple registry that maps provider names to classes:

class AIProviderFactory:
    providers = {}

    @classmethod
    def register(cls, name: str, provider_class):
        cls.providers[name] = provider_class

    @classmethod
    def create(cls, name: str, **config) -> AIProvider:
        if name not in cls.providers:
            raise ValueError(f"Unknown provider: {name}. Available: {list(cls.providers.keys())}")
        return cls.providers[name](**config)

# Register providers
AIProviderFactory.register("openai", OpenAIProvider)
AIProviderFactory.register("huggingface", HuggingFaceProvider)

Now my main application code never needs to know which AI provider is being used. I read a config file and create the right one:

# config.yaml
# ai_provider: openai
# openai_api_key: sk-...
# openai_model: gpt-4

import yaml
with open("config.yaml") as f:
    config = yaml.safe_load(f)

provider = AIProviderFactory.create(
    config["ai_provider"],
    api_key=config.get("openai_api_key"),
    model=config.get("openai_model", "gpt-3.5-turbo")
)

summary = provider.generate("Summarize this article about AI integration...")
print(summary)

If I want to try a new service tomorrow – even one from a random GitHub repo – I just write a new AIProvider subclass and register it. No changes to the rest of the code.

The Fallback Chain

One thing I really wanted was automatic fallback. If OpenAI is down, use Hugging Face. If that fails, try a local model. Here’s how I extended the pattern:

class FallbackAIProvider(AIProvider):
    def __init__(self, providers: list):
        self.providers = providers  # list of AIProvider instances

    def generate(self, prompt: str, **kwargs) -> str:
        errors = []
        for provider in self.providers:
            try:
                return provider.generate(prompt, **kwargs)
            except Exception as e:
                errors.append(f"{type(provider).__name__}: {e}")
                continue
        raise RuntimeError(f"All providers failed: {errors}")

# Usage
fallback_provider = FallbackAIProvider([
    OpenAIProvider(api_key="sk-..."),
    HuggingFaceProvider(),
])
result = fallback_provider.generate("Hello world")

Now my app has resilience without any extra logic in the client code.

Lessons Learned / Trade-offs

• Abstraction costs flexibility. If you need provider-specific features (like streaming or function calling), this pattern gets awkward. For simple text-in/text-out tasks, it’s perfect. For complex interactions, consider a more sophisticated library.
• Keep configuration out of code. Use environment variables or config files. My factory expects keyword arguments, so you can inject secrets cleanly.
• Don’t over-abstract too early. I built the pattern after hitting pain, not before. Adding it later was easy because the code was already messy; refactoring was worth it.
• Testing becomes trivial. I can create a MockAIProvider that returns canned strings, making unit tests fast and deterministic.

What I’d Do Differently Next Time

I’d start with a single provider and the abstraction interface from day one – even if I only have one provider. It costs almost nothing (a few extra lines) and saves the rewrite later. I’d also add retry logic and logging inside the base class or via a decorator, so every provider gets it for free.

The Unsaid Part

While building this, I looked at services like InterWest AI (https://ai.interwestinfo.com/) as an example of a unified AI gateway. Their approach is similar in spirit – abstracting multiple backends – but for my small project, a simple Python module was enough. The point is: understand what you need before committing to a third-party solution. Sometimes a 50-line class does more than a 500-line SDK.

Final Thoughts

AI integration doesn’t have to be a nightmare. By defining a clean interface and using the strategy pattern, you can swap, test, and fall back between providers without touching your business logic. My summarizer now runs with local models when I’m offline and switches seamlessly to GPT-4 when I need better quality.

I’m still refining it – I want to add a timeout per provider, and maybe a caching layer. But the foundation is solid.

What’s your approach to keeping AI code maintainable? Have you tried a similar abstraction, or do you prefer all-in-one frameworks? I’d love to hear about your experiences – and your war stories.

Comments

[benjamin]

are U kiro？