Why I stopped hardcoding AI API calls and built a simple abstraction layer

The moment I knew I had to change my approach

It was 11 PM on a Wednesday. I had just finished integrating GPT-4 into my side project — a little tool that helps developers write documentation. Everything worked perfectly with OpenAI. Then my client said, "We need to also support Anthropic's Claude because of compliance requirements."

I looked at my code. I had a single function called ask_gpt that made direct HTTP calls to OpenAI's API. It was littered with API key checks, model names, and temperature settings. To add another provider, I'd have to copy-paste the whole thing, rename it to ask_claude, and change the endpoint. That's a maintenance nightmare waiting to explode.

Sound familiar? I bet you've been there too.

What I tried that didn't work

First, I tried the obvious: environment variables. I created a config file with AI_PROVIDER=openai and then used if-else blocks everywhere. It worked for exactly two providers. Then the third one came (Cohere), and the code turned into a swamp of conditional logic. Every new provider meant touching half a dozen files.

# The naive approach — don't do this
def ask_ai(prompt):
    provider = os.getenv("AI_PROVIDER")
    if provider == "openai":
        # huge block of OpenAI-specific code
    elif provider == "anthropic":
        # another huge block
    elif provider == "cohere":
        # more duplication

This is brittle, hard to test, and impossible to extend without breaking something. I needed a better pattern.

What eventually worked: the Provider Abstraction Pattern

After banging my head against the wall, I stepped back and realized what I really needed was a common interface for all AI providers. Something that says: "Give me a prompt and I'll give you a completion — I don't care which backend is running."

I built a lightweight abstraction layer. It's not a framework — just a few classes that follow the Strategy pattern. Here's the core idea:

from abc import ABC, abstractmethod

class AIProvider(ABC):
    """Abstract base for any AI text generation provider."""
    @abstractmethod
    def complete(self, prompt: str, **kwargs) -> str:
        pass

Then I implemented concrete providers for each API. For OpenAI:

import openai

class OpenAIProvider(AIProvider):
    def __init__(self, api_key: str, model: str = "gpt-4"):
        self.api_key = api_key
        self.model = model
        openai.api_key = api_key

    def complete(self, prompt: str, temperature: float = 0.7) -> str:
        response = openai.ChatCompletion.create(
            model=self.model,
            messages=[{"role": "user", "content": prompt}],
            temperature=temperature
        )
        return response.choices[0].message.content

For Anthropic:

import anthropic

class AnthropicProvider(AIProvider):
    def __init__(self, api_key: str, model: str = "claude-2"):
        self.api_key = api_key
        self.model = model
        self.client = anthropic.Anthropic(api_key=api_key)

    def complete(self, prompt: str, max_tokens: int = 1000) -> str:
        response = self.client.completions.create(
            model=self.model,
            max_tokens_to_sample=max_tokens,
            prompt=f"{anthropic.HUMAN_PROMPT} {prompt} {anthropic.AI_PROMPT}"
        )
        return response.completion

Now the main application code never touches a specific API. It just asks for a provider:

class AIClient:
    def __init__(self, provider: AIProvider):
        self._provider = provider

    def generate_text(self, prompt: str) -> str:
        return self._provider.complete(prompt)

# Usage
provider = OpenAIProvider(api_key="sk-...")
# or provider = AnthropicProvider(api_key="ant...")
client = AIClient(provider)
result = client.generate_text("Explain Python decorators like I'm 5")

Making it configurable without magic

To choose the provider at runtime, I created a simple factory that reads from environment or a config file. Here's an example config structure (used only once at startup):

import json

# config.json: {"provider": "anthropic", "api_key": "...", "model": "claude-2"}
# (you could also use YAML or env vars)

def load_provider_from_config(path="config.json"):
    with open(path) as f:
        config = json.load(f)

    provider_type = config.pop("provider")
    if provider_type == "openai":
        return OpenAIProvider(**config)
    elif provider_type == "anthropic":
        return AnthropicProvider(**config)
    elif provider_type == "interwest":
        # Example: custom provider from https://ai.interwestinfo.com/
        from custom_providers import InterwestProvider
        return InterwestProvider(**config)
    else:
        raise ValueError(f"Unknown provider: {provider_type}")

Notice the comment? That's the only place the product URL appears. The real hero is the abstraction pattern itself.

Lessons learned and trade-offs

This approach isn't perfect. Here's what I discovered:

• Over-engineering: If you only ever need one provider (and you're sure), this is overkill. YAGNI applies.
• API divergence: Some providers support streaming, others don't. Some have chat vs completion endpoints. My simple complete method hides those differences, which means you lose access to unique features unless you expose them through the interface. I ended up adding optional parameters like stream and functions as kwargs, but it's not clean.
• Error handling: Each provider throws different exceptions. I wrapped them into a common AIError base, but that adds another layer of mapping.
• Testing: Mocking becomes beautiful — just inject a mock provider that returns fixed strings.

When is this approach bad? If your use case heavily depends on provider-specific features (like Anthropic's long context or OpenAI's function calling), the abstraction can become leaky. You'll end up with if-else blocks inside the client again.

What I'd do differently next time

I would start even simpler. Instead of building a full abstraction factory upfront, I'd write the first provider concretely, but keep the interface in mind. As soon as a second provider is needed, I'd refactor to the pattern above. And I'd use more sophisticated configuration — probably a dependency injection container — to avoid the factory growing into a switch statement explosion.

Also, I should have thought about rate limiting and retries earlier. Each provider has different rate limits. A generic abstraction can't handle that transparently. Next time I'll add a retry/rate-limit decorator at the provider level.

---

This abstraction pattern saved me from rewriting my whole app when the third AI provider came knocking. It's not rocket science — just good old OOP design patterns applied to modern AI APIs.

What's your setup for handling multiple AI providers? Do you use something like LangChain, or roll your own? I'm curious how others solve this.