Put the Model Call Behind a Port Before You Swap Providers

The cost of changing model providers is decided long before the change, at the moment someone drops an SDK call into a request handler. A single narrow port, one interface that the rest of the code depends on, converts that switch into an adapter edit plus a fixture refresh. The refactor below is illustrated with code that has not been executed for this article, and every command is a sketch of a verification step rather than a reported result.

The coupling hides in plain sight

Most services begin with something like this, and it works well enough that nobody revisits it.

# illustrative only — representative shape, not a real SDK
def handle_summary(payload: dict) -> dict:
    client = ProviderClient(os.environ["PROVIDER_KEY"])
    for attempt in range(3):
        try:
            raw = client.chat(model=DEFAULT_MODEL, messages=[{"role": "user", "content": payload["text"]}])
            return json.loads(raw.text)
        except Exception:
            time.sleep(2 ** attempt)
    raise RuntimeError("summary failed")

Four separate decisions are welded together here: which provider, which model, how retries work, and how output is parsed. Changing any one of them forces a change to the others, and none of them can be tested without a network round trip. The retry loop is the most damaging part, because it spends budget and wall-clock time on failures that a second attempt cannot repair.

A port small enough to keep

The interface should describe what the application needs, not what any provider happens to offer. Text in, text out, plus the accounting fields the caller has to log.

# illustrative only
from dataclasses import dataclass
from typing import Protocol

@dataclass(frozen=True)
class CompletionRequest:
    prompt: str
    max_output_tokens: int
    tag: str  # for local logs; never sent to a provider

@dataclass(frozen=True)
class CompletionResponse:
    text: str
    tokens_in: int
    tokens_out: int
    provider_request_id: str

class Completer(Protocol):
    def complete(self, req: CompletionRequest) -> CompletionResponse: ...

The port also needs its own error vocabulary, or provider exceptions will leak into business logic within a week. Three classes cover most cases: Retryable for transient transport and rate limits, Budget for an exhausted allowance or a refused payment, and Contract for a response that violates the expected shape. The adapter maps provider status codes into those three, since only the adapter knows what a given code means for that provider.

Two adapters, one of them free

The first adapter wraps the paid provider and keeps a small mapping table from status codes to the port's errors. The second adapter points at a free tier, which is where the design gets tested honestly, because limits arrive faster on free capacity than on a metered account. The operator of MonkeyCode describes the project as open source with free model access, a free tier allowance on the order of ten million tokens, and a free server option; confirm the current figures, terms, and availability on the official site, since allowances of this kind change.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

Free capacity is useful here for a specific reason: budget pressure surfaces the omissions in the port earlier, so a missing accounting field gets noticed in a day rather than a quarter. Keeping a shared replay stub on the free server option gives a small team one endpoint to point at during development, while the contract suite stays independent of whichever provider sits behind it.

Contract tests both adapters must pass

The suite asserts behaviour rather than implementation, so it runs unchanged against any adapter that satisfies the port. Each test is skipped loudly when credentials are absent, and the skip summary is read on every build instead of being ignored.

# illustrative only — pytest style
import pytest
from app.ports import CompletionRequest, Contract, Budget

ADAPTERS = ["paid", "free"]  # a fixture resolves each name to a factory

@pytest.mark.parametrize("name", ADAPTERS)
def test_returns_text_and_accounting(name, completer):
    r = completer(name).complete(CompletionRequest(prompt="say ok", max_output_tokens=16, tag="contract"))
    assert r.text.strip()
    assert r.tokens_out > 0
    assert r.provider_request_id

@pytest.mark.parametrize("name", ADAPTERS)
def test_oversized_request_raises_port_error(name, completer):
    with pytest.raises((Contract, Budget)):
        completer(name).complete(CompletionRequest(prompt="x" * 200_000, max_output_tokens=16, tag="contract"))

The second test is deliberately loose about which error appears, because a provider may treat an oversized prompt as invalid input or as a budget matter. What matters is that the adapter raises something from the port's vocabulary instead of leaking a raw transport exception to the caller.

# illustrative verification step
pytest -q tests/contract -rs
# read the skip summary: a skipped adapter is untested behaviour, not passing behaviour

Fixtures replace the network in CI

A narrow port makes recording trivial, because a fixture is just a mapping from request fields to a response body. Hash the stable fields to build the key, store the response body, and replay it whenever credentials are missing from the build environment.

# illustrative only
import hashlib, json
from app.ports import CompletionResponse, Contract

def key(req) -> str:
    blob = json.dumps({"p": req.prompt, "m": req.max_output_tokens}, sort_keys=True)
    return hashlib.sha256(blob.encode()).hexdigest()[:16]

class ReplayCompleter:
    def __init__(self, path):
        self.data = json.load(open(path, encoding="utf-8"))
    def complete(self, req):
        try:
            return CompletionResponse(**self.data[key(req)])
        except KeyError as exc:
            raise Contract(f"no fixture for {key(req)}") from exc

Fixtures go stale whenever a prompt changes, so store a fixture version beside the prompt hash and fail the build on a mismatch instead of silently replaying an old answer. Treat a missing fixture as a failure rather than a skip, or the offline suite will quietly shrink until it proves nothing.

Migration order that keeps reviews small

Start by adding the port and one adapter behind a flag, leaving the original call site untouched until the new path passes the contract suite. Move a single call site next, run both paths in shadow for a short window, and compare the fields the caller actually consumes. The old SDK import should disappear last, and that deletion is the moment to measure how much of the codebase still reaches for the provider directly.

# illustrative check — count remaining direct provider imports
grep -rn "ProviderClient" --include="*.py" . | wc -l

The count should fall to one, inside the adapter package, and any remaining hit outside that package is a leak worth its own review. Shadow comparison also catches a subtler failure, where two providers return text that parses but means something different.

When the port earns its keep

Situation	Verdict
Two or more providers, or a free tier used during development	Worth the interface; the swap is the point
Streaming, tool calling, or provider-specific features in active use	Design the port for them first, or leave those paths provider-aware
A single script run a few times a month	Ceremony; call the SDK directly
A strict output schema enforced downstream	Strong fit, since the adapter owns parsing

Limits and who should not adopt this

Streaming is the hardest interaction to place behind a port, because a token stream behaves differently from a single response, and retrofitting it later costs more than designing for it now. Provider-specific features such as tool schemas and structured output also vary enough that a narrow interface hides real capability, so widen the boundary deliberately or accept that those call sites stay provider-aware. Anyone writing a throwaway script, or prototyping an idea that may be deleted this week, is better served by calling the SDK directly and keeping the freedom to experiment.

The refactor does not improve output quality on its own, and it will not rescue a prompt that asks for the wrong thing. It only makes the provider boundary cheap to move, which is the property that matters when the next swap arrives with short notice.

If a scratch environment for adapter work would help, the free tier is a reasonable place to build the second adapter without spending anything.