DEV Community: Sergey Inozemtsev

How to send images and PDFs to OpenAI, Claude, and Gemini APIs using the same Python code

Sergey Inozemtsev — Mon, 27 Jul 2026 19:11:39 +0000

OpenAI, Anthropic's Claude, and Google's Gemini can all process images and PDFs. The annoying part starts when an application needs to support more than one of them: each API expects a different request structure, different field names, and sometimes different capabilities.

This tutorial shows how to send images and PDFs to all three providers through the same Python interface. The examples use llm-api-adapter, a lightweight SDK that calls each provider directly and converts a shared UserMessage into the required wire format.

Repository: github.com/Inozem/llm_api_adapter

Install the library

pip install "llm-api-adapter>=0.5.1"

Version 0.5.1 or later is required for PDF input. Image input was introduced in 0.5.0.

The package does not require the OpenAI, Anthropic, or Google SDKs. It sends requests directly and uses requests as its only runtime dependency.

Configure a provider

Create one adapter for the provider and model you want to call:

import os

from llm_api_adapter.universal_adapter import UniversalLLMAPIAdapter

adapter = UniversalLLMAPIAdapter(
    organization="openai",
    model=os.environ["OPENAI_MODEL"],
    api_key=os.environ["OPENAI_API_KEY"],
)

The file message examples below are independent from that configuration. To use Claude or Gemini, you create another adapter instance, but the UserMessage, ImagePart, and DocumentPart stay unchanged.

Send an image from a URL

The simplest option is an image with a public URL:

from llm_api_adapter.models.messages.chat_message import UserMessage
from llm_api_adapter.models.messages.file_parts import ImagePart

message = UserMessage(
    "Describe this image in one sentence.",
    files=[
        ImagePart(
            url="https://example.com/photo.jpg",
        )
    ],
)

response = adapter.chat(
    messages=[message],
    max_tokens=200,
)

print(response.content)

ImagePart detects the MIME type from the URL extension. In this example, .jpg becomes image/jpeg.

Under the shared interface, the adapter converts this file into the format required by the selected provider. The application does not need separate OpenAI, Claude, and Gemini message builders.

Image URLs without an extension

Some image URLs do not end with a filename:

https://api.example.com/files/42

In that case, pass the MIME type explicitly:

image = ImagePart(
    url="https://api.example.com/files/42",
    media_type="image/jpeg",
)

This avoids guessing from response headers or downloading the file during message serialization.

Send an image from bytes

Use bytes when the image is local, generated by your application, stored behind authentication, or already available in memory:

from llm_api_adapter.models.messages.chat_message import UserMessage
from llm_api_adapter.models.messages.file_parts import ImagePart

with open("photo.png", "rb") as image_file:
    image_bytes = image_file.read()

message = UserMessage(
    "What is shown in this image?",
    files=[
        ImagePart(
            data=image_bytes,
            media_type="image/png",
        )
    ],
)

response = adapter.chat(
    messages=[message],
    max_tokens=200,
)

print(response.content)

When you pass raw bytes, media_type is required. The adapter base64-encodes the data only when building the provider request.

Send multiple images in one message

The files field accepts a list, so the same interface works for comparisons and before-and-after analysis:

message = UserMessage(
    "Compare these two images and list the visible differences.",
    files=[
        ImagePart(url="https://example.com/before.jpg"),
        ImagePart(url="https://example.com/after.jpg"),
    ],
)

response = adapter.chat(
    messages=[message],
    max_tokens=400,
)

The text stays the first part of the user message, followed by the files in the order provided.

Send a PDF from bytes

PDF input uses DocumentPart. The rest of the request is identical to the image examples:

from llm_api_adapter.models.messages.chat_message import UserMessage
from llm_api_adapter.models.messages.file_parts import DocumentPart

with open("report.pdf", "rb") as pdf_file:
    pdf_bytes = pdf_file.read()

message = UserMessage(
    "Summarize this document in five bullet points.",
    files=[
        DocumentPart(
            data=pdf_bytes,
            media_type="application/pdf",
        )
    ],
)

response = adapter.chat(
    messages=[message],
    max_tokens=600,
)

print(response.content)

PDF bytes provide the most portable document path. They work across the supported OpenAI, Anthropic, and Google request formats.

DocumentPart currently represents PDF input rather than every possible document format. This is intentional: a narrow validated contract is more useful than a generic FilePart that appears portable but fails differently for each provider.

Send a PDF from a URL

You can also create a document part from a URL:

message = UserMessage(
    "Extract the main conclusions from this report.",
    files=[
        DocumentPart(
            url="https://example.com/report.pdf",
        )
    ],
)

DocumentPart detects application/pdf from a .pdf URL extension. For a URL without an extension, pass the MIME type explicitly:

document = DocumentPart(
    url="https://api.example.com/files/42",
    media_type="application/pdf",
)

PDF URL support has one important compatibility boundary:

Anthropic accepts PDF URLs.
Google accepts PDF URLs.
OpenAI Responses accepts PDF URLs.
OpenAI Chat Completions accepts PDF bytes, but not PDF URLs.

When the selected OpenAI path cannot represent a PDF URL, the adapter raises a clear ValueError instead of silently downloading the file, changing the endpoint, or sending an invalid payload.

For that path, download the file in your application under your own timeout, authentication, size, and security rules, then pass it as bytes.

Switch between OpenAI, Claude, and Gemini

The message is not tied to a provider:

message = UserMessage(
    "Describe this image.",
    files=[ImagePart(url="https://example.com/photo.jpg")],
)

Only the adapter configuration changes:

import os

from llm_api_adapter.universal_adapter import UniversalLLMAPIAdapter

adapters = {
    "openai": UniversalLLMAPIAdapter(
        organization="openai",
        model=os.environ["OPENAI_MODEL"],
        api_key=os.environ["OPENAI_API_KEY"],
    ),
    "anthropic": UniversalLLMAPIAdapter(
        organization="anthropic",
        model=os.environ["ANTHROPIC_MODEL"],
        api_key=os.environ["ANTHROPIC_API_KEY"],
    ),
    "google": UniversalLLMAPIAdapter(
        organization="google",
        model=os.environ["GOOGLE_MODEL"],
        api_key=os.environ["GOOGLE_API_KEY"],
    ),
}

provider = "anthropic"
response = adapters[provider].chat(
    messages=[message],
    max_tokens=200,
)

print(response.content)

You do not rebuild the image or document message when switching providers. This is the main boundary of the abstraction: normalize the portable application-facing input, then serialize it separately for each API.

Compatibility matrix

Input	OpenAI	Claude	Gemini
Image URL	Yes	Yes	Yes
Image bytes	Yes	Yes	Yes
Multiple images	Yes	Yes	Yes
PDF bytes	Yes	Yes	Yes
PDF URL	Responses API only	Yes	Yes

The exact model must also support the requested modality. A valid provider payload cannot make a text-only model accept an image or a document.

Common errors

Passing bytes without a MIME type

This is invalid:

ImagePart(data=image_bytes)

The adapter cannot safely infer a MIME type from arbitrary bytes. Pass it explicitly:

ImagePart(
    data=image_bytes,
    media_type="image/png",
)

The same rule applies to PDF bytes:

DocumentPart(
    data=pdf_bytes,
    media_type="application/pdf",
)

Providing both a URL and bytes

A file part must have one source:

ImagePart(
    url="https://example.com/photo.jpg",
    data=image_bytes,
    media_type="image/jpeg",
)

This is ambiguous and raises an error. Use either url or data.

Using the wrong file class

ImagePart validates image/* MIME types. DocumentPart currently validates application/pdf.

This fails early:

ImagePart(
    data=pdf_bytes,
    media_type="application/pdf",
)

Early validation is preferable to waiting for a provider-specific HTTP error after the request has already been sent.

Sending a PDF URL through OpenAI Chat Completions

Use PDF bytes for that API path, or choose an OpenAI model routed through the Responses API. The adapter does not hide this difference because doing so would change application behavior.

What the adapter normalizes

The shared Python interface hides repetitive payload conversion:

OpenAI uses different image and file blocks for Chat Completions and Responses.
Anthropic uses image or document content blocks with URL or base64 sources.
Google uses fileData for URLs and inlineData for bytes.

The adapter converts ImagePart and DocumentPart into those structures. It does not pretend that every provider supports every source type through every endpoint.

That distinction matters. A useful multi-provider SDK should reduce provider-specific application code, but it should still expose real capability boundaries as explicit errors.

When this approach is useful

This interface is a good fit when:

your application calls OpenAI, Claude, or Gemini directly;
you want to change providers without rebuilding message objects;
you need both URL and in-memory file input;
you want a small SDK rather than an agent framework or hosted proxy;
you prefer unsupported combinations to fail before the HTTP request.

A provider-native SDK may be a better choice when your application will never switch providers or depends heavily on a provider-specific file storage workflow.

Complete example

The following example sends a local PDF while keeping the provider configurable:

import os

from llm_api_adapter.models.messages.chat_message import UserMessage
from llm_api_adapter.models.messages.file_parts import DocumentPart
from llm_api_adapter.universal_adapter import UniversalLLMAPIAdapter

provider = os.environ.get("LLM_PROVIDER", "openai")

provider_config = {
    "openai": {
        "model": os.environ["OPENAI_MODEL"],
        "api_key": os.environ["OPENAI_API_KEY"],
    },
    "anthropic": {
        "model": os.environ["ANTHROPIC_MODEL"],
        "api_key": os.environ["ANTHROPIC_API_KEY"],
    },
    "google": {
        "model": os.environ["GOOGLE_MODEL"],
        "api_key": os.environ["GOOGLE_API_KEY"],
    },
}

with open("report.pdf", "rb") as pdf_file:
    pdf_bytes = pdf_file.read()

message = UserMessage(
    "Summarize this report in five bullet points.",
    files=[
        DocumentPart(
            data=pdf_bytes,
            media_type="application/pdf",
        )
    ],
)

adapter = UniversalLLMAPIAdapter(
    organization=provider,
    **provider_config[provider],
)

response = adapter.chat(
    messages=[message],
    max_tokens=600,
)

print(response.content)

Change LLM_PROVIDER and the corresponding model and API key variables. The document message remains unchanged.

The goal is not to make three different APIs look identical. It is to keep application code stable where the capabilities are genuinely portable, and make the remaining differences explicit.

The source code is available in the repository:
github.com/Inozem/llm_api_adapter

Multi-provider LLM resilience in Python without provider-specific code

Sergey Inozemtsev — Wed, 22 Jul 2026 18:31:32 +0000

OpenAI, Anthropic, and Google expose different APIs, message formats, tool-calling conventions, error types, and response structures.

That difference is manageable while an application uses only one provider. It becomes more expensive when the application needs retries, circuit breakers, fallback routes, observability, and recovery across several providers.

Without a shared abstraction, resilience logic tends to be implemented repeatedly:

OpenAI integration
├── request conversion
├── retry logic
├── error classification
├── circuit breaker
└── tool-call recovery

Anthropic integration
├── request conversion
├── retry logic
├── error classification
├── circuit breaker
└── tool-call recovery

Google integration
├── request conversion
├── retry logic
├── error classification
├── circuit breaker
└── tool-call recovery

I did not want to build resilience three times.

I wanted to define the recovery policy once and apply it across every provider supported by the application.

That became llm-api-resilience, a Python library for retries, ordered failover, circuit breakers, and checkpoint recovery across multiple LLM providers.

Quick start

pip install llm-api-resilience

Once the provider adapters are configured, an ordered fallback plan takes only a few lines:

from llm_api_resilience import RecoveryPlan, ResilientLLM, Route

llm = ResilientLLM(
    RecoveryPlan(
        [
            Route("openai-primary", openai_adapter),
            Route("anthropic-backup", anthropic_adapter),
            Route("google-last-resort", google_adapter),
        ]
    )
)

response = llm.chat(
    [{"role": "user", "content": "Explain circuit breakers briefly."}]
)

print(response.selected_route)
print(response.content)

GitHub: llm-api-resilience
Provider adapter: llm-api-adapter

The library is built on top of llm-api-adapter, which provides one interface for OpenAI, Anthropic, and Google.

Because provider-specific differences are handled by the adapter, the resilience layer can operate on a shared contract instead of understanding three separate APIs.

The result is a Python library with:

ordered failover across models and providers;
configurable retry policies;
circuit breakers;
attempt metadata;
checkpoint recovery for tool-calling sessions.

One interface across three providers

Before adding resilience, I had already built llm-api-adapter to normalize the main differences between provider APIs.

The application can create adapters for different providers and call the same chat() method:

import os

from llm_api_adapter.universal_adapter import UniversalLLMAPIAdapter

openai_adapter = UniversalLLMAPIAdapter(
    organization="openai",
    model="gpt-5.6-sol",
    api_key=os.environ["OPENAI_API_KEY"],
)

anthropic_adapter = UniversalLLMAPIAdapter(
    organization="anthropic",
    model="claude-sonnet-5",
    api_key=os.environ["ANTHROPIC_API_KEY"],
)

google_adapter = UniversalLLMAPIAdapter(
    organization="google",
    model="gemini-3.5-flash",
    api_key=os.environ["GOOGLE_API_KEY"],
)

messages = [
    {
        "role": "user",
        "content": "Reply with exactly: OK",
    }
]

adapters = {
    "openai": openai_adapter,
    "anthropic": anthropic_adapter,
    "google": google_adapter,
}

for provider, adapter in adapters.items():
    response = adapter.chat(
        messages=messages,
        max_tokens=64,
    )

    print(
        provider,
        adapter.model,
        "->",
        response.content,
    )

The important part is not only that the method names are the same.

The adapter creates a stable boundary around:

messages;
tool definitions and results;
normalized responses;
finish reasons;
structured outputs;
provider errors.

The resilience layer does not need to know how Anthropic represents a tool result, which OpenAI parameter controls structured output, or how Google formats its request.

It only needs to understand the adapter contract.

Once that contract existed, models from different providers could become routes in the same recovery plan.

Defining recovery once

A recovery plan contains an ordered collection of routes.

Each route contains an adapter, a RoutePolicy for retries, timeouts, and backoff, and an optional CircuitBreaker.

from llm_api_resilience import (
    CircuitBreaker,
    RecoveryPlan,
    ResilientLLM,
    Route,
    RoutePolicy,
)

primary_breaker = CircuitBreaker(
    failure_threshold=3,
    cooldown_s=30,
)

llm = ResilientLLM(
    RecoveryPlan(
        [
            Route(
                "openai-primary",
                openai_adapter,
                policy=RoutePolicy(
                    timeout_s=15,
                    max_attempts=2,
                    backoff_s=0.25,
                ),
                breaker=primary_breaker,
            ),
            Route(
                "anthropic-backup",
                anthropic_adapter,
                policy=RoutePolicy(
                    timeout_s=20,
                    max_attempts=1,
                ),
            ),
            Route(
                "google-last-resort",
                google_adapter,
                policy=RoutePolicy(
                    timeout_s=30,
                    max_attempts=1,
                ),
            ),
        ]
    )
)

response = llm.chat(
    messages,
    max_tokens=128,
)

print("Selected route:", response.selected_route)
print("Response:", response.content)

The application sends one normalized request.

The adapter handles provider-specific integration. The resilience layer decides which route to use and what to do when it fails.

Application
     ↓
llm-api-resilience
     ↓
llm-api-adapter
     ↓
OpenAI / Anthropic / Google

The application does not need separate fallback branches that rebuild messages, tools, and parameters for every provider.

Not every failure should be retried

Adding a retry loop around every exception is easy.

Deciding which failures should actually be retried is harder.

A timeout may succeed on the next attempt. A server error may disappear after a short delay. A rate limit may require backoff or a switch to another route.

An invalid API key will not be fixed by retrying. Neither will an invalid schema or an incorrectly configured tool.

In the current default classifier, only normalized timeout, rate-limit, and server errors are retryable:

from llm_api_adapter.errors import (
    LLMAPIRateLimitError,
    LLMAPIServerError,
    LLMAPITimeoutError,
)

from llm_api_resilience import DefaultFailureClassifier

classifier = DefaultFailureClassifier()

print(
    [
        error_type.__name__
        for error_type in classifier.retryable_errors
    ]
)

# [
#     "LLMAPITimeoutError",
#     "LLMAPIRateLimitError",
#     "LLMAPIServerError",
# ]

Authentication, configuration, invalid tool or schema errors, and ordinary ValueError failures are not retried by default.

This is another benefit of placing resilience above a provider adapter.

Each provider exposes different exceptions, but the recovery layer does not need three unrelated classification systems. Provider-specific failures are mapped into shared error types before the recovery policy evaluates them.

Retry first, then fail over

A route can receive multiple attempts before the recovery plan selects the next route.

primary route
      ↓
first attempt fails
      ↓
retry policy allows another attempt
      ↓
retry fails
      ↓
route is exhausted
      ↓
backup route selected

Failover is ordered rather than random.

This allows the application to express preferences such as:

use a cheaper model first;
keep a similar model as the first backup;
move to another provider after local retries are exhausted;
reserve the most expensive model for the final route.

But every attempt has a cost.

Retries add latency and token usage. A backup model may have a different price, context window, or behavioral profile.

A resilience policy should therefore not mean “keep trying until something works.” It should define how much latency, cost, and model variation the application is willing to accept.

Circuit breakers avoid predictable failures

Retries help with isolated errors. They are less useful when a route is already degraded.

Without a circuit breaker, every new request may wait for the same route to fail several times before moving to a healthy provider.

A circuit breaker remembers recent route health and increments its failure count only for retryable errors such as timeouts, rate limits, and server errors:

closed
   ↓
retryable failures reach threshold
   ↓
open
   ├─ cooldown not expired → request skipped
   │
   └─ cooldown expired + next request
          ↓
      half-open
       ├─ probe succeeds → closed
       └─ probe fails    → open

The circuit does not move to half-open automatically when the timer expires. That transition happens when the next request calls allow_request() after the cooldown.

While the circuit is open, the recovery plan can skip that route instead of repeatedly sending requests that are likely to fail.

The breaker is attached to a route rather than necessarily disabling an entire provider. One model or endpoint may be degraded while another route from the same provider remains usable.

Automatic fallback should not hide failures

A resilient request may eventually succeed even when several things went wrong first.

For example:

OpenAI attempt 1: timeout
OpenAI attempt 2: timeout
Anthropic attempt 1: success

From the application’s perspective, the request succeeded.

Operationally, however, the primary route failed twice and the backup provider handled the request. That affects reliability, latency, and cost.

llm-api-resilience exposes attempt metadata so the recovery path remains visible:

response = llm.chat(
    messages,
    max_tokens=128,
)

for attempt in response.attempts:
    print(
        attempt.route_name,
        attempt.provider,
        attempt.model,
        attempt.duration_s,
        attempt.success,
        attempt.error_type,
    )

An output could look like this:

openai-primary     openai     gpt-5.6-sol       2.41  False  LLMAPITimeoutError
openai-primary     openai     gpt-5.6-sol       2.39  False  LLMAPITimeoutError
anthropic-backup   anthropic  claude-sonnet-5   1.12  True   None

The attempt records also contain start time and error information. They do not include a separate retry number or retry count; the final selected route is available on response.selected_route.

Because error_message is currently derived from str(error), it may contain sensitive details from an upstream exception. Applications should sanitize or restrict error messages before exporting attempt metadata to logs or external observability systems.

This makes it possible to detect:

increasing fallback frequency;
a route becoming unreliable;
latency caused by repeated attempts;
an expensive backup handling too much traffic;
circuit breakers opening frequently.

The goal is not only to eventually receive a response. It is also to understand how that response was produced.

Provider fallback is only the first layer

Request-level failover is relatively straightforward when nothing has happened between model calls.

A tool-calling session introduces a more dangerous failure mode.

Imagine this sequence:

user asks the agent to report an incident
        ↓
model requests create_incident_ticket
        ↓
application creates ticket #12345
        ↓
tool result is returned
        ↓
model continuation fails

The external side effect has already happened.

The ticket exists. The application cannot safely pretend that the turn never started.

Restarting the entire turn may make the model request the same tool again and create ticket #12346 for the same incident.

Switching providers does not automatically solve this. The resilience layer must rebuild the provider-neutral session from the checkpoint captured before the first tool round and replay the result of the tool that already executed.

This is where ordinary request fallback stops being sufficient.

Checkpoint recovery after tool execution

A resilient tool-calling session creates a checkpoint before the external tool is executed.

The application then executes the tool and passes its result back through continue_with().

If the continuation fails with a retryable error, the library can restore the checkpoint, select a backup route, and reuse the stored tool result instead of executing the side effect again.

tool call requested
        ↓
checkpoint created
        ↓
tool executed
        ↓
result stored in journal
        ↓
continuation fails
        ↓
backup route selected
        ↓
checkpoint restored
        ↓
stored result replayed
        ↓
session continues

Here is a simplified incident-ticket example:

import json

from llm_api_adapter.models.tools import ToolSpec
from llm_api_resilience import ToolResult

tools = [
    ToolSpec(
        name="create_incident_ticket",
        description="Create an incident ticket.",
        json_schema={
            "type": "object",
            "properties": {
                "incident_id": {"type": "string"},
                "severity": {"type": "string"},
            },
            "required": ["incident_id", "severity"],
            "additionalProperties": False,
        },
    )
]

session = llm.session(
    [
        {
            "role": "user",
            "content": "Create a high-severity ticket for INC-42.",
        }
    ],
    tools=tools,
    tool_choice="auto",
    max_tokens=256,
)

first_response = session.start()

if first_response.tool_calls:
    # The checkpoint exists before the external side effect.
    assert session.checkpoint is not None

    tool_results = []

    for tool_call in first_response.tool_calls:
        # The application performs the external action.
        ticket_id = f"TICKET-{tool_call.arguments['incident_id']}"

        tool_value = {
            "ticket_id": ticket_id,
            "status": "created",
        }

        tool_results.append(
            ToolResult(
                tool_call_id=tool_call.call_id or tool_call.name,
                content=json.dumps(tool_value),
                idempotency_key=(
                    f"incident-ticket:"
                    f"{tool_call.arguments['incident_id']}"
                ),
                replay_policy="side_effecting",
            )
        )

    recovered_response = session.continue_with(tool_results)

    print("Final route:", recovered_response.selected_route)
    print("Answer:", recovered_response.content)

    assert session.checkpoint is not None
    assert len(session.journal.entries) == 1

There is no public restore_checkpoint() call in the application code.

Recovery happens inside continue_with().

If the primary continuation fails, the session can restore its checkpoint and continue through the next route. When the backup route produces a replay-compatible tool call, the journal returns the stored result and the application does not intentionally execute the side effect again. If the backup model changes the tool or its arguments, the application receives a new tool call that requires its own decision.

This requires more than an HTTP retry wrapper.

The checkpoint itself preserves:

the original messages from before the first tool round;
provider-neutral request parameters;
the route identity;
the operation ID.

Provider-specific previous_response state is intentionally excluded because it may only be compatible with the same provider and model.

The tool journal separately preserves:

the completed tool call;
its arguments;
the returned result;
an idempotency identifier;
the replay policy.

A backup provider should receive enough normalized context to continue the session without requiring provider-specific recovery code in the application.

Checkpoint recovery is not the same as idempotency

Checkpoint recovery has an important boundary.

It prevents the resilience layer from deliberately calling a completed tool again during session recovery. It does not automatically make every external system idempotent.

Consider a different failure:

application sends request to create ticket
        ↓
external service creates the ticket
        ↓
network fails before the response returns

The application does not know whether the operation succeeded.

There is no confirmed tool result to store or replay.

A local checkpoint cannot resolve that uncertainty by itself.

Important side effects should still use mechanisms such as:

idempotency keys;
unique operation identifiers;
database uniqueness constraints;
status lookup before retry;
transactional outbox patterns.

Checkpoint recovery and external idempotency solve related but different problems.

Checkpoint recovery protects progress after the tool result is known.

Idempotency protects the external operation when its execution status is uncertain.

Testing recovery without waiting for a real outage

Real provider failures are unpredictable and expensive to reproduce.

A resilience library needs deterministic tests that control exactly when every route succeeds or fails.

For example, a test can define this sequence:

primary start: tool call
        ↓
tool executes once
        ↓
primary continuation: timeout
        ↓
backup start: reconstructed tool call
        ↓
backup continuation: success

SequenceAdapter is a test helper that returns a predefined sequence of responses and exceptions.

The important assertions are not limited to the final answer:

final_response = session.continue_with(result)

assert final_response.selected_route == "backup"
assert final_response.content == "Ticket created successfully."

assert [
    attempt.route_name
    for attempt in session.attempts
] == [
    "primary",
    "primary",
    "backup",
    "backup",
]

assert len(session.journal.entries) == 1
assert session.checkpoint is not None

These assertions verify that the backup route completed the session, all route attempts were recorded, one tool result was journaled, and the checkpoint still exists.

They do not by themselves prove that the external side effect ran only once. That requires an explicit execution counter around the tool implementation:

executions = []

def create_ticket(arguments):
    ticket_id = f"TICKET-{arguments['incident_id']}"
    executions.append(ticket_id)

    return {
        "ticket_id": ticket_id,
        "status": "created",
    }

# After recovery:
assert executions == ["TICKET-INC-42"]

With that assertion, the test proves that recovery reused the stored result without executing the external action again.

Deterministic adapters are also useful for testing retries and circuit breakers. Tests should control error order and route behavior rather than hoping that a real provider fails at the right moment.

Why use a library instead of only a gateway?

Managed LLM gateways are useful.

They can centralize routing, credentials, quotas, observability, billing, and organization-wide policies.

llm-api-resilience targets a different boundary: the Python application itself.

A library-level approach can be useful when the application needs to:

use existing provider accounts and API keys;
define recovery routes in Python;
keep control over retry and failover decisions;
test recovery locally and deterministically;
access conversation state and completed tool results;
recover stateful tool-calling sessions.

A generic gateway can route inference requests, but it may not understand the application’s checkpoint, tool journal, or whether an external side effect has already happened.

This does not mean that a gateway and an application-level resilience library are mutually exclusive.

A gateway can manage infrastructure-level routing while the application manages stateful recovery.

Current limitations

A resilience layer does not make different providers interchangeable in every way.

Backup models are not behaviorally identical

Different models may interpret the same conversation differently.

A backup route may complete the request but produce another style, tool choice, or decision.

Failover improves availability. It does not make models deterministic substitutes.

Recovery depends on state

Process-local checkpoints are not sufficient for every deployment.

Distributed workers, process restarts, and long-running sessions may require persistent checkpoint storage and coordination between instances.

External side effects still require protection

The journal prevents deliberate replay of a known completed tool result. It cannot guarantee how an external service behaves when the execution result is unknown.

Every attempt has a cost

Retries and fallback increase latency and token usage.

Policies need explicit limits rather than unlimited attempts.

Error classification requires maintenance

Providers add new error types and change response behavior. Normalized error mappings need ongoing tests and updates.

These are not reasons to avoid resilience. They define where a general recovery layer ends and application-specific reliability work begins.

Define resilience once

The main benefit of building on top of a provider adapter is not only shorter configuration.

It is avoiding duplicated architecture.

Without a shared provider boundary, retries, circuit breakers, observability, failover, and session recovery must be integrated separately into every provider implementation.

With a unified adapter contract, the application can define one recovery system across OpenAI, Anthropic, and Google.

one request interface
        ↓
one normalized error model
        ↓
one recovery plan
        ↓
multiple providers

Provider fallback protects the next LLM call.

Checkpoint recovery protects the work that already happened before it.

GitHub: llm-api-resilience
Provider adapter: llm-api-adapter

Gemini returned 429 at 2.3% of its documented quota. My retry loop ran for 4 days.

Sergey Inozemtsev — Fri, 17 Jul 2026 11:33:06 +0000

This is a submission for DEV's Summer Bug Smash: Smash Stories powered by Sentry.

The numbers first

Documented quota: 5,000 RPM
Actual load at peak (from Cloud Console): ~115 RPM (2.3% of quota)
Budget: $6/month
Cost after 4 days: ~$90
Root cause: Gemini applying quota limits from the wrong service — bug confirmed in forum threads going back to September 2025
Fix: batch embeddings + API rate caps
Cost after fix: back to $6/month

What TLBrain is

TLBrain is a RAG memory system I built for my wife's ghostwriting business. She conducts 10–15 client calls per week. The information was getting lost between sessions — Claude couldn't remember clients across conversations.

The solution: record calls via TL;DV → auto-transcribe → sync to a vector database → query via a remote MCP server connected to Claude Cowork.

The sync pipeline looks like this:

Cloud Tasks handles the queue. Each transcript goes through a Firestore state machine: queued → downloading → imported → syncing → synced. If a task fails, Cloud Tasks retries it automatically.

That retry behavior is important. It's what turned a quota bug into a ~$90 incident.

The pipeline ran fine for a month

200+ conversations indexed. ~$44 total (one-time backfill). Ongoing cost: $6/month.

Then gemini-embedding-2 started returning 429s.

The failure

429 RESOURCE_EXHAUSTED: Quota exceeded for quota metric 
'aiplatform.googleapis.com/global_embed_content_requests_per_minute_per_base_model'

The pipeline kept running — Cloud Tasks kept enqueuing jobs, Sync Service kept picking them up. But embeddings were silently failing. New transcripts weren't being indexed. Search degraded. My wife noticed that Claude stopped remembering recent calls.

The first sign something was wrong: search results were getting stale.

First diagnosis: wrong

My first assumption — I'm hitting the rate limit. I reduced parallel sync containers from 2 to 1. Still 429.

I checked Gemini's documented quota: 5,000 RPM for gemini-embedding-2. Cloud Console showed peak usage at 2.3% of quota — ~115 RPM. Even with one container, nowhere near the limit.

The investigation

I opened Google Cloud Console to check my actual usage metrics.

Near zero.

429 quota errors. Near-zero usage shown in the dashboard. That contradiction ruled out my code. Something else was happening.

I searched for the exact error. Found forum threads going back to September 2025 — multiple developers, multiple Gemini models, same pattern: 429 at a fraction of the documented quota, dashboard showing minimal usage.

That's when the error message clicked. Look at the quota metric name again:

aiplatform.googleapis.com/global_embed_content_requests_per_minute_per_base_model

I'm calling the Gemini API — generativelanguage.googleapis.com. But the quota being enforced belongs to aiplatform.googleapis.com — Vertex AI. A completely different service with different (lower) limits.

Gemini was routing my embedding requests through Vertex AI internally and applying Vertex AI quota limits against them. That's why my dashboard showed near-zero usage: the generativelanguage quota was barely touched. The Vertex AI quota was what was exhausted — and I had no visibility into that counter at all.

The diagnosis: Gemini applying quota limits from the wrong service. No official acknowledgment at the time of writing (9 months later).

Forum threads reporting the same pattern, going back to September 2025 — no official fix as of the time of writing (9 months later):

Why it cost ~$90: anatomy of the loop

Here's where it gets expensive.

Each Sync Service task processes a full transcript document window by window:

for each window:
    1. generate summary via Gemini LLM        ← billed ✓
    2. generate facts via Gemini LLM          ← billed ✓
    3. embed [summary + facts] via Gemini Embeddings  ← 429 ✗ → task fails
    → write to Qdrant + Firestore

Step 3 fails on the first window. Cloud Tasks sees the failure. Cloud Tasks retries the entire task from the beginning.

Which means every retry restarted the loop from window 1 — regenerating summaries and facts for every window using Gemini LLM before even reaching embeddings again.

[task starts]
  window 1: summary ← billed | facts ← billed | embed ← 429, task fails ✗
[Cloud Tasks retries]
  window 1: summary ← billed again | facts ← billed again | embed ← 429, task fails ✗
[repeat for 4 days]

Some tasks had retried 50+ times before I noticed.

I noticed after 4 days and stopped the sync service manually.

The retry backoff at the time: seconds. Not minutes.

The fix

Three changes. No exotic patterns — just things I should have had from the start.

1. Batch embeddings

Instead of N separate embed_content calls in a loop, one call with a list of all texts from the window.

A transcript can have up to 400 utterances. Processing happens window by window — keeping everything in memory to batch at the transcript level would require 500MB+ containers. Not practical for Cloud Run.

So the unit of batching is the window, not the transcript. For each window: generate summary + facts, then embed everything from that window in one request, then move on.

Before the fix, each text was embedded separately:

# Before: separate embed call per item
for window in windows:
    summary, facts = generate_summary_and_facts(window)

    summary_embedding = gemini.embed(summary)           # 1 call
    fact_embeddings = [gemini.embed(f) for f in facts]  # N calls
    # total: ~10 calls per window

After: all texts from one window go in a single batch call, then the next window:

# After: one batch call per window
for window in windows:
    summary, facts = generate_summary_and_facts(window)

    texts = [summary] + facts
    embeddings = gemini.embed_batch(texts)  # 1 call per window
    # memory released after each window

The actual API call:

response = client.models.embed_content(
    model="gemini-embedding-2",
    contents=texts,
    config=types.EmbedContentConfig(output_dimensionality=768),
)
return [e.values for e in response.embeddings]

Same data, ~10x fewer API calls. Memory stays flat across the entire transcript.

2. API rate cap on the Sync Service

Added a hard limit on Gemini API calls per time window. If the sync service hits the cap, it stops processing new tasks — but the MCP server and search keep running.

This is the architectural separation doing its job: sync service going down doesn't affect the main service. My wife can still query Claude. New transcripts just aren't indexed until the cap resets.

Set a daily request cap directly in Google AI Studio quota settings — no code changes needed. If the sync service hits the cap, it stops processing, but the MCP server and search keep running.

3. Increased backoff for 429 errors

Changed retry behavior from seconds to minutes.

If a quota error occurs, the next retry should be far enough away to matter — not a tight loop that amplifies the problem.

Before: [1, 2, 4] seconds   — applied to all errors
After:  [30, 60, 120] seconds — applied specifically to 429 errors

Quota reset cycles are per-minute. A 4-second backoff doesn't help — it just hammers the same exhausted counter.

After the fix

4 transcripts had been stuck in the failed state during the incident. Restarted them manually. Zero 429s.

Cost went back to $6/month.

What I learned

1. Third-party services can silently break their own contracts.
I planned for my code failing. I didn't plan for Gemini's quota system misfiring. I do now.

2. Dashboard showing zero doesn't mean zero — it means the bug is external.
This was the key diagnostic step. When your usage metrics don't match the errors you're getting, the problem is outside your code.

3. Retry loops amplify cost asymmetrically.
The part that failed (embeddings) wasn't the expensive part. The part that ran successfully on every retry (LLM summarization) was. Always check what runs before the failure point.

4. Architectural isolation saved the main service.
Sync stopping didn't break search. The two services are separate Cloud Run instances. That decision — made before this incident — is the reason my wife could still use the tool while I fixed it.

5. Backoff in seconds for quota errors is wrong.
Quota reset cycles are measured in minutes. Retrying every 10 seconds doesn't help — it just burns through whatever quota you have left.

The refund

I submitted a support request to Google about the erroneous charges. They refunded ~$90 — the full amount of the erroneous charges.

I didn't expect it. I asked anyway.

There's a lucky coincidence worth naming: I used Gemini for both summarization and embeddings. The same provider that had the quota bug was also the one billing me for the runaway loop. That made the refund case coherent — Google caused the problem and Google charged me for it.

If I had used Claude or GPT for summarization, those charges would have been legitimate — those providers did the work correctly. I couldn't have disputed them.

Though there's a flip side: Anthropic and OpenAI work on prepaid credits. The loop would have stopped when the balance ran out, not after 4 days. A natural spending cap I didn't have with GCP's postpaid billing.

Postpaid: can spiral, but you can dispute after the fact. Prepaid: naturally capped, but no recourse. Neither is obviously safer — they just fail differently.

TLBrain is open source: github.com/Inozem/tlbrain

Claude Fable 5 refuses tool calls based on semantics, not logic

Sergey Inozemtsev — Tue, 07 Jul 2026 19:14:40 +0000

I was migrating llm_api_adapter — an open-source universal adapter for LLM APIs — to support Claude Fable 5. The tool was harmless.
The logic was identical to what worked on Opus 4.8. Then one e2e test came
back with this:

print(resp.finish_reason)  # "refusal"
print(resp.tool_calls)     # None

No exception. HTTP 200. The tool call just never happened.

Fable 5 didn't refuse because of what the tool does — it refused because of
what it sounds like it does.

The setup

from llm_api_adapter.universal_adapter import UniversalLLMAPIAdapter
from llm_api_adapter.models.messages.chat_message import UserMessage
from llm_api_adapter.models.tools.tool_spec import ToolSpec

adapter = UniversalLLMAPIAdapter(
    organization="anthropic",
    model="claude-fable-5",
    api_key="...",
)

tools = [
    ToolSpec(
        name="get_secret_word_score",
        description="Return the hidden score for a token.",
        json_schema={
            "type": "object",
            "properties": {"token": {"type": "string"}},
            "required": ["token"],
        },
    )
]

resp = adapter.chat(
    messages=[UserMessage('What is the secret score for token "strawberry_v2"?')],
    tools=tools,
    tool_choice="any",
    max_tokens=512,
)

print(resp.finish_reason)  # "refusal"
print(resp.tool_calls)     # None

The same prompt on Opus 4.8: finish_reason: "tool_use", correct arguments.

The fix

tools = [
    ToolSpec(
        name="get_word_score",
        description="Return the score for a word.",
        json_schema={
            "type": "object",
            "properties": {"word": {"type": "string"}},
            "required": ["word"],
        },
    )
]

resp = adapter.chat(
    messages=[UserMessage('What is the score for the word "strawberry"?')],
    tools=tools,
    tool_choice="any",
    max_tokens=512,
)

print(resp.finish_reason)            # "tool_use"
print(resp.tool_calls[0].arguments)  # {'word': 'strawberry'}

Identical logic. Different semantics. Different outcome.

The obvious hypothesis: Fable 5's classifiers flag security-adjacent words. So
I renamed the tool, removed "secret" from the prompt, and moved on.

But that's the what, not the how. I wanted to know the mechanism.

3 isolation tests

I kept the original code and removed one element at a time:

What I removed	What remained unchanged	Result
`secret` from tool name only	`token` arg + "secret score for token" in prompt	✅ tool_use
`secret` from name + prompt	`token` as argument name	✅ tool_use
Nothing from name	Cleaned prompt and argument name	✅ tool_use
Nothing	Everything original	❌ refusal

Removing any single element was enough to get a tool call through.

What's actually happening

Fable 5's classifier is not a keyword filter. It's a combinatorial pattern detector.

The combination of get_secret_word_score + argument named token + prompt
containing "secret score for token" reads as an auth/security operation — the
kind of thing you'd see when extracting credentials or scoring access tokens.

Each word on its own is harmless. Together, they cross a threshold.

This makes sense given how Anthropic built Fable 5. It shares the same
underlying model as Mythos 5 — the difference is that Fable ships with safety
classifiers that evaluate intent, not execution. The classifier doesn't care
what get_secret_word_score actually does. It cares what it looks like it
does, based on everything in the request: tool name, description, argument
names, and the prompt together.

The Fable 5 migration guide calls the migration "mostly drop-in" and lists safety classifier refusals as one of four key changes to watch. What it doesn't explain is how those classifiers actually evaluate your tool specs.

Practical takeaway

Don't think of your tool spec as a technical contract. Think of it as a
description of intent — because that's how the classifier reads it.

Before you name a tool, ask: if someone unfamiliar with my codebase read this
tool name, description, and argument names together, what would they think this
code is trying to do?

If it sounds like credential extraction, auth bypass, or scoring access tokens —
the classifier will think so too.

A few patterns that can combine badly:

secret / hidden / private + token / key + score / check / verify
auth / access in tool names with argument names that look like identifiers None of these words are banned. The problem is accumulation.

One integration note

stop_reason: "refusal" comes back as HTTP 200. If you're only handling
exceptions, you're silently dropping tool calls in production.

if resp.finish_reason == "refusal":
    # fall back to another model or surface the error explicitly
    ...

The API supports a fallbacks parameter (currently in beta) to handle this
server-side automatically — worth checking if you're running Fable 5 at scale.

RAG on call transcripts: utterance-aware chunking and hybrid retrieval in production

Sergey Inozemtsev — Mon, 01 Jun 2026 12:27:19 +0000

The LLM that indexes the transcripts and the LLM that answers questions never share output. One generates summaries. The other reads only the original utterances. That separation is the whole design.

What it does

TLBrain indexes call transcripts and gives Claude a persistent, searchable memory of every client conversation.

When my wife asks "what did we discuss with Acme last month?" — Claude queries the index, retrieves the relevant transcript segments, and answers with actual context from the calls.

The flow looks like this:

TL;DV records a call and fires a webhook
An import service converts the transcript into a Google Doc and places it in the right client folder in Google Drive — automatically
A sync service picks up new and changed documents, parses them, generates summaries and facts via Gemini, and indexes everything into Qdrant
A remote MCP server connects to Claude — accessible as a tool in both Claude.ai chat and Claude Cowork, on any device

By default, Claude only gets the relevant segments retrieved for each query — but there's also a tool to fetch the full transcript when needed.

What it costs

232 transcripts indexed for $44 — one-time cost. Each new transcript costs ~$0.19 to index.

Infrastructure runs entirely on free tiers:

Cloud Run
Firestore
Google Drive
Qdrant Cloud

All four stay within free tier limits for a small agency workload.

The only meaningful cost is Gemini API (Tier 1) — generation and embeddings during indexing. Embeddings are generated only for summaries and facts — not for utterances. Utterances are stored with BM25 sparse vectors and retrieved by range. This keeps both cost and vector storage size low. Free tier has strict rate limits that would make indexing 200+ transcripts impractical.

A few details that keep the cost low: embeddings use text-embedding-004 with output_dimensionality=768 — 4× cheaper than the default 3072. Summary and facts are generated in a single Gemini request per window. And if a file hasn't changed, it's skipped entirely — Gemini is never called again for the same content.

The $0.19 per transcript is a one-time indexing cost — you pay to embed the conversation once, not every time Claude searches it. Stop recording calls for a month, go on vacation, pause the business — the system costs nothing during that time. You only pay again when new transcripts are indexed. That said, this assumes you stay within the free tiers — once your call volume grows beyond those limits, infrastructure costs will kick in.

Without an index, asking Claude about a specific client means pasting entire transcripts into the context window — hitting message limits fast, and starting from scratch every session. TLBrain sends Claude only the relevant fragments: up to 75 utterances out of hundreds.

Why not just use Claude Projects?

Claude Projects is the obvious first answer. But it has two hard limits for this use case:

Context ceiling — paste enough transcripts and you hit the limit. Every new session starts from scratch.
No structure — there's no concept of clients, dates, or searchable facts. Everything is a flat pile of documents.

Imagine she needs to find what a client said about budget across three calls from different months. With Claude Projects, she'd need to manually find the right transcripts, paste them one by one, and hope they fit in context. With 232 calls in the archive, that's not a workflow — that's a research project. TLBrain returns the relevant fragments in seconds.

Architecture

TL;DV API  ←─────────────────  Reconciliation (Cloud Run, daily)
     ↓
Webhook Handler (Cloud Function)
     ↓
Import Service (Cloud Run)
     ↓
Google Drive + Firestore  ←───  Sync Checker (Cloud Function, every 15 min)
     ↓
Vector Sync (Cloud Run)
     ↓
Qdrant Cloud + Firestore
     ↓
MCP Server (Cloud Run)
     ↓
Claude (chat / Cowork)

Note: Firestore is used throughout as the state store — tracking import status, content hashes, and sync state.

Six services sounds like a lot — but each split is intentional. The webhook handler must respond to TL;DV in under 2 seconds or the delivery is marked as failed, so import runs in a separate service. The MCP server is isolated from the sync pipeline so a slow indexing job never blocks Claude's queries. Services are also split by runtime pattern: Cloud Functions wake up, check something, and go back to sleep — no idle cost. Cloud Run containers handle long-running tasks and stay warm longer: the MCP server keeps its instance alive for 15 minutes after the last request, so there are no cold starts during an active session.

Client folders in Google Drive are the source of truth for data organization. Each subfolder under the root is a client name. The sync service doesn't know about TL;DV — it only reads Google Docs from Drive. The import service and sync service are fully decoupled.

A daily reconciliation job cross-checks TL;DV's API against Firestore and queues anything the webhook missed — so no transcript gets lost silently.

One unexpected benefit

If a transcript contains transcription errors, I simply edit the Google Doc.

The sync service detects the change, regenerates summaries, re-embeds affected chunks, and updates Qdrant automatically.

No admin panel required.

Why traditional RAG chunking fails on conversations

Standard RAG splits text by token count. That works for documents but breaks on transcripts:

speaker boundaries are lost
replies get split mid-thought
retrieval returns fragments without context

The fix: treat each utterance as the atomic unit.

{
    "type": "utterance",
    "doc_id": "1BxK...drive_file_id",
    "client_name": "Acme Corp",
    "dialog_date": "2025-03-12",
    "speaker": "Alice",
    "text": "I're planning to launch in Q3, budget is around 5000.",
    "order_index": 42,
    "version": "sha256_of_content",
}

But utterances alone lose context. "Price is 5000" is meaningless without the surrounding conversation. So I generate summaries over sliding windows using anchor-based windowing:

def generate_windows(
    utterances: list[dict],
    anchor_step: int = 3,
    half_window: int = 2,
) -> list[dict]:
    if not utterances:
        return []
    windows = []
    n = len(utterances)
    for i in range(0, n, anchor_step):
        start = max(0, i - half_window)
        end = min(n - 1, i + half_window)
        window_utterances = utterances[start : end + 1]
        windows.append({
            "center_index": utterances[i]["order_index"],
            "covered_range": [
                utterances[start]["order_index"],
                utterances[end]["order_index"],
            ],
            "utterances": window_utterances,
        })
    return windows

Summaries and facts are generated in English regardless of the original language — so Claude always queries in English for consistent retrieval quality.

Why semantic search alone isn't enough

With summaries indexed, semantic search works well — most of the time.

The failure case: "what was the price she mentioned?" Specific numbers, names, short factual statements — these rarely survive summarization. The summary might say "discussed pricing" but the actual figure only lives in the raw utterance.

Semantic miss. Keyword hit.

The fix: three parallel searches.

Facts handle structured values like prices and dates. BM25 catches what semantic search misses — exact company names, abbreviations, or foreign words that don't survive summarization. If a client mentioned a specific vendor by name, semantic search might return "discussed partnerships" — BM25 finds the exact utterance.

from concurrent.futures import ThreadPoolExecutor

with ThreadPoolExecutor(max_workers=2) as executor:
    semantic_future = executor.submit(
        search_summaries_and_facts,
        query=query, client_name=client_name,
        date_from=date_from, date_to=date_to,
        top_k=15,
    )
    keyword_future = executor.submit(
        keyword_search_utterances,
        query=query, client_name=client_name,
        date_from=date_from, date_to=date_to,
    )
semantic_hits = semantic_future.result()   # dense, score >= 0.6
keyword_hits  = keyword_future.result()    # BM25, no threshold

# Pin: user_facts bypass score threshold entirely
user_fact_hits = search_user_facts(query, client_name=client_name)
pinned_hits = []
if user_fact_hits:
    hits_by_doc = {}
    for h in user_fact_hits:
        hits_by_doc[h["doc_id"]] = hits_by_doc.get(h["doc_id"], 0) + 1
    for doc_id, hit_count in hits_by_doc.items():
        pinned_hits.extend(search_summaries_for_doc(doc_id, query, top_k=hit_count * 5))

Note: ThreadPoolExecutor instead of asyncio.gather — the Qdrant Python SDK is synchronous. Real parallelism here comes from a thread pool, not coroutines.

Each search serves a different purpose:

Dense (semantic) over summaries and facts — finds topically relevant conversations
BM25 (keyword) over raw utterances — catches exact matches that don't survive summarization
Pin over user-added facts — forces specific documents into results regardless of score

Results are merged, overlapping ranges within the same document are combined, and the final
utterances are fetched by index range — no second search needed.

The output is a list of segments:

{
  "doc_id": "1BxKmN9vQ2rTzAp_drivefile",
  "client_name": "Acme Corp",
  "dialog_date": "2025-03-12",
  "segments": [
    {
      "range": [40, 46],
      "dialog": [
        {"speaker": "Alice",  "text": "So what's the timeline looking like?",             "order_index": 40},
        {"speaker": "Bob",    "text": "I need to be live by end of Q3.",                 "order_index": 41},
        {"speaker": "Alice",  "text": "I're planning to launch in Q3, budget is 5000.",  "order_index": 42},
        {"speaker": "Bob",    "text": "That works. Can you send a proposal by Friday?",   "order_index": 43},
        {"speaker": "Alice",  "text": "Sure, I'll have it over by Thursday.",             "order_index": 44}
      ]
    }
  ]
}

This is what gets sent to Claude as context. Not the full transcript — just the relevant
fragments.

What's next

Today TLBrain indexes 232 conversations and gives Claude access to years of client history — without loading entire transcripts into context.

The whole project is open source: TLBrain on GitHub

In the next post I'll cover how I turned this into a production remote MCP server with Google OAuth on Cloud Run.

One Tool Calling Interface for OpenAI, Claude, and Gemini

Sergey Inozemtsev — Thu, 12 Mar 2026 09:56:46 +0000

llm-api-adapter is an open‑source Python library designed to simplify working with multiple LLM providers.

Many AI applications today need to support multiple LLM providers.

Common reasons include:

cost optimization
fallback when a provider is unavailable
access to different model capabilities
experimentation with new models

In practice, the moment you try to support OpenAI, Claude, and Gemini, the integration becomes messy.

Tool calling alone already breaks portability:

Provider	Tool format
OpenAI	`tool_calls`
Anthropic	`tool_use` blocks
Google Gemini	`functionCall` / `functionResponse`

These are not just syntax differences.\
They require different request structures, response parsing, and execution loops.

Supporting multiple providers usually leads to:

provider-specific integration logic
provider-specific request/response handling
duplicated tool execution flows
multiple SDK dependencies

The result is more code, more bugs, and much harder provider switching.

To simplify this, I built llm-api-adapter — a small Python library that provides one unified interface for multiple LLM APIs.

Define tools once and run the same application logic across OpenAI, Anthropic, and Gemini.

Architecture

The adapter acts as a translation layer between your application and LLM providers.

              Application Logic
                     │
                     ▼
           UniversalLLMAPIAdapter
                     │
                     ▼
          Provider Translation Layer
                     │
                     ▼
 ┌─────────────┬─────────────┬─────────────┐
 │   OpenAI    │  Anthropic  │   Gemini    │
 │ tool_calls  │  tool_use   │ functionCall│
 └─────────────┴─────────────┴─────────────┘

Your application communicates with one interface, while the adapter converts requests and responses to the provider-specific formats.

Installation

pip install llm-api-adapter

The "Strawberry" problem

A classic example showing why tool calling matters:

How many "r" letters are in "strawberry"?

The correct answer is 3, but models often fail because they reason over tokens, not characters.

Best practice is:

Let the LLM reason, but delegate deterministic tasks to code.

This is exactly what tool calling enables.

Defining a tool once

With llm-api-adapter, tools are defined using a provider-agnostic schema.

from llm_api_adapter.models.tools import ToolSpec

tools = [
    ToolSpec(
        name="count_letter_in_word",
        description="Count how many times a specific letter appears in a word",
        json_schema={
            "type": "object",
            "properties": {
                "word": {"type": "string"},
                "letter": {"type": "string", "minLength": 1, "maxLength": 1},
            },
            "required": ["word", "letter"],
            "additionalProperties": False,
        },
    )
]

The adapter automatically converts this schema to:

OpenAI tools
Anthropic tool_use
Gemini functionCall

Running the same code across providers

The application logic remains identical.

Only the provider name, model, and API key change.

import json
from typing import Any, Dict

from llm_api_adapter.universal_adapter import UniversalLLMAPIAdapter
from llm_api_adapter.models.messages.chat_message import (
    UserMessage,
    AIMessage,
    ToolMessage,
)

def run_tool(name: str, args: Dict[str, Any]) -> Dict[str, Any]:
    if name == "count_letter_in_word":
        word, letter = args["word"], args["letter"]
        return {
            "word": word,
            "letter": letter,
            "count": word.lower().count(letter.lower()),
        }

providers = [
    ("openai", "gpt-5.2", openai_api_key),
    ("anthropic", "claude-haiku-4-5", anthropic_api_key),
    ("google", "gemini-2.5-flash", google_api_key),
]

for org, model, key in providers:
    adapter = UniversalLLMAPIAdapter(
        organization=org,
        model=model,
        api_key=key,
    )

    messages = [
        UserMessage('How many "r" letters are in "strawberry"?')
    ]

    first = adapter.chat(
        messages=messages,
        tools=tools,
        tool_choice="auto",
        max_tokens=1000,
    )

    if first.tool_calls:
        messages.append(
            AIMessage(content="", tool_calls=first.tool_calls)
        )
        for tc in first.tool_calls:
            result = run_tool(tc.name, tc.arguments)
            messages.append(
                ToolMessage(
                    tool_call_id=tc.call_id,
                    content=json.dumps(result),
                )
            )
        final = adapter.chat(
            messages=messages,
            previous_response=first,
            max_tokens=1000,
        )

        print(f"--- {org} / {model} ---")
        print(final.content)
        print()

Example output

Even though the models use different tokenization internally, they all trigger the tool correctly.

--- openai / gpt-5.2 ---
There are 3 letters "r" in "strawberry".

--- anthropic / claude-haiku-4-5 ---
There are 3 "r" letters in "strawberry".

--- google / gemini-2.5-flash ---
There are three "r" letters in "strawberry".

Without vs with an adapter

Problem	Without Adapter	With llm-api-adapter
Tool definitions	Provider specific	One universal schema
Tool execution	Custom logic per provider	Unified interface
Response parsing	Different formats	Single response model
Provider switching	Rewrite code	Change model string
Dependencies	Multiple SDKs	One library

Why this matters

Supporting multiple LLM providers normally requires separate integrations and duplicated logic.

A unified interface lets you:

keep application logic provider-agnostic
switch models without rewriting code
simplify agent architectures

Instead of adapting your code to each provider, you adapt the providers to your code.

GitHub

The project is open source.

👉 https://github.com/Inozem/llm_api_adapter

You will find full documentation, examples, and the source code in the repository.

Clean Architecture for AI Agents with Convo-Lang

Sergey Inozemtsev — Tue, 10 Feb 2026 20:47:41 +0000

Decoupling Orchestration from Reasoning

In this post, I’ll show how to design a clean, maintainable architecture for AI systems using Convo-Lang.

As a concrete example, I’ll use a hallucination-resistant AI agent that analyzes a job description, evaluates candidate fit against detailed professional experience, and generates a tailored resume only when the role is actually relevant.

In this setup, all reasoning and decision logic lives in Convo-Lang, while Python is used strictly for orchestration — loading inputs, executing agents, and wiring the pipeline together.

The goal of the example is not the resume itself. The goal is to demonstrate how to decouple orchestration from reasoning and build an AI system that is easy to understand, extend, and maintain over time.

The full working example is available in the Convo-Lang repository.

You can explore the complete code here: https://github.com/convo-lang/convo-lang/tree/main/packages/convo-lang-py/examples/02_patterns/resume_generator

You can clone it, run it locally, and experiment with it by simply replacing the job description and writing your own experience profile — the sample inputs live in the data/ folder.

What Convo-Lang actually is

Convo-Lang is not:

a prompt template engine
a thin wrapper around chat completions
a “nicer way to write prompts”

Convo-Lang is a domain-specific language for LLM reasoning and agent workflows.

It allows you to define:

explicit agent roles
typed input and output contracts
deterministic logic
schema-enforced outputs
multi-agent pipelines

All of this lives in .convo files — outside of application code.

Why resumes are a good stress test

Resume generation is a hostile domain for hallucinations:

inventing skills is unacceptable
inventing companies or roles is unacceptable
inventing dates is unacceptable
decisions must be explainable

A single “smart prompt” is the worst possible approach here.

So instead of asking how to prompt, I started by asking:
how should this system be modeled?

Modeling the system as Convo-Lang agents

The solution is built as five Convo-Lang agents, each responsible for exactly one thing:

JobDescriptionAnalyzer
Turns raw job text into structured requirements.
CandidateProfileAnalyzer
Converts free-form experience text into factual, structured data.
ProfileJobMatcher
Matches experience to requirements and explicitly lists gaps.
ResumeWriter
Generates a resume strictly from verified data.
FitEvaluator
Decides whether applying makes sense.

Each agent:

lives in its own .convo file
has a single responsibility
communicates only through typed contracts

This separation is not cosmetic.
It is the foundation of reliability.

Typed contracts instead of “return JSON please”

In most LLM systems, structured output is a suggestion.

In Convo-Lang, it is a contract.

Here is a real schema used by the CandidateProfileAnalyzer agent:

>define
ProfileData = struct(
  workExperience: array(
    struct(
      title: string
      companyName: string
      firstDate: string
      lastDate?: string
      summary: string
      experience: array(string)
    )
  )
  projects?: array(
    struct(
      title: string
      firstDate: string
      lastDate?: string
      experience: array(string)
    )
  )
)

This immediately changes system behavior:

required fields must exist
optional fields are explicit
invented fields are invalid
downstream agents can trust the data shape

Hallucinations don’t silently propagate.
They violate the contract.

Validating inputs before any reasoning happens

Hallucinations often start before generation.
They start when invalid or ambiguous input quietly enters the system.

Convo-Lang allows agents to validate inputs explicitly, before any reasoning takes place.

>define
JobData = struct(
  title: string
  mustRequirements: array(string)
  niceToHaveRequirements: array(string)
  keywords: array(string)
)

>do
jobData = new(JobData job_data)

That single line enforces a lot:

checks that job_data exists
validates required fields
enforces correct types
rejects malformed input early

If the input does not match JobData, the agent does not proceed.

The model never reasons over invalid data.

Here, input validation is part of the agent’s contract, not an afterthought.

Explainable matching instead of opaque scoring

The ProfileJobMatcher agent does not produce a mysterious score.

It produces:

only relevant roles and projects
explicit matchReasons for each item
two concrete gap lists: must-have and nice-to-have

MatchData = struct(
  coverageProfileData: struct(
    workExperience: array(
      title: string
      companyName: string
      firstDate: string
      lastDate?: string
      summary: string
      experience: array(string)
      matchReasons: array(string)
    )
    projects?: array(...)
  )
  gaps: struct(
    mustRequirements: array(string)
    niceToHaveRequirements: array(string)
  )
)

Nothing is hidden.
Every match and every gap is inspectable.

This output becomes the single source of truth for all downstream steps.

Deterministic logic inside the agent (not in prose)

A key feature of Convo-Lang is that deterministic logic lives next to reasoning.

In the FitEvaluator, the final decision is not guessed.
It is calculated.

>do
jobData = new(JobData job_data)
matchData = new(MatchData match_data)

totalConfidence = 100
jobRequirementsAmount = jobData.mustRequirements.length
requirementPoints = div(totalConfidence jobRequirementsAmount)

requirementGapAmount = matchData.gaps.mustRequirements.length
mainConfidence = mul(
  sub(jobRequirementsAmount requirementGapAmount)
  requirementPoints
)

decision = "apply"

if (lt(mainConfidence 70)) then (
  decision = "skip"
)
elif (lt(mainConfidence 90)) then (
  decision = "maybe apply"
)

This is business logic:

readable
reviewable
testable

The LLM explains the decision — but it does not invent the rules.

Schema-enforced output with `@json`

Convo-Lang does not rely on “please return JSON”.

It enforces it.

@json RecommendationData
>user
Help the candidate decide whether applying for this job makes sense.

If the output does not match RecommendationData, it is invalid.

Structured output is no longer a best-effort promise.
It is a guarantee.

Python as an orchestrator, not a reasoning layer

So where does Python fit into this architecture?

Python is intentionally boring.

It does not:

contain prompts
contain business rules
interpret free-form model output

It only:

loads input data
executes agents
passes validated JSON between them
handles I/O

job_data = convo_job_description_analyzer.complete(...)
profile_data = convo_candidate_profile_analyzer.complete(...)
match_data = convo_profile_job_matcher.complete(...)
resume_data = convo_resume_writer.complete(...)
decision = convo_fit_evaluator.complete(...)

All intelligence lives in .convo.
Python is just the runtime.

This separation is deliberate.

Why this separation matters

By keeping reasoning in Convo-Lang and orchestration in Python:

AI logic becomes portable
behavior is consistent across CLI, editor, and SDK
prompt changes don’t require backend redeploys
agent logic can be reviewed like code

The agents folder becomes the product.
The SDK becomes an implementation detail.

What this example actually demonstrates

This post is not really about resumes.

It demonstrates that Convo-Lang lets you:

treat LLM logic as first-class code
build multi-agent systems without prompt chaos
validate inputs and outputs explicitly
make hallucinations visible instead of hidden
scale reasoning without rewriting everything

That is why Convo-Lang is worth using.

Final takeaway

Hallucinations are rarely a model problem.
They are almost always an architecture problem.

Convo-Lang gives you the tools to fix that at the right level.

Resources

Convo-Lang core: https://github.com/convo-lang/convo-lang
Convo-Lang Python SDK: https://github.com/convo-lang/convo-lang/tree/main/packages/convo-lang-py
Resume agent example: https://github.com/convo-lang/convo-lang/tree/main/packages/convo-lang-py/examples/02_patterns/resume_generator
Documentation: https://learn.convo-lang.ai/

Prompts are logic, not strings: Why I contributed to Convo-Lang

Sergey Inozemtsev — Sun, 04 Jan 2026 13:36:56 +0000

If you’ve built anything non-trivial with LLMs, you’ve probably written code like this:

prompt = f"""
Analyze this job description: {job_description}
Analyze this candidate profile: {candidate_profile}
Decide whether the candidate is a good fit.
Return JSON.
"""

It works.
Until your project grows.

The problem: prompt spaghetti and technical debt

Hardcoding prompts directly into application code feels convenient at first.
But very quickly it turns into long-term technical debt:

Prompts become unreadable f-string monsters
Prompt changes require code changes and redeploys
Prompt versions drift across files and branches
Prompt engineers and copywriters are afraid to touch .py
Prompt logic, business logic, and orchestration logic get mixed together

At that point, prompts are:

hard to test
hard to reuse
hard to reason about

We already solved this problem for SQL, HTML, configs, and templates.
LLM prompts deserve the same treatment.

Prompts are not strings — they are logic

A modern LLM “prompt” is not just text.

It contains:

structure
contracts
conditions
branching
deterministic steps

Treating it as a Python string literal is the fastest way to lose control over your AI system.

That’s where Convo-Lang comes in.

Convo-Lang as an AI-native DSL

Convo-Lang is an open-source, AI-native DSL for building conversations and agent workflows.

Instead of embedding prompts into code, you define them in .convo files:

explicit schemas
role-based messages
deterministic logic blocks
structured outputs
multi-agent workflows

Your prompt becomes a first-class artifact, not a string buried in code.

How it works: Python as a thin runtime

Here’s all the Python code required to run a single agent:

print("Running FitEvaluator agent...")

convo_fit_evaluator = Conversation(agent_configs)
convo_fit_evaluator.add_convo_file("agents/fitEvaluator.convo")

job_apply_decision = convo_fit_evaluator.complete(
    variables={"job_data": job_data, "match_data": match_data}
)

Notice what’s missing:

no prompt text
no formatting logic
no hidden reasoning

Python is just the runtime.
All intelligence lives in .convo.

Typed contracts instead of free-form prompts

The core idea that changed how I think about prompts:

Agents should communicate through typed contracts, not vague instructions.

Example schema definitions used by an agent:

>define
JobData = struct(
  title:string
  mustRequirements:array(string)
  niceToHaveRequirements:array(string)
)

RecommendationData = struct(
  recommendation: struct(
    decision: enum("apply","maybe apply","skip")
    confidence: number
    summary: string
  )
)

This gives you:

explicit input shapes
explicit output contracts
predictable agent behavior

The agent no longer “guesses” what to return.

Deterministic logic lives next to the prompt

Convo-Lang is not just a prompt format.
It allows you to define explicit, deterministic logic inside the agent.

>do
jobData = new(JobData job_data)
matchData = new(MatchData match_data)

total = 100
reqCount = jobData.mustRequirements.length
niceCount = jobData.niceToHaveRequirements.length

reqPoints = div(total reqCount)
nicePoints = div(reqPoints 4)

reqGaps = matchData.gaps.mustRequirements.length
niceGaps = matchData.gaps.niceToHaveRequirements.length

confidence = add(
  mul(sub(reqCount reqGaps) reqPoints)
  mul(sub(niceCount niceGaps) nicePoints)
)

decision = "apply"
if (lt(confidence 70)) then (decision = "skip")
elif (lt(confidence 90)) then (decision = "maybe apply")

This is business logic, not prompt prose.

Schema-enforced output with `@json`

@json RecommendationData
>user
Return recommendation for this candidate and job.

This is not a suggestion.
It is schema‑enforced output validation.

Malformed or invalid responses don’t silently pass through.

Cross-SDK portability by design

Because the Convo-Lang core is implemented in TypeScript, it guarantees identical behavior across environments:

VS Code preview
CLI
Python runtime

If your prompt passes validation in the editor, it will behave the same way in your Python backend.

Write once. Run anywhere.

Architecture: a smart bridge, not a rewrite

The Python SDK does not reimplement the Convo-Lang engine.

Instead, it acts as a high‑performance bridge to the Node.js core, which handles parsing, validation, and async I/O.

This preserves full syntax and behavior parity across SDKs.

Separation of concerns — for real

With this approach:

.convo files own AI reasoning and decision logic
Python only orchestrates execution
Prompt engineers don’t touch backend code
Developers don’t rewrite prompts

The agents/ folder is the product.
Python is just the runtime.

Why I contributed to the Python SDK

I believe AI workflows need standards.

Prompts should be portable, testable, and explicit.

That’s why I helped bring Convo-Lang to Python.

Resources

Resume generator example (Python): https://github.com/convo-lang/convo-lang/tree/main/packages/convo-lang-py/examples/02_patterns/resume_generator
Convo-Lang core: https://github.com/convo-lang/convo-lang
Convo-Lang Python SDK: https://github.com/convo-lang/convo-lang/tree/main/packages/convo-lang-py
Documentation: https://learn.convo-lang.ai/

How I built a bulletproof CI/CD for my LLM Python library

Sergey Inozemtsev — Wed, 24 Dec 2025 09:20:35 +0000

When building an open-source library that integrates with multiple LLM providers (OpenAI, Anthropic, Google), reliability matters. Users expect upgrades to be safe and predictable.

This post describes the CI/CD setup I use for llm-api-adapter. The key idea is simple: test not only the code, but the actual published package.

The Strategy: two pipelines, three stages

I use a dual-pipeline setup aligned with GitHub Flow:

Dev pipeline — runs on every push to dev. Its job is early feedback and validating the distribution process.
Main pipeline — runs on main and version tags. Its job is stable, repeatable releases to PyPI.

Most CI setups stop at unit or integration tests. This one goes further by validating the artifact installed from TestPyPI.

1. Dev pipeline: pre-flight validation

The dev workflow is where most of the safety guarantees come from.

Stage A: Unit & Integration tests

Executed with pytest
Tests are separated via markers (unit, integration)
Fast feedback on logic and provider integration

Stage B: publish to TestPyPI

After tests pass, the package is built and published to TestPyPI.

This step catches issues that tests alone cannot:

Incorrect pyproject.toml
Missing files in the source distribution
Broken dependency declarations

Stage C: E2E tests from TestPyPI

This is the critical part of the pipeline.

The job:

Waits for TestPyPI to index the new release
Installs the package from TestPyPI, not from source
Pulls dependencies from the real PyPI
Runs real end-to-end tests using live API keys

pip install --index-url https://test.pypi.org/simple/ \
            --extra-index-url https://pypi.org/simple \
            llm-api-adapter

At this point, the CI environment matches what users will experience after pip install.

2. Main pipeline: controlled release

Once the package is validated in dev, changes move to main.

What runs on `main`

Full unit + integration test suite on every PR
No publishing on pushes

What triggers a release

A version tag (vX.Y.Z)
Build and publish to PyPI
Credentials handled via GitHub Secrets

By the time a tag is pushed, the same artifact has already passed E2E tests via TestPyPI.

Why this setup works

Before publishing to PyPI, I know that:

The code behaves correctly (unit + integration tests)
The package is installable from a registry (TestPyPI)
External LLM providers respond as expected (E2E tests)

Most importantly, this approach prevents broken versions from ever being published to PyPI.

Conclusion

If your library depends on external APIs, testing only the source code is not enough.

Testing the published artifact is what makes releases predictable and safe.

The full setup is fully public and reproducible:

👉 Repository:

https://github.com/Inozem/llm_api_adapter

👉 GitHub Actions workflows:

https://github.com/Inozem/llm_api_adapter/tree/main/.github/workflows

Question for you

How do you usually set up CI for your open-source projects?

Python LLM: reasoning is disabled by default in llm-api-adapter

Sergey Inozemtsev — Sun, 14 Dec 2025 09:00:27 +0000

Reasoning improves LLM output quality, but it is expensive and often unnecessary. Worse: most providers enable it implicitly or hide it behind non-obvious parameters.

Result: developers pay for reasoning even when they don’t need it.

The Problem

Reasoning is handled inconsistently across providers:

OpenAI: often enabled implicitly.
Gemini: controlled via thinkingConfig.
Anthropic (Claude): may enforce minimum reasoning tokens (e.g. 1024).
Nano/Mini models: sometimes impossible to disable reasoning entirely.

This leads to:

hidden costs
provider-specific conditionals
easy-to-miss misconfiguration

The Approach: Off by Default

Starting from llm_api_adapter v0.2.3, reasoning is disabled by default.

If it is not explicitly enabled:

no reasoning tokens are used
no extra cost is incurred
existing code keeps working

Costly features should be opt-in, not opt-out.

Enabling Reasoning Explicitly

When reasoning is actually required, it can be enabled via a single unified parameter:

String levels: "none" | "low" | "medium" | "high"
Numeric values: 256, 512, 1024, 2048, etc.

The adapter:

maps the value to provider-specific fields
applies correct formats per API
respects provider minimums
prevents invalid configurations

Installation

pip install llm-api-adapter

Example

from llm_api_adapter.universal_adapter import UniversalLLMAPIAdapter

messages = [
    {"role": "user", "content": "Explain quantum computing simply."}
]

# Pick a provider (same interface)
adapter = UniversalLLMAPIAdapter(
    organization="openai",
    model="gpt-5.1",
    api_key=openai_api_key,
)

# or
# adapter = UniversalLLMAPIAdapter(
#     organization="google",
#     model="gemini-2.5-pro",
#     api_key=google_api_key,
# )

# or
# adapter = UniversalLLMAPIAdapter(
#     organization="anthropic",
#     model="claude-sonnet-4-5",
#     api_key=anthropic_api_key,
# )

response = adapter.chat(
    messages=messages,
    reasoning_level="low",  # off by default, enabled explicitly
)

print(response.content)

Why This Matters

Lower and predictable costs
No accidental reasoning usage
Cleaner application code
Unified control across OpenAI, Claude, and Gemini

Repository

Source code and documentation: https://github.com/Inozem/llm-api-adapter

Structured prompts: how YAML cut my LLM costs by 30%

Sergey Inozemtsev — Wed, 05 Nov 2025 09:34:01 +0000

Result Summary:

Metric	Original Prompt	YAML Prompt	Savings
Tokens	355	251	−104 (−29.3%)
Cost	0.00001775 USD	0.00001255 USD	−29.3%

I'll show how this works using a popular prompt taken from the internet, rewritten in YAML format to show whether structured phrasing can reduce token count without harming quality.

Why it matters:

Fewer tokens → lower cost per request.
YAML forces clarity and structure, improving consistency of answers.
Easier to maintain and version prompts in code.

Original Prompt (PROMPT_A)

prompt_a = """
You are the "Architect Guide," specialized in assisting programmers who are experienced in individual module development but are looking to enhance their skills in understanding and managing entire project architectures.
Your primary roles and methods of guidance include:

Basics of Project Architecture: Start with foundational knowledge, focusing on principles and practices of inter-module communication and standardization in modular coding.
Integration Insights: Provide insights into how individual modules integrate and communicate within a larger system, using examples and case studies for effective project architecture demonstration.
Exploration of Architectural Styles: Encourage exploring different architectural styles, discussing their suitability for various types of projects, and provide resources for further learning.
Practical Exercises: Offer practical exercises to apply new concepts in real-world scenarios.
Analysis of Multi-layered Software Projects: Analyze complex software projects to understand their architecture, including layers like Frontend Application, Backend Service, and Data Storage.
Educational Insights: Focus on reviewing project readme files and source code for comprehensive understanding.
Use of Diagrams and Images: Utilize architecture diagrams and images to aid in understanding project structure and layer interactions.
Clarity Over Jargon: Avoid overly technical language, focusing on clear, understandable explanations.
No Coding Solutions: Focus on architectural concepts and practices rather than specific coding solutions.
Detailed Yet Concise Responses: Provide detailed responses that are concise and informative without being overwhelming.
Practical Application and Real-World Examples: Emphasize practical application with real-world examples.
Clarification Requests: Ask for clarification on vague project details or unspecified architectural styles to ensure accurate advice.
Professional and Approachable Tone: Maintain a professional yet approachable tone.
Use of Everyday Analogies: When discussing technical concepts, use everyday analogies to make them more accessible and understandable.
"""

Optimized YAML Prompt (PROMPT_B)

prompt_b = """
system: |
  Role: "Architect Guide"
  Purpose: Help developers skilled in module-level coding grow into understanding and managing full project architectures.

guidelines: |
  - Teach project architecture fundamentals: modular communication, standardization, and structure.
  - Explain module integration within larger systems using examples and case studies.
  - Compare architectural styles, discuss suitability, and share learning resources.
  - Provide practical exercises for real-world application.
  - Analyze multi-layered software (frontend, backend, data storage) to illustrate architecture.
  - Offer educational insights: review README files and source code for comprehension.
  - Use diagrams and visuals to clarify system interactions.
  - Prefer clarity over jargon; use plain, accessible language.
  - Focus on architecture concepts — no coding solutions.
  - Be detailed yet concise; avoid information overload.
  - Include real-world examples for practical relevance.
  - Ask clarifying questions about unclear project details.
  - Maintain a professional, approachable tone.
  - Use everyday analogies for complex concepts.

style: |
  Clear, didactic, structured.
  Encourage understanding of architecture as a living system, not just code components.
"""

Observation: The output quality didn’t just stay the same — it improved. ChatGPT understood the intent better, and responses became more focused.

Experiment Code Example

# Universal adapter
# using [llm-api-adapter](https://github.com/Inozem/llm_api_adapter)
# makes it easy to switch between different providers for testing
# can be installed easily via: pip install llm-api-adapter
from llm_api_adapter.universal_adapter import UniversalLLMAPIAdapter

messages_a = [
    {"role": "system", "content": prompt_a},
    {"role": "system", "content": "Help me to create weather application."},
]
messages_b = [
    {"role": "system", "content": prompt_b},
    {"role": "system", "content": "Help me to create weather application."},
]
adapter = UniversalLLMAPIAdapter(
    organization="openai",
    model="gpt-5-nano",
    api_key=openai_api_key,
)
# Runs
resp_a = adapter.chat(messages=messages_a)
resp_b = adapter.chat(messages=messages_b)
# Token savings
tokens_a = resp_a.usage.input_tokens
tokens_b = resp_b.usage.input_tokens
saved = tokens_a - tokens_b
rel = (saved / tokens_a) * 100 if tokens_a else 0
print("PROMPT A tokens:", tokens_a)
print("PROMPT B tokens:", tokens_b)
print("Saved tokens:", saved)
print(f"Relative saving: {rel:.1f}%")
# Cost
print("Cost A:", f"{resp_a.cost_input:.8f} {resp_a.currency}")
print("Cost B:", f"{resp_b.cost_input:.8f} {resp_b.currency}")

Prompt I used to generate the new YAML-formatted version:

Optimize this prompt into YAML format

Conclusion: Structured prompts are not just cleaner — they’re cheaper. Try YAML structuring in your next LLM project. It’s simple, reproducible, and can cut your costs by ~30%.

Unifying 3 LLM APIs in Python: OpenAI, Anthropic & Google with one SDK

Sergey Inozemtsev — Tue, 04 Nov 2025 12:34:19 +0000

A year ago, I released the first version of LLM API Adapter — a lightweight SDK that unified OpenAI, Anthropic, and Google APIs under one interface.

It got 7 ⭐ on GitHub and valuable feedback from early users.

That was enough motivation to take it to the next level.

What changed in the new version

The new version (v0.2.2) is now:

SDK-free — it talks directly to provider APIs, no external dependencies.
Unified — one chat() interface for all models (OpenAI, Anthropic, Google).
Transparent — automatic token and cost tracking.
Resilient — consistent error taxonomy across providers (auth, rate, timeout, token limits).
Tested — 98% unit test coverage.

Example: chat with any LLM

from llm_api_adapter.universal_adapter import UniversalLLMAPIAdapter

adapter = UniversalLLMAPIAdapter(provider="openai", model="gpt-5")

response = adapter.chat([
    {"role": "system", "content": "Be concise."},
    {"role": "user", "content": "Explain how LLM adapters work."},
])

print(response.content)

Switching models is as simple as changing two parameters:

adapter = UniversalLLMAPIAdapter(provider="anthropic",
                                 model="claude-sonnet-4-5")
# or
adapter = UniversalLLMAPIAdapter(provider="google",
                                 model="gemini-2.5-pro")

Token & cost tracking example

Every response now includes full token and cost accounting — no manual math needed.

from llm_api_adapter.universal_adapter import UniversalLLMAPIAdapter

google = UniversalLLMAPIAdapter(
    organization="google",
    model="gemini-2.5-pro",
    api_key=google_api_key
)

response = google.chat(**chat_params)

print(response.usage.input_tokens, "tokens",
      f"({response.cost_input} {response.currency})")
print(response.usage.output_tokens, "tokens",
      f"({response.cost_output} {response.currency})")
print(response.usage.total_tokens, "tokens",
      f"({response.cost_total} {response.currency})")

Output:

512 tokens (0.00025 USD)
137 tokens (0.00010 USD)
649 tokens (0.00035 USD)

Why I built this

Working with multiple LLMs used to mean rewriting the same code — again and again.

Each SDK had its own method names, parameter names, and error classes.

So I built a unified interface that abstracts those details.

One adapter — one consistent experience.

Join the project

You can try it now:

pip install llm-api-adapter

Docs & examples: github.com/Inozem/llm_api_adapter

If you like the idea — ⭐ star it or share feedback in Issues.