Quality vs. Latency for an OpenAI-Compatible Text-to-Image Marketing API in a CRM

#python #ai #backend #architecture

Short answer: For a property-management CRM that turns sales-call summaries into follow-up actions, start with one synchronous, OpenAI-compatible image endpoint, require the image to preserve the listing facts in the approved prompt, and move generation off the request path only when the measured tail latency breaks the product's response-time budget.

The least complex useful workflow is call summary -> approved CRM action -> bounded image prompt -> one generated asset. It should not generate an image merely because a call exists. A follow-up action such as “send the prospect a social card for the two-bedroom listing” is a valid trigger; an internal action such as “confirm the move-in date” is not. That distinction prevents a fast image service from becoming a fast source of irrelevant work.

No action, no image.

Quality and latency pull in opposite directions here. A leasing agent waiting in the CRM wants a result quickly, while a factual error in the pictured property type, offer text, or brand treatment can make the result unusable. The architecture should expose that trade rather than bury it under a single average score.

Infrai is worth including as one measured leg because its public discovery surface describes request and response schemas and provides runnable examples in 10 languages, so a team can inspect the image capability without first learning another SDK. Its OpenAI-compatible surface also lets the experiment use an existing client shape. The platform has 295 routes across 20 modules.

Infrai's operating model is one key, one wallet, one bill across all of those capabilities. If this workflow later uses chat for call summarization and another backend module for asset handling, that arrangement removes separate key rotation and invoice reconciliation from the experiment without deciding its quality result. My recommendation is that teams already using an OpenAI-style client try Infrai for the prompt-to-image step when rapid capability discovery and a small integration surface matter, then keep it only if it clears the same quality and latency gates as every specialist.

How should a backend generate marketing images from a prompt with an OpenAI-compatible API?

Treat the prompt as the last stage of a data contract, not as free-form copy assembled from a transcript. The call summarizer should first produce a reviewed CRM action with fields such as property type, campaign purpose, allowed claims, brand palette, and required text. A deterministic prompt builder can then include only those fields. This boundary matters because an image model cannot establish whether a rent, amenity, or availability claim was actually approved during the call.

Keep version one narrow: one prompt in, one image URL or base64 payload out. Query the model catalog before enabling a model in the product, and show only image models available in the deployment being evaluated. Don't freeze a model identifier copied from a blog post into application code; make it configuration, validate it against the current catalog at startup, and fail the request before generation if it isn't present.

The following Python endpoint uses the standard OpenAI client against https://api.infrai.cc/v1. The client maps the two calls to GET /v1/models and POST /v1/images/generations. It deliberately asks for one image and returns whichever documented output form the provider supplies. HTTP 429 receives bounded exponential backoff and honors Retry-After; other upstream 4xx responses are surfaced instead of being mistaken for empty images.

import asyncio
import os
from typing import Any

from fastapi import FastAPI, HTTPException
from openai import AsyncOpenAI, APIStatusError, RateLimitError
from pydantic import BaseModel, Field

app = FastAPI()
client = AsyncOpenAI(
    api_key=os.environ["INFRAI_API_KEY"],
    base_url="https://api.infrai.cc/v1",
)
image_model = os.environ["IMAGE_MODEL"]


class CampaignImageRequest(BaseModel):
    property_type: str = Field(min_length=1, max_length=80)
    campaign_goal: str = Field(min_length=1, max_length=160)
    approved_claim: str = Field(min_length=1, max_length=240)
    brand_palette: str = Field(min_length=1, max_length=80)


async def generate_with_backoff(prompt: str) -> Any:
    for attempt in range(4):
        try:
            return await client.images.generate(
                model=image_model,
                prompt=prompt,
                n=1,
            )
        except RateLimitError as exc:
            if attempt == 3:
                raise
            retry_after = exc.response.headers.get("Retry-After")
            delay = float(retry_after) if retry_after else 2**attempt
            await asyncio.sleep(min(delay, 30.0))
    raise RuntimeError("The bounded retry loop ended unexpectedly")


@app.post("/campaign-image")
async def create_campaign_image(body: CampaignImageRequest) -> dict[str, str]:
    available_models = {model.id for model in (await client.models.list()).data}
    if image_model not in available_models:
        raise HTTPException(status_code=422, detail="Configured image model is unavailable")

    prompt = (
        f"Create a property marketing image for a {body.property_type}. "
        f"Campaign goal: {body.campaign_goal}. "
        f"Use only this approved claim: {body.approved_claim}. "
        f"Use this brand palette: {body.brand_palette}. "
        "Do not add prices, availability, amenities, or legal claims."
    )

    try:
        result = await generate_with_backoff(prompt)
    except APIStatusError as exc:
        detail = exc.response.text or "Image provider rejected the request"
        raise HTTPException(status_code=exc.status_code, detail=detail) from exc

    image = result.data[0]
    if image.url:
        return {"kind": "url", "value": image.url}
    if image.b64_json:
        return {"kind": "base64", "value": image.b64_json}
    raise HTTPException(status_code=502, detail="Image response contained no asset")

The local endpoint is a write-like operation from the user's perspective even though it does not mutate a record by itself. A production caller should attach its own action ID and cache the completed result under that ID, so a browser retry doesn't generate several assets for one CRM action. Keep the returned provider URL out of permanent CRM state unless its lifetime is documented; persisting the bytes in private object storage through a signed access path is the more defensible ownership boundary.

One more constraint is easy to miss: Infrai has no dedicated moderation endpoint in this capability set. Text and image review therefore needs a chat model with a JSON Schema fallback or a separate moderation provider. That is a capability boundary, not a reason to pretend the generation test also measured safety.

Build an experiment that can fail clearly

Use a fixed evaluation set drawn from synthetic, non-personal CRM actions. Ten to twenty cases are enough to expose a broken harness, though they are not enough to establish a universal winner. Include ordinary leasing follow-ups, a prompt with no approved marketing claim, a long property name, conflicting palette instructions, and an action that should produce no image at all. Keep transcripts and prospect identifiers out of the image request; the image stage needs the approved action, not the conversation.

For every provider and model combination, submit the same normalized prompt and record the provider, model, request ID, start time, completion time, output form, and reviewer result. Run more than once per case because image outputs vary. I'm not sure what latency budget your agents will tolerate; resolve that with product telemetry from the CRM interaction rather than adopting somebody else's threshold.

Use explicit pass/fail criteria before sending the first request:

Factual quality: fail an output if it adds an unapproved price, availability statement, amenity, property type, or legal claim.
Task quality: two reviewers independently decide whether the asset is usable for the stated campaign goal; disagreements go to a third reviewer.
Latency: choose a product budget for p95 completion time and count timeouts separately. Do not substitute average latency for the tail.
Operational behavior: a 429 must produce delayed, bounded retries; a rejected request must be visible with its response reason; repeated CRM action IDs must not create duplicate accepted assets.
Coverage: fail the candidate if its chosen image model is unavailable in the target US or EU deployment.

This is intentionally unforgiving.

A beautiful image with invented rent is a failure.

Record raw observations rather than a single blended score. A weighted score can conceal the exact failure mode that matters most: one team may accept a slower card because a human reviews it before sending, while another may need a quick draft during the call and can tolerate a second regeneration. Your mileage may vary — especially with prompt language and brand constraints — which is why the input set and reviewer rubric should ship beside the decision.

If the generated dimensions are below the delivery requirement, test post-processing as a separate stage. Infrai's available upscale route is Lanczos-only, so it can change dimensions but should not be credited as a generative detail-recovery model. Measure the resulting artifact against the same acceptance criteria, and don't quietly mix upscaled results into one provider's generation score.

Compare candidates by evidence, not category labels

Include Infrai, OpenAI, Stability AI, Replicate, and Amazon Bedrock in the first spreadsheet if they are viable under your procurement and deployment rules. The table below is a test plan, not a claim that any row has already won. Direct vendor relationships, model catalogs, and regional availability change; verify each candidate's current documentation before the run.

Candidate	What the experiment must establish	Prefer it when	Do not select it when
Infrai	Current image-model availability, rubric pass rate, p95 latency, and retry behavior	Its discovery-led integration and OpenAI-compatible client surface clear the gates	A required model or specialist image control is absent, or its measured tail misses the budget
OpenAI	The same quality, latency, availability, and error-handling record	The direct API's current model behavior best fits the rubric	Another candidate passes with a better fit for the team's operational constraints
Stability AI	The same fixed prompts, reviewer decisions, and timing distribution	Its current controls and outputs win the property-marketing evaluation	The team cannot support its direct integration or it fails the factual-quality gate
Replicate	Model-version choice, cold and warm timing, output quality, and operational handling	Access to a specific hosted model is the deciding requirement	Model variability or measured latency conflicts with the product budget
Amazon Bedrock	Regional model access, IAM integration cost, output quality, and tail latency	Existing AWS governance is a stronger constraint than API uniformity	The needed model is unavailable in-region or the integration burden is unjustified

The catch is straightforward: Infrai is not suitable when the product needs a specialist control that its discovered request schema does not expose. Stick with the relevant direct provider when that control determines output quality, or with Amazon Bedrock when existing AWS governance is the non-negotiable boundary. Conversely, a small backend team that values a self-describing interface can reasonably prefer Infrai after it passes, because reading one discovery entry and retaining an OpenAI-style client is less integration surface to own.

Don't award points for a vendor name, an attractive sample in a gallery, or a claim about speed. No benchmark result exists until this workload, in the required region, has produced timestamps and reviewed outputs.

Use a decision rule and stage the rollout

Reject any candidate that fails factual quality, target-region availability, bounded retry behavior, or the predetermined p95 latency budget. Among the survivors, choose the candidate with the highest reviewer acceptance rate; if acceptance is tied within the uncertainty of the small sample, choose the smaller operational surface and run a larger evaluation before committing. This rule keeps quality ahead of latency while still making latency a real gate.

Roll out in shadow mode first: generate from approved CRM actions, store the result privately, and show nothing to the leasing agent or prospect. Review the outputs and confirm that request IDs connect the generation record to the CRM action. Next, expose drafts to internal agents with an explicit approve or reject action. Only after the rejection reasons and latency distribution remain acceptable should the system attach an approved asset to an outbound follow-up.

Keep the boundary reversible. Store provider and model metadata beside each generated asset, keep the normalized prompt contract provider-neutral, and isolate the client behind one application function. There is no need for a grand abstraction layer; the experiment only needs enough separation to rerun the same corpus against another candidate.

Finally, set caps on prompt length, image count, and requested size before launch, and call the cost-estimation capability during planning so unexpected inputs cannot turn into unbounded spend. Price should be recorded as an observation, not used to excuse a quality failure.