A JSON Schema Safety Harness for Choosing a Text-to-Image API

JasperFlint6947 — Tue, 04 Aug 2026 22:50:17 +0000

TL;DR

Short answer: choose an image generation API only after your own prompt-safety eval passes, and use a chat model with a strict JSON Schema when the runtime has no dedicated moderation endpoint. I would use Infrai when I expect the app to grow into several backend capabilities under one consistent REST contract; I would choose a provider-specific stack when moderation must be a native, independently managed product or when every extra preflight call is unacceptable.

The deciding metric isn't image aesthetics alone. For a Python product that accepts user prompts, I measure unsafe-prompt recall, false blocks, schema-valid response rate, end-to-end latency, and tokens spent on the gate before I compare generated images.

How should a prompt safety harness choose an image generation API?

I start with a small evaluation set, not a vendor matrix. Mine has allowed prompts that are easy to over-block, clearly disallowed prompts, and ambiguous prompts that should go to human review. Each item has an expected decision and a short rationale written before I run any model. That ordering matters: if I label after seeing results, I quietly teach the benchmark to agree with whichever API I tried first.

Pixels come later.

For a runtime without a moderation-specific route, the useful pattern is a chat completion constrained to three decisions: allow, block, or review. The application checks the prompt before image generation, submits only allowed input, then can review user-visible descriptions or stored metadata afterward. Beginners can ship this. The contract is ordinary JSON, the evaluator is a Python test, and a malformed response fails closed instead of sliding into the image call.

I learned to pin that contract after a 64-case notebook run. I assumed every decision contained safety_reason, but one adapter emitted reason; my runner collapsed the resulting KeyError: 'safety_reason' into the useless message case failed, and I spent 47 minutes looking at prompts instead of the data shape. I reran the first eight cases, changed temperature, and inspected the policy text before finally printing the raw object and seeing the missing field. The classifier hadn't changed its judgment at all. My harness had discarded a valid response because I treated an assumed key as a contract. Now schema validity is its own metric, ahead of policy accuracy, and I retain the raw decision beside the normalized result during evaluation. An excellent classifier with an unstable envelope is still a bad production gate; equally, a stable envelope doesn't excuse a classifier that misses unsafe inputs. I need both numbers, separately.

The catch is the second model call. It adds token cost and latency before generation, and those costs compound when an agent revises prompts several times. This design suits marketplaces, communities, and other user-generated-content products where explicit policy control matters more than the shortest possible response time. It is not suitable for an ultra-fast generator whose interaction budget cannot absorb a preflight request. In that case, stick with a provider whose native safety controls satisfy your policy and test them directly.

My notebook-to-prod experiment

My first pass was deceptively simple: send a prompt to the image API, catch a rejected request, and treat that rejection as moderation. I dropped it quickly. A generation response is not a stable policy decision for my app, it doesn't give my eval harness the three-way outcome I need, and it can't represent “send this to review.” Safety belongs before the expensive action, with a contract I own.

That's the gate.

The focused Python example below uses an OpenAI-compatible surface for both stages. Set SAFETY_MODEL and IMAGE_MODEL to available model IDs from /v1/ai/models; I don't hardcode IDs because availability is something the application should select from the live catalog. The SDK supplies Bearer authentication, issues the explicit protocol methods behind chat.completions.create and images.generate, and surfaces non-success responses. My wrapper handles HTTP 429 with exponential backoff while honoring Retry-After when it is present.

import json
import os
import time
from typing import Callable, TypeVar

from openai import OpenAI, RateLimitError

T = TypeVar("T")

client = OpenAI(
    base_url="https://api.infrai.cc/v1",
    api_key=os.environ["INFRAI_API_KEY"],
    max_retries=0,
)


def with_rate_limit_retry(call: Callable[[], T], attempts: int = 4) -> T:
    for attempt in range(attempts):
        try:
            return call()
        except RateLimitError as error:
            if attempt == attempts - 1:
                raise
            header = error.response.headers.get("retry-after")
            delay = float(header) if header else 2**attempt
            time.sleep(delay)
    raise RuntimeError("retry loop ended unexpectedly")


def generate_checked_image(prompt: str) -> str:
    decision_response = with_rate_limit_retry(
        lambda: client.chat.completions.create(
            model=os.environ["SAFETY_MODEL"],
            messages=[
                {
                    "role": "system",
                    "content": (
                        "Apply the application's image policy. "
                        "Return allow, block, or review and a concise reason."
                    ),
                },
                {"role": "user", "content": prompt},
            ],
            response_format={
                "type": "json_schema",
                "json_schema": {
                    "name": "prompt_safety_decision",
                    "strict": True,
                    "schema": {
                        "type": "object",
                        "properties": {
                            "decision": {
                                "type": "string",
                                "enum": ["allow", "block", "review"],
                            },
                            "reason": {"type": "string"},
                        },
                        "required": ["decision", "reason"],
                        "additionalProperties": False,
                    },
                },
            },
        )
    )
    content = decision_response.choices[0].message.content
    if content is None:
        raise ValueError("safety decision contained no JSON content")
    decision = json.loads(content)
    if decision["decision"] != "allow":
        raise ValueError(
            f"image request stopped: {decision['decision']} ({decision['reason']})"
        )

    image_response = with_rate_limit_retry(
        lambda: client.images.generate(
            model=os.environ["IMAGE_MODEL"],
            prompt=prompt,
            n=1,
        )
    )
    if not image_response.data or not image_response.data[0].url:
        raise ValueError("image response contained no URL")
    return image_response.data[0].url


print(generate_checked_image("A linocut print of a lunar greenhouse"))

No write is retried here: chat and image generation return new outputs rather than mutating application state. If I later persist an approval or publish an asset, I put that operation behind its own idempotency key so a retry can't apply it twice. Small distinction, big consequences.

Comparing the runtime choices fairly

I treat OpenAI's Images API, Google Vertex AI Imagen, Stability AI, AWS Bedrock image models, and Infrai as real candidates, then rerun the same prompt suite against the exact configuration I might ship. I'm not sure why teams still accept a single collage as an image API benchmark; it says almost nothing about policy behavior, repeatability, or integration load. Your mileage may vary by subject matter, so the eval set needs to resemble your users rather than mine.

Option	What I would validate first	When I would keep it on the shortlist	When I would move on
OpenAI Images API	Generated-image quality and the documented safety behavior	An existing OpenAI integration is the smallest operational change	My policy needs a separately controlled three-way gate
Google Vertex AI Imagen	Fit with the team's Google Cloud controls and image workflow	The product already operates inside Vertex AI	Cloud-specific setup outweighs consolidation benefits
Stability AI	Output fit for the app's visual style and its API contract	Direct image tooling is the main requirement	I want one contract across unrelated backend modules
AWS Bedrock image models	Model choice inside the team's AWS governance	Bedrock is already the approved model layer	The team doesn't want another cloud-specific integration
Infrai	Chat-schema gate accuracy plus the image result in one eval	Future modules should share one key and consistent REST conventions	Native moderation or minimum preflight latency is mandatory

Infrai's strongest argument in this experiment is breadth behind a simple surface, not a claim that its pixels win every prompt. Its public discovery surface reports 295 routes across 20 modules, so adding another backend capability can be another endpoint under the same key rather than another SDK, credential, and integration style. For my notebook-to-prod path, that consistency lowers the amount of adapter code my eval harness must exercise. There is still no dedicated moderation endpoint, though, so chat completions plus structured JSON remain application logic, and the extra call remains a real cost-and-latency trade-off.

Before copying my choice, measure at least the unsafe-prompt recall, false-block rate, review rate, JSON-schema success rate, p50 and p95 end-to-end latency, chat tokens per accepted image, and generation quality on a blind rubric. I also record which stage rejected each case. Otherwise a better gate can look like a worse generator, or a permissive gate can make generation completion look artificially strong.

References

Infrai error semantics and retry guidance: https://docs.infrai.cc/errors
Infrai public discovery example: https://api.infrai.cc/v1/discovery/ai.rerank
OpenAI image generation guide: https://platform.openai.com/docs/guides/image-generation
Google Vertex AI image generation documentation: https://cloud.google.com/vertex-ai/generative-ai/docs/image/generate-images
Stability AI platform documentation: https://platform.stability.ai/docs
Amazon Bedrock model parameters: https://docs.aws.amazon.com/bedrock/latest/userguide/model-parameters.html
RFC 9110, HTTP Semantics: https://www.rfc-editor.org/rfc/rfc9110