Image captioning has shifted from specialized CNN-to-RNN pipelines to unified vision-language models. Modern LLMs with vision encoders can ingest an image and emit a natural language description in a single API call. For engineering teams, the operational challenge is no longer model architecture but inference cost, especially when image inputs are encoded as thousands of tokens. Oxlo.ai addresses this with request-based pricing that stays flat regardless of input size, making it a practical backbone for high-volume captioning workloads.
The Architecture of LLM-Based Captioning
Contemporary vision-language models combine a vision encoder with an LLM decoder. The encoder converts image patches into embeddings that the language model processes alongside text tokens. This design lets you pass a base64-encoded image or URL directly into a chat/completions payload and receive a caption, a structured analysis, or a multi-turn conversation about visual content.
On Oxlo.ai, you can access vision-capable models such as Gemma 3 27B and Kimi VL A3B for general vision tasks, and Kimi K2.6 for advanced reasoning over images with its 131K context window. All are exposed through the standard chat/completions endpoint with no cold starts, so you can treat image captioning as just another text generation task with an extra content block.
A Minimal Captioning Request
Because Oxlo.ai is fully OpenAI SDK compatible, you can point your existing client at https://api.oxlo.ai/v1 and send vision requests immediately. The following Python example submits a base64 image to a vision model and requests a plain text caption.
from openai import OpenAI
import base64
def encode_image(path):
with open(path, "rb") as f:
return base64.b64encode(f.read()).decode("utf-8")
client = OpenAI(
base_url="https://api.oxlo.ai/v1",
api_key="YOUR_OXLO_API_KEY"
)
b64_image = encode_image("photo.jpg")
response = client.chat.completions.create(
model="gemma-3-27b", # Vision model available on Oxlo.ai
messages=[
{
"role": "user",
"content": [
{"type": "text", "text": "Write a concise, one-sentence caption for this image."},
{
"type": "image_url",
"image_url": {
"url": f"data:image/jpeg;base64,{b64_image}"
}
}
]
}
],
max_tokens=256
)
print(response.choices[0].message.content)
Why Token-Based Pricing Breaks for Vision
When you submit an image to an LLM, the provider converts it into a sequence of patches or tokens. Higher resolution means more tokens, and on token-based platforms the cost scales linearly with that token count. For product catalogs, medical imaging, or video frame extraction, a single image can consume as many tokens as a multi-page document, which makes per-token billing unpredictable at scale.
Unlike token-based providers such as Together AI, Fireworks AI, OpenRouter, Replicate, and Anyscale, Oxlo.ai uses request-based pricing. One flat cost per API request covers the entire inference call, no matter how many image patches or text tokens are in the prompt. For batch captioning, high-resolution photography, or agentic loops that pass images through multiple reasoning steps, this can be 10-100x cheaper than token-based billing for long-context workloads. See Oxlo.ai pricing for plan details.
Structured and Multi-Turn Workflows
Captions are rarely the end product. Most pipelines need structured metadata, such as a short label, a detailed description, and a list of tags. Oxlo.ai supports JSON mode on compatible models, so you can constrain the output to valid JSON and parse it without regex.
response = client.chat.completions.create(
model="gemma-3-27b",
messages=[
{
"role": "system",
"content": "You are a helpful assistant. Respond only with valid JSON containing keys: caption, tags, alt_text."
},
{
"role": "user",
"content": [
{"type": "text", "text": "Describe this image."},
{"type": "image_url", "image_url": {"url": f"data:image/jpeg;base64,{b64_image}"}}
]
}
],
response_format={"type": "json_object"},
max_tokens=512
)
import json
metadata = json.loads(response.choices[0].message.content)
print(metadata)
Beyond single-shot generation, you can use multi-turn conversations to refine captions iteratively, or function calling to trigger downstream tools when certain objects are detected. These features run on the same chat/completions endpoint with the same flat per-request cost.
Selecting a Vision Model on Oxlo.ai
Oxlo.ai hosts 45+ models across 7 categories. For image captioning, the Vision category includes Gemma 3 27B and Kimi VL A3B. If your task requires deeper reasoning, for example generating alt text that satisfies accessibility guidelines or analyzing complex diagrams, Kimi K2.6 offers advanced reasoning, agentic coding, and vision with a 131K context window. All models are served with no cold starts, so latency is consistent from the first request.
Building Production Pipelines
Image captioning at scale demands more than a model endpoint. It requires predictable costs, fast concurrency, and an API that does not force you to rewrite client libraries. Oxlo.ai provides a fully OpenAI API compatible base URL at https://api.oxlo.ai/v1, streaming responses, and vision inputs on standard chat/completions calls.
You can start with the Free plan, which includes 60 requests per day and access to 16+ free models, to prototype your pipeline. When you move to production, the Pro and Premium plans offer 1,000 and 5,000 requests per day respectively, while Enterprise customers can access dedicated GPUs and unlimited volume. Because every request is billed at a flat rate, your cost per caption stays constant even as image resolution or prompt complexity grows.
Top comments (0)