We'll build a lightweight Python CLI that recognizes objects, scenes, and text inside any JPEG or PNG by sending it to a vision-capable LLM on Oxlo.ai. It is useful for developers who need quick structured image metadata without training custom CV models or managing inference infrastructure.
What you'll need
- Python 3.10 or newer
pip install openai- An Oxlo.ai API key from https://portal.oxlo.ai
- A sample image file named
test-image.jpg
Step 1: Define the system prompt
The system prompt locks the vision model into returning only valid JSON with fixed fields. This removes guesswork when we parse the output later.
SYSTEM_PROMPT = """You are a precise image recognition engine. Analyze the provided image and return a single JSON object with exactly these keys:
- objects: list of visible objects
- scene: one sentence describing the setting
- text_detected: boolean, true if any text is visible
- safety_notes: list of any safety concerns, or an empty list
Return only the raw JSON object. Do not wrap it in markdown."""
Step 2: Configure the Oxlo.ai client
Oxlo.ai exposes an OpenAI-compatible endpoint, so the official SDK works after a single base_url change. I will use kimi-k2.6 because it supports vision and reasoning across a 131K context window.
from openai import OpenAI
client = OpenAI(base_url="https://api.oxlo.ai/v1", api_key="YOUR_OXLO_API_KEY")
print("Oxlo.ai client ready.")
Step 3: Encode the image
Vision endpoints accept images as base64 data URLs inside the message content array. This helper reads a local file and returns the payload chunk.
import base64
def encode_image(image_path):
with open(image_path, "rb") as f:
return base64.b64encode(f.read()).decode("utf-8")
image_path = "test-image.jpg"
base64_image = encode_image(image_path)
user_message = [
{"type": "text", "text": "Analyze this image and return the structured JSON."},
{"type": "image_url", "image_url": {"url": f"data:image/jpeg;base64,{base64_image}"}}
]
print(f"Encoded {image_path} ({len(base64_image)} characters).")
Step 4: Send the request and parse the response
Now we pass the system prompt and the image payload to the model. Oxlo.ai uses flat per-request pricing, so the cost stays the same even if we later send high-resolution images or multi-frame batches. That makes this approach predictable for production workloads.
from openai import OpenAI
import base64
import json
client = OpenAI(base_url="https://api.oxlo.ai/v1", api_key="YOUR_OXLO_API_KEY")
SYSTEM_PROMPT = """You are a precise image recognition engine. Analyze the provided image and return a single JSON object with exactly these keys:
- objects: list of visible objects
- scene: one sentence describing the setting
- text_detected: boolean, true if any text is visible
- safety_notes: list of any safety concerns, or an empty list
Return only the raw JSON object. Do not wrap it in markdown."""
def encode_image(image_path):
with open(image_path, "rb") as f:
return base64.b64encode(f.read()).decode("utf-8")
image_path = "test-image.jpg"
base64_image = encode_image(image_path)
user_message = [
{"type": "text", "text": "Analyze this image and return the structured JSON."},
{"type": "image_url", "image_url": {"url": f"data:image/jpeg;base64,{base64_image}"}}
]
response = client.chat.completions.create(
model="kimi-k2.6",
messages=[
{"role": "system", "content": SYSTEM_PROMPT},
{"role": "user", "content": user_message},
],
)
raw = response.choices[0].message.content.strip()
if raw.startswith("
```"):
raw = raw.split("```
")[1].replace("json", "").strip()
result = json.loads(raw)
print(json.dumps(result, indent=2))
Step 5: Wrap it in a reusable CLI
The final step adds argument parsing and error handling so the script behaves like a shipped tool. Keeping the Oxlo.ai client at module level means there are no cold starts on popular models when you process multiple images in a loop.
from openai import OpenAI
import base64
import json
import sys
client = OpenAI(base_url="https://api.oxlo.ai/v1", api_key="YOUR_OXLO_API_KEY")
SYSTEM_PROMPT = """You are a precise image recognition engine. Analyze the provided image and return a single JSON object with exactly these keys:
- objects: list of visible objects
- scene: one sentence describing the setting
- text_detected: boolean, true if any text is visible
- safety_notes: list of any safety concerns, or an empty list
Return only the raw JSON object. Do not wrap it in markdown."""
def encode_image(image_path):
with open(image_path, "rb") as f:
return base64.b64encode(f.read()).decode("utf-8")
def recognize(image_path):
base64_image = encode_image(image_path)
user_message = [
{"type": "text", "text": "Analyze this image and return the structured JSON."},
{"type": "image_url", "image_url": {"url": f"data:image/jpeg;base64,{base64_image}"}}
]
response = client.chat.completions.create(
model="kimi-k2.6",
messages=[
{"role": "system", "content": SYSTEM_PROMPT},
{"role": "user", "content": user_message},
],
)
raw = response.choices[0].message.content.strip()
if raw.startswith("
```"):
raw = raw.split("```
")[1].replace("json", "").strip()
return json.loads(raw)
if __name__ == "__main__":
if len(sys.argv) != 2:
print("Usage: python recognize.py ")
sys.exit(1)
output = recognize(sys.argv[1])
print(json.dumps(output, indent=2))
Run it
Save the final script as recognize.py, replace YOUR_OXLO_API_KEY with your key from https://portal.oxlo.ai, and run it against any image.
$ python recognize.py test-image.jpg
{
"objects": ["red bicycle", "wooden fence", "stop sign"],
"scene": "A suburban street corner on a cloudy day.",
"text_detected": true,
"safety_notes": ["cyclist near intersection"]
}
Next steps
Try swapping the model to gemma-3-27b if you need a smaller vision model for faster responses, or batch multiple images into a single conversation turn to build a photo deduplication pipeline. If you move to production, compare Oxlo.ai request-based pricing against token-based providers for long-context or high-volume image workloads on the pricing page.
Top comments (0)