You set up a custom AI provider. You configure Claude, GPT-4o, or Gemini — all vision-capable models. You send an image to your agent. The agent responds... but never mentions the image. No error. No warning. It just pretends the image doesn't exist.
This is #51869, and it's one of the cleanest examples of silent capability loss I've seen.
The One-Line Root Cause
In OpenClaw's onboard-custom.ts, when you configure a non-Azure custom provider, the generated model config always sets:
input: ["text"] as ["text"], // ← hardcoded, no image option
Every model under your custom provider is now text-only, regardless of what it actually supports.
The ironic part? The Azure path already handles this correctly with vision model detection.
The Silent Failure Chain
- User sends image via any channel
- Gateway receives attachment — everything looks normal
- Gateway checks model config — sees
input: ["text"] - Gateway silently drops the image
- Model responds to text only — valid-looking response
- No error anywhere — logs are clean
The user might think the model is bad at vision. They'll never suspect the onboarding wizard silently crippled their setup.
A Pattern We Keep Seeing
This is the fourth "silent failure" I've written about recently:
- #51857: imageModel doesn't route, Gemini returns 0 tokens
- #51209: Fallback chain doesn't cascade on 401/404
- #51251: Session model override persists across restarts
- #51869: Vision silently disabled for all custom providers
Common thread: the system behaves as if nothing is wrong. HTTP 200. Clean logs. Valid responses. Just... wrong ones.
Why Onboarding Defaults Matter
Most users configure once and never look at the JSON again. When the wizard silently downgrades capabilities:
- Users blame the model
- Users blame the channel
- Users open unrelated bug reports
- Users switch tools
Lessons for Agent Builders
- Audit your onboarding output. Manually review generated config.
- Capability loss should be loud. Log when images are dropped.
- Test the full path. Check the actual API payload, not just upload.
- Defaults should be generous. Declare more capabilities and let the model handle gracefully.
The Fix
If you're using a custom provider, check your openclaw.json and add "image" to the input array:
{
"id": "claude-sonnet-4-6",
"input": ["text", "image"]
}
One line that the wizard should have added for you.
Follow me for more AI agent failure mode analysis: blog | @realwulong
Top comments (0)