The question
Can a general-purpose vision model (ChatGPT) reliably tell which way a product photo needs to be rotated to be upright? We built a specialized model for this exact task (BlueVetaUpright) and ran a real comparison to find out.
Setup
- 400 real product photos, split across two independently-built 200-photo batches with zero shared source photos
- Each photo rotated to a random quadrant (90/180/270°); the task is predicting the correction needed
- ChatGPT used through its standard consumer chat interface (not a pinned API snapshot — more on why that matters below)
- Scored for quadrant accuracy only (0/90/180/270°), same task given to both systems
Result
BlueVetaUpright: 91.1%
ChatGPT: 75.9%
+15.2 percentage points, 95% CI [+10.4, +20.0] via a photo-cluster bootstrap (10,000 resamples, clustered by source photo so one confusing photo can't skew the result).
The first 200-photo batch alone was directionally positive but not statistically significant (its CI touched zero). We built a second, independent 200-photo batch specifically to resolve that, and the combined result holds.
A real, disclosed limitation
The two 200-photo batches did not score identically for ChatGPT — 82.1% on the first, 69.9% on the second, concentrated in the harder tilt buckets. BlueVetaUpright stayed consistent across both (88.2% / 94.0%). Our best explanation: a consumer chat UI gives no pinned model guarantee — no controlled snapshot, no guarantee against backend routing changes between sessions. That's a real limitation of comparing against a moving target, disclosed here rather than smoothed over.
Why this is harder than it looks
Rotation detection sounds trivial until you try it on real product photography: labels at odd angles, symmetric packaging, flat-lay shots with no obvious "up." A model trained specifically for this narrow task (ours fuses a quadrant classifier with a separate fine-tilt model and an arbitration layer) consistently beats a general-purpose vision model that was never specifically optimized for it.
Full methodology + paper
Full methodology, bucket breakdowns, and limitations are public: blueveta.com/upright/research. The underlying research is also archived with a DOI on Zenodo: 10.5281/zenodo.22975679.
Try the tool free: blueveta.com/upright
Top comments (0)