If you are building computer vision for physical retail, there is one architectural decision that determines whether your system reaches five hundred stores or stalls at fifty. Everything else — model choice, framework, cloud provider — matters far less than this.
The decision is where inference runs.
The intuitive architecture and why it breaks
The obvious design: cameras stream to the cloud, GPU instances run detection, results land in a database, dashboards read from it. Every component is managed, nothing runs on-premises, and the whole thing looks like a normal cloud application.
Then you do the arithmetic.
A single 1080p stream at a usable frame rate for detection sits in the low single-digit megabits per second. Call it 2 Mbps to be generous about encoding. A store with 20 cameras is pushing 40 Mbps continuously. Two hundred stores is 8 Gbps of sustained ingress, twenty-four hours a day.
That is before inference. You are now running detection on 4,000 concurrent streams. Even with efficient batching and modest models, that is a substantial standing GPU fleet that cannot scale to zero, because the cameras never stop.
The costs scale with cameras × stores, and they scale continuously rather than with usage. Meanwhile the value — the events you actually care about — scales with something much smaller: how often a shelf goes empty, how often a queue forms, how often a transaction looks wrong. You are paying to transport and process a torrent of pixels in order to extract a trickle of events.
The architecture that works
Push inference into the store. Send only structured events upstream.
[cameras] → [edge box: decode + detect + track] → [event buffer] → [cloud: event store, apps]
What leaves the building is not video. It is:
{
"store_id": "GB-0214",
"camera_id": "aisle-07-bay-3",
"event": "shelf_gap_detected",
"planogram_slot": "PG-88213-04",
"confidence": 0.91,
"observed_at": "2026-08-08T09:14:22Z",
"model_version": "gapdet-4.2.1",
"schema_version": 3
}
A few hundred bytes, emitted when something happens rather than continuously. Bandwidth becomes negligible. Cloud cost scales with events rather than pixels. The system keeps working through a connectivity outage because the edge box buffers locally. And most of your privacy exposure evaporates, because frames never leave the premises — which is a much better position to be in than having to explain your retention policy for footage of customers.
Two fields in that payload are doing quiet but essential work. model_version lets you attribute a spike in false positives to a specific rollout. schema_version stops a model update from silently breaking every downstream consumer — and it will, the first time you add a field and someone's parser is stricter than you expected.
The two-tier pattern
Here is the part that changed most recently.
The traditional approach needed a bespoke trained model per task: one for gap detection, one for queue counting, one per product category. Each carried its own labelling programme and its own retraining burden.
Vision-language models remove the training requirement. You can ask a VLM, in plain language, whether a shelf image shows a gap, or whether a promotional display matches a reference photo. No task-specific training at all — the work moves from collecting labelled data to writing precise evaluation sets.
But a VLM is far too slow and too expensive to run on every frame. So you run two tiers:
- Tier 1 — a small, fast, cheap detector running continuously on the edge box. Its job is not to be right about everything. Its job is to be cheap and to have high recall on "something here is worth a closer look."
- Tier 2 — a VLM invoked only on tier-1 triggers, ambiguous cases, and novel situations. It can run on the edge if the hardware allows, or in the cloud on the small fraction of frames that reach it.
The economic consequence is what makes this worth the complexity. Adding a new detection capability — "check whether the end-cap display matches the planogram photo" — used to be a quarter of work because it needed a labelling programme. With tier 2 it is a prompt, an eval set, and a few days. That difference decides whether an experiment gets funded.
What edge deployment actually costs you
I would be misrepresenting this if I only described the upside.
Running inference in stores means operating a fleet of physical devices across a retail estate. Provisioning them. Monitoring them. Pushing model and firmware updates without bricking any. Detecting when one has silently stopped. Replacing them when they fail, and — this genuinely happens — when a store manager unplugs one to charge something.
That is an ongoing operational function, not an installation project. It is the most consistently underestimated line item in these programmes, and it is the reason "just use the cloud" keeps sounding attractive right up until you model the bill.
The honest tradeoff: cloud inference has lower operational complexity and unacceptable unit economics at scale. Edge inference has workable unit economics and real fleet-management overhead. Above roughly fifty stores, the second is the only one that works.
Model ops is where this gets hard
The failure mode that catches teams is not architectural. It is that a model at 94% accuracy in the pilot store lands in the low seventies by store forty. Different lighting, different fixture heights, a camera knocked six degrees out of alignment during a shelf reset.
Two things follow. First, monitor accuracy per store, not in aggregate — an average across an estate will hide a store that has gone completely blind. Second, build a sampling pipeline that pulls real production frames, routes low-confidence cases to human review, and feeds corrections back into training. That labelling function runs permanently.
If you are evaluating a vendor for this, ask exactly one question: how do you detect that the model has degraded in store 214 specifically, and how does a fix reach that store? The quality of the answer predicts your second year far better than any benchmark number.
The full guide — use-case economics, accuracy targets per deployment, biometrics law, and a 90-day pilot design — is on the TechCirkle blog: Computer Vision in Retail: What Actually Ships in 2026. We cover the broader engineering side in computer vision development for business.
Frequently Asked Questions
Can I run retail computer vision entirely in the cloud?
For a handful of stores, yes. At scale the bandwidth and standing GPU costs grow with cameras multiplied by stores and continuously rather than with usage, which means you pay to transport a torrent of pixels to extract a trickle of events. Above roughly fifty stores the economics stop working.
What hardware do edge inference boxes need?
Enough GPU or NPU capacity to decode and run detection across the store's concurrent streams, plus local storage for event buffering during connectivity loss. Modern small-form-factor inference hardware handles typical store camera counts at a cost comparable to a point-of-sale terminal.
How do vision-language models fit into a real-time pipeline?
Not on every frame — they are too slow and too expensive. Use them as a second tier, invoked only when a cheap always-on detector flags something, or when a case is ambiguous or novel. The value is that adding a new capability needs a prompt and an eval set rather than a labelling programme.
Why version the event schema?
Because model updates change output, and a downstream consumer with a stricter parser will break silently. An explicit schema_version lets consumers handle multiple versions during a staged rollout instead of failing when the first updated store starts emitting.
How do you detect model drift across many stores?
Monitor accuracy per store rather than in aggregate — an estate-wide average will hide a single store that has effectively gone blind. Combine that with confidence-distribution monitoring, which shifts before accuracy visibly degrades and gives you earlier warning.
Does edge inference help with privacy compliance?
Substantially. If frames are processed locally and only non-identifying structured events leave the building, most data-protection exposure around retaining and transmitting customer footage simply does not arise. It is far easier to design that way than to retrofit it.
Top comments (0)