Your AI product recommendations and visual search features cost $0.30 per session. At 2 million sessions per month, that's $600K per year in inference spend for features your competitors are starting to run on-device for near zero marginal cost.
The gap between your cost structure and theirs will widen every quarter until you close it.
The Four Decisions That Determine Whether This Works
Recommendation vs search vs visual search. These three AI features have different on-device viability. Personalized recommendations can run on-device with a small embedding model. Visual search requires a larger model and more device compute. Keyword search augmentation is the easiest to migrate. Starting with the highest-cost, most-migratable feature delivers the fastest cost reduction without a multi-quarter project.
Catalog size constraints. On-device recommendation models index against a subset of your catalog, not your full SKU range. The index size you can cache on-device determines how broad the recommendation surface can be. For catalogs above 500K SKUs, hybrid architecture — on-device for frequency, cloud for tail catalog — is the practical answer. Designing for your actual catalog size before the integration sprint starts avoids a mid-project architectural pivot.
Personalization data locality. A recommendation model that learns from the user's in-session behavior can run on-device without transmitting behavioral data. A model that cross-references behavior against the broader user population needs to call a server. The personalization architecture determines the privacy story and the cost structure simultaneously — two outcomes from one decision.
Session-to-purchase attribution. Your analytics need to track whether on-device recommendations convert at the same rate as cloud recommendations. The A/B testing and attribution architecture has to be in place before you migrate, or you won't know if the cost reduction came with a revenue regression.
Most teams spend 4-6 months discovering these decisions by building the wrong version first. A team that has shipped this before compresses that to 1 week.
Why We Can Say That
We built Off Grid because we hit every one of these problems in production. Off Grid is the fastest-growing on-device AI application in the world, with 50,000+ users running it today. It's open source, with 1,650+ stars on GitHub and contributors from across the world. It has been cited in peer-reviewed clinical research on offline mobile edge AI. Every decision named above — model choice, platform, server boundary, compliance posture — we have made before, at scale, for real deployments.
How the Engagement Works
The engagement is four sprints. Each sprint is fixed-price. Each sprint has a named deliverable your team can put on a roadmap.
Discovery (Week 1, $5K): We resolve the four decisions — model, platform, server boundary, compliance posture. Deliverable: a 1-page architecture doc your CTO can take to the board and your Privacy Officer can take to Legal.
Integration (Weeks 2-3, $5K-$10K): We ship the on-device model into your app behind a feature flag. Deliverable: a working build your QA team can test against real workflows.
Optimization (Weeks 4-5, $5K-$10K): We hit the performance and compliance targets from the discovery doc. Deliverable: benchmarks signed off by your team.
Production hardening (Week 6, $5K): Edge cases, OS version coverage, app store and compliance review readiness. Deliverable: shippable build.
4-6 weeks total. $20K-$30K total. Money back if we don't hit the benchmarks. We have not had to refund.
"We're most impressed with Wednesday Solutions' flexibility." — Lucy Lai, Associate Engineering Director, Zalora
Ready to See the Numbers for Your App?
Worth 30 minutes? We'll walk you through what your current inference spend and usage volume mean for the business case, and what a realistic cost reduction target looks like. You'll leave with enough to run a planning meeting next week. No pitch deck. If we're not the right team, we'll tell you who is.
Top comments (0)