Hey devs 👋
We are building Fabio AI ChatBot, a WordPress plugin that adds an AI chatbot trained on a site's own content (posts, products, forum threads) using the owner's own API key (OpenAI, Gemini, Mistral, or Claude).
Something we've been thinking about: AI features are cheap to ship, but expensive to run at scale. Every full re-scan re-processes pages, products, forum posts — that's server load and tokens, multiplied across every site running the plugin.
So instead of shipping more AI features this cycle, we focused on cutting waste in the pipeline. First step: incremental scanning.
Before:
- Manual or full daily scan
- Every scan re-processes everything, even unchanged content
- Wasteful past a certain scale (1000+ products, 500+ articles)
After:
- Plugin detects when a specific post, page, or product is updated
- Only that item gets re-scanned and re-embedded
- No full re-scan needed unless explicitly triggered
Net effect: fewer redundant server cycles, fewer redundant tokens, more accurate answers — no more waiting on tonight's scan.
It's a small change, but it reflects a bigger question: we're still early in the AI product cycle, and most tooling optimizes for capability, not efficiency. That's fine at small scale. It gets costly — financially and environmentally — as adoption grows.
Are you actively designing for resource/token efficiency now, or pushing that to "optimize later"? Premature optimization at this stage, or worth baking in from day one?
Curious where the community lands on this.
Top comments (0)