Within 48 hours of launching Plant Finder on Google Play, we checked our backend telemetry to see what users were scanning.
We expected to see monsteras, tomatoes, rose bushes, and succulents.
Instead, we saw:
A worn-out leather sneaker.
A user’s sleeping cat.
A slice of pepperoni pizza.
A car dashboard.
Dozens of dark, blurry selfies.
When you hand people an app that uses multimodal AI vision to diagnose leaves, human nature takes over: people immediately try to break it.
While this curiosity is completely natural, it creates two major headaches for any mobile developer:
The Hallucination Problem: The AI tries way too hard to find a plant where none exists.
The Financial Drain: Every photo of a coffee mug or a carpet burns real money in visual AI tokens.
Here is how we tackled adversarial non-plant uploads, protected our bills, and stopped our app from diagnosing shoes.
The Sneaker That Had "Severe Root Rot"
General-purpose vision models are trained to be helpful and answer prompts directly.
Early on, if our mobile app sent a photo of a brown leather shoe with the prompt "Identify this plant and its health issues," the AI would genuinely attempt to fulfill the request.
The result? The model returned:
"Species: Ficus elastica (Rubber Tree). Health: 24%. Severe black spot and dehydration on the lower stem."
This was disastrous. If a user tested the app on an everyday object as a joke, seeing the AI confidently hallucinate an entire botanical diagnosis destroyed their trust in the product. If it can't tell the difference between a shoe and a ficus, why would a farmer trust it to identify tomato early blight?
The Token Cost of Human Curiosity
Running high-resolution multimodal vision calls through frontier models like Gemini 2.5 Flash is remarkably fast and capable, but it is not free.
In our first release, roughly 18% of all camera uploads were non-plant objects.
If you have thousands of active users, paying for 18% wasted API calls every single day quickly eats through your operating margins. We were essentially subsidizing people scanning their living room furniture with expensive AI inferences that delivered zero actual value.
We needed a strict gatekeeper.
Step 1: Giving the Model Permission to Say "No"
The first mistake we made was assuming the model knew it was dealing with a plant.
We updated our backend proxy instructions to introduce a strict two-stage verification process inside the prompt:
Stage 1 (The Guardrail): Examine the image purely to verify whether botanical tissue (leaves, stems, flowers, bark, or roots) is clearly present. If the image is not a plant, abort all diagnostic processing immediately and return a strict flag: NOT_A_PLANT.
Stage 2 (The Diagnosis): Only if Stage 1 passes should the model extract foliar pathology, soil pH symptoms, and care recommendations.
By explicitly instructing the model to reject non-plant images on the very first token, we stopped the hallucination loop in its tracks. A picture of a sneaker now simply returns an honest refusal instead of a fake plant name.
Step 2: Designing a Graceful Rejection UI
Catching non-plant images on the server is only half the battle. You also have to handle the user experience on the phone.
If an app abruptly crashes or throws a cold, generic error like "500 Internal Server Error," users assume the app is broken and leave a 1-star review.
Instead, we built an explicit rejection handler in our Flutter camera preview screen:
Clear Feedback: When the server returns NOT_A_PLANT, the app displays a clear, friendly card: "No botanical tissue detected. Please point your camera directly at leaves, stems, or flowers."
Credit Protection: In our virtual token economy ("Leaves"), a scan costs credits. If the system flags an image as a non-plant, we do not deduct their tokens. This prevents honest users who took a blurry or poorly lit photo from feeling cheated.
Step 3: Preventing Deliberate Token Spam
While protecting honest users is essential, you also have to protect your backend against malicious flooding or automated spam scripts.
To prevent bad actors from repeatedly uploading junk images and burning compute:
Server-Side IP Rate Limiting: Our cloud gateway tracks upload frequencies per IP address. If an IP exceeds 15 rapid-fire uploads in a 60-second window, requests are temporarily throttled at the network level.
Progressive Strike Lockout: If a single account or device repeatedly triggers consecutive NOT_A_PLANT rejections in a short period, the app introduces a cool-down timer. This eliminates automated script abuse while leaving normal gardeners unaffected.
What Other Mobile AI Developers Should Take Away
If you are building an AI camera app for food tracking, skin analysis, receipt scanning, or plant care, remember these three rules:
Users will test your limits: Always design your system assuming people will photograph their pets, faces, and surroundings first before testing your actual feature.
Teach your AI to say "I don't know": Models will confidently fabricate answers if your prompt doesn't give them a clear, graceful exit path.
Fail as early as possible: The faster your pipeline rejects an invalid image, the less money you spend on wasted cloud compute.
Project: Plant Finder
Platform: Android (Google Play)
Attribution: Designed by Code vairagi
Top comments (0)