I Built an AI That Wants You to Stop Using It 🌱

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 1: Touch Grass

What I Built

GrassQuest — an AI app that wants you to stop using it.

Most AI apps try to keep you on your screen.

GrassQuest does the opposite.

It is an offline AI-powered nature scavenger hunt. The app gives you a real-world challenge like:

🍃 Find a leaf.

You go outside, find one, take a photo, and GrassQuest uses a local open-weight vision model to check whether you've completed the challenge.

If you succeed:

🎉 Challenge Complete!
+10 XP

📱 Now put your phone away.

The idea is simple: AI should sometimes help you leave the screen, not spend more time on it.

It's designed for anyone who wants a small excuse to step outside — especially people who spend most of their day working or studying in front of a screen.

The game includes XP, levels, streaks, milestones, and different challenges to make going outside feel a little more fun.

Demo

GrassQuest works in one simple loop:

Get a challenge → Go outside → Find it → Take a photo → Local AI verifies it → Earn XP → Put your phone away.

In the demo, I complete a real challenge by finding a leaf and having the local vision model verify it.

The AI runs locally using SmolVLM2 + MLX, so the core image analysis doesn't depend on a cloud vision API. I also tested the local version with the internet disconnected after the model was downloaded.

The best part of the demo is what happens after the AI does its job:

“Challenge Complete 🎉 Now put your phone away.”

I also tested the application with the internet disconnected after the AI model had been downloaded.

Code

💻 GitHub: https://github.com/DiyaVj/grass-quest

The project is intentionally small and easy to run locally. There is no database, no user account, and no cloud AI API involved in the core experience.

How I Built It

GrassQuest uses SmolVLM2-2.2B-Instruct, an open-weight vision-language model, running locally through MLX and mlx-vlm.

The architecture is:

                 GrassQuest
                     │
                     ▼
              Next.js PWA
                     │
              Take a photo
                     │
                     ▼
             Local FastAPI API
                     │
                     ▼
          SmolVLM2-2.2B-Instruct
                + MLX
                     │
              Visual perception
                     │
                     ▼
          Deterministic Python
              validation
                     │
                     ▼
             Challenge result
                     │
                     ▼
             XP / Milestone
                     │
                     ▼
             📱 Put it away

The important part is that the AI is not just an optional feature added on top of the application.

It is the mechanism that makes the scavenger hunt work.

A problem I ran into

My first implementation asked the vision model something like:

"Does this image satisfy the challenge?"

That sounded reasonable.

It wasn't.

The model could see the image correctly but still hallucinate that the required object was present because the challenge itself was influencing its answer.

For example, an image containing a person could be incorrectly judged as containing a rock when the challenge was "Find a rock."

So I changed the architecture.

Instead of asking the model to make the final decision:

VLM → "Yes/No, did they complete the challenge?"

I now use:

VLM → "What target object is visibly present, and how many can you see?"

Then deterministic Python code makes the final decision.

For example:

Challenge:
Find two leaves.

AI perception:
leaf → count: 1

Python validation:
1 >= 2 → false

Result:
Challenge failed

This separation makes the system much more reliable and gives the AI a clearly defined role: visual perception, not business logic.

Why Does Open Innovation Matter?

I could have built this using a closed vision API.

But that would miss an important part of the idea.

GrassQuest is specifically designed around local AI.

Once the model has been downloaded, the image analysis can happen without sending the user's photos to a third-party AI service.

That gives the project several advantages:

• 🔒 Privacy — photos don't need to leave the local environment for AI analysis.
• 📡 Offline capability — the core AI inference doesn't depend on an internet connection.
• 💰 No per-image API cost — there is no cloud inference bill for every challenge.
• 🧩 Model freedom — the open-weight model can be replaced, optimized, or experimented with.
• 🔍 Transparency and control — I can inspect and change how the AI is used rather than treating a closed API as a black box.

For this particular project, open AI also makes the product concept stronger.

The whole point is to disconnect from the screen.

It felt contradictory to build an app encouraging people to go outside while requiring a constant connection to a cloud AI service.

Local inference makes that contradiction much smaller.

The idea behind GrassQuest

The easiest way to describe GrassQuest is:

What if the best AI app was one you were supposed to stop using?

AI is becoming very good at helping us do more things on our screens.

I wanted to build something that used AI for the opposite purpose.

Go outside. Find something real. Take one photo. Let AI verify it. Then put the phone away.

That's GrassQuest.