This is a submission for the Hacktoberfest Open-Source AI Challenge Week 1: Touch Grass
🌱 I Built an AI That Wants You to Stop Using It — Meet TerraLens
We are putting increasingly powerful AI into our pockets.
And then spending increasingly more time staring at those pockets.
Every app seems to want the same thing:
one more click, one more message, one more minute of attention.
So while thinking about the "Touch Grass" theme, I started with a slightly ridiculous question:
What if the best possible outcome for an AI app was that you stopped using it?
Not because it failed.
Because it succeeded.
That question became TerraLens.
TerraLens — AI that sends you outside.
TerraLens is an offline-first, open-weight AI field companion designed to help you notice more of the physical world while spending less time looking at a screen.
It gives you small real-world missions.
You put your phone away.
You walk.
You look.
You listen.
You discover something.
Then, and only then, you come back.
## What I Built
TerraLens turns an ordinary walk into an AI-guided expedition.
But I deliberately didn't build another chatbot.
There is no giant:
"What would you like to ask?"
box at the centre of the experience.
Instead, TerraLens asks:
Ready to go outside?
You choose an expedition style and duration.
For example:
- 🌿 Nature
- 📷 Photographer
- 🧭 Explorer
- 🎲 Surprise Me
- 😌 Chill
Then TerraLens gives you one real-world mission at a time.
Something like:
Find two leaves with visibly different edges.
Don't photograph the first thing you see.
Look around.
Compare.
Come back when you've found them.
And then you press:
GO FIND IT
That's when something unusual happens.
The application basically tells you to stop using the application.
📱 → 🌳 The TerraLens Loop
The whole product revolves around one simple loop:
Get a mission
↓
Put the phone away
↓
Explore the real world
↓
Notice something
↓
Capture an observation
↓
AI understands the context
↓
Get another physical-world action
↓
Put the phone away again
That last part matters.
The goal isn't:
AI → answer → another question → answer → another question
It's:
AI → action → world → discovery
🌱 Touch Grass Mode
I built a dedicated Touch Grass / Pocket Mode.
When you begin a mission, TerraLens intentionally removes most of the interface.
You might see:
YOUR MISSION
Find two leaves with
visibly different edges.
04:58 remaining
PUT YOUR PHONE AWAY 🌱
That's basically it.
No infinite feed.
No notifications asking you to come back.
No AI typing animation trying to hold your attention.
The mission is outside.
TerraLens will still be there when you return.
📷 🎙️ ✍️ Bring Back What You Discover
When you find something interesting, TerraLens lets you capture an observation as:
📷 A photo
See an interesting plant, leaf, flower, texture or object?
Capture it.
🎙️ A sound
Hear a bird call or an interesting environmental sound?
Record it.
✍️ A field note
Notice something that doesn't need a camera?
Write it down.
Those observations become part of your local nature journal and can be used by the Curiosity Engine to determine what you should investigate next.
🧠 The Curiosity Engine
This became one of my favourite parts of the project.
Most AI systems are optimized to answer.
TerraLens is optimized to create curiosity.
Suppose I photograph a flower.
A traditional assistant might respond with several paragraphs:
This appears to be Hibiscus rosa-sinensis. It belongs to the family Malvaceae...
Useful?
Absolutely.
But I'm still staring at my phone.
TerraLens should instead say something closer to:
Possible match: Hibiscus
Before I tell you more, look at the newest leaves near the end of the branch.
Are they the same colour as the older leaves?
Go check. I'll wait.
The difference is small technically but enormous conceptually.
One response makes me read.
The other makes me look.
And TerraLens keeps trying to turn information into physical actions:
LOOK
LISTEN
WALK
COMPARE
NOTICE
WAIT
SEARCH
REFLECT
The AI isn't supposed to become the experience.
It's supposed to trigger the experience.
🏆 Grass Score — Rewarding the Opposite of Screen Time
Then I ran into another interesting problem.
How should TerraLens measure success?
Normal software metrics didn't make sense.
Time in app?
Bad.
More sessions?
Not necessarily.
More clicks?
Definitely not.
So I created:
🌱 GRASS SCORE
A score from 0 to 1000 based on exploration behaviour.
For example:
GRASS SCORE
847 / 1000
🌱 EXPLORER
Outdoor time 42 min
Screen interaction 4 min
Distance 2.3 km
Observations 11
New discoveries 7
Missions completed 3
Screen Avoidance Bonus +120
The important part:
Grass Score is NOT generated by an LLM.
It is deterministic.
Actual expedition events feed a scoring algorithm.
That means TerraLens can't simply ask AI:
"Give this person a score."
and magically produce 847.
The score has a visible breakdown and can be explained.
📊 Nature / Screen Ratio
I also wanted one metric that captured the entire philosophy of TerraLens.
So I added:
Nature / Screen Ratio
Suppose you explored for:
42 minutes
but interacted with TerraLens for only:
4 minutes
Your ratio is roughly:
10.5×
And in TerraLens:
higher is better.
Think about how strange that is for software.
The product is celebrating the fact that you barely used it.
That's exactly the point.
📴 What Happens When the Internet Disappears?
This was one of the biggest reasons I wanted an open and offline-first architecture.
Imagine this:
You're walking along a trail.
You finally find something interesting.
You open your "AI nature assistant."
And it says:
Network error.
That felt completely backwards.
Nature doesn't come with guaranteed Wi-Fi.
So TerraLens treats offline operation as a normal state rather than a failure state.
The app can be installed as a Progressive Web App.
The core expedition experience uses local browser storage.
Missions, observations, scores and the journal can continue locally.
When connectivity disappears, the philosophy is:
You're offline. Good. Keep exploring.
And when connectivity returns, queued work can synchronize again.
🤖 Why I Chose Gemma
The main prize category I'm entering TerraLens for is:
🏆 Best Use of Gemma
Gemma isn't here because I needed an AI logo on the architecture diagram.
It fits the fundamental problem.
TerraLens needs an intelligence layer that can be:
- open-weight
- replaceable
- adaptable
- deployable in different environments
- moved closer to the user
- eventually optimized for local inference
The Curiosity Engine therefore has an AI adapter instead of hard-wiring the entire product into one closed provider.
Conceptually:
Observation
↓
AI Adapter
↓
Gemma
↓
Structured understanding
↓
Curiosity Engine
↓
Physical-world action
Gemma helps turn an observation into context that TerraLens can use to produce the next outdoor action.
And because the model layer is abstracted, TerraLens isn't permanently tied to one inference environment.
Gemma can be served through compatible runtimes while the rest of the application continues speaking the same structured language.
🧠 AI That Is Allowed To Say "I Don't Know"
There was another rule I cared about:
Never fake intelligence.
If the model doesn't answer, TerraLens doesn't invent an answer.
If an integration isn't configured, TerraLens doesn't display a fake green checkmark.
If an evaluation hasn't run, it doesn't invent benchmark numbers.
If there isn't enough data for a prediction, it doesn't ask an LLM to manufacture a percentage.
This sounds obvious.
But it becomes incredibly important in AI demos.
TerraLens uses explicit integration states such as:
READY
DEGRADED
NOT RUN YET
UNVERIFIED
NOT CONFIGURED
UNREACHABLE
One of my rules while building the project became:
Configured ≠ working.
A key existing in an environment variable isn't proof that an integration works.
🔬 Fine-Tuning Curiosity with Tinker
Another category TerraLens explores deeply is:
🧪 Best Use of Tinker
There's an interesting AI problem hidden inside TerraLens.
Generic assistants are trained to be helpful by answering.
But TerraLens sometimes needs to be helpful by not answering yet.
Consider:
Generic response
This appears to be a mango tree. Mango trees belong to the family Anacardiaceae...
versus:
TerraLens response
Looks like a mango tree.
Find the newest leaves near the end of a branch.
Are they the same colour as the older ones?
Go look before reading anything else.
That behaviour can be adapted.
So TerraLens includes a dedicated Curiosity Adapter fine-tuning/evaluation workflow.
The training examples map:
Observation
+
Context
+
Generic response
↓
Desired outdoor response
The evaluation isn't simply:
"Does the model sound good?"
It looks at things like:
- Outdoor Action Rate
- Screen Dependency Rate
- response length
- safety compliance
- mission relevance
- action diversity
The question becomes:
Can we measurably make an AI better at getting humans away from the AI?
I love the contradiction in that experiment.
🎙️ ElevenLabs — Because Voice Means Less Screen
Another natural fit was ElevenLabs.
If TerraLens wants you to stop looking at your screen, making you read every mission isn't ideal either.
So Pocket Mode supports voice narration.
Imagine walking with your phone in your pocket and hearing:
You've been exploring for eight minutes.
Stop here.
Listen carefully for ten seconds.
Can you hear two different natural sounds?
You don't need to unlock the phone.
You don't need to scroll.
You don't even need to look down.
Voice isn't being added because voice AI is cool.
It's being added because audio reduces the amount of visual attention TerraLens needs from you.
If the external voice service isn't available, TerraLens can fall back to browser speech capabilities.
🧭 Mastra — The Field Agent
Individual observations aren't very interesting if the system forgets everything five seconds later.
TerraLens therefore includes a higher-level Field Agent architecture using Mastra.
Its responsibilities are separated conceptually into areas such as:
FIELD AGENT
│
┌─────────────┼─────────────┐
│ │ │
▼ ▼ ▼
Observation Mission Safety
logic logic logic
│ │ │
└─────────────┼─────────────┘
│
▼
Curiosity
│
▼
Summary
This means an expedition can eventually be understood as a journey rather than a collection of unrelated AI calls.
⏳ Temporal — Because Outdoor Networks Fail
Imagine:
Start expedition
↓
Complete mission
↓
Record observation
↓
NETWORK DISAPPEARS
↓
Walk for 20 minutes
↓
Create more observations locally
↓
NETWORK RETURNS
↓
Synchronize
↓
Continue workflow
↓
Generate expedition recap
That's a very different environment from a normal web request.
So TerraLens uses a durable workflow architecture with Temporal for cloud-side work that should survive failures and process restarts.
The important idea is:
losing connectivity should pause cloud work, not destroy the expedition.
🔮 TabPFN — Predicting When Exploration Might Be Interesting
I also wanted TerraLens to eventually learn from structured outdoor observations.
That's where TabPFN fits.
Instead of asking an LLM:
"What are the chances I'll see birds this evening?"
and accepting whatever percentage it invents, structured information can be used.
For example:
Time
Temperature
Humidity
Season
Recent observations
Environment
Previous activity
↓
EXPLORATION FORECAST
Bird activity
HIGH
Insect activity
MEDIUM
Flower observations
HIGH
And if there isn't enough information?
TerraLens should simply say:
Not enough data yet.
"No prediction" is better than a fake prediction.
🗃️ MongoDB Atlas — Cloud Memory Without Making Cloud Mandatory
TerraLens stores the core expedition experience locally first.
That's deliberate.
But synchronized journals and cross-device history are useful too.
MongoDB Atlas provides the optional server-side observation layer.
The distinction matters:
DEVICE
Your current expedition
Your observations
Your score
Your journal
↓ optional sync
CLOUD
Backup / synchronized journal
Cross-device information
Agent context
Cloud storage enhances the experience.
It doesn't own the experience.
🐯 Tiger Data + pgvector — Nature Knowledge
Personal observations and nature knowledge aren't the same thing.
So TerraLens keeps those concerns separate.
Tiger Data / PostgreSQL + pgvector provides a retrieval layer for nature knowledge.
Conceptually:
Question / observation
↓
Embedding
↓
Vector similarity
↓
Relevant field-guide knowledge
↓
Gemma
↓
Grounded response
If vector retrieval isn't available, the architecture can fall back toward PostgreSQL text retrieval and ultimately bundled local knowledge.
Again:
degrade gracefully instead of simply breaking.
🧠 Backboard — Remembering the Explorer
Imagine TerraLens remembers:
Last weekend you spent most of your expedition photographing flowers.
Then next time it can say:
Last time you focused on colour.
Today let's ignore flowers for ten minutes.
Listen instead.
That's where long-term agent memory becomes interesting.
Backboard provides the integration path for richer long-term memory, while local memory remains available when the external service isn't configured.
🔎 SerpApi — Online Context, Not Online Dependency
Sometimes fresh web information is useful.
So TerraLens can optionally enrich observations using SerpApi.
But there's an important distinction in the UI:
LOCAL / MODEL RESULT
Possible identification:
...
ONLINE CONTEXT
Additional current information:
...
SerpApi isn't the identification engine.
It doesn't secretly turn TerraLens back into a cloud-only search wrapper.
The Internet enhances TerraLens.
The Internet doesn't enable TerraLens.
🔭 Sentry — Seeing What the AI Actually Did
AI systems become difficult to debug when all you know is:
"It felt slow."
So TerraLens includes observability designed around actual stages.
For example:
FIELD AGENT TRACE
Mission generation 320 ms
Gemma inference 1.8 sec
Safety check 120 ms
Mission update 190 ms
Voice generation 620 ms
Sentry provides the tracing/error path.
But TerraLens deliberately avoids sending things such as:
- photographs
- private field notes
- precise coordinates
into telemetry by default.
Debugging shouldn't quietly become surveillance.
🔐 Privacy Isn't a Settings Checkbox
TerraLens supports three conceptual privacy modes:
LOCAL ONLY
Keep the experience on the device wherever possible.
HYBRID
Local-first processing with selected synchronization.
CLOUD ENHANCED
Allow configured online capabilities.
Precise location isn't required to go exploring.
Photos are processed with privacy in mind.
External credentials remain server-side.
And cloud media processing isn't silently enabled just because an API key exists.
For a field companion, this matters.
A personal nature journal should feel personal.
🛡️ Safety
There is another problem with letting AI create outdoor missions:
AI can occasionally suggest stupid things.
😅
So generated activities have safety constraints.
TerraLens should never intentionally ask someone to:
- eat an unknown plant
- eat an unknown mushroom
- handle wildlife
- approach dangerous animals
- trespass
- climb unsafe structures
- enter dangerous water
- cross unsafe roads
- rely on AI identification for medical or food-safety decisions
The principle is simple:
Observe. Don't disturb.
Curiosity should make exploration better, not reckless.
🧪 Judge Mode & AI Lab
I also realized something while building this:
A hackathon judge might have only a few minutes.
Making them discover the architecture manually would be terrible UX.
So TerraLens includes a dedicated:
Judge Mode
It provides a short technical tour covering:
- the idea
- expeditions
- offline behaviour
- Gemma
- Grass Score
- Curiosity
- architecture
- integration status
- privacy
- open innovation
There's also an AI Lab for inspecting things like:
- model state
- evaluation
- offline capabilities
- observability
- architecture
I wanted the technical work to be visible instead of hiding everything behind the landing page.
🏗️ How I Built It
The application is built as a mobile-first Next.js + TypeScript Progressive Web App.
At a simplified level:
TERRALENS
│
┌────────▼────────┐
│ EXPEDITION │
└────────┬────────┘
│
┌──────────────┼──────────────┐
▼ ▼ ▼
MISSIONS OBSERVATIONS JOURNAL
│ │
└──────┬───────┘
▼
CURIOSITY ENGINE
│
┌───────┴───────┐
▼ ▼
LOCAL RULES GEMMA
│ │
└───────┬───────┘
▼
PHYSICAL ACTION
│
▼
POCKET MODE
│
▼
GRASS SCORE
When online, additional infrastructure can participate:
DEVICE
│
▼
SYNC QUEUE
│
▼
TEMPORAL
│
▼
MASTRA FIELD AGENT
│
┌──────────────┼───────────────┐
▼ ▼ ▼
MONGODB TIGER DATA TABPFN
│
┌──────────┼──────────┐
▼ ▼ ▼
BACKBOARD SERPAPI ELEVENLABS
And throughout the system:
SENTRY → observability
TINKER → model adaptation/evaluation
GEMMA → open-weight intelligence
The important thing is that the bottom diagram can disappear temporarily...
and the user can still go outside.
## Why Does Open Innovation Matter?
This is the question at the heart of the challenge.
For TerraLens, open innovation matters because of where the application is supposed to work.
Not inside a perfect office network.
Outside.
On a trail.
In a park.
In a garden.
On a village road.
Somewhere your connection may disappear completely.
A cloud-only AI architecture is incredibly powerful until the cloud becomes unreachable.
Open-weight AI gives us another direction.
We can move intelligence closer to the user.
We can change the runtime.
We can swap models.
We can inspect the behaviour.
We can fine-tune the model toward a strange objective like:
Stop answering so much and make the human investigate.
And we can design toward a future where more of that intelligence runs directly on the device.
There is also a privacy benefit.
A photograph doesn't automatically have to become an upload.
A field journal doesn't have to begin life on somebody else's server.
A user's exact location doesn't have to be the admission price for exploring nature.
And if an API disappears?
The entire idea doesn't have to disappear with it.
That's why open AI isn't decoration in TerraLens.
The environment TerraLens is designed for is exactly where open and local AI becomes most interesting.
## Demo
🌱 Live application
https://terralens.soumwadeepguha.com/
For the best experience, open TerraLens on a phone.
Try this:
- Start an expedition.
- Choose a mission.
- Enter Touch Grass Mode.
- Put the phone away.
- Actually complete the mission.
- Come back.
- Capture an observation.
- Continue exploring.
- Finish the expedition.
- Look at your Grass Score.
- Check your Nature / Screen Ratio.
- Browse the journal.
And yes:
actually take it outside.
That's where TerraLens makes sense.
## Code
TerraLens is open source.
💻 Source code
https://github.com/soumwadeep/TerraLens
The repository includes:
- the Next.js application
- PWA/offline infrastructure
- IndexedDB persistence
- expedition and mission engines
- deterministic Grass Score engine
- Curiosity Engine
- Gemma adapter
- Tinker workspace
- Mastra Field Agent
- Temporal worker
- TabPFN prediction service
- MongoDB integration
- Tiger Data retrieval layer
- ElevenLabs voice integration
- Backboard memory
- SerpApi enrichment
- Sentry observability
- unit/integration tests
- Playwright E2E tests
- CI configuration
- deployment configuration
Everything is available under the project's open-source license.
## My Agent Session
AI-assisted development played an important role in building TerraLens. I used Qoder as my coding agent throughout development—not just to generate code, but to help iterate on the architecture, investigate implementation problems, review different approaches, and turn the original idea into a working product.
The development process covered everything from the offline-first PWA architecture and IndexedDB persistence to the Curiosity Engine, Gemma integration, Touch Grass Mode, Grass Score, safety rules, synchronization, agent workflows, sponsor integrations, testing, and deployment.
Rather than starting with a finished specification and asking an agent to generate a landing page, TerraLens evolved through repeated cycles of idea → implementation → testing → review → improvement.
## Prize Categories
Rather than listing technologies just because they're sponsors, these are the categories that correspond to meaningful parts of TerraLens.
🏆 Best Use of Gemma — Primary Category
This is the category TerraLens is most strongly built around.
Gemma provides the open-weight intelligence path behind observation understanding and the Curiosity Engine.
More importantly, Gemma represents the architectural idea behind TerraLens:
AI that can move closer to the user instead of forcing the user closer to the cloud.
🏆 Best Use of Tinker
Tinker is used for the Curiosity Adapter experiment: adapting model behaviour away from long generic answers and toward safe, useful physical-world actions.
The interesting evaluation question is:
Can fine-tuning make an AI measurably better at getting someone to stop interacting with the AI?
🏆 Best Use of TabPFN
TabPFN powers the structured prediction path for exploration forecasting.
Rather than asking an LLM to invent probabilities, TerraLens can use structured historical/environmental information to make actual predictions.
🏆 Best Use of ElevenLabs
ElevenLabs gives Pocket Mode a voice.
This isn't just narration for decoration.
Voice directly supports TerraLens's goal of reducing visual screen interaction during an expedition.
🏆 Best Use of Mastra
Mastra powers the Field Agent architecture that reasons across an expedition rather than treating each observation as an unrelated interaction.
🏆 Best Use of MongoDB Atlas
MongoDB Atlas provides optional synchronized journal and observation storage while the browser remains capable of supporting the core offline experience.
🏆 Best Use of Temporal
Temporal makes the online workflow durable so cloud processing can survive unreliable connectivity and continue when an expedition reconnects.
🏆 Best Use of Tiger Data
Tiger Data/Postgres + pgvector powers the extended nature-knowledge retrieval layer.
🏆 Best Use of Backboard
Backboard provides long-term Field Agent memory so future expeditions can learn from previous exploration behaviour.
🏆 Best Use of SerpApi
SerpApi adds optional fresh online context while remaining deliberately separate from TerraLens's core identification/intelligence path.
🏆 Best Use of Sentry Agent Tracing
Sentry provides observability into AI and workflow execution while TerraLens's telemetry rules keep private observation content out of tracing.
🏆 Best Use of GitHub Copilot
GitHub and AI-assisted development supported the engineering workflow, while CI validates formatting, types, linting, tests and production builds.
🌱 What I Learned
The biggest thing I learned while building TerraLens wasn't about model size.
It wasn't about vector databases.
It wasn't about agents.
It wasn't even about offline AI.
It was this:
Sometimes the most helpful thing an AI can do is stop talking.
If I ask about a leaf, maybe I don't need six paragraphs.
Maybe I need:
Turn it over.
If I hear a bird:
Close your eyes.
If I find a flower:
Find another one that looks completely different.
If I've been walking while staring at my phone:
Put it away.
AI is becoming extremely good at generating more things for us to consume.
More text.
More images.
More video.
More answers.
More screen time.
TerraLens is a small experiment in the opposite direction.
Using intelligence not to create another digital world...
but to make the real one easier to notice.
One Last Thing
The funniest possible success metric for TerraLens would be this:
Someone opens the application.
Gets a mission.
Locks their phone.
Walks away.
And completely forgets TerraLens exists for the next thirty minutes.
For most apps, that would be terrible engagement.
For TerraLens?
That's a perfect session. 🌱
Ask once.
Look closer.
Listen longer.
Walk farther.
Put your phone away.
Go touch grass.
🌱 Try TerraLens:
https://terralens.soumwadeepguha.com/
💻 Source Code:
https://github.com/soumwadeep/TerraLens
TerraLens
AI that sends you outside.
Top comments (1)
Another Op product, in today's world we do need this kind of app very much to get some free time and have a nature walk in greenery.Thank you for building this auspicious web app Soumwadeep.