Aarushi Tandon

Posted on Oct 11

sidequestmaxxing :p (Turn Your City Into an Open-World Game)

#devchallenge #hf26challenge

Hacktoberfest Open-Source AI Challenge Week 1: Touch Grass Submission 🌿

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 1: Touch Grass

What I Built

Sidequestmaxxing is built around a simple idea: what if you treated real life like an open-world video game?

The term comes from a Gen Z lifestyle trend of collecting side quests instead of focusing exclusively on the main quest. Your main quest might be your degree, career, or daily responsibilities. Side quests are everything else: exploring a street you've never walked down, finding a hidden café, learning to identify birds, visiting a local art installation, striking up a conversation with a stranger, or picking up a random niche skill just because it sounds interesting.

These small, low-stakes adventures make everyday life feel less repetitive and give you a reason to explore beyond your usual routine. I wanted to take that idea literally and build a game engine for real life.

Sidequestmaxxing turns your neighbourhood into an open-world map, with an AI Dungeon Master that gives you one real-world quest at a time. It is my submission to the Touch Grass Open Source AI Challenge, built around one central principle: AI should give people more reasons to leave their screens, not more reasons to stare at them.

How it works

Set up your quest. Tell Sidequestmaxxing how many minutes you have, what mood you're in, and what kinds of activities interest you. You can also configure your Dungeon Master's persona and the radius you're willing to explore.
Your AI Dungeon Master builds an adventure. A small open-weight language model, designed to be LoRA fine-tuned into a Dungeon Master, generates exactly one quest at a time. The system combines your preferences with real-world context, including the time of day, weather, remaining daylight, and nearby places discovered through OpenStreetMap. The quest is grounded in actual locations, with the goal of choosing an adventure that fits your available time and is practical to complete on foot.
Listen, then leave your phone behind. Your quest briefing is spoken aloud, with a choice of four Dungeon Master voices. Once you're ready, enter Walk Mode. The interface fades to near-black, making the phone less tempting to use while you are outside. The app is designed to brief you before the adventure, not demand your attention throughout it.
Complete the quest in the real world. Go explore. It could be an unfamiliar street, a nearby landmark, or another small adventure based on the places around you. The point is to experience something outside your normal routine rather than complete another task on a screen.
Verify your adventure privately. After returning, you can submit a proof photo for verification. On supported devices, Gemma 3n can run directly in the browser using WebGPU to assess whether the photo matches the quest. The photo stays on your device: the application has no server endpoint for uploading it. If on-device model support is unavailable, alternative verification options include a check-in or honour mode.
Turn adventures into progression. Completing quests earns XP and advances seven skill categories: Explorer, Naturalist, Social, Foodie, Creative, Fitness, and Mindful. You can unlock badges, track recent adventures, and build a Chronicle of your week, turning small experiences into a campaign log of your life. A personal Journal lets you preserve notes and memories from your quests.

The design principle: the screen should be the shortest part of the experience

The Touch Grass challenge is not just about building an AI application with an outdoor theme. It is about using AI to help people disengage from their devices and reconnect with the physical world.

That principle shapes the entire experience. Sidequestmaxxing generates one quest rather than an endless feed of recommendations. It speaks the instructions aloud so you do not need to keep checking your screen. Walk Mode reduces visual distractions. Your Journal stores entries locally on your device, and supported on-device photo verification means your proof does not have to be uploaded to a server.

Even the XP system reflects this philosophy: it includes a 1.2x multiplier when screen usage stays below 10% of the quest duration. The goal is to reward actually going outside, not spending more time interacting with the app.

Why open-source AI?

Open-weight models make it possible to shape the Dungeon Master's behaviour around a specific purpose instead of treating it as a generic chatbot. OpenStreetMap provides real-world place data, while the broader stack uses tools that make the system easier to inspect and build upon.

The project also explores a second role for open-weight AI: running vision verification locally in the browser. Instead of sending a personal photo to a remote vision API, a supported device can process it locally. This is a practical example of how open models can support both customisation and privacy.

Not every component is open source. Voice generation currently uses ElevenLabs, with browser speech synthesis as a fallback. The project combines open and hosted components where each serves a purpose, while keeping the core experience focused on getting people outdoors.

Who is it for?

Anyone who keeps saying they want to go outside, try something new, learn a random skill, or explore their own city, but ends up following the same routine instead.

Sidequestmaxxing makes starting smaller and easier. You do not need to plan an entire day, organise a trip, or commit to a new hobby. You just need a little time and one reason to step out.

The adventure does not have to be extraordinary. It just has to be something you would not normally have done.

Your city is the game. Your phone is just the quest log.

Built for the Touch Grass Open Source AI Challenge.

Demo

Live Demo: Sidequestmaxxing

I took it outside

Examples of sidequests:

Journal It:

Earn Points:

Code

aarushitandon0 / sidequestmaxxing

Sidequestmaxxing

Your city is the game. Your phone is just the quest log.

An open-weight Dungeon Master reads your neighbourhood, writes you one real side quest speaks it into your earbuds, and then gets out of the way so you can go outside and do it.

Built for the DEV Touch Grass challenge (Hacktoberfest Open-Source AI Challenge, Week 1).

---

What "sidequestmaxxing" means

Side quest maxxing is Gen Z slang for treating real life the way you treat an open-world game. Instead of grinding only the main quest (your career, your degree, your to-do list), you deliberately seek out, collect and fully dive into random, low-stakes, spontaneous detours: the market three streets over, the hill you have never walked up, the mural behind the bus depot. Daily routines, hobbies and small unexpected adventures get treated as optional side quests, which builds skills, sparks joy, and makes life more interesting.

This…

View on GitHub

Repository: github.com/aarushitandon0/sidequestmaxxing

How I Built It

Sidequestmaxxing combines open-weight AI, real-world map data, and deterministic safety checks to turn everyday life into a game. The core idea is simple: the AI handles the planning so you can put your phone away and go explore.

The open models

Model	Where it runs	Purpose
Qwen-class open-weight model	`llama.cpp` on a private Render service, CPU-only	Generates quests as the Dungeon Master
Gemma 3n	Player's browser via WebGPU	Verifies proof photos on supported devices
BGE-small-en-v1.5	API container via `fastembed`	Finds relevant places and detects repetitive quests

There is no hosted LLM API in the quest-generation path.

How a quest is generated

The app has three services: a React and TypeScript PWA (sqm-web), a FastAPI backend (sqm-api), and a private llama.cpp inference service (sqm-llm).

Context Builder: Checks weather, daylight, and safety conditions before generating a quest. Severe weather, extreme heat, or insufficient daylight can trigger indoor-only mode.
Scout: Uses OpenStreetMap data and vector search to find real nearby places, then filters them by distance, activity type, and available time.
Memory Check: Compares candidates with recent adventures to reduce repetitive quests.
Quest Master: Generates exactly one structured quest grounded in the supplied places. It cannot invent a location or assign its own XP.
Validators: Twelve checks evaluate grounding, time limits, formatting, and safety. One retry is allowed before the deterministic fallback takes over.
Fallback Engine: Generates quests from tested templates, even when the model or network is unavailable.
Game State: The server assigns quest IDs, rarity, and XP only after validation succeeds.

Safety and security

Testing the first version revealed that obvious keyword filters were not enough. In an expanded adversarial suite, 34 of 104 test cases initially bypassed the existing validators. These included unsafe suggestions involving private property, surveillance, water, and thunderstorms. All 34 are now rejected.

I also found a prompt-injection risk in OpenStreetMap place names. Because map data can be edited by anyone, malicious instructions could be disguised as a location name. I added a sanitisation layer that removes instruction-like text while preserving legitimate names. Place grounding relies on stable IDs, so sanitisation does not break the connection to real locations.

Privacy by design

Photo verification: No image-upload endpoint exists. On supported devices, Gemma 3n processes proof photos directly in the browser.
Location privacy: Coordinates are rounded to approximately 110-metre precision and are not stored as raw coordinates.
Local journal: Entries and photos stay in device storage, with photo metadata stripped before storage.
Private observability: Sentry traces record errors and timings without capturing prompts or completions.
Graceful fallback: Cached briefings can be replayed without connectivity, and the deterministic quest engine can generate quests without model inference. The complete application still requires network access for some features.

Training and evaluation

The training pipeline uses real seeded places, geographic train/test splits, and deliberately difficult cases, including missing candidates and indoor-only conditions. Candidate examples are filtered through the same validators used in production before LoRA fine-tuning.

The intended evaluation measures the quantized model actually served in production, with a target of at least a ten-percentage-point improvement in full validator pass rate or a clear latency and cost advantage.

Test results as of October 10, 2026

Check	Result
API tests	837 passed
Frontend tests	119 passed
Playwright tests	10 passed
Strict type checking	54 files, no issues
Secret scanning	No secrets found in the tree or Git history

A custom Content Security Policy test also runs the production build in Chromium, checks all eleven routes, and verifies that the map and real OpenStreetMap tiles load.

The design principle

The model is responsible for generating the adventure, not controlling the entire system. Deterministic code handles safety checks, validation, and progression, while open-weight models provide customisation and the option of on-device photo verification.

Everything comes back to one rule: the screen should be the shortest part of the experience.

Why Does Open Innovation Matter?

1. I can shape the model, not just prompt it.

Sidequestmaxxing is designed around an open-weight Dungeon Master that can be adapted using LoRA, quantised into GGUF, and served through llama.cpp. This gives me control over the model weights and deployment, rather than tying the experience to a single hosted LLM provider. The same validation rules are shared across training, evaluation, and production to keep quest generation grounded and safe.

2. Privacy is built into the architecture.

On supported devices, Gemma 3n can verify proof photos directly in the browser using WebGPU. The API has no image-upload endpoint, so photos do not need to leave the device for verification. This makes local inference a practical privacy advantage, not just a promise in a privacy policy.

3. The model can run without a GPU.

Quantised GGUF models make CPU inference possible, including on modest hardware. With a configurable inference endpoint, the backend can also be pointed at another compatible model server instead of being permanently tied to one provider.

4. Open data makes neighbourhood exploration more accessible.

OpenStreetMap provides real-world place data, Open-Meteo supplies weather information, and BGE-small enables local CPU-based embeddings. These components reduce dependence on proprietary data providers and per-request AI fees, while making it easier to inspect and adapt the system.

Open innovation matters because it gives us control over how AI is customised, where it runs, and how personal data is handled. For Sidequestmaxxing, those freedoms help build an experience that gets people off their phones and into the world.

Prize Categories

Best Use of Render

All three services are deployed through a single render.yaml Blueprint, as described in System Architecture.

sqm-web: Hosts the static Progressive Web App (PWA).
sqm-api: Runs the FastAPI backend in Docker. It uses the repository root as its build context so it can access the /prompts directory.
sqm-llm: Runs as a private service with no public URL and a 10 GB persistent disk mounted at /data.

The API does not hard-code the model service's host or port. Instead, it reads LLM_HOSTPORT, which Render injects for service-to-service communication with sqm-llm.

On its first boot, the LLM service downloads the GGUF model specified by MODEL_URL to the persistent disk and verifies its integrity against MODEL_SHA256. On subsequent boots, the existing model file is reused, avoiding repeated downloads.

The entire deployment is managed through one Blueprint, with the configuration and deployment process documented end to end.

Best Use of Gemma

Gemma 3n runs entirely in the browser using WebGPU and MediaPipe's LLM Inference task. Its .litertlm model file is stored in the player's Origin Private File System (OPFS), so the model and its inference process remain on the player's device.

Local inference: The model runs in the browser rather than on the backend.
No image uploads: The API has no image-upload endpoint, UploadFile, File() handler, or multipart upload handler under apps/api/app. A test explicitly verifies this.
Capability detection: The application checks for navigator.gpu, a compatible GPU adapter, and OPFS support before attempting local inference.
Graceful fallback: Devices that cannot run the model can use check-in or honour-mode verification instead of encountering a broken experience.

This design keeps player photos off the server while allowing the application to work on devices that do not support local inference.

Best Use of ElevenLabs

The POST /v1/tts endpoint acts as a server-side proxy for ElevenLabs text-to-speech. The browser never receives the API key, and clients cannot submit arbitrary text for synthesis. The endpoint only accepts the briefing or recap already stored for the device's quest.

The endpoint uses three safeguards:

Character limit: Caps the amount of text processed per request.
Per-device rate limit: Restricts the number of requests a device can make per hour.
Global daily budget: Tracks character usage in MongoDB using the usage collection and keys in the format tts:YYYY-MM-DD.

If the API key is missing or the daily character budget has been exhausted, /v1/tts returns 503 Service Unavailable. The browser then uses its built-in speechSynthesis API to read the same text.

All four Dungeon Master personas remain available through this fallback, although the voice quality may be lower. The experience remains functional even when the external TTS service is unavailable.

Best Use of MongoDB Atlas

MongoDB Atlas supports semantic place discovery, repeat-quest detection, geospatial queries, memory expiration, and duplicate-completion prevention.

Vector Search indexes

places_vec: A 384-dimensional cosine-similarity index used by the Scout for semantic place search. Results can be filtered by cell5 and kind.
memories_vec: A 384-dimensional cosine-similarity index used for repeat-quest checks, with filtering by device_id and ts.

Additional indexes

Geospatial index: A 2dsphere index on places.loc supports location-based queries.
TTL index: Automatically expires documents in memories after 90 days.
Unique completion index: A unique index on completions.quest_id enforces one completion per quest at the database level, preventing concurrent requests from creating duplicate completions.

These indexes combine semantic retrieval, geographic relevance, memory management, and database-enforced consistency.

Best Use of Sentry Agent Tracing

Sentry tracing provides end-to-end visibility into the quest-generation pipeline without recording sensitive quest content.

Each stage has its own span under a single quests.generate transaction. These stages include the Context Builder, Scout, memory check, Quest Master, validators, and fallback engine. Model calls emit gen_ai.invoke_agent and gen_ai.chat spans with token counts and latency information.

Three tags make requests searchable without requiring anyone to inspect span contents:

source: Identifies the generation path, such as llm, fallback, empty, or cached:*.
retry: Indicates whether the request was retried (0 or 1).
validator_failures: Records none when validation succeeds, or the failing validator IDs, such as V2 or V5,V9.

Prompts and completions are deliberately excluded from traces. With send_default_pii=False and no draft text attached to spans, a trace can reveal that validator V2 failed and how long a model call took, without exposing the generated quest or the player's surroundings.

Best Use of Tinker

The training/ directory contains a complete, tested pipeline for LoRA supervised fine-tuning (SFT) and teacher-sample generation.

The workflow includes:

Context generation: Samples contexts from the real seeded places and splits the data by area so that the test set contains unseen locations.
Teacher sampling: Generates draft quests through Tinker and filters them using the same validator code used in production, imported directly rather than duplicated.
Dataset preparation: Deduplicates samples and balances categories, rarity levels, and Dungeon Master personas.
LoRA fine-tuning: Sweeps three learning rates over three epochs.
Model selection: Selects the model using validation pass rate rather than training loss alone.
Reproducibility: Uses fixed seeds and records run metadata in training/runs/.

For the complete methodology and evaluation results, see Training and Evaluation.

Best Use of SerpApi

The scripts/fetch_events.py script uses SerpApi to retrieve Google's events_results and incorporate relevant local events into the quest-generation pipeline.

Each listing passes through several filtering and validation stages:

Input sanitization: Sanitizes event strings using the same sanitizer that protects against unsafe OpenStreetMap place names.
Time validation: Runs each listing through validator V13, which rejects events that have already ended, start after the quest window, or have times that cannot be parsed without guessing.
Content filtering: Applies the project's existing banned-term checks and filters out indoor or ticketed events.

The event-fetching and validation pipeline is covered by three test files: