This is a submission for the Hacktoberfest Open-Source AI Challenge Week 1: Touch Grass
What I Built
Equipment Field Medic ā°ļøšļø is a hands-free Wilderness Gear Diagnosis and Repair Assistant built for backpackers, hikers, cyclists, and mountaineers.
When your gear breaks miles away from civilization, you want to keep your hands on your gear, not scrolling on your phone screen. Equipment Field Medic uses standard camera snapshots to accurately diagnose physical gear failures and walks you through emergency field repairs using Text-to-Speech narration and Hands-Free Voice Commands.
It perfectly embodies the "Touch Grass" theme because it is designed to make the screen the shortest part of the experience. You snap a photo, let the open-weight vision model diagnose the issue, and then the screen dims into a battery-saving OLED black mode. From there, you just listen to the voice guidance and use spoken commands like "Next Step" while you literally get your hands dirty fixing your bike or gear in the great outdoors.
Demo
- Live Application: Equipment Field Medic (Hosted on Vercel)
- Taking it Outside (Real-World Test): To earn the "Touch Grass" bonus points, I took my bicycle outside, manually dropped the chain, and tested the application trail-side! The model correctly diagnosed the chain issue from a quick photo and narrated the exact steps to reset the chain which was perfect, because my hands were covered in chain grease and I couldn't touch my phone anyway.
Code
The full project is open-source and available here:
Jayesh-JainX
/
Equipment-Field-Medic
Equipment Field Medic is a hands-free Wilderness Gear Diagnosis and Repair Assistant built for backpackers, hikers, and mountaineers.
Equipment Field Medic ā°ļøšļø
Equipment Field Medic is a hands-free Wilderness Gear Diagnosis and Repair Assistant built for backpackers, hikers, and mountaineers.
When your gear breaks miles away from civilization, you want to keep your hands on your gear, not scrolling on your phone screen. Equipment Field Medic uses standard camera snapshots to accurately diagnose failures (using open-weight Llama Vision models) and walks you through emergency field repair using Text-to-Speech narration and Hands-Free Voice Commands.
Key Features
- Hands-Free Operation: Say "Next Step", "Previous Step", or "Dim Screen" via Web Speech API built right into the browser.
-
Vision-Assisted Diagnostics: Harnesses Hugging Face models like
meta-llama/Llama-4-Scout-17B-[16E]-Instruct(or intelligent fallbacks) to identify broken components and suggest repairs using ONLY the tools currently packed in your Gear Locker. - Stealth Battery Saver: Dims the screen natively with high-contrast OLED black themes to preserve vital battery life when dealing with lengthy field repairs.
- ā¦
How I Built It
The application is engineered to be lightweight, resilient, and completely hands-free once the repair starts:
-
Open-Source AI Core: The diagnostic engine is powered by the open-weight multimodal model
meta-llama/Llama-4-Scout-17B-16E-Instruct(accessed via the Hugging Face Inference API / Novita). It analyzes the uploaded image of the gear failure and outputs structured, step-by-step emergency repair protocols based only on the tools the user has packed in their virtual Gear Locker. - Hands-Free Interface: The app uses the native browser Web Speech API for voice recognition (listening for commands like "Next Step", "Previous Step", or "Dim Screen") and Text-to-Speech (TTS) to dictate the instructions.
- Database & Persistence: Backed by MongoDB Atlas, which securely stores the user's "Gear Locker" inventory and repair history. It supports a graceful local-memory fallback when the user loses signal on the trail.
- Stealth Battery Saver: An automated high-contrast OLED black theme activates during lengthy repairs to preserve crucial phone battery life in the wilderness.
- Hosting: Deployed on Vercel for fast, low-latency edge delivery.
Why Does Open Innovation Matter?
When you're stranded on a trail with a broken bike or a snapped tent pole, relying on a closed, proprietary AI ecosystem is dangerous. Open-weight models are vital for outdoor utility tools for three major reasons:
- The Pathway to Offline-First: Closed APIs (like OpenAI or Claude) enforce a permanent dependency on the cloud. By architecting this application around open weights like Llama 4, the core logic can eventually be quantized and ported to run 100% locally on a mobile device via WebAssembly/WebGPU. Survival and emergency repair knowledge shouldn't require a 5G connection.
- Data Sovereignty: Real-world geolocation, physical inventory data, and movement habits shouldn't be harvested by closed corporate APIs to train proprietary models. Open-weight inference gives users complete sovereignty over their data.
- No Vendor Extortion: Open models allow the developer community to fine-tune the AI specifically on wilderness survival manuals and specialized cycling mechanics without paying expensive, unpredictable API token fees every time a hiker needs help.
Open innovation ensures that tools designed to help people safely explore the physical world remain accessible, auditable, and free from corporate walled gardens.
Prize Categories
- Best Use of MongoDB Atlas: I utilized MongoDB Atlas as the primary data layer to persist user Gear Locker inventories, offline-cached repair protocols, and diagnostic histories, ensuring critical data availability across sessions.
Top comments (1)
the fact that you manually dropped your bike chain and tested this trail-side with greasy hands is the ultimate "touch grass" validation. most people just demo on localhost.
the hands-free voice interface is brilliant for outdoor use. when your hands are covered in chain grease or holding a tent pole, you can't be tapping a screen. using the web speech api for "next step" commands while the screen dims to oled black to save battery is exactly how you design for the real-world environment.
curious about one edge case though: how does the web speech api handle ambient noise on a windy trail or near a river? do you have any noise-gating or confidence thresholds to prevent false triggers from wind gusts or other hikers talking nearby?
massive respect for the open-weight approach and the local-memory fallback when signal is lost. survival tools shouldn't require 5g. šÆšæ