EcoVision: Reviving an AI-Powered Mobile App for Visually Impaired Accessibility with YOLOv8

This is a submission for the GitHub Finish-Up-A-Thon Challenge

What I Built

EcoVision is an AI-powered mobile application designed to enhance autonomy for individuals with visual impairments. By combining modern computer vision with an accessible user interface, the app performs real-time edge inference directly on the smartphone to identify everyday household objects.
Initially started as a prototype during my university studies, the project combined Flutter, an older AI model, and an Arduino ultrasonic sensor via Bluetooth to calculate distances. This challenge provided the ultimate motivation to revive the repository, dust off the code, and transform an unstable codebase into a highly stable, production-ready application.

Demo

You can explore the entire codebase, development history, and configuration files directly on my GitHub repository here:
GitHub Repository - EcoVision
You can also view the application in action, including screenshots and a video walkthrough demonstrating the real-time object detection and audio feedback features, in the following public folder:
Google Drive Demo Folder

The Comeback Story

The main objective of reviving EcoVision was to bridge the gap between a fragile university prototype and a robust, production-ready assistive technology solution. Here is how the project underwent a massive evolution during this hackathon:

• Upgrading the Core Environment: The legacy prototype suffered from outdated package constraints. I completely overhauled the pubspec.yaml file, upgrading the SDK constraints to modern environment standards (>=3.3.4 <4.0.0).
• Massive Dataset Expansion (COCO): The previous version relied on a limited custom model that only recognized 4 objects (bed, stairs, table, and door) with distance output strictly in centimeters. I migrated the engine to a high-performance YOLOv8 model optimized with the COCO dataset, expanding the app's real-time detection capabilities to 80 different classes of everyday household items using float32 TensorFlow Lite (.tflite).
• Voice-Activated Commands with Spanish Pluralization: I engineered an interactive voice command system. Users can now activate the microphone and say "buscar [object]" (search for [object]) to target specific items. The system automatically reads the labels file dynamically and generates corresponding plural rules for all 80 COCO objects in Spanish (e.g., matching both "buscar botella" and "buscar botellas" to the singular class), a feature completely absent in the prototype.
• Intelligent Onboarding & Interactive Instructions: Developed a dedicated home screen before the camera interface. It triggers a step-by-step audio usage guide exclusively on the first launch so it isn't intrusive. It also features a styled pre-onboarding permissions guide explaining exactly why the app requires Camera, Microphone, Bluetooth, and Location permissions before system dialogs prompt the user.
• Advanced Settings & Categorization: Added a comprehensive Settings Panel where users can manually filter objects. The 80 COCO classes are organized into 8 logical subcategories (e.g., Kitchen, Furniture, Animals) using expandable menus (ExpansionTile) that automatically auto-expand the active filter category upon entering to minimize scrolling.
• Hardware Diagnostics & Bluetooth Sync: Integrated live status indicators to verify if the app is connected via Bluetooth to the Arduino module, complemented by a detailed diagnostic sensor panel that shows real-time status messages ("Con datos del sensor" or "Sin valor del sensor") with dynamic colors to ease hardware troubleshooting.
• Comprehensive Maker Guide & FAQ: Embedded an interactive hardware assembly manual detailing all required electronic components, circuit layouts, and the exact production-ready Arduino source code, alongside a dedicated Frequently Asked Questions (FAQ) section.
• TalkBack Screen Reader Optimization: Most importantly, the entire application interface, buttons, and settings have been rigorously optimized to work flawlessly with Android's TalkBack screen reader, ensuring a true, production-grade barrier-free accessibility experience that the original prototype completely lacked. ## My Experience with GitHub Copilot GitHub Copilot acted as an invaluable pair programmer throughout this intense revival process. Migrating an older Flutter project with broken dependencies can be an architectural nightmare. Copilot helped me rapidly debug dependency conflicts in the pubspec.yaml file, suggesting compatible versions for permission_handler and the core Vision package. Furthermore, it accelerated the integration of the float32 model inference pipeline by automatically generating boilerplate code for handling camera streams and managing asynchronous text-to-speech cues. It allowed me to focus on optimization and user experience rather than getting stuck on environment configuration.