AllergyShield (Edge Dietary Cross-Examiner)

This is a submission for the Hacktoberfest Weekend Challenge: Build for a Friend

What I Built
I built AllergyShield, an on-device OCR and ingredient verification tool for my roommate, who manages celiac disease along with severe sensitivities to common emulsifiers and hidden soy derivatives.

Grocery stores and international markets are notorious cellular dead zones. When shopping, closed cloud-based vision apps fail due to dropped connectivity, leaving my roommate manually inspecting obscure chemical names and manufacturing disclaimers on physical packaging. AllergyShield processes ingredient labels locally on an edge device or laptop without an active internet connection, flagging dangerous additives and synonymized allergens in seconds.

Demo
Live Demo / Screen Recording Link

Code

AWT-SRIRAM / SRM_AUTOSUB_BOT

🎬 SRM AUTOSUB BOT

An asynchronous, GPU-accelerated Telegram Bot engineered using Pyrogram and faster-whisper (distil-large-v3). Automatically extracts English audio streams, transcribes multi-channel media, and generates mobile-optimized, styled Advanced SubStation Alpha (.ass) English subtitles.

---

🔗 Quick Links

---

🚀 Key Features

⚡ High-Performance ASR Engine

• distil-large-v3 Architecture: Utilises quantized distil-large-v3 running on CUDA float16 hardware acceleration, achieving 5x–12x real-time transcription performance.
• Word-Level Timestamping: Generates exact word‑level timestamp alignments and applies centisecond rounding (round()‑based) to eliminate IEEE 754 drift on long files.
• FFmpeg Audio Extraction: Automatically probes multi‑track containers (.mkv, .mp4, .avi), prefers explicit English streams, and converts raw Audio to 16kHz mono PCM WAV with a 3x volume boost to capture quiet dialogue.

🎨 Styled .ASS Output (Custom Look)

• Embedded Styling: Outputs Advanced SubStation Alpha (.ass) files with a pre‑defined style – Trebuchet MS Italic…

View on GitHub

How I Built It
AllergyShield combines localized vision processing with a deterministic matching engine:

Visual Label Processing: Built on Moondream2 (a lightweight 1.86B parameter vision-language model) running via llama.cpp to transcribe ingredient lists from camera snapshots.

Allergen Verification Engine: Extracted text is tokenized and cross-examined using a local vector index in ChromaDB with BGE-small-en embeddings.

Rule-Based Guardrails: Rather than relying solely on probabilistic model completions for health-critical checks, the system pairs semantic similarity with hard-coded regex dictionaries of known allergen derivatives (e.g., maltodextrin, textured vegetable protein).

Why Does Open Innovation Matter?
Air-Gapped Reliability: In-store shopping frequently happens in basement grocers or shielded supermarkets with no cell service. Open-source local inference operates without an internet dependency.

Direct Architecture Control: Open models eliminate black-box behavioral drift. The extraction step is separated from deterministic rule validation, ensuring the tool never hallucinates dietary safety.

Model Portability: Using open-weight GGUF models allows the inference pipeline to run on consumer hardware with minimal memory overhead (under 4GB RAM).

Prize Categories
Open Source AI / Local Inference